Backend map / prompt audit / source index

# Nipux Project Atlas

A self-contained visual map of the current backend: entrypoints, daemon loop, worker prompt assembly, durable memory, tools, SQLite schema, UI control plane, tests, and every tracked source file with parsed functions/classes and line references.

Mind map

## Architecture

Click a node to jump into related detail.
Lifecycle

## Runtime Flow

What happens from terminal input to durable progress.

1. Startup pyproject entrypoint calls nipux_cli.cli:main. With no args, the chat/TUI opens on the focused job or first-run workspace.
2. Operator input Plain chat is stored as visible events and, when relevant, durable operator context for future worker prompts.
3. Daemon scheduling The daemon claims runnable jobs, keeps a lock/heartbeat, starts runs, and calls one bounded worker step repeatedly.
4. Prompt assembly worker.build_messages layers system prompt, program template, operator context, roadmaps, tasks, ledgers, experiments, memory, timeline, and recent steps.
5. Tool call The LLM selects one OpenAI-style tool. The registry executes it with ToolContext and stores input/output in steps/events.
6. Progress accounting Guards require artifacts, findings, tasks, experiments, or milestone validation when evidence or measurements appear.
7. Persistence Artifacts go to the job output directory. SQLite stores steps, events, ledgers, runtime state, and usage/cost metadata.
8. UI refresh The TUI reads timeline/events and compact job metrics, splitting chat from worker activity and status.

Exact text

## Prompt Surfaces

System/program prompts and instruction-like strings extracted from source.

### inline string

nipux_cli/chat_context.py:105

Context: `inline` · 640 chars

    You are Nipux, the chat model that controls a generic long-running agent workspace. You know the visible CLI state, focused job, job list, task queue, artifacts, memory, metrics, and recent activity. Answer directly from the visible job state. Do not claim hidden chain-of-thought. If the operator asks for work to be done, explain the concrete job/control action Nipux will take or how to run it from the Jobs/Status panel. If the operator asks where saved work is, explain that artifacts and history are visible from the Jobs/Status panel or direct CLI commands. Do not start replies with an introduction. Keep replies concise and useful.

### prompt_label

nipux_cli/chat_tui.py:106

Context: `assignment` · 1 chars

    ❯

### prompt_label

nipux_cli/chat_tui.py:109

Context: `assignment` · 1 chars

    ❯

### inline string

nipux_cli/cli.py:2857

Context: `inline` · 819 chars

    You are Nipux, the workspace chat model for a generic long-running agent CLI. Your job is to help the operator create, start, inspect, pause, resume, and steer worker jobs. You know the CLI concepts: jobs are long-running workers; artifacts are saved outputs; outcomes summarize durable progress; the updates page shows durable worker outcomes; the jobs page shows state, outputs, tasks, memory, findings, sources, experiments, and cost. Answer job-status questions from the job dossier. Mention concrete outputs, tasks, measurements, sources, blockers, and next branches when present. When the operator asks you to do new work, explain that Nipux will spin up a worker job; the harness will create the job from plain language. Keep replies concise, concrete, and operator-facing. Do not expose hidden chain-of-thought.

### prompt_label

nipux_cli/first_run_tui.py:81

Context: `assignment` · 5 chars

    Setup

### inline string

nipux_cli/templates.py:17

Context: `inline` · 1,339 chars

    ## Operating Rules

    - Work forever in bounded, resumable steps until the operator explicitly cancels or pauses the job.
    - Treat useful results as checkpoints, not endings: save the result, create the next branch, and continue.
    - Save important observations as artifacts.
    - Use report_update for short progress notes or blocked-state notes.
    - Use record_lesson when a source, mistake, operator preference, or strategy should affect future steps.
    - Use record_source and record_findings when those tools are available so the job improves its ledgers over time.
    - Use record_roadmap for broad work that needs milestones, feature groups, validation contracts, and roadmap-level checkpoints.
    - Use record_milestone_validation to validate milestones from evidence and create follow-up tasks when validation fails or blocks.
    - Use record_tasks to split broad objectives into durable branches with output contracts, acceptance criteria, required evidence, and stall behavior.
    - Use record_experiment whenever a branch produces measured results, comparisons, benchmarks, scores, or optimization data.
    - Use acknowledge_operator_context after incorporating or superseding active operator steering.
    - Use browser and web tools first. Do not assume memory is exact unless it points to an artifact.
    - Prefer quantity of attempts over one giant plan.

### inline string

nipux_cli/tools.py:1785

Context: `inline` · 329 chars

    Run a local shell command for CLI work. Use small read-only probes first. For long downloads, builds, training, crawls, or benchmarks, set a meaningful timeout, prefer resumable commands, and record or defer monitoring instead of repeatedly restarting short timed-out commands. Do not run destructive or high-risk cyber commands.

### prompt

nipux_cli/tui_layout.py:99

Context: `assignment` · 1 chars

### shell_guidance

nipux_cli/worker.py:306

Context: `assignment` · 48 chars

    Do not call shell_exec or do more research next.

### shell_guidance

nipux_cli/worker.py:308

Context: `assignment` · 185 chars

    Do not call broad shell_exec or do more research next. A single bounded shell_exec is allowed only when it validates one exact candidate path already listed in Candidate file discovery.

### inline string

nipux_cli/worker.py:1459

Context: `inline` · 472 chars

     Do not create new task branches. Either execute an existing high-priority branch, or use record_tasks only to update existing task titles to active, done, blocked, or skipped with concise result/evidence. If you have a near-duplicate task, update the closest existing task instead of inventing a fresh title. Consolidate branch sprawl into roadmap/milestones when useful. If this repeats, record_tasks is temporarily withheld so the worker must use a non-planning action.

### inline string

nipux_cli/worker.py:1829

Context: `inline` · 241 chars

    The last artifact read used a reference that does not exist. Do not invent or retry artifact ids. Use a valid recent artifact ref, call search_artifacts with a concrete query, or continue from already observed evidence with a durable record.

### inline string

nipux_cli/worker.py:5031

Context: `inline` · 305 chars

    Do not defer merely for a future worker turn to pick up ordinary work. Use defer_job only when waiting for a real external process, scheduled monitor interval, long-running command, or other time-based condition. Otherwise execute, measure, record a task/experiment/lesson, or mark the branch blocked now.

### inline string

nipux_cli/worker.py:5178

Context: `inline` · 351 chars

    A recent privileged/package-manager shell command failed due permission or authorization. Do not retry that class of command until the failure is accounted for. Use observed executable paths, user-writable installs, existing project files, or record_tasks/record_lesson/record_experiment to mark the branch blocked or choose a non-privileged recovery.

### guidance

nipux_cli/worker.py:5205

Context: `assignment` · 112 chars

    Use a different query, extract one of the prior result URLs, open a result in the browser, or write an artifact.

### guidance

nipux_cli/worker.py:5207

Context: `assignment` · 148 chars

    This artifact was already read. Do not read it again; use its content to inspect a concrete item, record findings/tasks, or write a report artifact.

### guidance

nipux_cli/worker.py:5212

Context: `assignment` · 161 chars

    This shell command was already run. Do not rerun discovery; use the previous output to inspect a specific file/item, write an artifact, or update findings/tasks.

### inline string

nipux_cli/worker.py:5239

Context: `inline` · 257 chars

    Browser automation is unavailable on this host. Do not retry browser tools until the runtime is installed or configured. Use web_search, web_extract, shell_exec, source/ledger tools, or record a blocked task/source and continue through a non-browser branch.

### inline string

nipux_cli/worker.py:5455

Context: `inline` · 334 chars

    This job already has many raw lessons and the connected memory graph is behind. Do not add another raw lesson. Use record_memory_graph to consolidate reusable strategy, mistake, constraint, decision, question, skill, or episode nodes with evidence links, or update existing tasks/roadmap/milestone state if this is only branch status.

### acceptance

nipux_cli/worker.py:6063

Context: `assignment` · 258 chars

    Use record_findings, record_source, record_experiment, record_tasks, record_roadmap, record_milestone_validation, or record_lesson to state exactly what the checkpoint proved, invalidated, changed, or failed to provide. Do not read the same checkpoint again.

### inline string

nipux_cli/worker.py:6064

Context: `inline` · 258 chars

    Use record_findings, record_source, record_experiment, record_tasks, record_roadmap, record_milestone_validation, or record_lesson to state exactly what the checkpoint proved, invalidated, changed, or failed to provide. Do not read the same checkpoint again.

### repair_prompt

nipux_cli/worker.py:7276

Context: `assignment` · 386 chars

    Your previous worker response did not call a tool. This worker must advance by calling exactly one available tool now. Do not answer in prose. Choose one bounded action that fits the current state, such as executing existing work, recording a measurement, updating an existing task, saving an evidence-backed output, recording a lesson/finding/source, or deferring only for a real wait.

### SYSTEM_PROMPT

nipux_cli/worker_policy.py:11

Context: `assignment` · 9,241 chars

    You are a long-running local work agent.

    Operate as a bounded worker, not a chat assistant. Choose one useful next step,
    call one of the available tools, and persist important evidence as artifacts.
    Do not claim the whole job is complete. A strong result is only a checkpoint:
    save it, report it, add the next tasks, and continue improving or broadening.

    Use a contract-first durable cycle. Read the objective, operator context,
    roadmap, active task, and recent evidence; choose the next action that satisfies
    the active output contract; produce or measure concrete evidence; update the
    right ledger; report the checkpoint; then open or continue the next branch.
    Research is only one possible contract. For action, experiment, monitor, report,
    or file-deliverable work, prefer execution, measurement, validation, or writing
    over more background collection. Keep moving forever until the operator pauses
    or cancels the job.
    The worker must not mark jobs completed or failed; use record_tasks,
    record_lesson, report_update, and artifacts to describe checkpoints, blockers,
    and next branches while the job stays runnable.

    Avoid loops. Do not repeat the same search query or the same exact tool call.
    If search results already exist, move forward by extracting source pages,
    opening a useful site in the browser, or saving a finding/evidence artifact.
    If a page has already been extracted and contains useful evidence, save that
    evidence with write_artifact before doing more searching or browsing.
    Only click or type browser refs from the most recent successful browser snapshot
    or navigation result. If a click/type fails with an unknown ref, use the fresh
    recovery snapshot or call browser_snapshot before retrying.
    If a source shows Cloudflare, login, paywall, or anti-bot verification, keep it
    visible in the trace. Do not bypass protections. Continue with normal visible
    browser actions when possible, persist what you have, or use alternate public
    sources if stuck.
    If a tool returns a list of actionable candidates such as files, packages,
    configurations, commands, sources, venues, records, branches, or options, do not
    keep re-listing the same candidate set with small formatting changes. Persist the
    candidate list once, choose the best candidate for the active contract, and move
    to execution, measurement, validation, or an explicit blocked decision.
    If a probe discovers a local/runtime candidate that might satisfy the active
    contract, promote that candidate immediately: record the fact, validate it with
    the smallest relevant action, and measure it before continuing external
    acquisition or research retries. Do not let an available local candidate fall
    out of context while pursuing lower-confidence external sources.
    If repeated external acquisition attempts fail with authentication, permission,
    quota, missing credentials, or unavailable resources, mark that branch blocked
    or low-yield and pivot to another source, local candidate, monitor/defer branch,
    or operator-visible credential requirement instead of retrying small variants.
    If a browser page says blocked, CAPTCHA, bot check, login required, paywall, or
    anti-bot, treat that page as a failed/low-yield source for the current job. Do
    not write an artifact that claims usable evidence exists unless the evidence is
    actually visible. Record the source outcome or pivot to another public source.
    Use report_update for short operator-readable progress notes when you need to
    say what you found or why you are blocked. Do not use report_update instead of
    write_artifact when you have durable evidence, findings, or report content to save.
    Use write_file when the objective requires a concrete file deliverable, source
    file, document, config, dataset, or other workspace output. If a measured
    experiment says the next action is to write, merge, update, compile, or insert
    content, prefer write_file or an execution command that actually changes the
    target over more read-only inspection.
    Use defer_job when the next useful step is to wait for an external process,
    scheduled check, long-running command, or monitor interval. Do not
    simulate waiting with repeated searches, reports, or shell probes.
    Use record_lesson when you learn something that should change future behavior:
    bad source patterns, task-specific success criteria, repeated mistakes, operator
    preferences, or a better strategy. Keep lessons short and reusable.
    Use record_memory_graph when work produces reusable connected knowledge: an
    episode worth remembering, a stable fact, a strategy, a reusable skill, an open
    question, a decision, or a constraint. Link nodes to their evidence and to each
    other. Treat this as the job's durable brain: recent events are fast episodic
    memory, while stable graph nodes are consolidated knowledge that should guide
    future branches without replaying raw history.
    Durable memory is not automatically true forever. If newer evidence contradicts
    an older memory-graph fact, constraint, strategy, or finding, update the older
    record as deprecated/resolved/stale and link the newer evidence before acting.
    Prefer fresh measured or directly observed evidence over stale summaries.
    Use record_source when a source is high-yield, low-yield, blocked, repetitive,
    or otherwise useful to score for future behavior.
    Use record_findings after finding durable candidates, facts, opportunities,
    experiments, files, bugs, sources, or other reusable outputs. Dedupe against the
    finding ledger and artifacts before saving.
    Use record_tasks to maintain a durable queue of objective-neutral branches:
    open work, active branch, blocked branch, completed branch, and skipped branch.
    Each task should include an output_contract (research, artifact, experiment,
    action, monitor, decision, or report), acceptance criteria, evidence needed,
    and stall behavior so progress is judged by evidence, not activity volume.
    Before marking a task or milestone done, audit the claim against the objective:
    list the requirement, the concrete artifact/file/finding/measurement/validation
    that proves it, and any remaining gap. If the audit is incomplete, keep the
    branch active or blocked and create the next smallest follow-up task.
    When the job is broad or starts looping, split it into tasks and move to the
    highest-priority open task rather than staying on one source or tactic forever.
    Use record_roadmap for broad, multi-phase, or ambiguous objectives that need a
    higher-level orchestration plan. A roadmap is generic: milestones group related
    features or work units; each milestone has acceptance criteria, evidence needed,
    and a validation contract. Use record_milestone_validation at milestone checkpoints
    to pass, fail, block, or create follow-up tasks from validation gaps. Keep the
    roadmap compact and update it from durable evidence, not from activity count.
    Use record_experiment for measurable trials, benchmarks, comparisons,
    optimization attempts, or hypothesis tests. A saved note, source, or artifact is
    not enough progress for a measurable objective: record the exact configuration,
    metric, result, whether higher or lower is better, and the next experiment. Keep
    improving against the best observed result instead of declaring victory after a
    single measurement.
    Use shell_exec for command-line work, repository inspection, diagnostics,
    benchmarks, repeatable experiments, and other command execution that the
    objective requires. Prefer small read-only probes before changing anything, use
    explicit timeouts, and save important command output with write_artifact before
    continuing. Do not run destructive or high-risk cyber commands.
    For long downloads, builds, training runs, crawls, benchmarks, or other slow
    actions, treat the action as a monitored branch: choose a timeout that can make
    meaningful progress, use resumable commands when available, record partial
    progress as an experiment/task/checkpoint, and use defer_job when the next useful
    step is to wait and check again. Do not repeatedly restart the same long action
    with short timeouts without recording what changed and how the next attempt will
    resume or differ.
    If a probe shows a partial output, incomplete file, running process, cache entry,
    checkpoint, or other unfinished artifact from an action branch, stop re-listing
    the same state. Either resume/continue the action with a resumable command,
    record a monitor/defer step for the still-running work, or record the branch as
    blocked with the concrete missing condition and next action.
    read_artifact only reads saved Nipux artifacts. Use shell_exec for repository,
    workspace, project, or filesystem files that are not saved artifacts.
    write_file writes workspace/local files directly; write_artifact writes Nipux's
    separate saved-output store. Use the right one for the operator-facing result.
    Operator messages are durable context from the human operator. Messages marked
    steer are active constraints until acknowledged or superseded. Messages marked
    follow_up are lower-priority queued work; keep them in the task queue and act on
    them after the current active branch has a durable checkpoint. Messages marked
    note are durable preferences. Use acknowledge_operator_context only after you
    have incorporated or intentionally superseded a steer/follow_up message.

### instruction

nipux_cli/worker_prompt_context.py:87

Context: `assignment` · 220 chars

    Take exactly one bounded next action. If recent state contains search results, do not search the same query again. If recent state contains extracted page evidence, write an artifact before doing more search or browsing.

### prompt_text

scripts/generate_project_atlas.py:592

Context: `assignment` · 102 chars

     prompt/instruction-like strings were extracted. Inspect this section after any agent-behavior change.

Tools

## Tool Registry

Static ToolSpec definitions from nipux_cli/tools.py.

Name | Description | Line
--- | --- | ---
`browser_navigate` | Navigate to a URL and return a compact browser snapshot. | 1735
`browser_snapshot` | Refresh the current page accessibility snapshot. | 1740
`browser_click` | Click an element by snapshot ref, for example @e5. | 1745
`browser_type` | Fill an input by snapshot ref. | 1750
`browser_scroll` | Scroll the current page up or down. | 1755
`browser_back` | Navigate back in browser history. | 1760
`browser_press` | Press a keyboard key in the browser. | 1761
`browser_console` | Read console errors or evaluate JavaScript in the current page. | 1766
`web_search` | Search the web for candidate sources. | 1775
`web_extract` | Extract markdown text from up to five URLs. | 1780
`shell_exec` | Run a local shell command for CLI work. Use small read-only probes first. For long downloads, builds, training, crawls, or benchmarks, set a meaningful timeout, prefer resumable commands, and record or defer monitoring instead of repeatedly restarting short timed-out commands. Do not run destructive or high-risk cyber commands. | 1785
`write_file` | Create, overwrite, or append a concrete workspace/local file for deliverables, code, documents, configs, or other file outputs. | 1795
`write_artifact` | Persist important findings, evidence, reports, or checkpoints to the job artifact store. | 1805
`read_artifact` | Read a saved artifact by artifact_id, visible number, exact saved path, or title. | 1816
`search_artifacts` | Search stored artifacts for exact evidence from prior steps. | 1826
`update_job_state` | Keep the current job runnable. Completion, failure, pausing, and cancellation are operator-only; workers should report checkpoints and continue. | 1831
`defer_job` | Wait before the next worker turn for this job. Use for long external processes, monitor/check-later tasks, or scheduled follow-up without completing or pausing the job. | 1839
`report_update` | Leave a short operator-readable progress note. Do not use this instead of write_artifact for durable evidence. | 1849
`record_lesson` | Save durable learning for this job: bad source patterns, success criteria, strategy changes, mistakes to avoid, or operator preferences. | 1858
`record_memory_graph` | Create or update the job | 1880
`search_memory_graph` | Search the job | 1943
`acknowledge_operator_context` | Acknowledge that durable operator steering has been incorporated or superseded. Use this after acting on a chat correction so it can leave the active context while remaining in history. | 1951
`record_source` | Update the source ledger with source quality, finding yield, failures, warnings, and last outcome. | 1960
`record_findings` | Update the finding ledger with evidence-backed useful results. Each finding needs an evidence anchor such as source_url/url, reason, evidence_artifact, or evidence metadata. | 1974
`record_tasks` | Create or update a durable queue of objective-neutral work branches. Use this to split long jobs into next actions, mark blocked branches, and keep the agent from cycling on one path. Missing task contract fields are filled with generic defaults from the task title. When the queue is saturated, near-duplicate task titles are folded into the matching existing task instead of creating another branch. | 2002
`record_roadmap` | Create or update a generic roadmap for broad work: milestones, features, success criteria, validation contract, scope, and current roadmap state. Use this before or during long-running work when task lists need higher-level structure. | 2033
`record_milestone_validation` | Record validation for a roadmap milestone and optionally create follow-up tasks for gaps. Use fresh evidence, acceptance criteria, and clear pass/fail/blocker reasons. | 2086
`record_experiment` | Track a measurable trial, benchmark, comparison, hypothesis test, or optimization attempt. Use this after any command or source produces a concrete result so future steps compare against the best observed result instead of treating notes as progress. Closed trials must include next_action so long-running work can continue from the result. | 2121
`send_digest_email` | Send or dry-run a digest email and save the body as an artifact. | 2140

Persistence

## SQLite Tables

CREATE TABLE blocks found in nipux_cli/db.py.

### schema_version

nipux_cli/db.py:24

- `version INTEGER NOT NULL`

### jobs

nipux_cli/db.py:28

- `id TEXT PRIMARY KEY`
- `title TEXT NOT NULL`
- `objective TEXT NOT NULL`
- `kind TEXT NOT NULL DEFAULT 'generic'`
- `status TEXT NOT NULL DEFAULT 'queued'`
- `priority INTEGER NOT NULL DEFAULT 0`
- `cadence TEXT`
- `created_at TEXT NOT NULL`
- `updated_at TEXT NOT NULL`
- `metadata_json TEXT NOT NULL DEFAULT '{}'`

### job_runs

nipux_cli/db.py:41

- `id TEXT PRIMARY KEY`
- `job_id TEXT NOT NULL REFERENCES jobs(id`

### steps

nipux_cli/db.py:53

- `id TEXT PRIMARY KEY`
- `job_id TEXT NOT NULL REFERENCES jobs(id`

### artifacts

nipux_cli/db.py:69

- `id TEXT PRIMARY KEY`
- `job_id TEXT NOT NULL REFERENCES jobs(id`

### evidence

nipux_cli/db.py:83

- `id TEXT PRIMARY KEY`
- `job_id TEXT NOT NULL REFERENCES jobs(id`

### memory_index

nipux_cli/db.py:94

- `id TEXT PRIMARY KEY`
- `job_id TEXT NOT NULL REFERENCES jobs(id`

### digests

nipux_cli/db.py:104

- `id TEXT PRIMARY KEY`
- `day TEXT NOT NULL`
- `target TEXT`
- `subject TEXT`
- `body_path TEXT`
- `sent_at TEXT`
- `status TEXT NOT NULL`
- `error TEXT`

### events

nipux_cli/db.py:115

- `id TEXT PRIMARY KEY`
- `job_id TEXT REFERENCES jobs(id`

Source index

## Important Files

95 Python modules plus docs/config files.

### AGENTS.md

32 lines

This repo is a focused long-running worker, not a broad assistant distribution.

0 symbols 0 TODOs
Imports and top symbols

Imports: none

Symbols: none

### README.md

378 lines

```text

0 symbols 0 TODOs
Imports and top symbols

Imports: none

Symbols: none

### RELEASE_CHECKLIST.md

38 lines

Use this before sharing the repository with outside users.

0 symbols 0 TODOs
Imports and top symbols

Imports: none

Symbols: none

### config.example.yaml

27 lines

model:

0 symbols 0 TODOs
Imports and top symbols

Imports: none

Symbols: none

### docs/long-running-memory-graph-design.md

62 lines

Nipux needs long-running workers that keep improving instead of flattening into repeated search, notes, or shallow checkpoints. The backend now treats each job as having a small durable "brain": a job-local memory graph made of connected nodes and links. It...

0 symbols 0 TODOs
Imports and top symbols

Imports: none

Symbols: none

### docs/pi-agent-core-port-plan.md

267 lines

Research date: 2026-04-30

0 symbols 0 TODOs
Imports and top symbols

Imports: none

Symbols: none

### nipux_cli/__init__.py

11 lines

Minimal daemon-first Nipux runtime. This package owns the daemon, state store, model adapter, artifact store, and fixed tool surface.

0 symbols 0 TODOs
Imports and top symbols

Imports: none

Symbols: none

### nipux_cli/__main__.py

9 lines

Run the Nipux CLI with ``python -m nipux_cli``.

0 symbols 0 TODOs
Imports and top symbols

Imports: __future__, nipux_cli.cli

Symbols: none

### nipux_cli/artifacts.py

138 lines

Artifact file storage for long-running jobs.

10 symbols 0 TODOs
Imports and top symbols

Imports: __future__, hashlib, re, dataclasses, pathlib, typing, nipux_cli.db

Symbols: safe_filename, sha256_text, StoredArtifact, ArtifactStore, __init__, job_dir, _assert_inside_home, write_text, read_text, search_text

### nipux_cli/browser.py

189 lines

Small `agent-browser` wrapper for the Nipux runtime.

15 symbols 0 TODOs
Imports and top symbols

Imports: __future__, json, os, shutil, subprocess, tempfile, hashlib, pathlib, typing, nipux_cli.config, nipux_cli.source_quality

Symbols: _find_agent_browser, _session_name, _profile_dir, _socket_dir, run_browser_command, navigate, snapshot, click, fill, scroll

### nipux_cli/chat_commands.py

286 lines

Slash-command dispatch for focused chat sessions.

3 symbols 0 TODOs
Imports and top symbols

Imports: __future__, argparse, dataclasses, typing, nipux_cli.config, nipux_cli.tui_style

Symbols: ChatCommandDeps, handle_chat_slash_command, _optional_int

### nipux_cli/chat_context.py

223 lines

Prompt context builder for the Nipux chat-side controller model.

8 symbols 0 TODOs
Imports and top symbols

Imports: __future__, typing, nipux_cli.db, nipux_cli.event_render, nipux_cli.metric_format, nipux_cli.operator_context, nipux_cli.tui_event_format, nipux_cli.tui_outcomes

Symbols: build_chat_messages, _durable_outcome_events, _durable_outcome_lines, _job_list_lines, _roadmap_lines, _experiment_line, _empty_section_text, _clip_chat_context

### nipux_cli/chat_controller.py

173 lines

Chat-controller behavior shared by the interactive CLI.

6 symbols 0 TODOs
Imports and top symbols

Imports: __future__, dataclasses, typing, nipux_cli.chat_intent

Symbols: ChatControllerDeps, handle_chat_message, chat_reply_text_and_metadata, handle_chat_control_intent, maybe_spawn_job_from_chat, queue_chat_note

### nipux_cli/chat_frame_runtime.py

571 lines

Terminal chat-frame runtime helpers.

27 symbols 0 TODOs
Imports and top symbols

Imports: __future__, queue, select, shutil, sys, termios, threading, time, tty, dataclasses, typing, typing

Symbols: ChatFrameDeps, compact_command_output, frame_next_job_id, next_chat_right_view, frame_refresh_interval, run_chat_frame, emit_frame_if_changed, _safe_render_frame, _fallback_chat_frame, _diff_frame_update

### nipux_cli/chat_intent.py

349 lines

Natural-language intent parsing for Nipux chat and shell control.

9 symbols 0 TODOs
Imports and top symbols

Imports: __future__, re

Symbols: natural_command_for, chat_control_command, _mentions_any, _looks_like_control_phrase, message_requests_immediate_run, message_requests_queued_job, extract_job_objective_from_message, looks_like_smalltalk, looks_like_job_objective

### nipux_cli/chat_tui.py

277 lines

Chat workspace terminal frame rendering.

10 symbols 0 TODOs
Imports and top symbols

Imports: __future__, typing, nipux_cli.config, nipux_cli.first_run_tui, nipux_cli.settings, nipux_cli.tui_commands, nipux_cli.tui_event_format, nipux_cli.tui_events, nipux_cli.tui_layout, nipux_cli.tui_outcomes, nipux_cli.tui_status, nipux_cli.tui_style

Symbols: build_chat_frame, _two_col_title, _two_col_line, _overlay_settings_modal, _settings_row, _rate_text, _metadata_records, _step_count, _step_line, _daemon_state_line

### nipux_cli/cli.py

3188 lines

Thin CLI for the Nipux agent runtime.

159 symbols 0 TODOs
Imports and top symbols

Imports: __future__, argparse, json, os, shlex, shutil, subprocess, sys, threading, time, contextlib, io

Symbols: _launch_agent_plist, _systemd_service_text, _db, _record_command_deps, cmd_init, cmd_update, cmd_uninstall, cmd_create, _ensure_model_setup_verified_for_workspace, _workspace_has_model_config

### nipux_cli/cli_help.py

92 lines

Help text and static branding for the Nipux command console.

2 symbols 0 TODOs
Imports and top symbols

Imports: __future__, typing

Symbols: print_shell_help, _print_group

### nipux_cli/cli_render.py

280 lines

Reusable text renderers for non-frame CLI commands.

22 symbols 0 TODOs
Imports and top symbols

Imports: __future__, json, os, shutil, textwrap, pathlib, typing, nipux_cli.event_render, nipux_cli.tui_event_format, nipux_cli.tui_status, nipux_cli.tui_style

Symbols: clip_json, print_step, print_artifact, print_run, print_wrapped, section_title, print_metric_grid, short_path, print_jobs_panel, next_operator_action

### nipux_cli/cli_state.py

126 lines

Persistent CLI focus state and job lookup helpers.

12 symbols 0 TODOs
Imports and top symbols

Imports: __future__, hashlib, json, datetime, pathlib, typing, nipux_cli.config, nipux_cli.db

Symbols: default_job_id, configured_focus_job_id, find_job, shell_state_path, read_shell_state, write_shell_state, setup_completed, mark_setup_completed, model_setup_fingerprint, model_setup_verified

### nipux_cli/compression.py

246 lines

Deterministic rolling memory summaries for long-running jobs.

7 symbols 0 TODOs
Imports and top symbols

Imports: __future__, nipux_cli.db, nipux_cli.memory_graph, nipux_cli.operator_context

Symbols: _clip_text, refresh_memory_index, _metadata_list, _rank_tasks, _compact_count, _context_fraction, _first_positive_int

### nipux_cli/config.py

271 lines

Configuration for the Nipux long-running agent runtime.

21 symbols 0 TODOs
Imports and top symbols

Imports: __future__, os, dataclasses, pathlib, typing, yaml

Symbols: get_agent_home, load_env_file, ensure_private_file_permissions, ensure_private_dir_permissions, write_private_text, ModelConfig, api_key, RuntimeConfig, state_db_path, jobs_dir

### nipux_cli/context_pressure.py

254 lines

Context-pressure signals for long-running worker prompts.

10 symbols 0 TODOs
Imports and top symbols

Imports: __future__, datetime, typing, nipux_cli.db

Symbols: context_pressure_for_prompt, usage_pressure_for_prompt, emit_usage_pressure_update, emit_context_pressure_update, compact_token_count, _usage_pressure_band, _context_pressure_band, _durable_usage_signal_count, _as_float, _as_int

### nipux_cli/daemon.py

695 lines

Daemon runner for restartable background jobs.

41 symbols 0 TODOs
Imports and top symbols

Imports: __future__, contextlib, fcntl, hashlib, json, os, signal, threading, time, dataclasses, datetime, email.utils

Symbols: DaemonAlreadyRunning, runtime_code_file_names, current_runtime_fingerprint, _runtime_code_fingerprint, _runtime_code_paths, runtime_stale, _parse_lock_metadata, daemon_lock_status, single_instance_lock, update_lock_metadata

### nipux_cli/daemon_control.py

243 lines

Daemon process control helpers used by CLI commands.

10 symbols 0 TODOs
Imports and top symbols

Imports: __future__, argparse, os, signal, subprocess, sys, time, pathlib, typing, nipux_cli.config, nipux_cli.cli_state, nipux_cli.daemon

Symbols: remote_model_preflight_failures, _recoverable_provider_preflight, recoverable_remote_model_preflight_failures, provider_preflight_is_recoverable, ensure_remote_model_ready_for_worker, cmd_start_impl, start_daemon_if_needed_impl, cmd_restart_impl, stop_daemon_process_impl, _find_single_daemon_process

### nipux_cli/dashboard.py

493 lines

Operator-facing dashboard state and rendering.

23 symbols 0 TODOs
Imports and top symbols

Imports: __future__, collections, datetime, pathlib, textwrap, typing, nipux_cli.config, nipux_cli.daemon, nipux_cli.db, nipux_cli.operator_context, nipux_cli.scheduling, nipux_cli.tools

Symbols: collect_dashboard_state, render_dashboard, render_overview, _select_focus_job, _job_card, _focus_state, _render_focus, _public_run, _public_step, _public_artifact

### nipux_cli/db.py

2752 lines

SQLite state store for the Nipux agent.

97 symbols 0 TODOs
Imports and top symbols

Imports: __future__, json, random, re, sqlite3, threading, time, uuid, datetime, pathlib, typing, nipux_cli.metric_format

Symbols: utc_now, new_id, _slugify, _unique_job_id, _json_dumps, _json_loads, _bounded_float, _merge_string_lists, _memory_edge_key, _as_int

### nipux_cli/digest.py

380 lines

Digest rendering and optional email delivery.

9 symbols 0 TODOs
Imports and top symbols

Imports: __future__, smtplib, datetime, email.message, pathlib, typing, nipux_cli.config, nipux_cli.db, nipux_cli.operator_context, nipux_cli.tui_layout

Symbols: _metadata_list, _active_operator_messages, _safe_int, _latest_run_model, _usage_lines, render_job_digest, send_digest_email, render_daily_digest, write_daily_digest

### nipux_cli/doctor.py

260 lines

Runtime checks for the Nipux agent.

15 symbols 0 TODOs
Imports and top symbols

Imports: __future__, json, shutil, urllib.error, urllib.request, dataclasses, pathlib, typing, urllib.parse, nipux_cli.config, nipux_cli.db, nipux_cli.tools

Symbols: Check, as_dict, _check_writable_dir, _check_db, _check_tool_surface, _check_model_config, _check_browser_runtime, _check_model_endpoint, _check_model_generation, _check_openrouter_auth

### nipux_cli/event_render.py

118 lines

Readable event rendering shared by CLI history and chat context.

4 symbols 0 TODOs
Imports and top symbols

Imports: __future__, shlex, typing, nipux_cli.metric_format, nipux_cli.tui_event_format, nipux_cli.tui_style

Symbols: event_line, event_display_parts, event_label, compact_time

### nipux_cli/first_run_controller.py

170 lines

First-run command decisions for the Nipux TUI.

7 symbols 0 TODOs
Imports and top symbols

Imports: __future__, shlex, contextlib, dataclasses, io, typing, nipux_cli.settings, nipux_cli.tui_commands, nipux_cli.frame_snapshot

Symbols: FirstRunFrameDeps, handle_first_run_action, handle_first_run_frame_line, first_run_chat_reply, create_first_run_job, capture_first_run_command, first_token

### nipux_cli/first_run_frame_runtime.py

436 lines

Terminal runtime for the first-run Nipux workspace.

17 symbols 0 TODOs
Imports and top symbols

Imports: __future__, select, shutil, sys, termios, time, tty, dataclasses, typing, urllib.parse, nipux_cli.config, nipux_cli.settings

Symbols: FirstRunRuntimeDeps, run_first_run_frame, clamp_selection, _safe_render_frame, _fallback_first_run_frame, _submit_first_run_line, _handle_first_run_escape, directional_first_run_action, _apply_first_run_action, _handle_edit_input

### nipux_cli/first_run_tui.py

571 lines

First-run terminal UI rendering for Nipux.

38 symbols 0 TODOs
Imports and top symbols

Imports: __future__, typing, nipux_cli.config, nipux_cli.settings, nipux_cli.tui_layout, nipux_cli.tui_style

Symbols: build_first_run_frame, first_run_columns, first_run_actions, first_run_themed_lines, _wizard_body_lines, _model_page_lines, _endpoint_page_lines, _api_page_lines, _access_page_lines, _doctor_page_lines

### nipux_cli/frame_snapshot.py

183 lines

Data loading contract for the interactive Nipux terminal frame.

5 symbols 0 TODOs
Imports and top symbols

Imports: __future__, typing, nipux_cli.config, nipux_cli.daemon, nipux_cli.db, nipux_cli.tui_outcomes

Symbols: load_frame_snapshot, load_workspace_frame_snapshot, _safe_job, _workspace_token_usage, _summary_events

### nipux_cli/llm.py

288 lines

LLM provider adapters for one bounded worker step.

23 symbols 0 TODOs
Imports and top symbols

Imports: __future__, json, urllib.error, urllib.parse, urllib.request, dataclasses, typing, openai, nipux_cli.config

Symbols: ToolCall, LLMResponse, LLMResponseError, __init__, StepLLM, next_action, OpenAIChatLLM, __init__, next_action, complete

### nipux_cli/measurement.py

141 lines

Measurement parsing helpers for generic progress accounting.

7 symbols 0 TODOs
Imports and top symbols

Imports: __future__, re, typing

Symbols: measurement_candidates, _table_measurement_candidates, _split_markdown_table_row, _is_markdown_separator_row, _table_measurement_label, measurement_candidates_are_diagnostic_only, _candidate_is_diagnostic_only

### nipux_cli/memory_graph.py

302 lines

Job-local memory graph helpers for long-running workers.

14 symbols 0 TODOs
Imports and top symbols

Imports: __future__, re, datetime, typing, nipux_cli.worker_prompt_format

Symbols: memory_graph_from_job, memory_graph_for_prompt, _node_contains_stale_token, _node_has_negative_memory_marker, _node_has_stale_id, search_memory_graph, rank_memory_nodes, _node_score, _edge_index, _tokens

### nipux_cli/memory_graph_view.py

299 lines

Self-contained HTML view for a job-local memory graph.

4 symbols 0 TODOs
Imports and top symbols

Imports: __future__, json, html, typing, nipux_cli.memory_graph

Symbols: render_memory_graph_html, _view_node, _view_edge, _string_list

### nipux_cli/metric_format.py

17 lines

Small formatting helpers for measured worker results.

1 symbols 0 TODOs
Imports and top symbols

Imports: __future__, typing

Symbols: format_metric_value

### nipux_cli/operator_context.py

83 lines

Generic filtering for operator messages that enter worker context.

6 symbols 0 TODOs
Imports and top symbols

Imports: __future__, re, typing

Symbols: operator_entry_is_active, operator_entry_is_prompt_relevant, active_prompt_operator_entries, inactive_prompt_operator_ids, _conversation_only, _actionable

### nipux_cli/parser_builder.py

356 lines

Argparse construction for Nipux CLI commands.

2 symbols 0 TODOs
Imports and top symbols

Imports: __future__, argparse, collections.abc

Symbols: _handler, build_arg_parser

### nipux_cli/planning.py

384 lines

Generic initial planning primitives for long-running jobs.

9 symbols 0 TODOs
Imports and top symbols

Imports: __future__, re, typing

Symbols: objective_profiles, initial_plan_for_objective, initial_task_contract, initial_roadmap_for_objective, _initial_summary_for_profiles, _initial_tasks_for_profiles, _initial_questions_for_profiles, _primary_execution_contract, format_initial_plan

### nipux_cli/progress.py

213 lines

Generic progress summaries for long-running jobs.

12 symbols 0 TODOs
Imports and top symbols

Imports: __future__, dataclasses, typing

Symbols: ProgressCheckpoint, build_progress_checkpoint, ledger_counts, ledger_update_counts, ledger_resolution_counts, recent_progress_bits, _metadata_list, _clip_text, _updated_existing_record, _record_after_checkpoint

### nipux_cli/provider_errors.py

64 lines

Generic model-provider error classification.

4 symbols 0 TODOs
Imports and top symbols

Imports: __future__, json, typing

Symbols: provider_error_text, provider_action_required, provider_action_required_note, provider_rate_limited

### nipux_cli/record_commands.py

542 lines

Read-only CLI commands for job records, ledgers, memory, and usage.

16 symbols 0 TODOs
Imports and top symbols

Imports: __future__, json, dataclasses, pathlib, typing, nipux_cli.artifacts, nipux_cli.cli_render, nipux_cli.cli_render, nipux_cli.daemon, nipux_cli.memory_graph, nipux_cli.memory_graph_view, nipux_cli.tui_status

Symbols: RecordCommandDeps, cmd_findings_impl, cmd_tasks_impl, cmd_roadmap_impl, cmd_experiments_impl, cmd_sources_impl, cmd_memory_impl, _write_memory_graph_view, cmd_metrics_impl, cmd_usage_impl

### nipux_cli/scheduling.py

76 lines

Shared scheduling helpers for deferred long-running work.

6 symbols 0 TODOs
Imports and top symbols

Imports: __future__, datetime, typing

Symbols: job_deferred_until, job_is_deferred, job_provider_blocked, provider_retry_metadata, operator_resume_metadata, _metadata_time

### nipux_cli/service_install.py

179 lines

OS service installation helpers for the Nipux daemon.

7 symbols 0 TODOs
Imports and top symbols

Imports: __future__, os, shlex, shutil, subprocess, sys, argparse, pathlib, nipux_cli.config

Symbols: launch_agent_path, launch_agent_plist, systemd_service_path, systemd_service_text, cmd_autostart, cmd_service, xml_escape

### nipux_cli/settings.py

153 lines

Inline config editing helpers for Nipux slash commands.

12 symbols 0 TODOs
Imports and top symbols

Imports: __future__, os, pathlib, typing, yaml, nipux_cli.config, nipux_cli.cli_state, nipux_cli.tui_commands

Symbols: config_field_value, save_config_field, inline_setting_notice, edit_target_label, edit_target_hint, edit_target_masks_input, _config_path, _load_config_yaml, _save_config_yaml, _save_env_secret

### nipux_cli/settings_commands.py

84 lines

Slash-command handlers for inline Nipux configuration.

5 symbols 0 TODOs
Imports and top symbols

Imports: __future__, shlex, contextlib, io, nipux_cli.config, nipux_cli.settings, nipux_cli.tui_commands

Symbols: handle_chat_setting_command, config_summary_lines, _rate_text, _cost_limit_text, capture_setting_command

### nipux_cli/shell_tools.py

348 lines

Shell and workspace file tools for Nipux workers.

22 symbols 0 TODOs
Imports and top symbols

Imports: __future__, contextlib, json, os, re, signal, subprocess, time, datetime, pathlib, typing

Symbols: write_file, shell_exec, cleanup_registered_shell_processes, _shell_error, _shell_success_anomaly, _missing_executable_probe, _empty_observation_probe, _shell_missing_command_anomaly, _shell_sudo_password_anomaly, _shell_build_error_anomaly

### nipux_cli/source_quality.py

32 lines

Source quality checks for web and browser tools.

1 symbols 0 TODOs
Imports and top symbols

Imports: __future__

Symbols: anti_bot_reason

### nipux_cli/task_match.py

102 lines

Task title matching helpers for long-running job queues.

4 symbols 0 TODOs
Imports and top symbols

Imports: __future__, re, typing

Symbols: task_key, find_semantic_task_match, _task_tokens, _task_similarity

### nipux_cli/templates.py

67 lines

Program templates for generic long-running jobs.

3 symbols 0 TODOs
Imports and top symbols

Imports: __future__

Symbols: program_for_job, _generic_template, _research_paper_template

### nipux_cli/tools.py

2229 lines

Static tool registry for the Nipux agent.

68 symbols 0 TODOs
Imports and top symbols

Imports: __future__, json, re, time, dataclasses, datetime, typing, nipux_cli.artifacts, nipux_cli.config, nipux_cli.db, nipux_cli.metric_format, nipux_cli.digest

Symbols: ToolContext, ToolSpec, as_openai_tool, _missing_argument, _placeholder_argument, _schema_placeholder_arguments, _schema_missing_arguments, _json, _write_artifact, _read_artifact

### nipux_cli/tui_commands.py

296 lines

Slash command metadata and command-palette helpers for the TUI.

7 symbols 0 TODOs
Imports and top symbols

Imports: __future__, nipux_cli.tui_style

Symbols: slash_suggestion_lines, autocomplete_slash, slash_completion_for_submit, _slash_argument_text, _slash_argument_footer, cycle_slash, _slash_command_matches

### nipux_cli/tui_event_format.py

262 lines

Shared event formatting helpers for Nipux terminal renderers.

18 symbols 0 TODOs
Imports and top symbols

Imports: __future__, os, re, shlex, pathlib, typing, nipux_cli.metric_format, nipux_cli.tui_style

Symbols: event_tool_args, shell_write_target, event_title_body, experiment_metric_text, event_clock, event_hour, friendly_error_text, brief_reflection_text, generic_display_text, clean_step_summary

### nipux_cli/tui_events.py

371 lines

Compact event rendering helpers for the Nipux terminal UI.

16 symbols 0 TODOs
Imports and top symbols

Imports: __future__, re, textwrap, typing, nipux_cli.tui_event_format, nipux_cli.tui_style

Symbols: chat_event_parts, append_chat_output, chat_pane_lines, _chat_item_lines, _flatten_chat_blocks, _chat_label, _normalized_chat_body, _is_low_value_chat_notice, _is_waiting_notice, _is_generic_chat_notice

### nipux_cli/tui_input.py

80 lines

Terminal input helpers for full-screen Nipux frames.

5 symbols 0 TODOs
Imports and top symbols

Imports: __future__, os, re, select, sys, time

Symbols: read_terminal_char, read_escape_sequence, terminal_escape_complete, decode_terminal_escape, drain_pending_input

### nipux_cli/tui_layout.py

234 lines

Reusable terminal layout primitives for Nipux frames.

16 symbols 0 TODOs
Imports and top symbols

Imports: __future__, typing, nipux_cli.tui_style

Symbols: _top_bar, _two_col_title, _two_col_line, _edge_line, _triple_line, _compose_bar, _metric_strip, _pill, _token_usage_topline, _model_cost_is_zero

### nipux_cli/tui_outcomes.py

508 lines

Durable outcome summaries for the Nipux terminal UI.

18 symbols 0 TODOs
Imports and top symbols

Imports: __future__, textwrap, typing, nipux_cli.tui_event_format, nipux_cli.tui_style

Symbols: model_update_event_parts, is_summary_event_candidate, latest_durable_outcome_line, latest_hour_outcome_summary_line, visible_outcome_summary_line, job_outcome_summary, outcome_counts, recent_model_update_lines, chat_updates_pane_lines, _wrapped_label_line

### nipux_cli/tui_status.py

540 lines

Status and work-pane renderers for the Nipux terminal UI.

23 symbols 0 TODOs
Imports and top symbols

Imports: __future__, textwrap, typing, nipux_cli.config, nipux_cli.operator_context, nipux_cli.scheduling, nipux_cli.tui_event_format, nipux_cli.tui_events, nipux_cli.tui_outcomes, nipux_cli.tui_layout, nipux_cli.tui_style

Symbols: worker_label, job_display_state, active_operator_messages, right_pane_lines, _is_workspace_placeholder, _empty_workspace_status_lines, chat_work_pane_lines, chat_settings_pane_lines, frame_jobs_lines, _job_compact_work_lines

### nipux_cli/tui_style.py

153 lines

Small terminal styling helpers shared by the CLI frame renderers.

15 symbols 0 TODOs
Imports and top symbols

Imports: __future__, os, re, sys, typing

Symbols: _fancy_ui, _style, _accent, _muted, _bold, _one_line, _strip_ansi, _fit_ansi, _center_ansi, _themed_lines

### nipux_cli/uninstall.py

217 lines

Uninstall helpers for local Nipux runtime state.

11 symbols 0 TODOs
Imports and top symbols

Imports: __future__, os, shutil, subprocess, dataclasses, pathlib, typing, nipux_cli.config, nipux_cli.service_install

Symbols: UninstallPlan, build_uninstall_plan, uninstall_runtime, uninstall_installed_tool, installed_tool_paths, _disable_services, _run_command, _process_lines, _dedupe_paths, _assert_safe_delete_target

### nipux_cli/updater.py

179 lines

Self-update helpers for source checkouts and installed tools.

10 symbols 0 TODOs
Imports and top symbols

Imports: __future__, os, shutil, subprocess, collections.abc, pathlib

Symbols: find_checkout_root, update_checkout, _update_uv_tool_install, _verify_updated_command, _uv_tool_update_spec, _run_git, _run_command, _process_lines, _git_text, _short_path

### nipux_cli/updates.py

132 lines

Readable durable progress reports for jobs.

4 symbols 0 TODOs
Imports and top symbols

Imports: __future__, shlex, typing, nipux_cli.config, nipux_cli.daemon, nipux_cli.db, nipux_cli.tui_outcomes, nipux_cli.tui_status, nipux_cli.tui_style

Symbols: render_updates_report, render_all_updates_report, _metadata_list, _metadata_count

### nipux_cli/usage.py

78 lines

Formatting helpers for model token and cost usage.

4 symbols 0 TODOs
Imports and top symbols

Imports: __future__, typing, nipux_cli.tui_layout

Symbols: format_usage_report, _safe_int, _safe_float, _safe_optional_float

### nipux_cli/web.py

121 lines

Small web search/extract helpers without external web tool dependencies.

11 symbols 0 TODOs
Imports and top symbols

Imports: __future__, html, re, urllib.parse, urllib.request, html.parser, typing, nipux_cli.source_quality

Symbols: _TextExtractor, __init__, handle_starttag, handle_endtag, handle_data, text, _request, _strip_html, _duckduckgo_link, web_search

### nipux_cli/worker.py

7538 lines

Bounded worker loop for one restartable agent step.

241 symbols 1 TODOs
Imports and top symbols

Imports: __future__, json, re, shlex, signal, threading, time, dataclasses, datetime, pathlib, typing, urllib.parse

Symbols: StepExecution, build_messages, _acknowledge_non_prompt_operator_context, _measured_progress_guard_for_prompt, _deliverable_progress_guard_for_prompt, _experiment_stagnation_guard_for_prompt, _measurement_obligation_for_prompt, _recent_measurement_evidence_for_prompt, _file_validation_obligation_for_prompt, _current_execution_focus_for_prompt

### nipux_cli/worker_policy.py

489 lines

Static worker prompt and loop policy constants.

0 symbols 0 TODOs
Imports and top symbols

Imports: __future__, re

Symbols: none

### nipux_cli/worker_prompt_context.py

921 lines

Prompt-context renderers for the Nipux worker loop.

35 symbols 0 TODOs
Imports and top symbols

Imports: __future__, re, typing, nipux_cli.memory_graph, nipux_cli.metric_format, nipux_cli.operator_context, nipux_cli.tui_outcomes, nipux_cli.worker_policy, nipux_cli.worker_prompt_format

Symbols: _memory_entries_for_prompt, _render_worker_prompt, _redact_stale_tokens_for_prompt, _match_inside_path_like_span, _operator_messages_for_prompt, _operator_message_line, _lessons_for_prompt, _lesson_prompt_text, _positive_durable_lines_for_lesson_conflicts, _negative_lesson_conflict_tokens

### nipux_cli/worker_prompt_format.py

223 lines

Prompt-facing summaries for worker history and tool observations.

8 symbols 0 TODOs
Imports and top symbols

Imports: __future__, json, re, pathlib, typing, nipux_cli.metric_format, nipux_cli.source_quality, nipux_cli.worker_policy

Symbols: compact, clip_text, format_step_for_prompt, observation_for_prompt, clean_prompt_candidate_path, browser_candidates_for_prompt, _looks_like_metric_cell, _looks_like_service_description

### nipux_cli/worker_tool_summary.py

99 lines

Compact result summaries for worker tool executions.

1 symbols 0 TODOs
Imports and top symbols

Imports: __future__, typing, nipux_cli.metric_format

Symbols: summarize_tool_result

### nipux_cli/worker_usage.py

48 lines

Usage accounting for worker model turns.

3 symbols 0 TODOs
Imports and top symbols

Imports: __future__, json, typing, nipux_cli.llm

Symbols: turn_usage_metadata, estimate_token_count, _as_int

### plans/nipux-runtime-notes.md

50 lines

Nipux is a narrow, restartable worker for long-running browser, web research,

0 symbols 0 TODOs
Imports and top symbols

Imports: none

Symbols: none

### pyproject.toml

54 lines

[build-system]

0 symbols 0 TODOs
Imports and top symbols

Imports: none

Symbols: none

### scripts/generate_project_atlas.py

619 lines

Generate docs/project-atlas.html from the tracked Nipux source tree.

32 symbols 2 TODOs
Imports and top symbols

Imports: __future__, ast, html, re, subprocess, dataclasses, pathlib, typing

Symbols: SourceFile, Symbol, Prompt, main, load_source_files, tracked_paths, git, extract_symbols, call_names, dotted_name

### scripts/live_memory_graph_smoke.py

226 lines

Run an opt-in real-model smoke test for memory-graph tool calling. This script is intentionally outside the normal Nipux runtime path. It creates an isolated temporary Nipux home, seeds generic durable job state, and verifies that a configured OpenAI-compat...

5 symbols 0 TODOs
Imports and top symbols

Imports: __future__, argparse, json, os, shutil, sys, tempfile, pathlib, typing, nipux_cli.config, nipux_cli.db, nipux_cli.memory_graph

Symbols: main, _seed_metadata, _execution_summary, _finish, _human_summary

### scripts/render_nipux_ascii_video.py

523 lines

Render a Nipux ASCII-art CLI intro as an MP4. The renderer is dependency-light on purpose: it draws a small embedded bitmap font into raw RGB frames and pipes those frames directly to ffmpeg.

26 symbols 0 TODOs
Imports and top symbols

Imports: __future__, argparse, math, random, shutil, subprocess, dataclasses, pathlib

Symbols: Cell, TextGrid, __init__, set, put, center, box, clamp, ease, mix

### tests/nipux_cli/test_artifacts.py

34 lines

No module docstring.

2 symbols 0 TODOs
Imports and top symbols

Imports: pytest, nipux_cli.artifacts, nipux_cli.db

Symbols: test_artifact_store_writes_reads_and_searches, test_artifact_store_rejects_paths_outside_home

### tests/nipux_cli/test_browser_web.py

118 lines

No module docstring.

11 symbols 0 TODOs
Imports and top symbols

Imports: json, nipux_cli.browser, nipux_cli.tools, nipux_cli.web, nipux_cli.artifacts, nipux_cli.config, nipux_cli.db

Symbols: test_session_name_is_stable_and_safe, test_long_session_name_is_short_and_hashed, test_strip_html_removes_scripts_and_keeps_text, test_browser_marks_anti_bot_interstitial_as_warning, test_browser_marks_captcha_block_as_warning, test_web_extract_marks_anti_bot_pages_as_warning, fake_request, test_browser_tool_uses_native_wrapper, fake_navigate, test_browser_click_adds_recovery_snapshot_for_stale_ref

### tests/nipux_cli/test_cli.py

4970 lines

No module docstring.

231 symbols 0 TODOs
Imports and top symbols

Imports: json, queue, subprocess, time, pathlib, nipux_cli.artifacts, nipux_cli, nipux_cli.chat_frame_runtime, nipux_cli.chat_frame_runtime, nipux_cli.chat_frame_runtime, nipux_cli.chat_frame_runtime, nipux_cli.chat_frame_runtime

Symbols: _mode, _mark_test_model_ready, test_cli_has_operator_commands, test_cli_version_flag, test_main_catches_keyboard_interrupt_without_traceback, interrupt, test_python_module_entrypoint_uses_cli_main, test_init_openrouter_writes_secret_free_config_and_env_template, test_init_defaults_to_local_endpoint, test_init_openrouter_defaults_to_generic_route

### tests/nipux_cli/test_cli_model_preflight.py

86 lines

No module docstring.

12 symbols 0 TODOs
Imports and top symbols

Imports: types, nipux_cli.cli, nipux_cli.doctor

Symbols: _config, test_remote_model_preflight_blocks_rejected_auth, fake_doctor, test_remote_model_preflight_allows_recovery_monitor_for_quota, fake_doctor, test_remote_model_preflight_skips_fake_runs, fake_doctor, test_model_preflight_checks_local_endpoints, fake_doctor, test_start_does_not_spawn_daemon_when_model_preflight_fails

### tests/nipux_cli/test_compression.py

101 lines

No module docstring.

1 symbols 0 TODOs
Imports and top symbols

Imports: nipux_cli.compression, nipux_cli.db

Symbols: test_refresh_memory_index_includes_durable_progress_ledgers

### tests/nipux_cli/test_config.py

143 lines

No module docstring.

7 symbols 0 TODOs
Imports and top symbols

Imports: pathlib, nipux_cli.config

Symbols: _mode, test_load_config_defaults_to_local_endpoint, test_load_config_from_yaml, test_load_config_reads_local_env_file, test_load_config_tightens_local_env_permissions, test_default_config_yaml_allows_provider_template_without_secret, test_config_example_matches_default_local_endpoint

### tests/nipux_cli/test_daemon.py

560 lines

No module docstring.

41 symbols 0 TODOs
Imports and top symbols

Imports: datetime, json, threading, time, pytest, nipux_cli.config, nipux_cli.daemon, nipux_cli.daemon_control, nipux_cli.db, nipux_cli.worker, nipux_cli.doctor

Symbols: test_single_instance_lock_rejects_second_holder, test_daemon_lock_status_reports_free_lock, test_lock_metadata_can_be_updated_while_held, test_lock_metadata_update_restores_missing_process_fields, test_daemon_lock_heartbeat_updates_while_worker_turn_runs, SlowDaemon, run_once, test_stop_daemon_recovers_pidless_lock_from_process_list, PsResult, test_daemon_lock_status_detects_stale_runtime

### tests/nipux_cli/test_dashboard.py

86 lines

No module docstring.

3 symbols 0 TODOs
Imports and top symbols

Imports: datetime, nipux_cli.artifacts, nipux_cli.config, nipux_cli.dashboard, nipux_cli.db

Symbols: test_dashboard_collects_jobs_steps_and_artifacts, test_overview_marks_idle_daemon_as_ready_for_work, test_overview_marks_old_heartbeat_as_busy_for_running_step

### tests/nipux_cli/test_db.py

609 lines

No module docstring.

19 symbols 0 TODOs
Imports and top symbols

Imports: nipux_cli.db

Symbols: test_db_job_run_step_and_artifact_roundtrip, test_create_job_uses_unique_readable_slug_ids, test_step_numbers_increment_across_runs_for_a_job, test_job_token_usage_aggregates_message_usage, test_append_operator_message_roundtrip, test_claim_operator_messages_marks_one_message_at_a_time, test_acknowledge_operator_messages_marks_delivered_context, test_rename_job_updates_title_without_changing_id, test_delete_job_removes_related_rows, test_append_lesson_roundtrip

### tests/nipux_cli/test_digest.py

43 lines

No module docstring.

1 symbols 0 TODOs
Imports and top symbols

Imports: nipux_cli.config, nipux_cli.db, nipux_cli.digest

Symbols: test_daily_digest_includes_ledgers_lessons_sources_and_strategy

### tests/nipux_cli/test_doctor.py

157 lines

No module docstring.

13 symbols 0 TODOs
Imports and top symbols

Imports: io, json, urllib.error, nipux_cli.config, nipux_cli.doctor

Symbols: FakeHTTPResponse, __init__, __enter__, __exit__, read, test_doctor_checks_local_runtime_without_model_call, test_doctor_warns_when_remote_model_key_is_missing, test_doctor_reports_openrouter_auth_failure, fake_urlopen, test_doctor_reports_generation_limit_after_model_listing

### tests/nipux_cli/test_generic_runtime_audit.py

35 lines

No module docstring.

1 symbols 0 TODOs
Imports and top symbols

Imports: pathlib

Symbols: test_runtime_code_has_no_task_specific_literals

### tests/nipux_cli/test_live_memory_graph_smoke.py

37 lines

No module docstring.

3 symbols 0 TODOs
Imports and top symbols

Imports: importlib.util, sys, pathlib, nipux_cli.memory_graph

Symbols: _load_live_smoke, test_live_memory_graph_smoke_fails_cleanly_without_key, test_live_memory_graph_smoke_seed_pushes_generic_consolidation

### tests/nipux_cli/test_llm.py

151 lines

No module docstring.

23 symbols 0 TODOs
Imports and top symbols

Imports: types, nipux_cli.config, nipux_cli.llm

Symbols: _FakeCompletions, __init__, create, test_chat_llm_requires_tool_choice_for_worker_actions, FakeOpenAI, __init__, test_chat_llm_retries_without_tool_choice_when_provider_rejects_it, RejectingCompletions, create, FakeOpenAI

### tests/nipux_cli/test_measurement.py

32 lines

No module docstring.

2 symbols 0 TODOs
Imports and top symbols

Imports: nipux_cli.measurement

Symbols: test_measurement_candidates_extract_markdown_table_unit_columns, test_measurement_candidates_extract_generic_table_metrics

### tests/nipux_cli/test_metric_format.py

11 lines

No module docstring.

2 symbols 0 TODOs
Imports and top symbols

Imports: nipux_cli.metric_format

Symbols: test_format_metric_value_spaces_named_units, test_format_metric_value_keeps_attached_symbol_units

### tests/nipux_cli/test_operator_context.py

30 lines

No module docstring.

4 symbols 0 TODOs
Imports and top symbols

Imports: nipux_cli.operator_context

Symbols: _entry, test_conversation_only_operator_messages_do_not_enter_worker_prompt, test_actionable_operator_messages_remain_worker_constraints, test_inactive_prompt_operator_ids_returns_only_conversation_active_messages

### tests/nipux_cli/test_planning.py

90 lines

No module docstring.

8 symbols 0 TODOs
Imports and top symbols

Imports: nipux_cli.planning

Symbols: test_initial_task_contracts_are_generic_and_complete, test_initial_roadmap_uses_valid_generic_contracts, test_initial_plan_adapts_to_measurable_objectives, test_initial_plan_adapts_to_deliverable_objectives, test_initial_plan_treats_generated_files_as_deliverables, test_initial_plan_adapts_to_monitoring_objectives, test_initial_plan_does_not_add_meta_progress_update_task, test_objective_profiles_stay_generic

### tests/nipux_cli/test_progress.py

214 lines

No module docstring.

7 symbols 0 TODOs
Imports and top symbols

Imports: nipux_cli.progress

Symbols: test_progress_checkpoint_reports_deltas_and_recent_durable_work, test_progress_checkpoint_for_saved_output_is_concise, test_progress_checkpoint_without_delta_is_activity_not_progress, test_progress_checkpoint_counts_existing_record_updates_as_progress, test_progress_checkpoint_ignores_non_substantive_record_touches, test_progress_checkpoint_counts_roadmap_updates_and_validations, test_progress_helpers_ignore_malformed_metadata

### tests/nipux_cli/test_project_atlas.py

46 lines

No module docstring.

3 symbols 0 TODOs
Imports and top symbols

Imports: importlib.util, sys, pathlib

Symbols: _load_generator, test_project_atlas_generator_maps_prompts_tools_and_source_without_self_embedding, test_project_atlas_redacts_secret_assignments_from_rendered_source

### tests/nipux_cli/test_provider_errors.py

21 lines

No module docstring.

3 symbols 0 TODOs
Imports and top symbols

Imports: nipux_cli.provider_errors

Symbols: ProviderPayloadError, test_provider_action_required_detects_payload_and_status_text, test_provider_rate_limited_detects_transient_rate_text

### tests/nipux_cli/test_templates.py

15 lines

No module docstring.

1 symbols 0 TODOs
Imports and top symbols

Imports: nipux_cli.templates

Symbols: test_generic_template_pushes_artifacts_and_updates

### tests/nipux_cli/test_tools.py

2166 lines

No module docstring.

69 symbols 0 TODOs
Imports and top symbols

Imports: json, os, signal, subprocess, time, nipux_cli.artifacts, nipux_cli.config, nipux_cli.db, nipux_cli.shell_tools, nipux_cli.tools

Symbols: test_static_tool_surface_is_focused, test_tool_registry_validates_required_arguments, test_tool_registry_blocks_truncated_reference_arguments, test_tool_access_config_filters_worker_schema_and_blocks_calls, test_artifact_tools_roundtrip, test_read_artifact_missing_ref_returns_valid_recent_refs, test_defer_job_records_resume_time_without_pausing, test_shell_exec_tool_runs_bounded_command, test_shell_exec_flags_masked_auth_failure_output, test_write_file_tool_writes_and_appends_workspace_file

### tests/nipux_cli/test_uninstall.py

136 lines

No module docstring.

10 symbols 0 TODOs
Imports and top symbols

Imports: subprocess, pathlib, nipux_cli.uninstall

Symbols: _completed, test_uninstall_plan_includes_runtime_and_legacy_state, test_uninstall_plan_includes_configured_runtime_home, test_uninstall_runtime_removes_state_and_service_files, test_uninstall_runtime_dry_run_keeps_files, test_uninstall_installed_tool_uses_uv_when_available, runner, test_uninstall_installed_tool_falls_back_to_safe_uv_paths, which, test_installed_tool_paths_ignore_non_user_tool

### tests/nipux_cli/test_worker.py

11339 lines

No module docstring.

359 symbols 0 TODOs
Imports and top symbols

Imports: json, pathlib, nipux_cli.artifacts, nipux_cli.config, nipux_cli.db, nipux_cli.llm, nipux_cli.worker

Symbols: SnapshotRegistry, openai_tools, handle, SuccessRegistry, openai_tools, handle, MeasuredShellRegistry, openai_tools, handle, DiagnosticShellRegistry

Functions and classes

## Symbol Map

Parsed with Python AST.

### safe_filename

function nipux_cli/artifacts.py:16

No docstring.

Calls: _SAFE_NAME_RE.sub, strip, value.strip

### sha256_text

function nipux_cli/artifacts.py:21

No docstring.

Calls: hashlib.sha256, hexdigest, text.encode

### StoredArtifact

class nipux_cli/artifacts.py:26

No docstring.

Calls: dataclass

### ArtifactStore

class nipux_cli/artifacts.py:34

No docstring.

Calls: Path, StoredArtifact, ValueError, artifact.get, content.lower, find, join, len, lower, max, new_id, path.exists, path.mkdir, path.resolve, path.write_text, query.lower

### __init__

function nipux_cli/artifacts.py:35

No docstring.

Calls: Path, self.home.mkdir

### job_dir

function nipux_cli/artifacts.py:40

No docstring.

Calls: path.mkdir

### _assert_inside_home

function nipux_cli/artifacts.py:45

No docstring.

Calls: ValueError, path.resolve, resolved.relative_to, self.home.resolve

### write_text

function nipux_cli/artifacts.py:54

No docstring.

Calls: StoredArtifact, new_id, path.write_text, replace, safe_filename, self.db.add_artifact, self.job_dir, sha256_text, utc_now

### read_text

function nipux_cli/artifacts.py:88

No docstring.

Calls: Path, path.exists, safe_path.read_text, self._assert_inside_home, self.db.get_artifact

### search_text

function nipux_cli/artifacts.py:95

No docstring.

Calls: artifact.get, content.lower, find, join, len, lower, max, query.lower, results.append, self.db.list_artifacts, self.read_text, str, strip

### _find_agent_browser

function nipux_cli/browser.py:18

No docstring.

Calls: FileNotFoundError, shutil.which

### _session_name

function nipux_cli/browser.py:27

No docstring.

Calls: ch.isalnum, hashlib.sha1, hexdigest, join, len, task_id.encode

### _profile_dir

function nipux_cli/browser.py:35

No docstring.

Calls: _session_name

### _socket_dir

function nipux_cli/browser.py:39

No docstring.

Calls: Path, _session_name, os.environ.get

### run_browser_command

function nipux_cli/browser.py:44

No docstring.

Calls: Path, _find_agent_browser, _profile_dir, _session_name, _socket_dir, isinstance, json.loads, proc.kill, proc.wait, profile_dir.mkdir, result.setdefault, socket_dir.mkdir, stderr_path.open, stderr_path.read_text, stdout_path.open, stdout_path.read_text

### navigate

function nipux_cli/browser.py:110

No docstring.

Calls: _annotate_source_quality, get, result.get, run_browser_command, snapshot.get

### snapshot

function nipux_cli/browser.py:121

No docstring.

Calls: _annotate_source_quality, run_browser_command

### click

function nipux_cli/browser.py:125

No docstring.

Calls: _with_recovery_snapshot, ref.startswith, run_browser_command

### fill

function nipux_cli/browser.py:130

No docstring.

Calls: _with_recovery_snapshot, ref.startswith, run_browser_command

### scroll

function nipux_cli/browser.py:135

No docstring.

Calls: run_browser_command

### back

function nipux_cli/browser.py:139

No docstring.

Calls: run_browser_command

### press

function nipux_cli/browser.py:143

No docstring.

Calls: run_browser_command

### console

function nipux_cli/browser.py:147

No docstring.

Calls: bool, console_result.get, errors_result.get, run_browser_command

### _annotate_source_quality

function nipux_cli/browser.py:160

No docstring.

Calls: anti_bot_reason, data.get, isinstance, result.get, str, warnings.append

### _with_recovery_snapshot

function nipux_cli/browser.py:180

No docstring.

Calls: _annotate_source_quality, error.lower, result.get, run_browser_command, str

### ChatCommandDeps

class nipux_cli/chat_commands.py:14

No docstring.

Calls: dataclass

### handle_chat_slash_command

function nipux_cli/chat_commands.py:53

No docstring.

Calls: _one_line, _optional_int, argparse.Namespace, bool, db.append_lesson, db.close, db.get_job, deps.activity, deps.artifact, deps.artifacts, deps.cancel, deps.create_job, deps.db_factory, deps.delete, deps.digest, deps.doctor

### _optional_int

function nipux_cli/chat_commands.py:285

No docstring.

Calls: int, isdigit

### build_chat_messages

function nipux_cli/chat_context.py:22

Build bounded visible-state context for conversational job control.

Calls: _clip_chat_context, _durable_outcome_events, _durable_outcome_lines, _empty_section_text, _experiment_line, _job_list_lines, _roadmap_lines, active_prompt_operator_entries, artifact.get, clean_step_summary, db.list_artifacts, db.list_jobs, db.list_steps, db.list_timeline_event...

### _durable_outcome_events

function nipux_cli/chat_context.py:127

No docstring.

Calls: db.list_events, event.get, is_summary_event_candidate, len, merged.values, sorted, str

### _durable_outcome_lines

function nipux_cli/chat_context.py:142

No docstring.

Calls: hourly_outcome_summary, join, label.lower, len, lines.append, model_update_event_parts, outcome_counts, reversed, seen.add, set

### _job_list_lines

function nipux_cli/chat_context.py:165

No docstring.

Calls: entry.get, enumerate, join, len, lines.append, rstrip, split, str

### _roadmap_lines

function nipux_cli/chat_context.py:180

No docstring.

Calls: entry.get, isinstance, join, roadmap.get, strip

### _experiment_line

function nipux_cli/chat_context.py:199

No docstring.

Calls: entry.get, format_metric_value

### _empty_section_text

function nipux_cli/chat_context.py:214

No docstring.

Calls: title.startswith

### _clip_chat_context

function nipux_cli/chat_context.py:218

No docstring.

Calls: len, max, rstrip, str

### ChatControllerDeps

class nipux_cli/chat_controller.py:17

No docstring.

Calls: dataclass

### handle_chat_message

function nipux_cli/chat_controller.py:28

No docstring.

Calls: chat_reply_text_and_metadata, db.append_agent_update, db.append_event, db.close, deps.db_factory, deps.friendly_error_text, handle_chat_control_intent, maybe_spawn_job_from_chat, print, queue_chat_note, reply_callable, reply_text.strip, type

### chat_reply_text_and_metadata

function nipux_cli/chat_controller.py:79

No docstring.

Calls: getattr, isinstance, str

### handle_chat_control_intent

function nipux_cli/chat_controller.py:96

No docstring.

Calls: chat_control_command, command.lstrip, deps.capture_command, deps.compact_command_output, join, print

### maybe_spawn_job_from_chat

function nipux_cli/chat_controller.py:114

No docstring.

Calls: db.append_agent_update, db.append_operator_message, db.close, deps.create_job, deps.db_factory, deps.start_daemon, deps.write_shell_state, extract_job_objective_from_message, message_requests_immediate_run, message_requests_queued_job, print

### queue_chat_note

function nipux_cli/chat_controller.py:156

No docstring.

Calls: db.append_operator_message, db.close, deps.db_factory, entry.get, print

### ChatFrameDeps

class nipux_cli/chat_frame_runtime.py:40

No docstring.

Calls: dataclass

### compact_command_output

function nipux_cli/chat_frame_runtime.py:50

No docstring.

Calls: _one_line, compacted.append, join, line.split, line.startswith, line.strip, output.splitlines

### frame_next_job_id

function nipux_cli/chat_frame_runtime.py:60

No docstring.

Calls: ids.index, isinstance, job.get, len, snapshot.get, str

### next_chat_right_view

function nipux_cli/chat_frame_runtime.py:74

No docstring.

Calls: keys.index, len

### frame_refresh_interval

function nipux_cli/chat_frame_runtime.py:83

No docstring.

Calls: none

### run_chat_frame

function nipux_cli/chat_frame_runtime.py:89

No docstring.

Calls: _append_notice, _drain_async_notices, _frame_enter_sequence, _frame_exit_sequence, _handle_chat_escape, _handle_chat_submit, _handle_edit_input, _has_active_state_notice, _one_line, _safe_render_frame, autocomplete_slash, char.isprintable, deps.load_snapshot, deps.write_shell_...

### emit_frame_if_changed

function nipux_cli/chat_frame_runtime.py:244

No docstring.

Calls: _diff_frame_update, print

### _safe_render_frame

function nipux_cli/chat_frame_runtime.py:253

No docstring.

Calls: _append_notice, _display_notices, _fallback_chat_frame, _one_line, deps.render_frame, print, type

### _fallback_chat_frame

function nipux_cli/chat_frame_runtime.py:283

No docstring.

Calls: _fit_plain, _one_line, isinstance, job.get, join, lines.extend, max, shutil.get_terminal_size, snapshot.get, str

### _diff_frame_update

function nipux_cli/chat_frame_runtime.py:301

No docstring.

Calls: frame.splitlines, join, len, max, output.append, previous_frame.splitlines, range

### _fit_plain

function nipux_cli/chat_frame_runtime.py:315

No docstring.

Calls: _one_line, _strip_ansi, len, max, str

### _append_notice

function nipux_cli/chat_frame_runtime.py:322

No docstring.

Calls: notices.append

### _append_thinking_notice

function nipux_cli/chat_frame_runtime.py:327

No docstring.

Calls: _append_notice, _has_thinking_notice

### _append_waiting_notice

function nipux_cli/chat_frame_runtime.py:332

No docstring.

Calls: _append_notice, _has_waiting_notice

### _has_thinking_notice

function nipux_cli/chat_frame_runtime.py:337

No docstring.

Calls: any, notice.startswith

### _has_waiting_notice

function nipux_cli/chat_frame_runtime.py:341

No docstring.

Calls: any, notice.startswith

### _has_active_state_notice

function nipux_cli/chat_frame_runtime.py:345

No docstring.

Calls: _has_thinking_notice, _has_waiting_notice

### _clear_thinking_notices

function nipux_cli/chat_frame_runtime.py:349

No docstring.

Calls: notice.startswith

### _display_notices

function nipux_cli/chat_frame_runtime.py:357

No docstring.

Calls: int, len, rendered.append, time.monotonic

### _handle_edit_input

function nipux_cli/chat_frame_runtime.py:374

No docstring.

Calls: _append_notice, char.isprintable, decode_terminal_escape, inline_setting_notice, read_escape_sequence

### _handle_chat_submit

function nipux_cli/chat_frame_runtime.py:405

No docstring.

Calls: _append_notice, _append_thinking_notice, _append_waiting_notice, _looks_like_waiting_output, _one_line, _post_submit_snapshot_job_id, _start_chat_message_worker, buffer.strip, compact_command_output, deps.capture_chat_command, deps.is_plain_chat_line, deps.load_snapshot, join,...

### _post_submit_snapshot_job_id

function nipux_cli/chat_frame_runtime.py:468

Return the job id to refresh after a submitted command or message.

Calls: line.strip, lower, split, text.startswith

### _start_chat_message_worker

function nipux_cli/chat_frame_runtime.py:480

No docstring.

Calls: _one_line, async_messages.put, deps.handle_chat_message, thread.start, threading.Thread, type

### run

function nipux_cli/chat_frame_runtime.py:487

No docstring.

Calls: _one_line, async_messages.put, deps.handle_chat_message, type

### _drain_async_notices

function nipux_cli/chat_frame_runtime.py:500

No docstring.

Calls: _append_notice, _append_waiting_notice, _clear_thinking_notices, _looks_like_waiting_output, async_messages.get_nowait

### _looks_like_waiting_output

function nipux_cli/chat_frame_runtime.py:520

No docstring.

Calls: join, lower, normalized.startswith, split, str

### _handle_chat_escape

function nipux_cli/chat_frame_runtime.py:533

No docstring.

Calls: _append_notice, buffer.startswith, cycle_slash, decode_terminal_escape, deps.load_snapshot, deps.page_click, deps.write_shell_state, drain_pending_input, frame_next_job_id, get, isinstance, next_chat_right_view, read_escape_sequence

### natural_command_for

function nipux_cli/chat_intent.py:106

No docstring.

Calls: NATURAL_COMMANDS.get, join, lower, split, text.strip

### chat_control_command

function nipux_cli/chat_intent.py:110

No docstring.

Calls: NATURAL_COMMANDS.get, _looks_like_control_phrase, _mentions_any, join, line.strip, rstrip, split, text.lower

### _mentions_any

function nipux_cli/chat_intent.py:242

No docstring.

Calls: re.escape, re.search

### _looks_like_control_phrase

function nipux_cli/chat_intent.py:253

No docstring.

Calls: text.startswith

### message_requests_immediate_run

function nipux_cli/chat_intent.py:272

No docstring.

Calls: bool, join, lower, message.strip, message_requests_queued_job, re.match, re.search, split

### message_requests_queued_job

function nipux_cli/chat_intent.py:281

No docstring.

Calls: bool, join, lower, message.strip, re.search, split

### extract_job_objective_from_message

function nipux_cli/chat_intent.py:291

No docstring.

Calls: join, looks_like_job_objective, looks_like_smalltalk, match.group, message.strip, re.match, split, strip, text.lower

### looks_like_smalltalk

function nipux_cli/chat_intent.py:314

No docstring.

Calls: lowered.endswith

### looks_like_job_objective

function nipux_cli/chat_intent.py:318

No docstring.

Calls: any, len, re.escape, re.search, text.lower, text.split

### build_chat_frame

function nipux_cli/chat_tui.py:23

No docstring.

Calls: _compose_bar, _daemon_state_line, _metadata_records, _overlay_settings_modal, _step_count, _step_line, _top_bar, _two_col_line, _two_col_title, bool, chat_pane_lines, chat_updates_pane_lines, counts.get, edit_target_hint, edit_target_label, edit_target_masks_input

### _two_col_title

function nipux_cli/chat_tui.py:170

No docstring.

Calls: _bold, _fit_ansi, _muted, left.upper, right.upper

### _two_col_line

function nipux_cli/chat_tui.py:174

No docstring.

Calls: _fit_ansi, _muted

### _overlay_settings_modal

function nipux_cli/chat_tui.py:178

No docstring.

Calls: _accent, _bold, _fit_ansi, _muted, _rate_text, _settings_row, _strip_ansi, box.append, enumerate, int, len, load_config, max, min, str

### _settings_row

function nipux_cli/chat_tui.py:242

No docstring.

Calls: _bold, _muted, _one_line, label.ljust

### _rate_text

function nipux_cli/chat_tui.py:247

No docstring.

Calls: none

### _metadata_records

function nipux_cli/chat_tui.py:251

No docstring.

Calls: isinstance, job.get, metadata.get

### _step_count

function nipux_cli/chat_tui.py:259

No docstring.

Calls: int, max, step.get

### _step_line

function nipux_cli/chat_tui.py:264

No docstring.

Calls: _one_line, clean_step_summary, step.get

### _daemon_state_line

function nipux_cli/chat_tui.py:271

No docstring.

Calls: isinstance, lock.get, metadata.get

### _launch_agent_plist

function nipux_cli/cli.py:187

No docstring.

Calls: _service_launch_agent_plist

### _systemd_service_text

function nipux_cli/cli.py:191

No docstring.

Calls: _service_systemd_service_text

### _db

function nipux_cli/cli.py:256

No docstring.

Calls: AgentDB, config.ensure_dirs, load_config

### _record_command_deps

function nipux_cli/cli.py:262

No docstring.

Calls: RecordCommandDeps

### cmd_init

function nipux_cli/cli.py:270

No docstring.

Calls: Path, config.ensure_dirs, default_config_yaml, env_path.exists, expanduser, getattr, load_config, path.exists, path.parent.mkdir, print, write_private_text

### cmd_update

function nipux_cli/cli.py:304

No docstring.

Calls: SystemExit, _one_line, argparse.Namespace, cmd_restart, config.ensure_dirs, daemon_after.get, daemon_before.get, daemon_lock_status, getattr, load_config, print, str, update_checkout

### cmd_uninstall

function nipux_cli/cli.py:336

No docstring.

Calls: _stop_daemon_process, bool, build_uninstall_plan, float, getattr, input, load_config, lower, path.expanduser, print, strip, uninstall_installed_tool, uninstall_runtime

### cmd_create

function nipux_cli/cli.py:376

No docstring.

Calls: SystemExit, _create_job, _ensure_model_setup_verified_for_workspace, print

### _ensure_model_setup_verified_for_workspace

function nipux_cli/cli.py:388

No docstring.

Calls: _auto_verify_model_setup, _model_setup_verified, _workspace_has_model_config, load_config, print

### _workspace_has_model_config

function nipux_cli/cli.py:400

No docstring.

Calls: _read_shell_state, bool, exists, get

### _auto_verify_model_setup

function nipux_cli/cli.py:404

No docstring.

Calls: _clear_model_setup_verified, _mark_model_setup_verified, all, run_doctor

### _create_job

function nipux_cli/cli.py:414

No docstring.

Calls: _db, _write_shell_state, db.append_agent_update, db.append_roadmap_record, db.append_task_record, db.close, db.create_job, db.update_job_status, enumerate, format_initial_plan, initial_plan_for_objective, initial_roadmap_for_objective, initial_task_contract, max, objective.str...

### cmd_jobs

function nipux_cli/cli.py:458

No docstring.

Calls: _configured_focus_job_id, _db, _print_jobs_panel, bool, daemon_lock_status, db.close, db.list_jobs, load_config, print, str

### cmd_focus

function nipux_cli/cli.py:472

No docstring.

Calls: _db, _default_job_id, _find_job, _job_display_state, _write_shell_state, bool, daemon_lock_status, db.close, db.get_job, join, load_config, print

### cmd_rename

function nipux_cli/cli.py:495

No docstring.

Calls: _db, _job_ref_text, _resolve_job_id, _write_shell_state, content.endswith, content.splitlines, db.close, db.get_job, db.rename_job, join, print, program.exists, program.read_text, program.write_text, startswith

### cmd_delete

function nipux_cli/cli.py:521

No docstring.

Calls: Path, _db, _job_ref_text, _read_shell_state, _resolve_job_id, _write_shell_state, counts.get, db.close, db.delete_job, job_dir.exists, path.exists, path.unlink, print, result.get, shutil.rmtree, state.get

### cmd_chat

function nipux_cli/cli.py:561

No docstring.

Calls: _db, _ensure_model_setup_verified_for_workspace, _enter_chat, _job_ref_text, _resolve_job_id, _write_shell_state, db.close, print

### cmd_home

function nipux_cli/cli.py:578

No docstring.

Calls: _auto_verify_model_setup, _enter_first_run_setup, _enter_workspace_chat, _has_saved_jobs, _install_readline_history, _model_setup_verified, _start_interactive_daemon_if_possible, _workspace_has_model_config, load_config

### _enter_first_run_setup

function nipux_cli/cli.py:592

No docstring.

Calls: _enter_first_run_frame, _frame_chat_enabled, print

### _enter_empty_workspace

function nipux_cli/cli.py:601

No docstring.

Calls: _db, _one_line, _read_shell_state, _rule, db.close, db.list_jobs, get, job.get, print, str

### _has_saved_jobs

function nipux_cli/cli.py:624

No docstring.

Calls: _db, bool, db.close, db.list_jobs

### _enter_workspace_chat

function nipux_cli/cli.py:632

No docstring.

Calls: _enter_chat_frame, _enter_empty_workspace, _frame_chat_enabled

### _print_first_run_menu

function nipux_cli/cli.py:639

No docstring.

Calls: _short_path, load_config, print

### _handle_first_run_menu_line

function nipux_cli/cli.py:654

No docstring.

Calls: _extract_job_objective_from_message, _first_run_chat_reply, _first_token, _print_first_run_menu, _run_shell_line, argparse.Namespace, cmd_doctor, cmd_init, cmd_jobs, line.lower, line.startswith, line.strip, lowered.startswith, print, strip

### _prompt_first_run_value

function nipux_cli/cli.py:697

No docstring.

Calls: input, print, strip

### _first_run_create_and_open

function nipux_cli/cli.py:705

No docstring.

Calls: _create_job, _ensure_model_setup_verified_for_workspace, _enter_workspace_chat, _start_interactive_daemon_if_possible, _write_shell_state, print

### _first_token

function nipux_cli/cli.py:716

No docstring.

Calls: _controller_first_token

### _enter_first_run_frame

function nipux_cli/cli.py:720

No docstring.

Calls: _enter_workspace_chat, _first_run_runtime_deps, _run_first_run_frame, _start_interactive_daemon_if_possible, _write_shell_state

### _first_run_runtime_deps

function nipux_cli/cli.py:730

No docstring.

Calls: FirstRunRuntimeDeps, _first_run_click_action, _render_first_run_frame

### _first_run_actions

function nipux_cli/cli.py:747

No docstring.

Calls: _first_run_tui_actions

### _clamp_first_run_selection

function nipux_cli/cli.py:751

No docstring.

Calls: _clamp_first_run_runtime_selection, _first_run_actions

### _handle_first_run_action

function nipux_cli/cli.py:755

No docstring.

Calls: _controller_handle_first_run_action, _first_run_frame_deps

### _first_run_click_action

function nipux_cli/cli.py:759

No docstring.

Calls: _first_run_actions, len, max, min, shutil.get_terminal_size

### _chat_page_click

function nipux_cli/cli.py:780

No docstring.

Calls: int, max, min, shutil.get_terminal_size

### _handle_first_run_frame_line

function nipux_cli/cli.py:797

No docstring.

Calls: _controller_handle_first_run_frame_line, _first_run_frame_deps

### _first_run_chat_reply

function nipux_cli/cli.py:801

No docstring.

Calls: _controller_first_run_chat_reply

### _create_first_run_job

function nipux_cli/cli.py:805

No docstring.

Calls: _controller_create_first_run_job, _first_run_frame_deps

### _capture_first_run_command

function nipux_cli/cli.py:809

No docstring.

Calls: _controller_capture_first_run_command

### _first_run_frame_deps

function nipux_cli/cli.py:813

No docstring.

Calls: FirstRunFrameDeps, _model_setup_verified, load_config

### _current_default_job_id

function nipux_cli/cli.py:826

No docstring.

Calls: _db, _default_job_id, db.close

### _render_first_run_frame

function nipux_cli/cli.py:834

No docstring.

Calls: _build_first_run_frame, _emit_frame_if_changed, shutil.get_terminal_size

### _build_first_run_frame

function nipux_cli/cli.py:856

No docstring.

Calls: _build_first_run_tui_frame, _daemon_state_line, _db, _first_run_columns, _short_path, daemon_lock_status, db.close, db.list_jobs, load_config, max

### _enter_chat

function nipux_cli/cli.py:893

No docstring.

Calls: _chat_handle_line, _chat_prompt, _db, _default_job_id, _ensure_model_setup_verified_for_workspace, _enter_chat_frame, _fancy_ui, _frame_chat_enabled, _install_readline_history, _one_line, _print_chat_composer, _print_startup_history, _rule, _shell_summary, _start_chat_live_fee...

### _frame_chat_enabled

function nipux_cli/cli.py:948

No docstring.

Calls: os.environ.get, sys.stdin.isatty, sys.stdout.isatty

### _enter_chat_frame

function nipux_cli/cli.py:957

No docstring.

Calls: _chat_frame_deps, _run_chat_frame

### _chat_frame_deps

function nipux_cli/cli.py:961

No docstring.

Calls: ChatFrameDeps, _chat_page_click, _handle_chat_message, _load_frame_snapshot, _render_chat_frame

### _capture_chat_command

function nipux_cli/cli.py:982

No docstring.

Calls: StringIO, _chat_handle_line, _run_workspace_command_line, chat_control_command, line.strip, lstrip, raw.startswith, redirect_stdout, stream.getvalue, strip

### _run_workspace_command_line

function nipux_cli/cli.py:994

No docstring.

Calls: _append_workspace_chat_event, _create_workspace_job_from_chat, _extract_job_objective_from_message, _print_workspace_chat_help, _run_shell_line, _run_workspace_run_command, _run_workspace_setting_command, _workspace_command_should_create_worker, join, len, print, shlex.split, ...

### _run_workspace_setting_command

function nipux_cli/cli.py:1040

No docstring.

Calls: _handle_chat_setting_command

### _workspace_command_should_create_worker

function nipux_cli/cli.py:1048

No docstring.

Calls: _db, _extract_job_objective_from_message, _find_job, db.close

### _run_workspace_run_command

function nipux_cli/cli.py:1059

No docstring.

Calls: build_parser, parse_args, parsed.func, print

### _print_workspace_chat_help

function nipux_cli/cli.py:1072

No docstring.

Calls: print

### _start_worker_from_chat_context

function nipux_cli/cli.py:1078

Start the daemon from the TUI without dumping preflight internals into chat.

Calls: StringIO, _one_line, _start_daemon_if_needed, output.lower, print, redirect_stdout, report, str, stream.getvalue, type

### report

function nipux_cli/cli.py:1087

No docstring.

Calls: print

### _start_worker_from_chat_namespace

function nipux_cli/cli.py:1118

No docstring.

Calls: _start_worker_from_chat_context, bool, float, getattr

### _is_plain_chat_line

function nipux_cli/cli.py:1127

No docstring.

Calls: chat_control_command, line.strip, lower, lowered.split, shlex.split, stripped.lower, stripped.startswith

### _load_frame_snapshot

function nipux_cli/cli.py:1143

No docstring.

Calls: _db, _default_job_id, _workspace_chat_events, db.close, load_frame_snapshot

### _render_chat_frame

function nipux_cli/cli.py:1158

No docstring.

Calls: _build_chat_frame, _emit_frame_if_changed, shutil.get_terminal_size

### _build_chat_frame

function nipux_cli/cli.py:1184

No docstring.

Calls: _build_chat_tui_frame

### _resolve_job_id

function nipux_cli/cli.py:1209

No docstring.

Calls: _default_job_id, _find_job, _job_ref_text, str

### _activate_job_if_planning

function nipux_cli/cli.py:1217

No docstring.

Calls: db.append_agent_update, db.get_job, db.update_job_status, job.get

### _ensure_job_runnable

function nipux_cli/cli.py:1226

No docstring.

Calls: _activate_job_if_planning, db.append_agent_update, db.get_job, db.update_job_status, job.get, job_provider_blocked, operator_resume_metadata, str

### cmd_steer

function nipux_cli/cli.py:1247

No docstring.

Calls: _db, _job_ref_text, _resolve_job_id, db.append_operator_message, db.close, db.get_job, join, print, strip

### cmd_pause

function nipux_cli/cli.py:1267

No docstring.

Calls: _db, _resolve_control_job_and_note, db.close, db.get_job, db.update_job_status, print

### cmd_resume

function nipux_cli/cli.py:1282

No docstring.

Calls: _db, _job_ref_text, _resolve_job_id, db.close, db.get_job, db.update_job_status, operator_resume_metadata, print

### cmd_cancel

function nipux_cli/cli.py:1297

No docstring.

Calls: _db, _resolve_control_job_and_note, db.close, db.get_job, db.update_job_status, print

### cmd_status

function nipux_cli/cli.py:1312

No docstring.

Calls: _db, _job_ref_text, _resolve_job_id, _terminal_width, collect_dashboard_state, db.close, json.dumps, print, render_dashboard, render_overview

### cmd_health

function nipux_cli/cli.py:1331

No docstring.

Calls: _daemon_event_line, _daemon_state_line, _db, _default_job_id, _job_display_state, _launch_agent_path, _one_line, _rule, _step_count, _step_line, _worker_label, bool, config.ensure_dirs, daemon_lock_status, db.close, db.get_job

### cmd_history

function nipux_cli/cli.py:1388

No docstring.

Calls: _db, _event_line, _job_ref_text, _print_event_card, _public_event, _resolve_job_id, _rule, db.close, db.get_job, db.list_timeline_events, json.dumps, max, print

### cmd_events

function nipux_cli/cli.py:1419

No docstring.

Calls: _db, _event_line, _job_ref_text, _print_event_card, _public_event, _resolve_job_id, _rule, db.close, db.get_job, db.list_timeline_events, emit, event.get, json.dumps, print, seen.add, set

### emit

function nipux_cli/cli.py:1433

No docstring.

Calls: _event_line, _print_event_card, _public_event, _rule, db.list_timeline_events, event.get, json.dumps, print, seen.add, str

### cmd_dashboard

function nipux_cli/cli.py:1462

No docstring.

Calls: _db, _job_ref_text, _resolve_job_id, _terminal_width, collect_dashboard_state, db.close, print, render_dashboard, time.sleep

### cmd_artifacts

function nipux_cli/cli.py:1483

No docstring.

Calls: _db, _generic_display_text, _job_ref_text, _one_line, _resolve_job_id, _rule, artifact.get, db.close, db.get_job, db.list_artifacts, enumerate, print

### cmd_artifact

function nipux_cli/cli.py:1513

No docstring.

Calls: ArtifactStore, _db, _job_ref_text, _resolve_artifact_ref, _resolve_job_id, _rule, content.endswith, db.close, getattr, len, print, resolved.get, store.read_text

### cmd_lessons

function nipux_cli/cli.py:1535

No docstring.

Calls: _db, _job_ref_text, _print_lessons, _resolve_job_id, db.close, db.get_job, print

### cmd_learn

function nipux_cli/cli.py:1549

No docstring.

Calls: _db, _job_ref_text, _one_line, _resolve_job_id, db.append_lesson, db.close, db.get_job, join, print, strip

### cmd_findings

function nipux_cli/cli.py:1570

No docstring.

Calls: _record_command_deps, cmd_findings_impl

### cmd_tasks

function nipux_cli/cli.py:1574

No docstring.

Calls: _record_command_deps, cmd_tasks_impl

### cmd_roadmap

function nipux_cli/cli.py:1578

No docstring.

Calls: _record_command_deps, cmd_roadmap_impl

### cmd_experiments

function nipux_cli/cli.py:1582

No docstring.

Calls: _record_command_deps, cmd_experiments_impl

### cmd_sources

function nipux_cli/cli.py:1586

No docstring.

Calls: _record_command_deps, cmd_sources_impl

### cmd_memory

function nipux_cli/cli.py:1590

No docstring.

Calls: _record_command_deps, cmd_memory_impl

### cmd_metrics

function nipux_cli/cli.py:1594

No docstring.

Calls: _record_command_deps, cmd_metrics_impl

### cmd_usage

function nipux_cli/cli.py:1598

No docstring.

Calls: _record_command_deps, cmd_usage_impl

### _remote_model_preflight_failures

function nipux_cli/cli.py:1602

No docstring.

Calls: _daemon_remote_model_preflight_failures

### _recoverable_remote_model_preflight_failures

function nipux_cli/cli.py:1606

No docstring.

Calls: _daemon_recoverable_remote_model_preflight_failures

### _provider_preflight_is_recoverable

function nipux_cli/cli.py:1610

No docstring.

Calls: _daemon_provider_preflight_is_recoverable

### _ensure_remote_model_ready_for_worker

function nipux_cli/cli.py:1614

No docstring.

Calls: _daemon_ensure_remote_model_ready

### cmd_start

function nipux_cli/cli.py:1618

No docstring.

Calls: _cmd_start_impl, _ensure_remote_model_ready_for_worker, _stop_daemon_process

### _start_daemon_if_needed

function nipux_cli/cli.py:1626

No docstring.

Calls: _start_daemon_if_needed_impl, _stop_daemon_process

### _start_interactive_daemon_if_possible

function nipux_cli/cli.py:1639

Best-effort daemon start for the full-screen UI without printing over the frame.

Calls: StringIO, _start_daemon_if_needed, redirect_stdout, stream.getvalue

### cmd_restart

function nipux_cli/cli.py:1651

No docstring.

Calls: _cmd_restart_impl, _stop_daemon_process

### _stop_daemon_process

function nipux_cli/cli.py:1659

No docstring.

Calls: _stop_daemon_process_impl

### cmd_stop

function nipux_cli/cli.py:1663

No docstring.

Calls: _db, _job_ref_text, _resolve_job_id, _stop_daemon_process, db.close, db.get_job, db.update_job_status, getattr, load_config, print

### cmd_browser_dashboard

function nipux_cli/cli.py:1684

No docstring.

Calls: Path, Path.cwd, SystemExit, _find_agent_browser, config.ensure_dirs, expanduser, load_config, log_path.open, log_path.parent.mkdir, print, str, subprocess.Popen, subprocess.call, subprocess.run

### _print_startup_history

function nipux_cli/cli.py:1715

No docstring.

Calls: _db, _important_startup_events, _print_event_card, _print_session_overview, _section_title, bool, daemon_lock_status, db.close, db.get_job, db.list_artifacts, db.list_jobs, db.list_memory, db.list_steps, db.list_timeline_events, enumerate, len

### _print_session_overview

function nipux_cli/cli.py:1752

No docstring.

Calls: _job_display_state, _metadata_records, _next_operator_action, _print_jobs_panel, _print_metric_grid, _print_wrapped, _section_title, _short_path, _status_badge, _step_count, _terminal_width, _worker_label, isinstance, job.get, len, metadata.get

### _print_chat_composer

function nipux_cli/cli.py:1816

No docstring.

Calls: _accent, _fancy_ui, _section_title, _terminal_width, max, min, print

### _chat_prompt

function nipux_cli/cli.py:1827

No docstring.

Calls: _accent

### _start_chat_live_feed

function nipux_cli/cli.py:1831

No docstring.

Calls: os.environ.get, sys.stdin.isatty, sys.stdout.isatty, thread.start, threading.Event, threading.Thread

### _chat_live_feed_loop

function nipux_cli/cli.py:1845

No docstring.

Calls: _db, _default_job_id, _minimal_live_event_line, _print_live_line, db.close, db.list_events, event.get, initialized_jobs.add, seen.add, seen.update, seen_by_job.setdefault, set, stop.wait, str

### _print_live_line

function nipux_cli/cli.py:1875

No docstring.

Calls: _chat_prompt, _fancy_ui, _live_badge, print

### _resolve_control_job_and_note

function nipux_cli/cli.py:1885

No docstring.

Calls: _default_job_id, _find_job, _job_ref_text, _note_text, _resolve_job_id, getattr, hasattr, join, len, range, str, strip

### _pid_is_alive

function nipux_cli/cli.py:1900

No docstring.

Calls: os.kill

### _step_by_id

function nipux_cli/cli.py:1908

No docstring.

Calls: db.list_steps

### _step_count

function nipux_cli/cli.py:1915

No docstring.

Calls: int, max, step.get

### _job_lessons

function nipux_cli/cli.py:1920

No docstring.

Calls: _metadata_records

### _metadata_records

function nipux_cli/cli.py:1924

No docstring.

Calls: isinstance, job.get, metadata.get

### _print_lessons

function nipux_cli/cli.py:1930

No docstring.

Calls: _job_lessons, _one_line, _rule, enumerate, isinstance, len, lesson.get, max, print

### _resolve_artifact_ref

function nipux_cli/cli.py:1946

No docstring.

Calls: ArtifactStore, Path, artifact.get, artifacts.extend, db.get_artifact, db.get_job, db.list_artifacts, db.list_jobs, expanduser, index_ref.isdigit, int, join, len, lower, ordered_jobs.append, ordered_jobs.extend

### cmd_logs

function nipux_cli/cli.py:2008

No docstring.

Calls: _clean_step_summary, _db, _job_display_state, _job_ref_text, _one_line, _print_step, _resolve_job_id, artifact.get, bool, daemon_lock_status, db.close, db.get_job, db.list_artifacts, db.list_runs, db.list_steps, load_config

### cmd_activity

function nipux_cli/cli.py:2052

No docstring.

Calls: _db, _event_line, _job_display_state, _print_event_details, _resolve_job_id, _rule, bool, daemon_lock_status, db.close, db.get_job, db.list_timeline_events, emit, event.get, isinstance, load_config, metadata.get

### emit

function nipux_cli/cli.py:2066

No docstring.

Calls: _event_line, _print_event_details, _rule, db.list_timeline_events, event.get, isinstance, metadata.get, print, seen_events.add, str

### cmd_updates

function nipux_cli/cli.py:2095

No docstring.

Calls: _db, _resolve_job_id, db.close, getattr, join, print, render_all_updates_report, render_updates_report

### cmd_watch

function nipux_cli/cli.py:2131

No docstring.

Calls: _db, _job_display_state, _job_ref_text, _print_artifact, _print_run, _print_step, _resolve_job_id, bool, daemon_lock_status, db.close, db.get_job, db.list_artifacts, db.list_runs, db.list_steps, emit_snapshot, list

### emit_snapshot

function nipux_cli/cli.py:2150

No docstring.

Calls: _print_artifact, _print_run, _print_step, db.get_job, db.list_artifacts, db.list_runs, db.list_steps, list, print, reversed, seen_artifacts.add, seen_runs.add, seen_steps.add

### cmd_run_one

function nipux_cli/cli.py:2194

No docstring.

Calls: LLMResponse, ScriptedLLM, ToolCall, _activate_job_if_planning, _db, _ensure_model_setup_verified_for_workspace, _job_ref_text, _model_setup_verified, _resolve_job_id, db.close, json.dumps, print, run_one_step

### cmd_work

function nipux_cli/cli.py:2234

No docstring.

Calls: LLMResponse, ScriptedLLM, ToolCall, _activate_job_if_planning, _db, _ensure_model_setup_verified_for_workspace, _model_setup_verified, _print_step, _resolve_job_id, _step_by_id, _terminal_width, collect_dashboard_state, db.close, db.get_job, json.dumps, print

### _pause_job_for_recoverable_provider_preflight

function nipux_cli/cli.py:2295

No docstring.

Calls: _recoverable_remote_model_preflight_failures, db.append_agent_update, db.get_job, db.update_job_metadata, db.update_job_status, get, job.get, job_provider_blocked, join, str, utc_now

### cmd_run

function nipux_cli/cli.py:2345

No docstring.

Calls: _db, _default_job_id, _ensure_job_runnable, _find_job, _job_display_state, _job_ref_text, _model_setup_verified, _pause_job_for_recoverable_provider_preflight, _provider_preflight_is_recoverable, _remote_model_preflight_failures, _start_daemon_if_needed, _write_shell_state, ar...

### cmd_digest

function nipux_cli/cli.py:2421

No docstring.

Calls: _db, _job_ref_text, _resolve_job_id, db.close, print, render_job_digest

### cmd_daily_digest

function nipux_cli/cli.py:2444

No docstring.

Calls: _db, db.close, json.dumps, print, write_daily_digest

### cmd_daemon

function nipux_cli/cli.py:2453

No docstring.

Calls: Daemon.open, SystemExit, _ensure_remote_model_ready_for_worker, daemon.close, daemon.run_forever, daemon.run_once, json.dumps, load_config, print, str

### cmd_doctor

function nipux_cli/cli.py:2470

No docstring.

Calls: SystemExit, _clear_model_setup_verified, _mark_model_setup_verified, all, load_config, print, run_doctor

### _verify_model_setup_from_first_run

function nipux_cli/cli.py:2487

No docstring.

Calls: StringIO, argparse.Namespace, cmd_doctor, item.split, item.strip, join, print, redirect_stdout, splitlines, stream.getvalue

### _chat_handle_line

function nipux_cli/cli.py:2501

No docstring.

Calls: _chat_command_deps, _db, _handle_chat_message, _handle_chat_slash_command, argparse.Namespace, cmd_focus, cmd_jobs, db.close, db.get_job, line.startswith, line.strip, print, shlex.split

### _handle_chat_message

function nipux_cli/cli.py:2559

No docstring.

Calls: _chat_controller_deps, _controller_handle_chat_message, _handle_workspace_chat_message, _model_setup_verified, load_config, print

### _chat_reply_text_and_metadata

function nipux_cli/cli.py:2578

No docstring.

Calls: _controller_reply_text_and_metadata

### _workspace_chat_events

function nipux_cli/cli.py:2582

No docstring.

Calls: _read_shell_state, get, isinstance

### _append_workspace_chat_event

function nipux_cli/cli.py:2589

No docstring.

Calls: _workspace_chat_events, _write_shell_state, events.append, int, len, time.time, utc_now

### _handle_workspace_chat_message

function nipux_cli/cli.py:2605

No docstring.

Calls: _append_workspace_chat_event, _capture_chat_command, _chat_reply_text_and_metadata, _compact_command_output, _create_workspace_job_from_chat, _extract_job_objective_from_message, _friendly_error_text, _reply_to_workspace_chat, chat_control_command, control_command.lstrip, join...

### _create_workspace_job_from_chat

function nipux_cli/cli.py:2644

No docstring.

Calls: _create_job, _db, _refine_job_objective_for_worker, _start_worker_from_chat_context, _write_shell_state, db.append_agent_update, db.append_operator_message, db.close, message_requests_immediate_run, message_requests_queued_job

### _refine_job_objective_for_worker

function nipux_cli/cli.py:2670

No docstring.

Calls: OpenAIChatLLM, _db, _db_handle.close, _durable_job_objective, complete, len, strip

### _durable_job_objective

function nipux_cli/cli.py:2703

No docstring.

Calls: _one_line, join, split, str, strip

### _workspace_chat_job_dossier

function nipux_cli/cli.py:2718

Compact job context for the left-side workspace chat model.

Calls: _metadata_records, _one_line, _safe_job_counts, _safe_list_artifacts, _safe_list_events, _safe_list_steps, _step_line, _workspace_current_task, _workspace_recent_outcomes, artifact.get, counts.get, enumerate, isinstance, job.get, join, len

### _safe_job_counts

function nipux_cli/cli.py:2770

No docstring.

Calls: db.job_record_counts

### _safe_list_artifacts

function nipux_cli/cli.py:2779

No docstring.

Calls: db.list_artifacts

### _safe_list_events

function nipux_cli/cli.py:2786

No docstring.

Calls: db.list_timeline_events

### _safe_list_steps

function nipux_cli/cli.py:2793

No docstring.

Calls: db.list_steps

### _workspace_current_task

function nipux_cli/cli.py:2800

No docstring.

Calls: _one_line, get, int, str, task.get, visible.sort

### _workspace_recent_outcomes

function nipux_cli/cli.py:2821

No docstring.

Calls: _model_update_event_parts, _one_line, label.lower, len, outcomes.append, reversed, seen.add, set

### _reply_to_workspace_chat

function nipux_cli/cli.py:2841

No docstring.

Calls: OpenAIChatLLM, _db, _one_line, _workspace_chat_events, _workspace_chat_job_dossier, chr, complete_response, db.close, db.list_jobs, event.get, join

### _handle_chat_control_intent

function nipux_cli/cli.py:2883

No docstring.

Calls: _chat_controller_deps, _controller_handle_chat_control_intent

### _maybe_spawn_job_from_chat

function nipux_cli/cli.py:2887

No docstring.

Calls: _chat_controller_deps, _controller_maybe_spawn_job_from_chat

### _queue_chat_note

function nipux_cli/cli.py:2891

No docstring.

Calls: _chat_controller_deps, _controller_queue_chat_note

### _chat_controller_deps

function nipux_cli/cli.py:2895

No docstring.

Calls: ChatControllerDeps

### _chat_command_deps

function nipux_cli/cli.py:2908

No docstring.

Calls: ChatCommandDeps

### _reply_to_chat

function nipux_cli/cli.py:2949

No docstring.

Calls: OpenAIChatLLM, _build_chat_messages, _db, complete_response, db.close, db.get_job

### cmd_shell

function nipux_cli/cli.py:2961

No docstring.

Calls: _install_readline_history, _print_shell_header, _print_shell_status, _run_shell_line, _shell_prompt, input, print

### _print_shell_header

function nipux_cli/cli.py:2980

No docstring.

Calls: _rule, _shell_summary, print

### _shell_summary

function nipux_cli/cli.py:2989

No docstring.

Calls: _db, _default_job_id, _job_display_state, _worker_label, bool, daemon_lock_status, db.close, db.get_job

### _shell_prompt

function nipux_cli/cli.py:3006

No docstring.

Calls: _db, _default_job_id, _worker_label, bool, daemon_lock_status, db.close, db.get_job, job.get, load_config, str, strip

### _install_readline_history

function nipux_cli/cli.py:3023

No docstring.

Calls: atexit.register, config.ensure_dirs, load_config, readline.read_history_file

### _print_shell_status

function nipux_cli/cli.py:3039

No docstring.

Calls: _db, _terminal_width, collect_dashboard_state, db.close, print, render_dashboard

### _print_shell_help

function nipux_cli/cli.py:3049

No docstring.

Calls: _render_shell_help

### _run_shell_line

function nipux_cli/cli.py:3053

No docstring.

Calls: _print_shell_help, _steer_default_job, build_parser, isinstance, join, len, line.strip, lower, natural_command_for, parsed.func, parser.parse_args, print, shlex.split

### _steer_default_job

function nipux_cli/cli.py:3101

No docstring.

Calls: _db, _default_job_id, db.append_operator_message, db.close, db.get_job, print

### build_parser

function nipux_cli/cli.py:3116

No docstring.

Calls: build_arg_parser

### main

function nipux_cli/cli.py:3173

No docstring.

Calls: argparse.Namespace, args.func, build_parser, cmd_home, parser.parse_args, print

### print_shell_help

function nipux_cli/cli_help.py:18

No docstring.

Calls: _print_group, print, rule

### _print_group

function nipux_cli/cli_help.py:88

No docstring.

Calls: print

### clip_json

function nipux_cli/cli_render.py:18

No docstring.

Calls: json.dumps, len

### print_step

function nipux_cli/cli_render.py:25

No docstring.

Calls: _one_line, checkpoint.get, clean_step_summary, clip_json, isinstance, lesson.get, output_data.get, print, source.get, step.get, update.get

### print_artifact

function nipux_cli/cli_render.py:63

No docstring.

Calls: artifact.get, print

### print_run

function nipux_cli/cli_render.py:69

No docstring.

Calls: print, run.get

### print_wrapped

function nipux_cli/cli_render.py:75

No docstring.

Calls: join, len, max, min, prefix.rstrip, print, split, str, textwrap.wrap

### section_title

function nipux_cli/cli_render.py:87

No docstring.

Calls: _accent, _fancy_ui, _one_line, len, max, min, terminal_width, title.upper

### print_metric_grid

function nipux_cli/cli_render.py:99

No docstring.

Calls: join, len, ljust, max, min, print, range, rstrip, terminal_width

### short_path

function nipux_cli/cli_render.py:108

No docstring.

Calls: Path.home, len, max, str, text.startswith

### print_jobs_panel

function nipux_cli/cli_render.py:119

No docstring.

Calls: _one_line, _status_badge, enumerate, item.get, job_display_state, len, print, section_title, str, worker_label

### next_operator_action

function nipux_cli/cli_render.py:136

No docstring.

Calls: job.get, str

### important_startup_events

function nipux_cli/cli_render.py:155

No docstring.

Calls: event.get, len, reversed, selected.append, selected.sort, str

### print_event_card

function nipux_cli/cli_render.py:186

No docstring.

Calls: _event_badge, _muted, _one_line, artifact_indexes.get, event.get, event_display_parts, print, str

### public_event

function nipux_cli/cli_render.py:197

No docstring.

Calls: dict, public.pop

### print_event_details

function nipux_cli/cli_render.py:203

No docstring.

Calls: _one_line, event.get, isinstance, json.dumps, metadata.get, metadata.items, print

### step_line

function nipux_cli/cli_render.py:220

No docstring.

Calls: _one_line, clean_step_summary, step.get

### terminal_width

function nipux_cli/cli_render.py:227

No docstring.

Calls: shutil.get_terminal_size

### rule

function nipux_cli/cli_render.py:231

No docstring.

Calls: min, terminal_width

### json_default

function nipux_cli/cli_render.py:235

No docstring.

Calls: str

### daemon_state_line

function nipux_cli/cli_render.py:239

No docstring.

Calls: isinstance, lock.get, metadata.get

### daemon_event_line

function nipux_cli/cli_render.py:248

No docstring.

Calls: _one_line, event.get, job_titles.get, join, pieces.append, str, strip

### job_ref_text

function nipux_cli/cli_render.py:264

No docstring.

Calls: isinstance, join, str, text.split

### note_text

function nipux_cli/cli_render.py:275

No docstring.

Calls: isinstance, join, str, strip

### default_job_id

function nipux_cli/cli_state.py:15

No docstring.

Calls: configured_focus_job_id, db.list_jobs, job.get, str

### configured_focus_job_id

function nipux_cli/cli_state.py:27

No docstring.

Calls: db.get_job, get, isinstance, read_shell_state

### find_job

function nipux_cli/cli_state.py:38

No docstring.

Calls: db.list_jobs, job.get, join, lower, query.split, str

### shell_state_path

function nipux_cli/cli_state.py:55

No docstring.

Calls: config.ensure_dirs, load_config

### read_shell_state

function nipux_cli/cli_state.py:61

No docstring.

Calls: isinstance, json.loads, path.exists, path.read_text, shell_state_path

### write_shell_state

function nipux_cli/cli_state.py:72

No docstring.

Calls: json.dumps, read_shell_state, shell_state_path, state.update, write_text

### setup_completed

function nipux_cli/cli_state.py:80

No docstring.

Calls: bool, get, read_shell_state

### mark_setup_completed

function nipux_cli/cli_state.py:84

No docstring.

Calls: write_shell_state

### model_setup_fingerprint

function nipux_cli/cli_state.py:88

No docstring.

Calls: config.model.api_key.encode, encode, hashlib.sha256, hexdigest, json.dumps, load_config

### model_setup_verified

function nipux_cli/cli_state.py:100

No docstring.

Calls: isinstance, marker.get, model_setup_fingerprint, read_shell_state, state.get

### mark_model_setup_verified

function nipux_cli/cli_state.py:108

No docstring.

Calls: datetime.now, isoformat, load_config, model_setup_fingerprint, write_shell_state

### clear_model_setup_verified

function nipux_cli/cli_state.py:125

No docstring.

Calls: write_shell_state

### _clip_text

function nipux_cli/compression.py:10

No docstring.

Calls: join, len, max, rstrip, split, str

### refresh_memory_index

function nipux_cli/compression.py:17

Write a compact, artifact-referenced job memory entry. This is deliberately deterministic. A local model can later improve the prose, but the daemon should always have a cheap c...

Calls: _clip_text, _compact_count, _context_fraction, _first_positive_int, _metadata_list, _rank_tasks, active_prompt_operator_entries, artifact.get, db.get_job, db.job_token_usage, db.list_artifacts, db.list_steps, db.upsert_memory, entry.get, experiment.get, finding.get

### _metadata_list

function nipux_cli/compression.py:193

No docstring.

Calls: isinstance, metadata.get

### _rank_tasks

function nipux_cli/compression.py:200

No docstring.

Calls: int, sorted, status_rank.get, str, task.get

### _compact_count

function nipux_cli/compression.py:212

No docstring.

Calls: float, int, str

### _context_fraction

function nipux_cli/compression.py:224

No docstring.

Calls: _first_positive_int, float, usage.get

### _first_positive_int

function nipux_cli/compression.py:238

No docstring.

Calls: float, int

### get_agent_home

function nipux_cli/config.py:22

Return the Nipux agent home directory.

Calls: Path, Path.home, expanduser, os.environ.get, strip

### load_env_file

function nipux_cli/config.py:29

Load KEY=value pairs from a local env file without overriding the shell.

Calls: Path, ensure_private_file_permissions, env_path.exists, env_path.read_text, expanduser, key.strip, line.split, line.startswith, raw_line.strip, splitlines, strip, value.strip

### ensure_private_file_permissions

function nipux_cli/config.py:47

Best-effort POSIX privacy for local config/secret files.

Calls: Path, chmod

### ensure_private_dir_permissions

function nipux_cli/config.py:58

Best-effort POSIX privacy for the local Nipux state directory.

Calls: Path, chmod

### write_private_text

function nipux_cli/config.py:69

Write text with private file permissions from creation time.

Calls: Path, ensure_private_file_permissions, expanduser, handle.write, os.close, os.fdopen, os.open, target.parent.mkdir

### ModelConfig

class nipux_cli/config.py:87

No docstring.

Calls: dataclass, os.environ.get

### api_key

function nipux_cli/config.py:97

No docstring.

Calls: os.environ.get

### RuntimeConfig

class nipux_cli/config.py:102

No docstring.

Calls: dataclass, field

### state_db_path

function nipux_cli/config.py:112

No docstring.

Calls: none

### jobs_dir

function nipux_cli/config.py:116

No docstring.

Calls: none

### logs_dir

function nipux_cli/config.py:120

No docstring.

Calls: none

### digests_dir

function nipux_cli/config.py:124

No docstring.

Calls: none

### ToolAccessConfig

class nipux_cli/config.py:129

No docstring.

Calls: dataclass

### EmailConfig

class nipux_cli/config.py:137

No docstring.

Calls: dataclass, os.environ.get

### password

function nipux_cli/config.py:148

No docstring.

Calls: os.environ.get

### AppConfig

class nipux_cli/config.py:153

No docstring.

Calls: dataclass, directory.mkdir, ensure_private_dir_permissions, field

### ensure_dirs

function nipux_cli/config.py:159

No docstring.

Calls: directory.mkdir, ensure_private_dir_permissions

### _as_dict

function nipux_cli/config.py:170

No docstring.

Calls: isinstance

### _optional_float

function nipux_cli/config.py:174

No docstring.

Calls: float

### load_config

function nipux_cli/config.py:180

Load config.yaml, falling back to a local OpenAI-compatible endpoint.

Calls: AppConfig, EmailConfig, ModelConfig, Path, RuntimeConfig, ToolAccessConfig, _as_dict, _optional_float, bool, cfg_path.exists, cfg_path.read_text, email_raw.get, expanduser, float, get_agent_home, int

### default_config_yaml

function nipux_cli/config.py:234

Return a starter config file for an OpenAI-compatible model server.

Calls: base_url.rstrip

### context_pressure_for_prompt

function nipux_cli/context_pressure.py:34

No docstring.

Calls: _as_float, compact_token_count, isinstance, job.get, metadata.get, pressure.get, str

### usage_pressure_for_prompt

function nipux_cli/context_pressure.py:54

No docstring.

Calls: _as_float, _as_int, _durable_usage_signal_count, _usage_pressure_band, bits.append, bool, compact_token_count, int, isinstance, join, max, usage.get

### emit_usage_pressure_update

function nipux_cli/context_pressure.py:95

No docstring.

Calls: _as_float, _as_int, _usage_pressure_band, bool, compact_token_count, datetime.now, db.append_agent_update, db.get_job, db.update_job_metadata, int, isinstance, isoformat, job.get, max, metadata.get, previous.get

### emit_context_pressure_update

function nipux_cli/context_pressure.py:139

No docstring.

Calls: _as_float, _as_int, _context_pressure_band, compact_token_count, datetime.now, db.append_agent_update, db.get_job, db.update_job_metadata, isinstance, isoformat, job.get, max, metadata.get, previous.get, round, str

### compact_token_count

function nipux_cli/context_pressure.py:177

No docstring.

Calls: _as_int, str

### _usage_pressure_band

function nipux_cli/context_pressure.py:186

No docstring.

Calls: _as_float, _as_int, bool, usage.get

### _context_pressure_band

function nipux_cli/context_pressure.py:209

No docstring.

Calls: none

### _durable_usage_signal_count

function nipux_cli/context_pressure.py:216

No docstring.

Calls: isinstance, job.get, lower, metadata.get, milestone.get, roadmap.get, str, sum, task.get

### _as_float

function nipux_cli/context_pressure.py:243

No docstring.

Calls: float

### _as_int

function nipux_cli/context_pressure.py:250

No docstring.

Calls: float, int

### DaemonAlreadyRunning

class nipux_cli/daemon.py:29

No docstring.

Calls: none

### runtime_code_file_names

function nipux_cli/daemon.py:36

No docstring.

Calls: Path, _runtime_code_paths, resolve, tuple

### current_runtime_fingerprint

function nipux_cli/daemon.py:42

Return a stable fingerprint for code that affects daemon behavior.

Calls: DEFAULT_REGISTRY.names, DEFAULT_REGISTRY.openai_tools, SYSTEM_PROMPT.encode, _runtime_code_fingerprint, encode, hashlib.sha256, hexdigest, json.dumps, len, lru_cache, payload.items

### _runtime_code_fingerprint

function nipux_cli/daemon.py:68

No docstring.

Calls: Path, _runtime_code_paths, digest, digest.hexdigest, digest.update, hashlib.sha256, lru_cache, max, mtimes.append, name.encode, path.read_bytes, path.stat, resolve

### _runtime_code_paths

function nipux_cli/daemon.py:84

No docstring.

Calls: package_dir.glob, path.is_file, sorted

### runtime_stale

function nipux_cli/daemon.py:93

No docstring.

Calls: current_runtime_fingerprint, get, isinstance, metadata.get, recorded.get

### _parse_lock_metadata

function nipux_cli/daemon.py:102

No docstring.

Calls: isinstance, json.loads, raw.strip

### daemon_lock_status

function nipux_cli/daemon.py:113

Return whether another process currently holds the daemon lock.

Calls: Path, _parse_lock_metadata, contextlib.suppress, current_runtime_fingerprint, fcntl.flock, handle.fileno, handle.read, handle.seek, path.open, path.parent.mkdir, runtime_stale, str

### single_instance_lock

function nipux_cli/daemon.py:146

Hold an exclusive non-blocking daemon lock for this state directory.

Calls: DaemonAlreadyRunning, Path, current_runtime_fingerprint, datetime.now, fcntl.flock, handle.fileno, handle.flush, handle.seek, handle.truncate, handle.write, isoformat, json.dumps, os.getpid, path.open, path.parent.mkdir

### update_lock_metadata

function nipux_cli/daemon.py:171

No docstring.

Calls: _parse_lock_metadata, datetime.now, handle.flush, handle.read, handle.seek, handle.truncate, handle.write, isoformat, json.dumps, metadata.setdefault, metadata.update, os.getpid

### _work_heartbeat

function nipux_cli/daemon.py:184

Keep daemon lock metadata fresh while a worker turn is in progress.

Calls: current_runtime_fingerprint, datetime.now, float, isoformat, max, stop.set, stop.wait, thread.join, thread.start, threading.Event, threading.Thread, update_metadata

### beat

function nipux_cli/daemon.py:197

No docstring.

Calls: current_runtime_fingerprint, datetime.now, isoformat, stop.wait, update_metadata

### append_daemon_event

function nipux_cli/daemon.py:221

Append a small daemon event that the CLI can tail without parsing stdout.

Calls: config.ensure_dirs, datetime.now, handle.write, isoformat, json.dumps, path.open

### read_daemon_events

function nipux_cli/daemon.py:236

No docstring.

Calls: events.append, isinstance, json.loads, path.exists, path.read_text, splitlines

### fake_step_llm

function nipux_cli/daemon.py:253

No docstring.

Calls: LLMResponse, ScriptedLLM, ToolCall, datetime.now, isoformat

### Daemon

class nipux_cli/daemon.py:273

No docstring.

Calls: AgentDB, _exception_backoff, _exception_payload, _is_digest_due, _model_generation_ready, _provider_probe_due, _sleep_or_stop, _step_failure_backoff, _work_heartbeat, append_daemon_event, astimezone, cleanup_registered_shell_processes, cls, config.ensure_dirs, current_runtime_...

### open

function nipux_cli/daemon.py:278

No docstring.

Calls: AgentDB, cls, config.ensure_dirs, load_config

### lock_path

function nipux_cli/daemon.py:284

No docstring.

Calls: none

### close

function nipux_cli/daemon.py:287

No docstring.

Calls: self.db.close

### next_runnable_job

function nipux_cli/daemon.py:290

Return the next runnable job by priority/age. UI focus is intentionally not used here. Focus is for the operator's chat view; the daemon should keep all runnable jobs advancing.

Calls: datetime.now, job_is_deferred, job_provider_blocked, self._maybe_recover_provider_blocked_jobs, self.db.append_agent_update, self.db.list_jobs, self.db.update_job_status

### _maybe_recover_provider_blocked_jobs

function nipux_cli/daemon.py:324

No docstring.

Calls: _model_generation_ready, _provider_probe_due, append_daemon_event, job_provider_blocked, len, now.isoformat, self.db.append_agent_update, self.db.list_jobs, self.db.update_job_status

### idle_sleep_seconds

function nipux_cli/daemon.py:368

Return the next idle sleep, capped by the nearest deferred job wake.

Calls: astimezone, datetime.now, due_times.append, job_deferred_until, max, min, self.db.list_jobs, total_seconds

### run_once

function nipux_cli/daemon.py:383

No docstring.

Calls: fake_step_llm, print, run_one_step, self.next_runnable_job

### send_due_daily_digest

function nipux_cli/daemon.py:395

No docstring.

Calls: _is_digest_due, datetime.now, isoformat, now.date, self.db.digest_exists, write_daily_digest

### run_forever

function nipux_cli/daemon.py:407

No docstring.

Calls: _exception_backoff, _exception_payload, _sleep_or_stop, _step_failure_backoff, _work_heartbeat, append_daemon_event, cleanup_registered_shell_processes, current_runtime_fingerprint, datetime.now, isoformat, json.dumps, len, locked_update_metadata, max, os.getpid, print

### locked_update_metadata

function nipux_cli/daemon.py:421

No docstring.

Calls: update_lock_metadata

### _is_digest_due

function nipux_cli/daemon.py:562

No docstring.

Calls: configured_time.split, int

### _raise_keyboard_interrupt

function nipux_cli/daemon.py:572

No docstring.

Calls: none

### _exception_payload

function nipux_cli/daemon.py:576

No docstring.

Calls: str, type

### _failure_backoff

function nipux_cli/daemon.py:583

No docstring.

Calls: max, min

### _step_failure_backoff

function nipux_cli/daemon.py:588

Return a retry delay for failed worker steps. Worker failures are recorded as failed steps rather than escaping as daemon exceptions, so they use the same generic throttling pat...

Calls: _failure_backoff

### _exception_backoff

function nipux_cli/daemon.py:599

No docstring.

Calls: _failure_backoff, _is_rate_limit_error, _retry_after_seconds, max, min

### _is_rate_limit_error

function nipux_cli/daemon.py:609

No docstring.

Calls: _is_rate_limit_text, getattr, type

### _is_rate_limit_text

function nipux_cli/daemon.py:616

No docstring.

Calls: provider_rate_limited

### _retry_after_seconds

function nipux_cli/daemon.py:620

No docstring.

Calls: _exception_headers, _parse_retry_after, headers.items, key.lower

### _exception_headers

function nipux_cli/daemon.py:631

No docstring.

Calls: dict, getattr, items, str

### _parse_retry_after

function nipux_cli/daemon.py:639

No docstring.

Calls: contextlib.suppress, float, max, parsed.replace, parsed.timestamp, parsedate_to_datetime, str, strip, time.time

### _provider_probe_due

function nipux_cli/daemon.py:658

No docstring.

Calls: contextlib.suppress, datetime.fromisoformat, isinstance, job.get, metadata.get, now.astimezone, previous.astimezone, previous.replace, raw.replace, str, strip, total_seconds

### _model_generation_ready

function nipux_cli/daemon.py:671

No docstring.

Calls: join, run_doctor

### _sleep_or_stop

function nipux_cli/daemon.py:680

No docstring.

Calls: time.sleep

### _focused_job_id

function nipux_cli/daemon.py:686

No docstring.

Calls: isinstance, json.loads, parsed.get, path.exists, path.read_text

### remote_model_preflight_failures

function nipux_cli/daemon_control.py:27

No docstring.

Calls: doctor_fn

### _recoverable_provider_preflight

function nipux_cli/daemon_control.py:33

No docstring.

Calls: failure.split, provider_action_required, provider_rate_limited, strip

### recoverable_remote_model_preflight_failures

function nipux_cli/daemon_control.py:45

No docstring.

Calls: _recoverable_provider_preflight, remote_model_preflight_failures

### provider_preflight_is_recoverable

function nipux_cli/daemon_control.py:54

No docstring.

Calls: _recoverable_provider_preflight

### ensure_remote_model_ready_for_worker

function nipux_cli/daemon_control.py:58

No docstring.

Calls: _recoverable_provider_preflight, clear_model_setup_verified, mark_model_setup_verified, print, remote_model_preflight_failures

### cmd_start_impl

function nipux_cli/daemon_control.py:85

No docstring.

Calls: Path, Path.cwd, SystemExit, bool, command.append, config.ensure_dirs, daemon_lock_status, expanduser, load_config, log_path.open, log_path.parent.mkdir, metadata.get, print, process.poll, ready_fn, status.get

### start_daemon_if_needed_impl

function nipux_cli/daemon_control.py:140

No docstring.

Calls: argparse.Namespace, config.ensure_dirs, daemon_lock_status, load_config, metadata.get, print, start_fn, status.get, stop_fn, time.sleep

### cmd_restart_impl

function nipux_cli/daemon_control.py:165

No docstring.

Calls: argparse.Namespace, config.ensure_dirs, float, load_config, start_fn, stop_fn, time.sleep

### stop_daemon_process_impl

function nipux_cli/daemon_control.py:179

No docstring.

Calls: SystemExit, _find_single_daemon_process, daemon_lock_status, isinstance, metadata.get, os.kill, pid_alive, print, status.get, time.sleep, time.time

### _find_single_daemon_process

function nipux_cli/daemon_control.py:213

Best-effort recovery for older locks that lost pid metadata.

Calls: Path, candidates.append, command.split, int, join, len, line.partition, normalized.split, os.getpid, raw_line.strip, result.stdout.splitlines, set, sorted, subprocess.run

### collect_dashboard_state

function nipux_cli/dashboard.py:19

Build a serializable snapshot for status and dashboard commands.

Calls: DEFAULT_REGISTRY.names, _focus_state, _job_card, _select_focus_job, daemon_lock_status, datetime.now, db.list_jobs, isoformat, len, str

### render_dashboard

function nipux_cli/dashboard.py:48

Render a compact terminal dashboard.

Calls: _compact_time, _daemon_text, _job_state_text, _one_line, _render_focus, bool, daemon.get, job.get, join, latest.get, len, lines.append, lines.extend, ljust, max, min

### render_overview

function nipux_cli/dashboard.py:86

Render a human-sized status view for the interactive shell.

Calls: _active_operator_messages, _daemon_health_text, _job_state_text, _one_line, _worker_text, agent_update.get, artifact.get, bool, counts.get, daemon.get, focus.get, get, isinstance, job.get, join, latest_step.get

### _select_focus_job

function nipux_cli/dashboard.py:157

No docstring.

Calls: db.get_job, job.get

### _job_card

function nipux_cli/dashboard.py:167

No docstring.

Calls: _public_step, _step_count, db.list_artifacts, db.list_runs, db.list_steps, len, step.get, sum

### _focus_state

function nipux_cli/dashboard.py:185

No docstring.

Calls: Counter, _active_operator_messages, _public_artifact, _public_run, _public_step, _step_count, db.list_artifacts, db.list_memory, db.list_runs, db.list_steps, dict, endswith, entry.get, isinstance, job.get, len

### _render_focus

function nipux_cli/dashboard.py:256

No docstring.

Calls: _compact_value, _job_state_text, _one_line, _tool_mix, artifact.get, counts.get, entry.get, finding.get, focus.get, get, isinstance, job.get, join, len, lesson.get, lines.append

### _public_run

function nipux_cli/dashboard.py:334

No docstring.

Calls: run.get

### _public_step

function nipux_cli/dashboard.py:345

No docstring.

Calls: _clean_step_summary, input_data.get, isinstance, step.get

### _public_artifact

function nipux_cli/dashboard.py:362

No docstring.

Calls: artifact.get

### _step_count

function nipux_cli/dashboard.py:373

No docstring.

Calls: int, max, step.get

### _active_operator_messages

function nipux_cli/dashboard.py:378

No docstring.

Calls: active_prompt_operator_entries, entry.get, isinstance, metadata.get, str

### _daemon_text

function nipux_cli/dashboard.py:389

No docstring.

Calls: daemon.get, metadata.get

### _daemon_health_text

function nipux_cli/dashboard.py:399

No docstring.

Calls: _age_seconds, _one_line, daemon.get, int, metadata.get, running_step.get

### _worker_text

function nipux_cli/dashboard.py:433

No docstring.

Calls: job.get, job_deferred_until, str

### _job_state_text

function nipux_cli/dashboard.py:442

No docstring.

Calls: job.get, job_deferred_until, str

### _age_seconds

function nipux_cli/dashboard.py:451

No docstring.

Calls: datetime.fromisoformat, datetime.now, max, parsed.astimezone, total_seconds, value.replace

### _compact_time

function nipux_cli/dashboard.py:459

No docstring.

Calls: datetime.fromisoformat, parsed.astimezone, strftime, value.replace

### _one_line

function nipux_cli/dashboard.py:467

No docstring.

Calls: join, max, shorten, split, str

### _clean_step_summary

function nipux_cli/dashboard.py:472

No docstring.

Calls: join, split, str, text.split, text.startswith

### _compact_value

function nipux_cli/dashboard.py:479

No docstring.

Calls: isinstance, join, sorted, str

### _tool_mix

function nipux_cli/dashboard.py:486

No docstring.

Calls: join, tool_counts.items

### resolve_artifact_path

function nipux_cli/dashboard.py:492

No docstring.

Calls: Path, expanduser, str

### utc_now

function nipux_cli/db.py:136

No docstring.

Calls: datetime.now, isoformat

### new_id

function nipux_cli/db.py:140

No docstring.

Calls: uuid.uuid4

### _slugify

function nipux_cli/db.py:144

No docstring.

Calls: new_id, re.sub, strip, value.lower

### _unique_job_id

function nipux_cli/db.py:149

No docstring.

Calls: _slugify, conn.execute, fetchone

### _json_dumps

function nipux_cli/db.py:159

No docstring.

Calls: json.dumps

### _json_loads

function nipux_cli/db.py:163

No docstring.

Calls: isinstance, json.loads

### _bounded_float

function nipux_cli/db.py:171

No docstring.

Calls: float, max, min

### _merge_string_lists

function nipux_cli/db.py:179

No docstring.

Calls: isinstance, join, source.strip, split, str, values.append

### _memory_edge_key

function nipux_cli/db.py:195

No docstring.

Calls: edge.get, str

### _as_int

function nipux_cli/db.py:202

No docstring.

Calls: float, int

### _as_float

function nipux_cli/db.py:209

No docstring.

Calls: float

### _nested_value

function nipux_cli/db.py:216

No docstring.

Calls: current.get, isinstance

### _metadata_list

function nipux_cli/db.py:225

No docstring.

Calls: isinstance, metadata.get

### _change_fingerprint

function nipux_cli/db.py:232

No docstring.

Calls: _json_dumps, entry.get

### _norm_key

function nipux_cli/db.py:236

No docstring.

Calls: re.sub, strip, value.lower

### _clean_status

function nipux_cli/db.py:240

No docstring.

Calls: lower, replace, value.strip

### _experiment_metric_value

function nipux_cli/db.py:245

No docstring.

Calls: entry.get, float

### _same_metric_group

function nipux_cli/db.py:255

No docstring.

Calls: _experiment_metric_value, bool, entry.get, lower, metric_name.strip, metric_unit.strip, str, strip

### _best_experiment_for_metric

function nipux_cli/db.py:270

No docstring.

Calls: _experiment_metric_value, _same_metric_group, experiment.get, max, min

### _metric_delta

function nipux_cli/db.py:294

No docstring.

Calls: _experiment_metric_value, float, round

### _mark_best_experiments

function nipux_cli/db.py:313

No docstring.

Calls: _experiment_metric_value, append, bool, experiment.get, groups.items, groups.setdefault, item.get, lower, max, min, str, strip, winners.append

### _row_to_dict

function nipux_cli/db.py:337

No docstring.

Calls: dict, json.loads, key.removesuffix

### _insert_event

function nipux_cli/db.py:350

No docstring.

Calls: _json_dumps, body.strip, conn.execute, event_type.strip, lower, new_id, ref_id.strip, ref_table.strip, title.strip, utc_now

### _projected_event

function nipux_cli/db.py:394

No docstring.

Calls: none

### AgentDB

class nipux_cli/db.py:420

Small SQLite wrapper with WAL and jittered write retries.

Calls: KeyError, Path, RuntimeError, ValueError, _as_float, _as_int, _best_experiment_for_metric, _bounded_float, _change_fingerprint, _clean_status, _insert_event, _json_dumps, _json_loads, _mark_best_experiments, _memory_edge_key, _merge_string_lists

### __init__

function nipux_cli/db.py:425

No docstring.

Calls: Path, self._conn.execute, self._init_schema, self.path.parent.mkdir, sqlite3.connect, str, threading.RLock

### close

function nipux_cli/db.py:440

No docstring.

Calls: self._conn.close, self._conn.execute

### _init_schema

function nipux_cli/db.py:449

No docstring.

Calls: RuntimeError, fetchone, int, self._conn.execute, self._conn.executescript

### _write

function nipux_cli/db.py:458

No docstring.

Calls: fn, lower, random.uniform, range, self._conn.commit, self._conn.execute, self._conn.rollback, sqlite3.OperationalError, str, time.sleep

### append_event

function nipux_cli/db.py:479

No docstring.

Calls: _insert_event, self._write

### op

function nipux_cli/db.py:491

No docstring.

Calls: _insert_event

### list_events

function nipux_cli/db.py:506

No docstring.

Calls: _row_to_dict, fetchall, filters.append, int, join, lower, params.append, params.extend, self._conn.execute, str, strip

### list_timeline_events

function nipux_cli/db.py:538

Return visible job history, combining durable events with old projected state.

Calls: _metadata_list, _projected_event, artifact.get, entry.get, enumerate, event.get, experiment.get, finding.get, format_metric_value, int, isinstance, job.get, lesson.get, list, max, memory.get

### create_job

function nipux_cli/db.py:724

No docstring.

Calls: _insert_event, _json_dumps, _unique_job_id, conn.execute, objective.strip, self._write, splitlines, utc_now

### op

function nipux_cli/db.py:737

No docstring.

Calls: _insert_event, _json_dumps, _unique_job_id, conn.execute

### get_job

function nipux_cli/db.py:759

No docstring.

Calls: KeyError, _row_to_dict, fetchone, self._conn.execute

### list_jobs

function nipux_cli/db.py:766

No docstring.

Calls: _row_to_dict, fetchall, join, list, self._conn.execute

### update_job_status

function nipux_cli/db.py:778

No docstring.

Calls: KeyError, _insert_event, _json_dumps, conn.execute, current.update, fetchone, get, json.loads, self._write, str, utc_now

### op

function nipux_cli/db.py:781

No docstring.

Calls: KeyError, _insert_event, _json_dumps, conn.execute, current.update, fetchone, get, json.loads, str

### update_job_metadata

function nipux_cli/db.py:809

No docstring.

Calls: KeyError, _json_dumps, conn.execute, current.update, fetchone, json.loads, self._write, utc_now

### op

function nipux_cli/db.py:812

No docstring.

Calls: KeyError, _json_dumps, conn.execute, current.update, fetchone, json.loads

### claim_operator_messages

function nipux_cli/db.py:825

No docstring.

Calls: KeyError, _insert_event, _json_dumps, claimed.append, conn.execute, dict, entry.get, fetchone, isinstance, json.loads, len, lower, metadata.get, mode.strip, replace, self._write

### op

function nipux_cli/db.py:835

No docstring.

Calls: KeyError, _insert_event, _json_dumps, claimed.append, conn.execute, dict, entry.get, fetchone, isinstance, json.loads, len, lower, metadata.get, replace, str, strip

### acknowledge_operator_messages

function nipux_cli/db.py:883

No docstring.

Calls: KeyError, _insert_event, _json_dumps, acknowledged.append, conn.execute, dict, entry.get, fetchone, isinstance, json.loads, len, lower, metadata.get, replace, self._write, status.strip

### op

function nipux_cli/db.py:897

No docstring.

Calls: KeyError, _insert_event, _json_dumps, acknowledged.append, conn.execute, dict, entry.get, fetchone, isinstance, json.loads, len, lower, metadata.get, replace, str, strip

### rename_job

function nipux_cli/db.py:955

No docstring.

Calls: KeyError, ValueError, _insert_event, _row_to_dict, conn.execute, dict, fetchone, self._write, title.strip, utc_now

### op

function nipux_cli/db.py:961

No docstring.

Calls: KeyError, _insert_event, _row_to_dict, conn.execute, dict, fetchone

### delete_job

function nipux_cli/db.py:982

No docstring.

Calls: KeyError, _row_to_dict, conn.execute, fetchall, fetchone, self._write, str

### op

function nipux_cli/db.py:983

No docstring.

Calls: KeyError, _row_to_dict, conn.execute, fetchall, fetchone, str

### append_operator_message

function nipux_cli/db.py:1012

No docstring.

Calls: KeyError, ValueError, _insert_event, _json_dumps, conn.execute, fetchone, isinstance, json.loads, lower, message.strip, messages.append, metadata.get, mode.strip, replace, self._write, utc_now

### op

function nipux_cli/db.py:1029

No docstring.

Calls: KeyError, _insert_event, _json_dumps, conn.execute, fetchone, isinstance, json.loads, messages.append, metadata.get

### append_agent_update

function nipux_cli/db.py:1058

No docstring.

Calls: KeyError, ValueError, _insert_event, _json_dumps, category.strip, conn.execute, fetchone, isinstance, job_metadata.get, json.loads, message.strip, self._write, updates.append, utc_now

### op

function nipux_cli/db.py:1077

No docstring.

Calls: KeyError, _insert_event, _json_dumps, conn.execute, fetchone, isinstance, job_metadata.get, json.loads, updates.append

### append_lesson

function nipux_cli/db.py:1106

No docstring.

Calls: KeyError, ValueError, _insert_event, _json_dumps, _norm_key, category.strip, conn.execute, current.get, existing.get, fetchone, int, isinstance, item.get, job_metadata.get, json.loads, lesson.strip

### op

function nipux_cli/db.py:1128

No docstring.

Calls: KeyError, _insert_event, _json_dumps, _norm_key, conn.execute, current.get, existing.get, fetchone, int, isinstance, item.get, job_metadata.get, json.loads, lessons.append, merged.update, next

### append_memory_graph_records

function nipux_cli/db.py:1188

No docstring.

Calls: KeyError, ValueError, _bounded_float, _insert_event, _json_dumps, _memory_edge_key, _merge_string_lists, _metadata_list, _norm_key, conn.execute, current.get, edge.get, existing_edge_keys.add, fetchone, int, isinstance

### op

function nipux_cli/db.py:1201

No docstring.

Calls: KeyError, _bounded_float, _insert_event, _json_dumps, _memory_edge_key, _merge_string_lists, _metadata_list, _norm_key, conn.execute, current.get, edge.get, existing_edge_keys.add, fetchone, int, isinstance, job_metadata.get

### append_source_record

function nipux_cli/db.py:1349

No docstring.

Calls: KeyError, ValueError, _change_fingerprint, _insert_event, _json_dumps, _metadata_list, _norm_key, conn.execute, current.get, dict.fromkeys, entry.get, fetchone, float, int, isinstance, json.loads

### op

function nipux_cli/db.py:1368

No docstring.

Calls: KeyError, _change_fingerprint, _insert_event, _json_dumps, _metadata_list, conn.execute, current.get, dict.fromkeys, entry.get, fetchone, float, int, isinstance, json.loads, list, merged_metadata.update

### append_finding_record

function nipux_cli/db.py:1454

No docstring.

Calls: KeyError, ValueError, _change_fingerprint, _insert_event, _json_dumps, _metadata_list, _norm_key, category.strip, conn.execute, contact.strip, current.get, entry.get, evidence_artifact.strip, fetchone, findings.append, isinstance

### op

function nipux_cli/db.py:1478

No docstring.

Calls: KeyError, _change_fingerprint, _insert_event, _json_dumps, _metadata_list, category.strip, conn.execute, contact.strip, current.get, entry.get, evidence_artifact.strip, fetchone, findings.append, isinstance, items, json.loads

### append_roadmap_record

function nipux_cli/db.py:1569

No docstring.

Calls: KeyError, ValueError, _change_fingerprint, _clean_status, _insert_event, _json_dumps, _norm_key, bool, conn.execute, current.get, current_milestone.strip, entry.get, existing_features.append, feature.get, fetchone, int

### merge_feature

function nipux_cli/db.py:1589

No docstring.

Calls: _change_fingerprint, _clean_status, _norm_key, current.get, entry.get, existing_features.append, feature.get, isinstance, items, lower, merged.update, next, replace, str, strip

### op

function nipux_cli/db.py:1654

No docstring.

Calls: KeyError, _change_fingerprint, _clean_status, _insert_event, _json_dumps, _norm_key, bool, conn.execute, current.get, current_milestone.strip, entry.get, fetchone, int, isinstance, items, job_metadata.get

### append_milestone_validation_record

function nipux_cli/db.py:1860

No docstring.

Calls: KeyError, ValueError, _clean_status, _insert_event, _json_dumps, _norm_key, conn.execute, current.get, entry.get, evidence.strip, fetchone, isinstance, job_metadata.get, json.loads, merged_metadata.update, milestone.strip

### op

function nipux_cli/db.py:1880

No docstring.

Calls: KeyError, _insert_event, _json_dumps, _norm_key, conn.execute, current.get, entry.get, evidence.strip, fetchone, isinstance, job_metadata.get, json.loads, merged_metadata.update, milestones.append, next, next_action.strip

### append_task_record

function nipux_cli/db.py:1980

No docstring.

Calls: KeyError, ValueError, _change_fingerprint, _insert_event, _json_dumps, _metadata_list, _norm_key, acceptance_criteria.strip, conn.execute, current.get, entry.get, evidence_needed.strip, fetchone, goal.strip, int, isinstance

### op

function nipux_cli/db.py:2009

No docstring.

Calls: KeyError, _change_fingerprint, _insert_event, _json_dumps, _metadata_list, _norm_key, acceptance_criteria.strip, conn.execute, current.get, entry.get, evidence_needed.strip, fetchone, goal.strip, int, isinstance, items

### append_experiment_record

function nipux_cli/db.py:2117

No docstring.

Calls: KeyError, ValueError, _best_experiment_for_metric, _change_fingerprint, _insert_event, _json_dumps, _mark_best_experiments, _metadata_list, _metric_delta, _norm_key, bool, conn.execute, current.get, entry.get, evidence_artifact.strip, experiments.append

### op

function nipux_cli/db.py:2145

No docstring.

Calls: KeyError, _best_experiment_for_metric, _change_fingerprint, _insert_event, _json_dumps, _mark_best_experiments, _metadata_list, _metric_delta, bool, conn.execute, current.get, entry.get, evidence_artifact.strip, experiments.append, fetchone, format_metric_value

### append_reflection

function nipux_cli/db.py:2276

No docstring.

Calls: KeyError, ValueError, _insert_event, _json_dumps, _metadata_list, conn.execute, fetchone, json.loads, reflections.append, self._write, strategy.strip, summary.strip, utc_now

### op

function nipux_cli/db.py:2295

No docstring.

Calls: KeyError, _insert_event, _json_dumps, _metadata_list, conn.execute, fetchone, json.loads, reflections.append, strategy.strip

### start_run

function nipux_cli/db.py:2322

No docstring.

Calls: _insert_event, conn.execute, new_id, self._write, utc_now

### op

function nipux_cli/db.py:2326

No docstring.

Calls: _insert_event, conn.execute

### finish_run

function nipux_cli/db.py:2350

No docstring.

Calls: conn.execute, self._write, utc_now

### op

function nipux_cli/db.py:2353

No docstring.

Calls: conn.execute

### mark_interrupted_running

function nipux_cli/db.py:2361

No docstring.

Calls: _json_dumps, conn.execute, int, self._write, utc_now

### op

function nipux_cli/db.py:2365

No docstring.

Calls: _json_dumps, conn.execute, int

### next_step_no

function nipux_cli/db.py:2392

No docstring.

Calls: fetchone, int, self._conn.execute

### add_step

function nipux_cli/db.py:2396

No docstring.

Calls: _insert_event, _json_dumps, conn.execute, new_id, self._write, self.next_step_no, utc_now

### op

function nipux_cli/db.py:2411

No docstring.

Calls: _insert_event, _json_dumps, conn.execute

### finish_step

function nipux_cli/db.py:2434

No docstring.

Calls: _insert_event, _json_dumps, conn.execute, fetchone, self._write, utc_now

### op

function nipux_cli/db.py:2445

No docstring.

Calls: _insert_event, _json_dumps, conn.execute, fetchone

### list_steps

function nipux_cli/db.py:2480

No docstring.

Calls: _row_to_dict, fetchall, int, self._conn.execute

### job_record_counts

function nipux_cli/db.py:2519

No docstring.

Calls: fetchone, int, self._conn.execute

### job_token_usage

function nipux_cli/db.py:2537

No docstring.

Calls: _as_float, _as_int, _json_loads, _nested_value, bool, fetchall, isinstance, metadata.get, self._conn.execute, str, usage.get

### list_runs

function nipux_cli/db.py:2592

No docstring.

Calls: _row_to_dict, fetchall, self._conn.execute

### add_artifact

function nipux_cli/db.py:2599

No docstring.

Calls: _insert_event, _json_dumps, conn.execute, new_id, self._write, str, utc_now

### op

function nipux_cli/db.py:2615

No docstring.

Calls: _insert_event, _json_dumps, conn.execute, str

### get_artifact

function nipux_cli/db.py:2650

No docstring.

Calls: KeyError, _row_to_dict, fetchone, self._conn.execute

### list_artifacts

function nipux_cli/db.py:2657

No docstring.

Calls: _row_to_dict, fetchall, self._conn.execute

### upsert_memory

function nipux_cli/db.py:2664

No docstring.

Calls: _insert_event, _json_dumps, conn.execute, fetchone, new_id, self._write, str, utc_now

### op

function nipux_cli/db.py:2675

No docstring.

Calls: _insert_event, _json_dumps, conn.execute, fetchone, str

### list_memory

function nipux_cli/db.py:2704

No docstring.

Calls: _row_to_dict, fetchall, self._conn.execute

### digest_exists

function nipux_cli/db.py:2711

No docstring.

Calls: fetchone, self._conn.execute

### record_digest

function nipux_cli/db.py:2718

No docstring.

Calls: _insert_event, conn.execute, new_id, self._write, str, utc_now

### op

function nipux_cli/db.py:2731

No docstring.

Calls: _insert_event, conn.execute, str, utc_now

### _metadata_list

function nipux_cli/digest.py:17

No docstring.

Calls: isinstance, job.get, metadata.get

### _active_operator_messages

function nipux_cli/digest.py:23

No docstring.

Calls: active_prompt_operator_entries, entry.get, str

### _safe_int

function nipux_cli/digest.py:32

No docstring.

Calls: float, int

### _latest_run_model

function nipux_cli/digest.py:39

No docstring.

Calls: db.list_runs, get, str

### _usage_lines

function nipux_cli/digest.py:46

No docstring.

Calls: _format_compact_count, _format_usage_cost, _latest_run_model, _safe_int, bool, db.job_token_usage, lines.append, usage.get

### render_job_digest

function nipux_cli/digest.py:86

No docstring.

Calls: _active_operator_messages, _metadata_list, _usage_lines, artifact.get, bool, db.get_job, db.list_artifacts, db.list_steps, entry.get, experiment.get, finding.get, float, int, item.get, join, len

### send_digest_email

function nipux_cli/digest.py:212

No docstring.

Calls: EmailMessage, ValueError, all, message.set_content, smtp.login, smtp.send_message, smtp.starttls, smtplib.SMTP

### render_daily_digest

function nipux_cli/digest.py:232

No docstring.

Calls: _active_operator_messages, _metadata_list, _usage_lines, artifact.get, db.list_artifacts, db.list_jobs, db.list_steps, entry.get, experiment.get, finding.get, float, int, item.get, join, len, lesson.get

### write_daily_digest

function nipux_cli/digest.py:340

No docstring.

Calls: Path, body_path.write_text, config.runtime.digests_dir.mkdir, date.today, db.digest_exists, db.record_digest, email_result.get, isoformat, render_daily_digest, send_digest_email, str

### Check

class nipux_cli/doctor.py:20

No docstring.

Calls: dataclass

### as_dict

function nipux_cli/doctor.py:25

No docstring.

Calls: none

### _check_writable_dir

function nipux_cli/doctor.py:29

No docstring.

Calls: Check, path.mkdir, probe.unlink, probe.write_text, str

### _check_db

function nipux_cli/doctor.py:40

No docstring.

Calls: AgentDB, Check, db.close, str

### _check_tool_surface

function nipux_cli/doctor.py:49

No docstring.

Calls: Check, DEFAULT_REGISTRY.names, join, len, set, sorted

### _check_model_config

function nipux_cli/doctor.py:57

No docstring.

Calls: Check, host.endswith, lower, urlparse

### _check_browser_runtime

function nipux_cli/doctor.py:72

No docstring.

Calls: Check, shutil.which

### _check_model_endpoint

function nipux_cli/doctor.py:86

No docstring.

Calls: Check, _check_model_generation, _check_openrouter_auth, _model_available, config.model.base_url.rstrip, data.get, decode, isinstance, json.loads, len, response.read, urllib.request.Request, urllib.request.urlopen

### _check_model_generation

function nipux_cli/doctor.py:116

No docstring.

Calls: Check, _extract_error_message, config.model.base_url.rstrip, data.get, decode, encode, exc.read, isinstance, json.dumps, json.loads, response.read, str, urllib.request.Request, urllib.request.urlopen

### _check_openrouter_auth

function nipux_cli/doctor.py:163

No docstring.

Calls: Check, _extract_error_message, decode, exc.read, response.read, str, urllib.request.Request, urllib.request.urlopen

### _extract_error_message

function nipux_cli/doctor.py:182

No docstring.

Calls: _extract_error_raw, _metadata_value, body.strip, data.get, details.append, error.get, isinstance, join, json.loads, primary.strip, str, strip

### _metadata_value

function nipux_cli/doctor.py:212

No docstring.

Calls: isinstance, metadata.get, str, strip

### _extract_error_raw

function nipux_cli/doctor.py:221

No docstring.

Calls: _extract_error_message, isinstance, json.dumps, json.loads, metadata.get, str, strip

### _model_available

function nipux_cli/doctor.py:242

No docstring.

Calls: data.get, isinstance, item.get, str

### run_doctor

function nipux_cli/doctor.py:249

No docstring.

Calls: _check_browser_runtime, _check_db, _check_model_config, _check_model_endpoint, _check_tool_surface, _check_writable_dir, checks.append, load_config

### event_line

function nipux_cli/event_render.py:13

No docstring.

Calls: _one_line, event_display_parts

### event_display_parts

function nipux_cli/event_render.py:19

No docstring.

Calls: _one_line, body.startswith, clean_step_summary, compact_time, event.get, event_label, format_metric_value, generic_display_text, isinstance, kind.startswith, metadata.get, min, shlex.quote, str, strip

### event_label

function nipux_cli/event_render.py:77

No docstring.

Calls: kind.endswith, kind.startswith, kind.upper, labels.get, metadata.get, str

### compact_time

function nipux_cli/event_render.py:114

No docstring.

Calls: _one_line, len, value.replace

### FirstRunFrameDeps

class nipux_cli/first_run_controller.py:25

No docstring.

Calls: dataclass

### handle_first_run_action

function nipux_cli/first_run_controller.py:36

No docstring.

Calls: TOGGLE_SETTING_COMMANDS.get, action.split, action.startswith, bool, config_field_value, deps.capture_command, deps.capture_setting_command, deps.model_setup_verified, deps.verify_model_setup

### handle_first_run_frame_line

function nipux_cli/first_run_controller.py:80

No docstring.

Calls: deps.capture_command, deps.capture_setting_command, deps.current_default_job_id, deps.extract_objective, deps.model_setup_verified, deps.verify_model_setup, first_run_chat_reply, first_token, line.strip, lowered.startswith, original.lower, original.startswith, strip

### first_run_chat_reply

function nipux_cli/first_run_controller.py:134

No docstring.

Calls: none

### create_first_run_job

function nipux_cli/first_run_controller.py:139

No docstring.

Calls: deps.create_job, deps.model_setup_verified, objective.strip

### capture_first_run_command

function nipux_cli/first_run_controller.py:153

No docstring.

Calls: StringIO, item.split, item.strip, join, print, redirect_stdout, run_shell_line, splitlines, stream.getvalue

### first_token

function nipux_cli/first_run_controller.py:165

No docstring.

Calls: line.split, lower, shlex.split

### FirstRunRuntimeDeps

class nipux_cli/first_run_frame_runtime.py:28

No docstring.

Calls: dataclass

### run_first_run_frame

function nipux_cli/first_run_frame_runtime.py:36

No docstring.

Calls: _append_notice, _apply_first_run_action, _frame_enter_sequence, _frame_exit_sequence, _handle_edit_input, _handle_first_run_escape, _one_line, _safe_render_frame, _submit_first_run_line, autocomplete_slash, char.isprintable, clamp_selection, deps.actions, next_first_run_view_a...

### clamp_selection

function nipux_cli/first_run_frame_runtime.py:178

No docstring.

Calls: len, max, min

### _safe_render_frame

function nipux_cli/first_run_frame_runtime.py:184

No docstring.

Calls: _append_notice, _fallback_first_run_frame, _one_line, deps.render_frame, print, type

### _fallback_first_run_frame

function nipux_cli/first_run_frame_runtime.py:203

No docstring.

Calls: _fit_plain, _one_line, join, lines.extend, max, shutil.get_terminal_size

### _submit_first_run_line

function nipux_cli/first_run_frame_runtime.py:219

No docstring.

Calls: buffer.strip, clamp_selection, deps.actions, deps.handle_action, deps.handle_line, line.startswith

### _handle_first_run_escape

function nipux_cli/first_run_frame_runtime.py:237

No docstring.

Calls: _apply_first_run_action, buffer.startswith, clamp_selection, cycle_slash, decode_terminal_escape, deps.actions, deps.click_action, deps.handle_action, drain_pending_input, isinstance, len, read_escape_sequence

### directional_first_run_action

function nipux_cli/first_run_frame_runtime.py:295

Return the setup-screen action for left/right navigation.

Calls: key.startswith, label.lower, reversed

### _apply_first_run_action

function nipux_cli/first_run_frame_runtime.py:309

No docstring.

Calls: _append_notice, isinstance, notices.clear, str, strip

### _handle_edit_input

function nipux_cli/first_run_frame_runtime.py:340

No docstring.

Calls: _append_notice, _save_first_run_edit, char.isprintable, decode_terminal_escape, read_escape_sequence

### required_first_run_edit_field

function nipux_cli/first_run_frame_runtime.py:372

No docstring.

Calls: get

### next_first_run_view_after_edit

function nipux_cli/first_run_frame_runtime.py:380

No docstring.

Calls: get

### _save_first_run_edit

function nipux_cli/first_run_frame_runtime.py:388

No docstring.

Calls: _is_local_endpoint, endswith, inline_setting_notice, load_config, parsed.path.rstrip, raw_value.strip, urlparse, value.lower

### _is_local_endpoint

function nipux_cli/first_run_frame_runtime.py:415

No docstring.

Calls: host.endswith, lower, urlparse

### _append_notice

function nipux_cli/first_run_frame_runtime.py:420

No docstring.

Calls: notices.append

### _one_line

function nipux_cli/first_run_frame_runtime.py:425

No docstring.

Calls: join, len, max, split, str

### _fit_plain

function nipux_cli/first_run_frame_runtime.py:432

No docstring.

Calls: _one_line, len, max, str

### build_first_run_frame

function nipux_cli/first_run_tui.py:55

No docstring.

Calls: _clamp_first_run_selection, _compose_bar, _first_run_edit_hint, _first_run_hint, _first_run_prompt_label, _normalize_first_run_view, _wizard_body_lines, edit_target_masks_input, first_run_themed_lines, join, len, max

### first_run_columns

function nipux_cli/first_run_tui.py:109

No docstring.

Calls: int, max, min

### first_run_actions

function nipux_cli/first_run_tui.py:118

No docstring.

Calls: _normalize_first_run_view

### first_run_themed_lines

function nipux_cli/first_run_tui.py:122

No docstring.

Calls: _themed_lines

### _wizard_body_lines

function nipux_cli/first_run_tui.py:126

No docstring.

Calls: _access_page_lines, _api_page_lines, _append_notice_block, _doctor_page_lines, _endpoint_page_lines, _fit_page, _model_page_lines

### _model_page_lines

function nipux_cli/first_run_tui.py:155

No docstring.

Calls: _accent, _bold, _center_ansi, _muted, _one_line, _panel, _step_count_label, _step_header, max, min

### _endpoint_page_lines

function nipux_cli/first_run_tui.py:178

No docstring.

Calls: _bold, _center_ansi, _form_panel, _muted, _step_count_label, _step_header, min

### _api_page_lines

function nipux_cli/first_run_tui.py:198

No docstring.

Calls: _bold, _center_ansi, _muted, _panel, _step_count_label, _step_header, _style, min

### _access_page_lines

function nipux_cli/first_run_tui.py:223

No docstring.

Calls: _access_row, _action_cards, _bold, _center_ansi, _muted, _panel, _step_count_label, _step_header, first_run_actions, min

### _doctor_page_lines

function nipux_cli/first_run_tui.py:243

No docstring.

Calls: _accent, _action_cards, _bold, _center_ansi, _fit_ansi, _muted, _panel, _step_count_label, _step_header, first_run_actions, min

### _stepper_lines

function nipux_cli/first_run_tui.py:269

No docstring.

Calls: _accent, _fit_ansi, _muted, _step_state, lines.append

### _step_header

function nipux_cli/first_run_tui.py:278

No docstring.

Calls: _accent, _bold, _center_ansi, _muted, enumerate, join, parts.append

### _action_cards

function nipux_cli/first_run_tui.py:290

No docstring.

Calls: _action_tile, _center_ansi, _join_many_cards, enumerate, len, max, min, row.rstrip

### _action_tile

function nipux_cli/first_run_tui.py:306

No docstring.

Calls: _accent, _action_value, _bold, _fit_ansi, _muted, _one_line, border, max

### _panel

function nipux_cli/first_run_tui.py:329

No docstring.

Calls: _center_ansi, _fit_ansi, _muted, len, lines.append, max

### _form_panel

function nipux_cli/first_run_tui.py:340

No docstring.

Calls: _accent, _bold, _muted, _one_line, _panel, max

### _choice_card

function nipux_cli/first_run_tui.py:352

No docstring.

Calls: _accent, _bold, _fit_ansi, _muted, border, max

### _join_cards

function nipux_cli/first_run_tui.py:366

No docstring.

Calls: _center_ansi, _strip_ansi, len, max, range, rows.append

### _join_many_cards

function nipux_cli/first_run_tui.py:376

No docstring.

Calls: _fit_ansi, _strip_ansi, gap_text.join, len, max, range, row_parts.append, rows.append

### _append_notice_block

function nipux_cli/first_run_tui.py:389

No docstring.

Calls: _accent, _bold, _fit_ansi, _one_line, len, max, min, notice_lines.append

### _fit_page

function nipux_cli/first_run_tui.py:400

No docstring.

Calls: _fit_ansi, fitted.extend, len, range

### _action_line

function nipux_cli/first_run_tui.py:407

No docstring.

Calls: _accent, _action_value, _bold, _fit_ansi, _muted, _one_line, max

### _screen_value_lines

function nipux_cli/first_run_tui.py:425

No docstring.

Calls: _large_value, _muted

### _large_value

function nipux_cli/first_run_tui.py:444

No docstring.

Calls: _accent, _bold, _fit_ansi, _muted, _one_line, len, max

### _action_value

function nipux_cli/first_run_tui.py:449

No docstring.

Calls: bool, config_field_value, key.split, key.startswith, str

### _step_state

function nipux_cli/first_run_tui.py:465

No docstring.

Calls: _is_local_endpoint, _one_line, bool, sum

### _first_run_hint

function nipux_cli/first_run_tui.py:480

No docstring.

Calls: none

### _first_run_edit_hint

function nipux_cli/first_run_tui.py:494

No docstring.

Calls: edit_target_hint

### _first_run_prompt_label

function nipux_cli/first_run_tui.py:504

No docstring.

Calls: edit_target_label

### _left_title

function nipux_cli/first_run_tui.py:514

No docstring.

Calls: _screen_heading

### _screen_heading

function nipux_cli/first_run_tui.py:518

No docstring.

Calls: get

### _screen_copy

function nipux_cli/first_run_tui.py:528

No docstring.

Calls: get

### _install_summary

function nipux_cli/first_run_tui.py:538

No docstring.

Calls: _is_local_endpoint, _muted, _one_line

### _normalize_first_run_view

function nipux_cli/first_run_tui.py:544

No docstring.

Calls: none

### _step_count_label

function nipux_cli/first_run_tui.py:548

No docstring.

Calls: keys.index, len

### _access_row

function nipux_cli/first_run_tui.py:557

No docstring.

Calls: _accent, _fit_ansi, _muted

### _is_local_endpoint

function nipux_cli/first_run_tui.py:562

No docstring.

Calls: lowered.startswith, value.lower

### _clamp_first_run_selection

function nipux_cli/first_run_tui.py:567

No docstring.

Calls: first_run_actions, len, max, min

### load_frame_snapshot

function nipux_cli/frame_snapshot.py:16

Return the compact state bundle rendered by the chat TUI.

Calls: _summary_events, daemon_lock_status, db.get_job, db.job_record_counts, db.job_token_usage, db.list_artifacts, db.list_events, db.list_jobs, db.list_memory, db.list_steps, item.get, load_workspace_frame_snapshot, max, str

### load_workspace_frame_snapshot

function nipux_cli/frame_snapshot.py:75

Return the chat/control frame before any worker job is focused.

Calls: _safe_job, _summary_events, _workspace_token_usage, daemon_lock_status, db.job_record_counts, db.job_token_usage, db.list_artifacts, db.list_events, db.list_jobs, db.list_memory, db.list_steps, focused_job.get, item.get, len, list, max

### _safe_job

function nipux_cli/frame_snapshot.py:139

No docstring.

Calls: db.get_job

### _workspace_token_usage

function nipux_cli/frame_snapshot.py:148

No docstring.

Calls: none

### _summary_events

function nipux_cli/frame_snapshot.py:161

No docstring.

Calls: db.list_events, event.get, is_summary_event_candidate, len, max, merged.values, sorted, str

### ToolCall

class nipux_cli/llm.py:18

No docstring.

Calls: dataclass, field

### LLMResponse

class nipux_cli/llm.py:25

No docstring.

Calls: dataclass, field

### LLMResponseError

class nipux_cli/llm.py:33

Raised when a provider returns an OpenAI-shaped response without choices.

Calls: __init__, super

### __init__

function nipux_cli/llm.py:36

No docstring.

Calls: __init__, super

### StepLLM

class nipux_cli/llm.py:41

No docstring.

Calls: none

### next_action

function nipux_cli/llm.py:42

No docstring.

Calls: none

### OpenAIChatLLM

class nipux_cli/llm.py:46

OpenAI-compatible chat-completions adapter.

Calls: LLMResponse, LLMResponseError, OpenAI, ToolCall, _enrich_openrouter_generation_usage, _is_unsupported_tool_choice_error, _response_id, _response_model, _response_payload, _response_usage, calls.append, dict, error.get, fallback_request.pop, isinstance, json.loads

### __init__

function nipux_cli/llm.py:51

No docstring.

Calls: OpenAI

### next_action

function nipux_cli/llm.py:67

No docstring.

Calls: LLMResponse, LLMResponseError, ToolCall, _enrich_openrouter_generation_usage, _is_unsupported_tool_choice_error, _response_id, _response_model, _response_payload, _response_usage, calls.append, dict, error.get, fallback_request.pop, isinstance, json.loads, payload.get

### complete

function nipux_cli/llm.py:115

No docstring.

Calls: self.complete_response

### complete_response

function nipux_cli/llm.py:118

No docstring.

Calls: LLMResponse, LLMResponseError, _enrich_openrouter_generation_usage, _response_id, _response_model, _response_payload, _response_usage, error.get, isinstance, payload.get, self._openai.chat.completions.create, str

### ScriptedLLM

class nipux_cli/llm.py:146

Tiny deterministic LLM used by tests and CLI dry runs.

Calls: LLMResponse, list, self.responses.pop

### __init__

function nipux_cli/llm.py:149

No docstring.

Calls: list

### next_action

function nipux_cli/llm.py:152

No docstring.

Calls: LLMResponse, self.responses.pop

### _response_payload

function nipux_cli/llm.py:159

No docstring.

Calls: hasattr, isinstance, repr, response.model_dump, response.to_dict

### _response_usage

function nipux_cli/llm.py:169

No docstring.

Calls: _estimate_token_count, _response_payload, dict, getattr, hasattr, isinstance, json.dumps, payload.get, usage_obj.model_dump

### _enrich_openrouter_generation_usage

function nipux_cli/llm.py:200

No docstring.

Calls: _safe_float, _safe_int, bool, data.get, decode, dict, enriched.get, int, isinstance, json.loads, payload.get, response.read, urllib.parse.quote, urllib.parse.urlparse, urllib.request.Request, urllib.request.urlopen

### _safe_float

function nipux_cli/llm.py:242

No docstring.

Calls: float

### _safe_int

function nipux_cli/llm.py:251

No docstring.

Calls: float, int

### _is_unsupported_tool_choice_error

function nipux_cli/llm.py:260

No docstring.

Calls: any, lower, type

### _estimate_token_count

function nipux_cli/llm.py:275

No docstring.

Calls: len, max

### _response_model

function nipux_cli/llm.py:281

No docstring.

Calls: _response_payload, getattr, payload.get, str

### _response_id

function nipux_cli/llm.py:286

No docstring.

Calls: _response_payload, getattr, payload.get, str

### measurement_candidates

function nipux_cli/measurement.py:38

No docstring.

Calls: EXPLICIT_RESULT_UNIT_PATTERN.search, MEASUREMENT_INTENT_PATTERN.search, MEASUREMENT_PATTERN.finditer, _candidate_is_diagnostic_only, _table_measurement_candidates, bool, candidates.append, join, len, match.end, match.group, match.start, min, output.get, split, str

### _table_measurement_candidates

function nipux_cli/measurement.py:68

No docstring.

Calls: TABLE_NUMBER_PATTERN.search, TABLE_UNIT_PATTERN.search, _is_markdown_separator_row, _split_markdown_table_row, _table_measurement_label, candidates.append, enumerate, header.strip, len, line.strip, number.group, splitlines, startswith, str, strip

### _split_markdown_table_row

function nipux_cli/measurement.py:99

No docstring.

Calls: cell.strip, join, raw.endswith, raw.split, raw.startswith, split, str, strip

### _is_markdown_separator_row

function nipux_cli/measurement.py:108

No docstring.

Calls: all, bool, cell.strip, re.fullmatch

### _table_measurement_label

function nipux_cli/measurement.py:112

No docstring.

Calls: TABLE_NUMBER_PATTERN.fullmatch, enumerate, header.strip, len, lower, min, range, strip

### measurement_candidates_are_diagnostic_only

function nipux_cli/measurement.py:126

No docstring.

Calls: MEASUREMENT_INTENT_PATTERN.search, _candidate_is_diagnostic_only, all, bool, str

### _candidate_is_diagnostic_only

function nipux_cli/measurement.py:131

No docstring.

Calls: ACTION_MEASUREMENT_PATTERN.search, DIAGNOSTIC_MEASUREMENT_PATTERN.search, EXPLICIT_RESULT_UNIT_PATTERN.search, LABELED_MEASUREMENT_PATTERN.search, bool, re.search

### memory_graph_from_job

function nipux_cli/memory_graph.py:53

No docstring.

Calls: graph.get, isinstance, job.get, metadata.get

### memory_graph_for_prompt

function nipux_cli/memory_graph.py:65

No docstring.

Calls: _clean_string_list, _durable_signal_count, _edge_index, _node_contains_stale_token, _node_has_stale_id, clip_text, edge_index.get, isinstance, job.get, join, len, lines.append, max, memory_graph_from_job, metadata.get, node.get

### _node_contains_stale_token

function nipux_cli/memory_graph.py:132

No docstring.

Calls: _clean_string_list, _node_has_negative_memory_marker, join, node.get, re.escape, re.search, str, strip, text.lower, token.startswith

### _node_has_negative_memory_marker

function nipux_cli/memory_graph.py:157

No docstring.

Calls: any

### _node_has_stale_id

function nipux_cli/memory_graph.py:161

No docstring.

Calls: node.get, str, strip

### search_memory_graph

function nipux_cli/memory_graph.py:171

No docstring.

Calls: edge.get, graph.get, isinstance, max, node.get, rank_memory_nodes, str

### rank_memory_nodes

function nipux_cli/memory_graph.py:185

No docstring.

Calls: _node_score, _tokens, max, sorted

### _node_score

function nipux_cli/memory_graph.py:193

No docstring.

Calls: _clean_string_list, _float_between, _recency_score, int, join, lower, min, node.get, str

### _edge_index

function nipux_cli/memory_graph.py:226

No docstring.

Calls: append, edge.get, index.setdefault, str

### _tokens

function nipux_cli/memory_graph.py:239

No docstring.

Calls: re.findall, value.lower

### _float_between

function nipux_cli/memory_graph.py:243

No docstring.

Calls: float, max, min

### _recency_score

function nipux_cli/memory_graph.py:251

No docstring.

Calls: datetime.fromisoformat, datetime.now, max, total_seconds, value.replace

### _clean_string_list

function nipux_cli/memory_graph.py:268

No docstring.

Calls: isinstance, join, split, str, strip

### _durable_signal_count

function nipux_cli/memory_graph.py:274

No docstring.

Calls: isinstance, job.get, metadata.get, roadmap.get, sum

### render_memory_graph_html

function nipux_cli/memory_graph_view.py:12

Return a standalone clickable graph page for a job's memory graph.

Calls: _view_edge, _view_node, escape, graph.get, job.get, json.dumps, memory_graph_from_job, replace, str

### _view_node

function nipux_cli/memory_graph_view.py:271

No docstring.

Calls: _string_list, node.get, str

### _view_edge

function nipux_cli/memory_graph_view.py:287

No docstring.

Calls: edge.get, str

### _string_list

function nipux_cli/memory_graph_view.py:296

No docstring.

Calls: isinstance, join, split, str, strip

### format_metric_value

function nipux_cli/metric_format.py:8

Return a readable metric string such as ``score=0.82`` or ``tokens=4200 tokens``.

Calls: metric_unit.startswith, str, strip

### operator_entry_is_active

function nipux_cli/operator_context.py:27

No docstring.

Calls: entry.get, lower, replace, str, strip

### operator_entry_is_prompt_relevant

function nipux_cli/operator_context.py:36

No docstring.

Calls: _actionable, _conversation_only, entry.get, lower, replace, str, strip

### active_prompt_operator_entries

function nipux_cli/operator_context.py:50

No docstring.

Calls: isinstance, operator_entry_is_prompt_relevant

### inactive_prompt_operator_ids

function nipux_cli/operator_context.py:59

No docstring.

Calls: entry.get, ids.append, isinstance, operator_entry_is_active, operator_entry_is_prompt_relevant, str

### _conversation_only

function nipux_cli/operator_context.py:74

No docstring.

Calls: any, join, message.split, pattern.search

### _actionable

function nipux_cli/operator_context.py:79

No docstring.

Calls: _conversation_only, any, join, message.split, pattern.search

### _handler

function nipux_cli/parser_builder.py:13

No docstring.

Calls: none

### build_arg_parser

function nipux_cli/parser_builder.py:17

No docstring.

Calls: _handler, activity.add_argument, activity.set_defaults, argparse.ArgumentParser, artifact.add_argument, artifact.set_defaults, artifacts.add_argument, artifacts.set_defaults, autostart.add_argument, autostart.set_defaults, browser_dashboard.add_argument, browser_dashboard.set_...

### objective_profiles

function nipux_cli/planning.py:99

Infer generic work profiles from an objective without binding to a domain.

Calls: _PROFILE_PRIORITY.get, _PROFILE_TERMS.items, len, objective.lower, re.findall, scores.append, scores.sort, set

### initial_plan_for_objective

function nipux_cli/planning.py:115

No docstring.

Calls: _initial_questions_for_profiles, _initial_summary_for_profiles, _initial_tasks_for_profiles, join, objective.split, objective_profiles

### initial_task_contract

function nipux_cli/planning.py:131

No docstring.

Calls: any, task_title.lower

### initial_roadmap_for_objective

function nipux_cli/planning.py:222

No docstring.

Calls: _primary_execution_contract, join, objective_profiles

### _initial_summary_for_profiles

function nipux_cli/planning.py:277

No docstring.

Calls: none

### _initial_tasks_for_profiles

function nipux_cli/planning.py:292

No docstring.

Calls: tasks.extend

### _initial_questions_for_profiles

function nipux_cli/planning.py:339

No docstring.

Calls: questions.append, questions.insert

### _primary_execution_contract

function nipux_cli/planning.py:359

No docstring.

Calls: none

### format_initial_plan

function nipux_cli/planning.py:373

No docstring.

Calls: isinstance, join, lines.append, lines.extend, plan.get, str

### ProgressCheckpoint

class nipux_cli/progress.py:10

No docstring.

Calls: dataclass

### build_progress_checkpoint

function nipux_cli/progress.py:23

Create the operator-facing checkpoint text from durable ledger deltas.

Calls: ProgressCheckpoint, _as_int, _count_phrase, bool, changed_parts.extend, deltas.items, join, ledger_counts, ledger_resolution_counts, ledger_update_counts, metadata.get, previous.get, recent_progress_bits, resolutions.items, str, updates.items

### ledger_counts

function nipux_cli/progress.py:76

No docstring.

Calls: _metadata_list, isinstance, len, metadata.get, roadmap.get

### ledger_update_counts

function nipux_cli/progress.py:89

Count durable ledger updates that do not increase ledger size.

Calls: _as_int, _record_after_checkpoint, _updated_existing_record, isinstance, metadata.get, record_map.items, roadmap.get

### ledger_resolution_counts

function nipux_cli/progress.py:116

Count durable branch resolutions so task updates do not look like empty churn.

Calls: _record_after_checkpoint, _updated_existing_record, experiment.get, isinstance, lower, metadata.get, str, task.get

### recent_progress_bits

function nipux_cli/progress.py:135

No docstring.

Calls: _as_int, _clip_text, _metadata_list, bits.append, entry.get, experiment.get, finding.get, get, isinstance, join, lower, metadata.get, milestone.get, roadmap.get, sorted, str

### _metadata_list

function nipux_cli/progress.py:171

No docstring.

Calls: isinstance, metadata.get

### _clip_text

function nipux_cli/progress.py:178

No docstring.

Calls: join, len, max, rstrip, value.split

### _updated_existing_record

function nipux_cli/progress.py:185

No docstring.

Calls: _record_after_checkpoint, isinstance, record.get

### _record_after_checkpoint

function nipux_cli/progress.py:194

No docstring.

Calls: bool, record.get, str

### _count_phrase

function nipux_cli/progress.py:201

No docstring.

Calls: bits.append, join, key.endswith

### _as_int

function nipux_cli/progress.py:209

No docstring.

Calls: int

### provider_error_text

function nipux_cli/provider_errors.py:43

No docstring.

Calls: error.lower, getattr, isinstance, join, json.dumps, lower, parts.append, str, type

### provider_action_required

function nipux_cli/provider_errors.py:53

No docstring.

Calls: any, provider_error_text

### provider_action_required_note

function nipux_cli/provider_errors.py:58

No docstring.

Calls: provider_action_required

### provider_rate_limited

function nipux_cli/provider_errors.py:62

No docstring.

Calls: any, provider_error_text

### RecordCommandDeps

class nipux_cli/record_commands.py:22

No docstring.

Calls: dataclass

### cmd_findings_impl

function nipux_cli/record_commands.py:28

No docstring.

Calls: _metadata_records, _one_line, _resolve_or_print, db.close, db.get_job, deps.db_factory, enumerate, finding.get, float, isinstance, join, json.dumps, len, print, rule, sorted

### cmd_tasks_impl

function nipux_cli/record_commands.py:68

No docstring.

Calls: _metadata_records, _one_line, _resolve_or_print, db.close, db.get_job, deps.db_factory, enumerate, int, join, json.dumps, len, lower, print, rule, sorted, status.strip

### cmd_roadmap_impl

function nipux_cli/record_commands.py:119

No docstring.

Calls: _one_line, _print_milestones, _resolve_or_print, db.close, db.get_job, deps.db_factory, isinstance, job.get, json.dumps, len, metadata.get, print, roadmap.get, rule

### cmd_experiments_impl

function nipux_cli/record_commands.py:151

No docstring.

Calls: _metadata_records, _one_line, _resolve_or_print, bool, db.close, db.get_job, deps.db_factory, enumerate, experiment.get, join, json.dumps, len, lower, print, rule, sorted

### cmd_sources_impl

function nipux_cli/record_commands.py:208

No docstring.

Calls: _metadata_records, _one_line, _resolve_or_print, db.close, db.get_job, deps.db_factory, enumerate, float, int, isinstance, join, json.dumps, len, print, rule, sorted

### cmd_memory_impl

function nipux_cli/record_commands.py:252

No docstring.

Calls: _metadata_records, _print_memory_sections, _resolve_or_print, _write_memory_graph_view, active_operator_messages, bool, db.close, db.get_job, db.list_memory, deps.db_factory, getattr, isinstance, job.get, json.dumps, len, memory_graph_from_job

### _write_memory_graph_view

function nipux_cli/record_commands.py:296

No docstring.

Calls: ArtifactStore, Path, db.add_artifact, expanduser, len, path.parent.mkdir, path.write_text, print, render_memory_graph_html, sha256_text, store.write_text, str

### cmd_metrics_impl

function nipux_cli/record_commands.py:340

No docstring.

Calls: _metadata_records, _print_best_records, _resolve_or_print, _step_count, artifact.get, bool, daemon_lock_status, db.close, db.get_job, db.list_artifacts, db.list_steps, deps.db_factory, isinstance, job.get, len, lower

### cmd_usage_impl

function nipux_cli/record_commands.py:381

No docstring.

Calls: _resolve_or_print, db.close, db.get_job, db.job_token_usage, deps.db_factory, format_usage_report, int, job.get, join, json.dumps, print, str

### _resolve_or_print

function nipux_cli/record_commands.py:407

No docstring.

Calls: deps.job_ref_text, deps.resolve_job_id, print

### _metadata_records

function nipux_cli/record_commands.py:416

No docstring.

Calls: isinstance, job.get, metadata.get

### _print_milestones

function nipux_cli/record_commands.py:424

No docstring.

Calls: _one_line, enumerate, feature.get, int, isinstance, join, len, max, milestone.get, min, print, sorted, status_order.get, str, sum

### _print_memory_sections

function nipux_cli/record_commands.py:467

No docstring.

Calls: _one_line, entry.get, graph.get, isinstance, join, lesson.get, min, node.get, pending_measurement.get, print, reflection.get, str

### _print_best_records

function nipux_cli/record_commands.py:519

No docstring.

Calls: _one_line, best.get, best_experiment.get, best_finding.get, experiment.get, finding.get, float, max, print, source.get

### _step_count

function nipux_cli/record_commands.py:540

No docstring.

Calls: int, max, step.get

### job_deferred_until

function nipux_cli/scheduling.py:9

No docstring.

Calls: astimezone, datetime.fromisoformat, datetime.now, isinstance, job.get, metadata.get, raw_until.replace, str, strip, until.astimezone, until.replace

### job_is_deferred

function nipux_cli/scheduling.py:25

No docstring.

Calls: job_deferred_until

### job_provider_blocked

function nipux_cli/scheduling.py:29

Return true when provider calls need operator action before retrying.

Calls: _metadata_time, isinstance, job.get, metadata.get, str, strip

### provider_retry_metadata

function nipux_cli/scheduling.py:46

Metadata patch used when the operator explicitly retries provider work.

Calls: datetime.now, isoformat

### operator_resume_metadata

function nipux_cli/scheduling.py:55

Metadata patch used when the operator explicitly makes a job runnable.

Calls: patch.update, provider_retry_metadata

### _metadata_time

function nipux_cli/scheduling.py:69

No docstring.

Calls: datetime.fromisoformat, parsed.astimezone, parsed.replace, value.replace

### launch_agent_path

function nipux_cli/service_install.py:16

No docstring.

Calls: Path.home

### launch_agent_plist

function nipux_cli/service_install.py:20

No docstring.

Calls: Path.cwd, command.append, config.ensure_dirs, join, load_config, str, xml_escape

### systemd_service_path

function nipux_cli/service_install.py:64

No docstring.

Calls: Path.home

### systemd_service_text

function nipux_cli/service_install.py:68

No docstring.

Calls: Path.cwd, command.append, config.ensure_dirs, join, load_config, shlex.quote, str

### cmd_autostart

function nipux_cli/service_install.py:102

No docstring.

Calls: SystemExit, launch_agent_path, launch_agent_plist, os.getuid, path.exists, path.parent.mkdir, path.unlink, path.write_text, print, str, subprocess.run

### cmd_service

function nipux_cli/service_install.py:139

No docstring.

Calls: SystemExit, path.exists, path.parent.mkdir, path.unlink, path.write_text, print, result.stderr.strip, result.stdout.strip, shutil.which, subprocess.run, systemd_service_path, systemd_service_text

### xml_escape

function nipux_cli/service_install.py:178

No docstring.

Calls: replace, value.replace

### config_field_value

function nipux_cli/settings.py:16

No docstring.

Calls: load_config, str, values.get

### save_config_field

function nipux_cli/settings.py:40

No docstring.

Calls: _coerce_config_value, _load_config_yaml, _save_config_yaml, data.setdefault, field.split, isinstance

### inline_setting_notice

function nipux_cli/settings.py:53

No docstring.

Calls: _save_env_secret, _short_path, clear_model_setup_verified, get_agent_home, load_config, raw_value.strip, save_config_field

### edit_target_label

function nipux_cli/settings.py:71

No docstring.

Calls: none

### edit_target_hint

function nipux_cli/settings.py:77

No docstring.

Calls: config_field_value, load_config

### edit_target_masks_input

function nipux_cli/settings.py:86

No docstring.

Calls: none

### _config_path

function nipux_cli/settings.py:90

No docstring.

Calls: get_agent_home

### _load_config_yaml

function nipux_cli/settings.py:94

No docstring.

Calls: _config_path, default_config_yaml, isinstance, path.exists, path.read_text, yaml.safe_load

### _save_config_yaml

function nipux_cli/settings.py:103

No docstring.

Calls: _config_path, write_private_text, yaml.safe_dump

### _save_env_secret

function nipux_cli/settings.py:108

No docstring.

Calls: current.strip, env_path.exists, env_path.parent.mkdir, env_path.read_text, existing.items, get_agent_home, join, key.strip, raw.split, raw.strip, splitlines, startswith, write_private_text

### _coerce_config_value

function nipux_cli/settings.py:124

No docstring.

Calls: Path, SETTINGS_FIELD_TYPES.get, ValueError, expanduser, float, int, raw_value.strip, str, value.lower

### _short_path

function nipux_cli/settings.py:145

No docstring.

Calls: Path.home, len, max, str, text.startswith

### handle_chat_setting_command

function nipux_cli/settings_commands.py:14

No docstring.

Calls: config_field_value, config_summary_lines, inline_setting_notice, join, load_config, print

### config_summary_lines

function nipux_cli/settings_commands.py:38

No docstring.

Calls: _cost_limit_text, _rate_text, load_config

### _rate_text

function nipux_cli/settings_commands.py:64

No docstring.

Calls: none

### _cost_limit_text

function nipux_cli/settings_commands.py:68

No docstring.

Calls: none

### capture_setting_command

function nipux_cli/settings_commands.py:72

No docstring.

Calls: StringIO, handle_chat_setting_command, item.split, item.strip, join, line.startswith, print, redirect_stdout, shlex.split, splitlines, stream.getvalue

### write_file

function nipux_cli/shell_tools.py:17

No docstring.

Calls: Path, Path.cwd, _json, args.get, bool, expanduser, fh.write, lower, path.exists, path.is_absolute, path.is_dir, path.open, path.parent.mkdir, path.stat, str, strip

### shell_exec

function nipux_cli/shell_tools.py:47

No docstring.

Calls: Path, _json, _kill_process_group, _register_shell_process, _shell_error, _terminate_process_group, _truncate_output, _unregister_shell_process, args.get, dict, exists, expanduser, float, int, isinstance, max

### cleanup_registered_shell_processes

function nipux_cli/shell_tools.py:126

No docstring.

Calls: _as_int, _pid_exists, _read_shell_process_registry, _shell_process_registry_path, _write_shell_process_registry, cleaned.append, contextlib.suppress, datetime.now, dict, isoformat, os.killpg, record.get, survivors.append, time.sleep

### _shell_error

function nipux_cli/shell_tools.py:157

No docstring.

Calls: _missing_executable_probe, _shell_success_anomaly, combined.lower, combined.split, join, part.strip

### _shell_success_anomaly

function nipux_cli/shell_tools.py:173

No docstring.

Calls: _empty_observation_probe, _missing_executable_probe, _shell_build_error_anomaly, _shell_http_error_anomaly, _shell_missing_command_anomaly, _shell_sudo_password_anomaly, any, combined.lower, combined.split, join, part.strip

### _missing_executable_probe

function nipux_cli/shell_tools.py:212

No docstring.

Calls: combined_output.lower, combined_output.strip, match.group, re.match, str, strip

### _empty_observation_probe

function nipux_cli/shell_tools.py:222

No docstring.

Calls: re.match, str, strip

### _shell_missing_command_anomaly

function nipux_cli/shell_tools.py:231

No docstring.

Calls: bool, re.search

### _shell_sudo_password_anomaly

function nipux_cli/shell_tools.py:240

No docstring.

Calls: none

### _shell_build_error_anomaly

function nipux_cli/shell_tools.py:244

No docstring.

Calls: bool, re.search, text.lower

### _shell_http_error_anomaly

function nipux_cli/shell_tools.py:253

No docstring.

Calls: any

### _terminate_process_group

function nipux_cli/shell_tools.py:259

No docstring.

Calls: os.killpg

### _kill_process_group

function nipux_cli/shell_tools.py:266

No docstring.

Calls: os.killpg

### _register_shell_process

function nipux_cli/shell_tools.py:273

No docstring.

Calls: _shell_process_registry_path, datetime.now, getattr, handle.write, isoformat, json.dumps, path.open, path.parent.mkdir

### _unregister_shell_process

function nipux_cli/shell_tools.py:291

No docstring.

Calls: _as_int, _read_shell_process_registry, _shell_process_registry_path, _write_shell_process_registry, record.get

### _shell_process_registry_path

function nipux_cli/shell_tools.py:297

No docstring.

Calls: Path, expanduser

### _read_shell_process_registry

function nipux_cli/shell_tools.py:301

No docstring.

Calls: isinstance, json.loads, path.exists, path.read_text, records.append, splitlines

### _write_shell_process_registry

function nipux_cli/shell_tools.py:315

No docstring.

Calls: contextlib.suppress, join, json.dumps, path.parent.mkdir, path.unlink, path.write_text

### _pid_exists

function nipux_cli/shell_tools.py:324

No docstring.

Calls: os.kill

### _as_int

function nipux_cli/shell_tools.py:332

No docstring.

Calls: int

### _truncate_output

function nipux_cli/shell_tools.py:339

No docstring.

Calls: isinstance, len, str, value.decode

### _json

function nipux_cli/shell_tools.py:347

No docstring.

Calls: json.dumps

### anti_bot_reason

function nipux_cli/source_quality.py:19

Return a short reason if text looks like an anti-bot interstitial.

Calls: join, lower

### task_key

function nipux_cli/task_match.py:30

No docstring.

Calls: lower, re.sub, strip

### find_semantic_task_match

function nipux_cli/task_match.py:34

No docstring.

Calls: _task_similarity, _task_tokens, isinstance, len, lower, replace, round, str, strip, task.get, task_key

### _task_tokens

function nipux_cli/task_match.py:87

No docstring.

Calls: len, lower, re.findall, str

### _task_similarity

function nipux_cli/task_match.py:96

No docstring.

Calls: len, max, min

### program_for_job

function nipux_cli/templates.py:6

No docstring.

Calls: _TEMPLATES.get, body, lower, strip

### _generic_template

function nipux_cli/templates.py:12

No docstring.

Calls: none

### _research_paper_template

function nipux_cli/templates.py:37

No docstring.

Calls: none

### ToolContext

class nipux_cli/tools.py:25

No docstring.

Calls: dataclass

### ToolSpec

class nipux_cli/tools.py:58

No docstring.

Calls: dataclass

### as_openai_tool

function nipux_cli/tools.py:64

No docstring.

Calls: none

### _missing_argument

function nipux_cli/tools.py:75

No docstring.

Calls: isinstance, re.fullmatch, stripped.lower, value.strip

### _placeholder_argument

function nipux_cli/tools.py:96

No docstring.

Calls: _missing_argument, bool, isinstance, re.search, strip, value.strip

### _schema_placeholder_arguments

function nipux_cli/tools.py:109

No docstring.

Calls: REFERENCE_LIKE_FIELD_PATTERN.search, _placeholder_argument, _schema_placeholder_arguments, enumerate, isinstance, placeholders.append, placeholders.extend, properties.items, schema.get, str, value.get

### _schema_missing_arguments

function nipux_cli/tools.py:129

No docstring.

Calls: _missing_argument, _schema_missing_arguments, enumerate, isinstance, missing.append, missing.extend, properties.items, schema.get, str, value.get

### _json

function nipux_cli/tools.py:161

No docstring.

Calls: json.dumps

### _write_artifact

function nipux_cli/tools.py:165

No docstring.

Calls: _json, args.get, ctx.artifacts.write_text, isinstance, str

### _read_artifact

function nipux_cli/tools.py:187

No docstring.

Calls: _json, _recent_artifact_refs, _resolve_artifact_ref, args.get, ctx.artifacts.read_text, resolved.get, str

### _recent_artifact_refs

function nipux_cli/tools.py:211

No docstring.

Calls: artifact.get, ctx.db.list_artifacts, enumerate, refs.append, str

### _resolve_artifact_ref

function nipux_cli/tools.py:224

No docstring.

Calls: artifact.get, artifact_ref.strip, ctx.db.list_artifacts, int, join, len, lower, ref.isdigit, ref.lower, str, strip

### _search_artifacts

function nipux_cli/tools.py:248

No docstring.

Calls: _json, args.get, ctx.artifacts.search_text, int, str

### _update_job_state

function nipux_cli/tools.py:254

No docstring.

Calls: _append_completion_audit_task, _json, args.get, ctx.db.append_agent_update, ctx.db.update_job_status, follow_up_task.get, lower, str, strip

### _defer_job

function nipux_cli/tools.py:297

No docstring.

Calls: _defer_until, _json, args.get, ctx.db.append_agent_update, ctx.db.get_job, ctx.db.update_job_status, job.get, str, strip, until.isoformat

### _defer_until

function nipux_cli/tools.py:332

No docstring.

Calls: args.get, datetime.fromisoformat, datetime.now, float, max, parsed.astimezone, parsed.replace, raw_until.replace, str, strip, timedelta

### _report_update

function nipux_cli/tools.py:350

No docstring.

Calls: _append_completion_audit_task, _json, _perpetual_checkpoint_message, args.get, ctx.db.append_agent_update, follow_up_task.get, isinstance, lower, metadata.get, str, strip

### _append_completion_audit_task

function nipux_cli/tools.py:371

No docstring.

Calls: ctx.db.append_task_record

### _perpetual_checkpoint_message

function nipux_cli/tools.py:409

Keep worker reports checkpoint-oriented without hiding the underlying audit trail.

Calls: join, leading_claim.search, leading_claim.sub, re.compile, split, str, strip, whole_job_claim.search, whole_job_claim.sub

### _record_lesson

function nipux_cli/tools.py:432

No docstring.

Calls: _json, _lesson_explains_measurement_obligation, _pending_measurement, _resolve_measurement_obligation, args.get, ctx.db.append_lesson, float, isinstance, lower, str, strip

### _lesson_explains_measurement_obligation

function nipux_cli/tools.py:463

No docstring.

Calls: any, isinstance, join, lower, metadata.values, parts.append, str

### _record_memory_graph

function nipux_cli/tools.py:512

No docstring.

Calls: _json, args.get, ctx.db.append_agent_update, ctx.db.append_memory_graph_records, isinstance, record.get

### _search_memory_graph

function nipux_cli/tools.py:536

No docstring.

Calls: _json, args.get, ctx.db.get_job, int, memory_graph_from_job, search_memory_graph, str

### _acknowledge_operator_context

function nipux_cli/tools.py:545

No docstring.

Calls: _acknowledgeable_operator_messages, _json, args.get, ctx.db.acknowledge_operator_messages, ctx.db.append_agent_update, ctx.db.get_job, entry.get, isinstance, lower, result.get, str, strip

### _acknowledgeable_operator_messages

function nipux_cli/tools.py:583

No docstring.

Calls: entry.get, isinstance, job.get, lower, metadata.get, pending.append, replace, str, strip

### _record_source

function nipux_cli/tools.py:605

No docstring.

Calls: _json, _source_has_assessment, args.get, ctx.db.append_source_record, float, int, isinstance, str, strip

### _source_has_assessment

function nipux_cli/tools.py:649

No docstring.

Calls: any, bool, metadata.values, outcome.strip, str, strip

### _record_findings

function nipux_cli/tools.py:669

No docstring.

Calls: _finding_has_evidence, _json, args.get, ctx.db.append_agent_update, ctx.db.append_finding_record, ctx.db.append_source_record, entry.get, finding.get, float, isinstance, len, min, rejected.append, round, source_records.append, source_yields.get

### _finding_has_evidence

function nipux_cli/tools.py:779

No docstring.

Calls: any, evidence_artifact.strip, metadata.get, reason.strip, source_url.strip, str, strip, url.strip

### _record_tasks

function nipux_cli/tools.py:802

No docstring.

Calls: _complete_task_contract, _json, _pending_measurement, _resolve_measurement_obligation, _semantic_task_match_under_pressure, _task_queue_pressure_active, _task_targets_measurement_obligation, _task_would_be_unchanged, _validated_task_status, args.get, ctx.db.append_agent_update...

### _task_targets_measurement_obligation

function nipux_cli/tools.py:977

No docstring.

Calls: _metadata_scalar_text, _text_mentions_measurement, _text_mentions_measurement_accounting, join, lower, output_contract.strip

### _task_would_be_unchanged

function nipux_cli/tools.py:1006

No docstring.

Calls: _task_change_fingerprint, acceptance_criteria.strip, after.get, ctx.db.get_job, dict, entry.get, evidence_needed.strip, goal.strip, int, isinstance, items, job.get, job_metadata.get, lower, merged_metadata.update, next

### _task_change_fingerprint

function nipux_cli/tools.py:1081

No docstring.

Calls: entry.get, json.dumps

### _task_queue_pressure_active

function nipux_cli/tools.py:1085

No docstring.

Calls: _task_is_guard_recovery, ctx.db.get_job, isinstance, job.get, len, lower, metadata.get, replace, str, strip, task.get

### _semantic_task_match_under_pressure

function nipux_cli/tools.py:1101

No docstring.

Calls: _task_is_guard_recovery, ctx.db.get_job, find_semantic_task_match, isinstance, job.get, metadata.get

### _task_is_guard_recovery

function nipux_cli/tools.py:1115

No docstring.

Calls: bool, isinstance, lower, metadata.get, startswith, str, strip, task.get

### _text_mentions_measurement

function nipux_cli/tools.py:1120

No docstring.

Calls: any

### _text_mentions_measurement_accounting

function nipux_cli/tools.py:1135

No docstring.

Calls: any

### _metadata_scalar_text

function nipux_cli/tools.py:1173

No docstring.

Calls: isinstance, join, metadata.values, parts.append, str

### _complete_task_contract

function nipux_cli/tools.py:1181

No docstring.

Calls: acceptance_criteria.strip, dict, evidence_needed.strip, inferred.append, initial_task_contract, isinstance, metadata.get, output_contract.strip, set, sorted, stall_behavior.strip, str

### _validated_task_status

function nipux_cli/tools.py:1212

No docstring.

Calls: _recent_deliverable_evidence, _task_contract_completion_has_evidence, _task_metadata_has_completion_evidence, dict, lower, output_contract.strip, replace, result.strip, status.strip

### _task_contract_completion_has_evidence

function nipux_cli/tools.py:1245

No docstring.

Calls: _shell_command_counts_as_action_evidence, _task_metadata_has_completion_evidence, args.get, ctx.db.list_steps, input_data.get, isinstance, recent_evidence_tools.get, reversed, step.get, str

### _task_metadata_has_completion_evidence

function nipux_cli/tools.py:1275

No docstring.

Calls: any, evidence_keys.update, metadata.get, str, strip

### _recent_deliverable_evidence

function nipux_cli/tools.py:1295

No docstring.

Calls: _artifact_args_look_like_deliverable, _shell_command_looks_like_write, args.get, ctx.db.list_steps, input_data.get, isinstance, reversed, step.get, str

### _artifact_args_look_like_deliverable

function nipux_cli/tools.py:1313

No docstring.

Calls: any, args.get, join, lower, str

### _shell_command_looks_like_write

function nipux_cli/tools.py:1322

No docstring.

Calls: any, command.strip, re.search

### _shell_command_counts_as_action_evidence

function nipux_cli/tools.py:1338

No docstring.

Calls: _shell_command_looks_like_write, bool, command.strip, re.compile, re.match, re.search, read_only.search

### _record_roadmap

function nipux_cli/tools.py:1361

No docstring.

Calls: _json, args.get, ctx.db.append_agent_update, ctx.db.append_roadmap_record, isinstance, len, roadmap.get, str, strip

### _record_milestone_validation

function nipux_cli/tools.py:1395

No docstring.

Calls: _json, _validation_has_positive_evidence, args.get, ctx.db.append_agent_update, ctx.db.append_milestone_validation_record, ctx.db.append_task_record, follow_up_tasks.append, get, int, isinstance, len, lower, replace, str, strip, task.get

### _validation_has_positive_evidence

function nipux_cli/tools.py:1490

No docstring.

Calls: any, evidence.strip, metadata.get, result.strip, str, strip

### _record_experiment

function nipux_cli/tools.py:1506

No docstring.

Calls: _experiment_has_closed_trial_context, _json, _optional_float, _resolve_measurement_obligation, args.get, bool, ctx.db.append_agent_update, ctx.db.append_experiment_record, format_metric_value, isinstance, lower, record.get, str, strip

### _experiment_has_closed_trial_context

function nipux_cli/tools.py:1595

No docstring.

Calls: any, args.get, config.values, isinstance, metadata.values, str, strip

### _optional_float

function nipux_cli/tools.py:1609

No docstring.

Calls: float, isinstance, replace, value.strip

### _pending_measurement

function nipux_cli/tools.py:1625

No docstring.

Calls: ctx.db.get_job, isinstance, job.get, metadata.get, obligation.get

### _resolve_measurement_obligation

function nipux_cli/tools.py:1637

No docstring.

Calls: _pending_measurement, ctx.db.append_agent_update, ctx.db.update_job_metadata, dict, resolved.update, time.gmtime, time.strftime

### _send_digest_email

function nipux_cli/tools.py:1672

No docstring.

Calls: _json, args.get, ctx.artifacts.write_text, send_digest_email, str

### _browser_call

function nipux_cli/tools.py:1691

No docstring.

Calls: KeyError, _json, args.get, bool, browser.back, browser.click, browser.console, browser.fill, browser.navigate, browser.press, browser.scroll, browser.snapshot, str

### _web_call

function nipux_cli/tools.py:1714

No docstring.

Calls: KeyError, _json, args.get, int, isinstance, str, web_extract, web_search

### _browser_handler

function nipux_cli/tools.py:1726

No docstring.

Calls: _browser_call

### _web_handler

function nipux_cli/tools.py:1730

No docstring.

Calls: _web_call

### ToolRegistry

class nipux_cli/tools.py:2151

No docstring.

Calls: REQUIRED_ARGUMENT_ALIASES.get, REQUIRED_ARGUMENT_GROUPS.get, _json, _missing_argument, _schema_missing_arguments, _schema_placeholder_arguments, _tool_access_group, _tool_enabled, aliases.get, all, args.get, as_openai_tool, dict, handler, isinstance, join

### __init__

function nipux_cli/tools.py:2152

No docstring.

Calls: none

### names

function nipux_cli/tools.py:2155

No docstring.

Calls: sorted

### openai_tools

function nipux_cli/tools.py:2158

No docstring.

Calls: _tool_enabled, as_openai_tool, self.names

### validate_arguments

function nipux_cli/tools.py:2161

No docstring.

Calls: REQUIRED_ARGUMENT_ALIASES.get, REQUIRED_ARGUMENT_GROUPS.get, _missing_argument, _schema_missing_arguments, _schema_placeholder_arguments, _tool_enabled, aliases.get, all, args.get, dict, isinstance, join, missing.append, missing.extend, spec.parameters.get, str

### handle

function nipux_cli/tools.py:2195

No docstring.

Calls: _json, _tool_access_group, _tool_enabled, handler

### _tool_enabled

function nipux_cli/tools.py:2208

No docstring.

Calls: _tool_access_group, bool, getattr

### _tool_access_group

function nipux_cli/tools.py:2217

No docstring.

Calls: name.startswith

### slash_suggestion_lines

function nipux_cli/tui_commands.py:163

No docstring.

Calls: SLASH_ARGUMENT_HINTS.get, _accent, _bold, _fit_ansi, _muted, _slash_argument_footer, enumerate, input_buffer.rstrip, input_buffer.startswith, len, lines.append, lower, max, min, next, split

### autocomplete_slash

function nipux_cli/tui_commands.py:227

No docstring.

Calls: _slash_command_matches, input_buffer.startswith

### slash_completion_for_submit

function nipux_cli/tui_commands.py:236

Return the buffer to use and whether Enter should submit it now.

Calls: _slash_argument_text, _slash_command_matches, input_buffer.rstrip, input_buffer.startswith, lower, split, strip

### _slash_argument_text

function nipux_cli/tui_commands.py:265

No docstring.

Calls: len, split

### _slash_argument_footer

function nipux_cli/tui_commands.py:270

No docstring.

Calls: len

### cycle_slash

function nipux_cli/tui_commands.py:276

No docstring.

Calls: _slash_command_matches, input_buffer.rstrip, input_buffer.startswith, len, matches.index

### _slash_command_matches

function nipux_cli/tui_commands.py:291

No docstring.

Calls: input_buffer.strip, lower, startswith

### event_tool_args

function nipux_cli/tui_event_format.py:15

No docstring.

Calls: input_data.get, isinstance, metadata.get

### shell_write_target

function nipux_cli/tui_event_format.py:21

No docstring.

Calls: candidate.startswith, command.split, command.strip, enumerate, re.search, redirect.group, shlex.split, strip, target.startswith

### event_title_body

function nipux_cli/tui_event_format.py:43

No docstring.

Calls: none

### experiment_metric_text

function nipux_cli/tui_event_format.py:49

No docstring.

Calls: format_metric_value, join, metadata.get, str

### event_clock

function nipux_cli/tui_event_format.py:59

No docstring.

Calls: _compact_time, _one_line, event.get, len, str

### event_hour

function nipux_cli/tui_event_format.py:66

No docstring.

Calls: _compact_time, event.get, len, str

### friendly_error_text

function nipux_cli/tui_event_format.py:75

No docstring.

Calls: _one_line, clean_step_summary, text.lower

### brief_reflection_text

function nipux_cli/tui_event_format.py:95

No docstring.

Calls: _one_line, clean_step_summary, counts.replace, match.group, re.search

### generic_display_text

function nipux_cli/tui_event_format.py:106

No docstring.

Calls: join, split, str

### clean_step_summary

function nipux_cli/tui_event_format.py:110

No docstring.

Calls: join, split, str, text.split, text.startswith

### chat_message_paragraphs

function nipux_cli/tui_event_format.py:117

No docstring.

Calls: join, paragraphs.append, raw.strip, re.sub, replace, split, str, text.splitlines

### chat_agent_message_text

function nipux_cli/tui_event_format.py:130

No docstring.

Calls: _one_line, body.split, clean_step_summary, len, re.findall, title.lower

### tool_live_summary

function nipux_cli/tui_event_format.py:145

No docstring.

Calls: _regex_group, _short_command, _short_url, args.get, clean_step_summary, event_tool_args, get, isinstance, len, metadata.get, output.get, short_path, str

### short_path

function nipux_cli/tui_event_format.py:213

No docstring.

Calls: Path.home, len, max, str, text.startswith

### _regex_group

function nipux_cli/tui_event_format.py:224

No docstring.

Calls: match.group, re.search

### _short_url

function nipux_cli/tui_event_format.py:229

No docstring.

Calls: replace, stripped.split, url.replace

### _short_command

function nipux_cli/tui_event_format.py:236

No docstring.

Calls: command.split, join, len, next, part.startswith, parts.index, remote.split, shlex.split, strip

### _compact_time

function nipux_cli/tui_event_format.py:258

No docstring.

Calls: _one_line, len, value.replace

### chat_event_parts

function nipux_cli/tui_events.py:74

No docstring.

Calls: _is_low_value_chat_notice, _is_waiting_notice, _normalized_chat_body, event.get, event_clock, friendly_error_text, isinstance, metadata.get, str, strip

### append_chat_output

function nipux_cli/tui_events.py:91

No docstring.

Calls: _chat_label, _fit_ansi, _muted, chat_message_paragraphs, len, lines.append, max, textwrap.wrap

### chat_pane_lines

function nipux_cli/tui_events.py:105

No docstring.

Calls: _chat_item_lines, _fit_ansi, _flatten_chat_blocks, _is_generic_chat_notice, _is_low_value_chat_notice, _is_waiting_notice, _muted, _normalized_chat_body, chat_empty_state_lines, chat_event_parts, items.append, len, max, min, notice.removeprefix, notice.startswith

### _chat_item_lines

function nipux_cli/tui_events.py:172

No docstring.

Calls: _accent, _fit_ansi, _muted, _style, append_chat_output, str

### _flatten_chat_blocks

function nipux_cli/tui_events.py:182

No docstring.

Calls: rows.append, rows.extend

### _chat_label

function nipux_cli/tui_events.py:191

No docstring.

Calls: _muted, _style, label.lower

### _normalized_chat_body

function nipux_cli/tui_events.py:203

No docstring.

Calls: join, split, str

### _is_low_value_chat_notice

function nipux_cli/tui_events.py:207

No docstring.

Calls: any, normalized.lower

### _is_waiting_notice

function nipux_cli/tui_events.py:212

No docstring.

Calls: lowered.startswith, normalized.lower

### _is_generic_chat_notice

function nipux_cli/tui_events.py:221

No docstring.

Calls: any, lowered.startswith, normalized.lower

### chat_empty_state_lines

function nipux_cli/tui_events.py:226

No docstring.

Calls: _accent, _bold, _center_ansi, _centered_wrapped_hint, len, max

### _centered_wrapped_hint

function nipux_cli/tui_events.py:238

No docstring.

Calls: _center_ansi, _muted, max, min, textwrap.wrap

### worker_activity_lines

function nipux_cli/tui_events.py:243

No docstring.

Calls: _one_line, get, int, item.get, items.append, live_badge, live_display_text, max, minimal_live_event_line, rendered.append, str

### minimal_live_event_line

function nipux_cli/tui_events.py:261

No docstring.

Calls: _one_line, brief_reflection_text, event.get, event_title_body, friendly_error_text, generic_display_text, isinstance, metadata.get, str, strip, tool_live_summary

### live_badge

function nipux_cli/tui_events.py:317

No docstring.

Calls: _style, badge_text.startswith, re.sub

### live_display_text

function nipux_cli/tui_events.py:346

No docstring.

Calls: len, text.startswith

### read_terminal_char

function nipux_cli/tui_input.py:12

No docstring.

Calls: data.decode, os.read

### read_escape_sequence

function nipux_cli/tui_input.py:17

No docstring.

Calls: len, max, min, read_terminal_char, select.select, sys.stdin.fileno, terminal_escape_complete, time.monotonic

### terminal_escape_complete

function nipux_cli/tui_input.py:34

No docstring.

Calls: len, re.match, sequence.startswith

### decode_terminal_escape

function nipux_cli/tui_input.py:46

No docstring.

Calls: csi_arrow.group, int, len, match.group, ord, re.match, sequence.startswith

### drain_pending_input

function nipux_cli/tui_input.py:74

No docstring.

Calls: os.read, select.select, sys.stdin.fileno

### _top_bar

function nipux_cli/tui_layout.py:18

No docstring.

Calls: _edge_line, _muted, _one_line, _style, _token_usage_topline, max

### _two_col_title

function nipux_cli/tui_layout.py:47

No docstring.

Calls: _fit_ansi, _muted, _style, left.upper, right.upper

### _two_col_line

function nipux_cli/tui_layout.py:53

No docstring.

Calls: _fit_ansi, _muted

### _edge_line

function nipux_cli/tui_layout.py:57

No docstring.

Calls: _fit_ansi, _strip_ansi, len, max

### _triple_line

function nipux_cli/tui_layout.py:65

No docstring.

Calls: _edge_line, _fit_ansi, _strip_ansi, join, len, max

### _compose_bar

function nipux_cli/tui_layout.py:83

No docstring.

Calls: _accent, _fit_ansi, _muted, len, lines.extend, max, min, title.strip

### _metric_strip

function nipux_cli/tui_layout.py:112

No docstring.

Calls: _bold, _muted, _one_line, _strip_ansi, join, len

### _pill

function nipux_cli/tui_layout.py:121

No docstring.

Calls: _muted, _style, any, str, value_text.lower

### _token_usage_topline

function nipux_cli/tui_layout.py:134

No docstring.

Calls: _format_compact_count, _format_usage_cost, _muted, _safe_int, _style, usage.get

### _model_cost_is_zero

function nipux_cli/tui_layout.py:164

No docstring.

Calls: base_url.lower, lowered_model.endswith, model.lower

### _format_usage_cost

function nipux_cli/tui_layout.py:175

No docstring.

Calls: _model_cost_is_zero, _safe_float, _safe_int, _safe_optional_float, bool, usage.get

### _format_compact_count

function nipux_cli/tui_layout.py:193

No docstring.

Calls: _safe_int, str

### _safe_int

function nipux_cli/tui_layout.py:204

No docstring.

Calls: float, int

### _safe_float

function nipux_cli/tui_layout.py:211

No docstring.

Calls: float

### _safe_optional_float

function nipux_cli/tui_layout.py:218

No docstring.

Calls: float

### _status_dot

function nipux_cli/tui_layout.py:227

No docstring.

Calls: _style

### model_update_event_parts

function nipux_cli/tui_outcomes.py:89

No docstring.

Calls: _durable_progress_event_parts, _outcome_text, brief_reflection_text, chat_agent_message_text, event.get, event_clock, event_title_body, event_tool_args, experiment_metric_text, generic_display_text, get, isinstance, max, metadata.get, output.get, shell_write_target

### is_summary_event_candidate

function nipux_cli/tui_outcomes.py:152

No docstring.

Calls: bool, event.get, event_tool_args, get, isinstance, metadata.get, shell_write_target, str

### latest_durable_outcome_line

function nipux_cli/tui_outcomes.py:170

No docstring.

Calls: _event_badge, _fit_ansi, _muted, _one_line, _strip_ansi, len, max, model_update_event_parts, reversed

### latest_hour_outcome_summary_line

function nipux_cli/tui_outcomes.py:191

Return a single compact count summary for the newest visible activity hour.

Calls: _bold, _fit_ansi, _muted, _one_line, _strip_ansi, event_hour, get, hourly_outcome_summary, int, len, max, model_update_event_parts, order.append

### visible_outcome_summary_line

function nipux_cli/tui_outcomes.py:217

Return a stable summary of the durable outcomes available to the pane.

Calls: _bold, _fit_ansi, _muted, _one_line, _strip_ansi, hourly_outcome_summary, len, max, outcome_counts

### job_outcome_summary

function nipux_cli/tui_outcomes.py:228

Return a short per-job durable outcome mix for compact job cards.

Calls: _one_line, hourly_outcome_summary, outcome_counts

### outcome_counts

function nipux_cli/tui_outcomes.py:238

No docstring.

Calls: counts.get, int, model_update_event_parts

### recent_model_update_lines

function nipux_cli/tui_outcomes.py:260

Render recent durable worker outcomes for the compact status pane.

Calls: _event_badge, _fit_ansi, _muted, _one_line, _strip_ansi, get, int, item.get, items.append, len, lines.append, max, model_update_event_parts, reversed, str, textwrap.wrap

### chat_updates_pane_lines

function nipux_cli/tui_outcomes.py:320

No docstring.

Calls: _bold, _fit_ansi, _muted, _one_line, _page_indicator, _wrapped_label_line, hourly_outcome_summary, job.get, len, lines.extend, max, outcome_counts, recent_model_update_lines

### _wrapped_label_line

function nipux_cli/tui_outcomes.py:343

No docstring.

Calls: _bold, _fit_ansi, _muted, _strip_ansi, len, lines.append, max, textwrap.wrap

### hourly_update_lines

function nipux_cli/tui_outcomes.py:355

No docstring.

Calls: _bold, _event_badge, _fit_ansi, _muted, _strip_ansi, append, counts.get, event_hour, hourly_outcome_summary, int, len, max, min, model_update_event_parts, order.append, rendered.append

### hourly_outcome_summary

function nipux_cli/tui_outcomes.py:406

No docstring.

Calls: OUTCOME_SUMMARY_NAMES.get, counts.get, int, join, label.lower, ordered.extend, pieces.append, set, sorted

### _durable_progress_event_parts

function nipux_cli/tui_outcomes.py:419

No docstring.

Calls: _count_map, _dominant_progress_key, _outcome_text, _progress_count_phrase, _progress_label_for_key, any, bool, deltas.get, generic_display_text, int, join, metadata.get, pieces.append, resolutions.get, set, totals.values

### _count_map

function nipux_cli/tui_outcomes.py:454

No docstring.

Calls: int, isinstance, str, value.items

### _dominant_progress_key

function nipux_cli/tui_outcomes.py:468

No docstring.

Calls: resolutions.get, totals.get

### _progress_label_for_key

function nipux_cli/tui_outcomes.py:478

No docstring.

Calls: none

### _progress_count_phrase

function nipux_cli/tui_outcomes.py:494

No docstring.

Calls: OUTCOME_SUMMARY_NAMES.get, _progress_label_for_key, join, label.endswith, parts.append

### _outcome_text

function nipux_cli/tui_outcomes.py:504

No docstring.

Calls: _one_line, generic_display_text

### worker_label

function nipux_cli/tui_status.py:36

No docstring.

Calls: job.get, job_deferred_until, job_provider_blocked, str

### job_display_state

function nipux_cli/tui_status.py:49

No docstring.

Calls: job.get, job_deferred_until, job_provider_blocked, str

### active_operator_messages

function nipux_cli/tui_status.py:60

No docstring.

Calls: active_prompt_operator_entries, entry.get, isinstance, metadata.get, str

### right_pane_lines

function nipux_cli/tui_status.py:71

No docstring.

Calls: _bold, _chat_workspace_lines, _defer_status_line, _empty_workspace_status_lines, _event_badge, _fit_ansi, _is_workspace_placeholder, _metric_strip, _metrics_grid_lines, _muted, _one_line, _yield_line, active_operator_messages, artifact.get, frame_jobs_lines, get

### _is_workspace_placeholder

function nipux_cli/tui_status.py:175

No docstring.

Calls: job.get, str

### _empty_workspace_status_lines

function nipux_cli/tui_status.py:179

No docstring.

Calls: _bold, _fit_ansi, _muted, _page_indicator

### chat_work_pane_lines

function nipux_cli/tui_status.py:192

No docstring.

Calls: _bold, _event_badge, _fit_ansi, _muted, _one_line, _page_indicator, _rank_visible_tasks, _status_badge, experiment.get, experiment_metric_text, job.get, len, lines.append, lines.extend, max, min

### chat_settings_pane_lines

function nipux_cli/tui_status.py:239

No docstring.

Calls: _bold, _fit_ansi, _muted, _page_indicator, _rate_text, _setting_line, str

### frame_jobs_lines

function nipux_cli/tui_status.py:277

No docstring.

Calls: _accent, _fit_ansi, _job_compact_work_lines, _muted, _one_line, _status_badge, enumerate, get, item.get, job_display_state, max, min, rendered.append, rendered.extend, str, worker_label

### _job_compact_work_lines

function nipux_cli/tui_status.py:315

No docstring.

Calls: _bold, _fit_ansi, _job_recent_non_output_pieces, _muted, _one_line, bool, counts.get, int, job_outcome_summary, latest.get, len, lines.append, max, second.get, str

### _job_recent_non_output_pieces

function nipux_cli/tui_status.py:348

No docstring.

Calls: _compact_outcome_label, _muted, _one_line, len, max, model_update_event_parts, pieces.append, reversed, seen.add, set

### _compact_outcome_label

function nipux_cli/tui_status.py:378

No docstring.

Calls: get, label.lower

### _defer_status_line

function nipux_cli/tui_status.py:395

No docstring.

Calls: _fit_ansi, _muted, _one_line, isinstance, job.get, job_deferred_until, max, metadata.get, str, strftime, strip, until.astimezone

### _rank_visible_tasks

function nipux_cli/tui_status.py:408

No docstring.

Calls: int, isinstance, sorted, status_order.get, str, task.get

### _chat_workspace_lines

function nipux_cli/tui_status.py:420

No docstring.

Calls: _bold, _context_pressure_line, _current_task_line, _muted, _one_line, _page_indicator, _status_badge, goal_lines.append, job.get, len, lines.append, max, str, textwrap.wrap

### _current_task_line

function nipux_cli/tui_status.py:451

No docstring.

Calls: _fit_ansi, _muted, _one_line, _rank_visible_tasks, _status_badge, isinstance, job.get, max, metadata.get, str, task.get

### _context_pressure_line

function nipux_cli/tui_status.py:469

No docstring.

Calls: _fit_ansi, _format_compact_count, _muted, _one_line, _safe_int, max, usage.get

### _metrics_grid_lines

function nipux_cli/tui_status.py:485

No docstring.

Calls: _fit_ansi, _metric_cell, _metric_strip, len, lines.append, max, range

### _metric_cell

function nipux_cli/tui_status.py:500

No docstring.

Calls: _bold, _fit_ansi, _muted

### _yield_line

function nipux_cli/tui_status.py:505

No docstring.

Calls: _fit_ansi, _muted, _safe_int, _status_badge, lookup.get

### _setting_line

function nipux_cli/tui_status.py:525

No docstring.

Calls: _bold, _fit_ansi, _muted, _one_line, max

### _rate_text

function nipux_cli/tui_status.py:532

No docstring.

Calls: none

### _safe_int

function nipux_cli/tui_status.py:536

No docstring.

Calls: float, int

### _fancy_ui

function nipux_cli/tui_style.py:11

No docstring.

Calls: os.environ.get, sys.stdout.isatty

### _style

function nipux_cli/tui_style.py:20

No docstring.

Calls: _fancy_ui, str

### _accent

function nipux_cli/tui_style.py:27

No docstring.

Calls: _style

### _muted

function nipux_cli/tui_style.py:31

No docstring.

Calls: _style

### _bold

function nipux_cli/tui_style.py:35

No docstring.

Calls: _style

### _one_line

function nipux_cli/tui_style.py:39

No docstring.

Calls: join, len, max, split, str

### _strip_ansi

function nipux_cli/tui_style.py:46

No docstring.

Calls: re.sub

### _fit_ansi

function nipux_cli/tui_style.py:50

No docstring.

Calls: _one_line, _strip_ansi, int, len, max, str

### _center_ansi

function nipux_cli/tui_style.py:60

No docstring.

Calls: _fit_ansi, _strip_ansi, len, max

### _themed_lines

function nipux_cli/tui_style.py:68

No docstring.

Calls: _fancy_ui, _fit_ansi, replace

### _frame_enter_sequence

function nipux_cli/tui_style.py:76

No docstring.

Calls: _fancy_ui

### _frame_exit_sequence

function nipux_cli/tui_style.py:81

No docstring.

Calls: none

### _page_indicator

function nipux_cli/tui_style.py:85

No docstring.

Calls: _accent, _bold, _muted, join, parts.append

### _status_badge

function nipux_cli/tui_style.py:95

No docstring.

Calls: _style, get, str

### _event_badge

function nipux_cli/tui_style.py:122

No docstring.

Calls: _style, get

### UninstallPlan

class nipux_cli/uninstall.py:21

No docstring.

Calls: dataclass

### build_uninstall_plan

function nipux_cli/uninstall.py:26

Return all local runtime paths that a full uninstall should remove.

Calls: Path.home, UninstallPlan, _dedupe_paths, get_agent_home, homes.append, launch_agent_path, runtime_home.expanduser, systemd_service_path, tuple

### uninstall_runtime

function nipux_cli/uninstall.py:37

Remove local Nipux state, logs, service files, and legacy state dirs.

Calls: _assert_safe_delete_target, _disable_services, build_uninstall_plan, lines.append, lines.extend, path.expanduser, shutil.rmtree, target.exists, target.is_dir, target.is_symlink, target.unlink

### uninstall_installed_tool

function nipux_cli/uninstall.py:66

Remove the installed `nipux` command from common uv-tool locations.

Calls: _process_lines, installed_tool_paths, lines.append, lines.extend, path.exists, path.is_dir, path.is_symlink, path.unlink, run, shutil.rmtree, shutil.which

### installed_tool_paths

function nipux_cli/uninstall.py:114

Return safe user-level paths for uv-tool Nipux installs.

Calls: Path, Path.home, _dedupe_paths, _is_safe_installed_tool_path, candidates.append, expanduser, path.expanduser, resolve, safe.append, shutil.which, tuple

### _disable_services

function nipux_cli/uninstall.py:133

No docstring.

Calls: launch_agent_path, launch_path.exists, lines.append, os.getuid, runner, service_path.exists, shutil.which, str, systemd_service_path

### _run_command

function nipux_cli/uninstall.py:168

No docstring.

Calls: list, subprocess.run

### _process_lines

function nipux_cli/uninstall.py:178

No docstring.

Calls: isinstance, line.rstrip, line.strip, splitlines

### _dedupe_paths

function nipux_cli/uninstall.py:184

No docstring.

Calls: path.expanduser, result.append, seen.add, set, str

### _assert_safe_delete_target

function nipux_cli/uninstall.py:196

No docstring.

Calls: Path, Path.home, ValueError, len, path.expanduser, resolve

### _is_safe_installed_tool_path

function nipux_cli/uninstall.py:206

No docstring.

Calls: expanded.resolve, path.expanduser

### find_checkout_root

function nipux_cli/updater.py:19

Return the nearest enclosing git checkout for the Nipux install.

Calls: Path, current.is_file, exists, expanduser, resolve

### update_checkout

function nipux_cli/updater.py:31

Update the current Nipux install and return output lines. Source checkouts are fast-forwarded with git. Installed tools are refreshed from the configured source repository so `n...

Calls: Path, _git_text, _process_lines, _short_path, _update_uv_tool_install, dirty.stdout.strip, exists, expanduser, find_checkout_root, lines.append, lines.extend, prefix.append, resolve, run, top_level.stdout.strip

### _update_uv_tool_install

function nipux_cli/updater.py:83

No docstring.

Calls: _process_lines, _uv_tool_update_spec, _verify_updated_command, lines.append, lines.extend, run, shutil.which

### _verify_updated_command

function nipux_cli/updater.py:114

No docstring.

Calls: _process_lines, join, runner, shutil.which, strip

### _uv_tool_update_spec

function nipux_cli/updater.py:125

Return the direct source uv should use for installed-tool updates.

Calls: os.environ.get, repo.startswith, strip

### _run_git

function nipux_cli/updater.py:139

No docstring.

Calls: list, subprocess.run

### _run_command

function nipux_cli/updater.py:150

No docstring.

Calls: list, subprocess.run

### _process_lines

function nipux_cli/updater.py:160

No docstring.

Calls: isinstance, line.rstrip, line.strip, output.splitlines

### _git_text

function nipux_cli/updater.py:165

No docstring.

Calls: isinstance, process.stdout.strip

### _short_path

function nipux_cli/updater.py:172

No docstring.

Calls: Path.home, len, max, str, text.startswith

### render_updates_report

function nipux_cli/updates.py:16

No docstring.

Calls: _metadata_list, _one_line, artifact.get, bool, daemon_lock_status, db.get_job, db.list_artifacts, db.list_timeline_events, hourly_update_lines, isinstance, job.get, job_display_state, latest.get, lesson.get, lines.append, lines.extend

### render_all_updates_report

function nipux_cli/updates.py:70

No docstring.

Calls: _metadata_count, _one_line, artifact.get, bool, counts.get, daemon_lock_status, db.job_record_counts, db.list_artifacts, db.list_events, db.list_jobs, job_display_state, join, len, lines.append, lines.extend, max

### _metadata_list

function nipux_cli/updates.py:124

No docstring.

Calls: isinstance, metadata.get

### _metadata_count

function nipux_cli/updates.py:129

No docstring.

Calls: isinstance, job.get, len, metadata.get

### format_usage_report

function nipux_cli/usage.py:10

No docstring.

Calls: _format_compact_count, _format_usage_cost, _safe_float, _safe_int, _safe_optional_float, bool, lines.append, max, usage.get

### _safe_int

function nipux_cli/usage.py:58

No docstring.

Calls: float, int

### _safe_float

function nipux_cli/usage.py:65

No docstring.

Calls: float

### _safe_optional_float

function nipux_cli/usage.py:72

No docstring.

Calls: float

### _TextExtractor

class nipux_cli/web.py:15

No docstring.

Calls: __init__, data.strip, html.unescape, join, re.sub, self.parts.append, strip, super

### __init__

function nipux_cli/web.py:16

No docstring.

Calls: __init__, super

### handle_starttag

function nipux_cli/web.py:21

No docstring.

Calls: self.parts.append

### handle_endtag

function nipux_cli/web.py:28

No docstring.

Calls: self.parts.append

### handle_data

function nipux_cli/web.py:34

No docstring.

Calls: data.strip, self.parts.append

### text

function nipux_cli/web.py:40

No docstring.

Calls: html.unescape, join, re.sub, strip

### _request

function nipux_cli/web.py:48

No docstring.

Calls: ValueError, body.decode, len, match.group, re.search, response.headers.get, response.read, urllib.request.Request, urllib.request.urlopen

### _strip_html

function nipux_cli/web.py:68

No docstring.

Calls: _TextExtractor, parser.feed, parser.text

### _duckduckgo_link

function nipux_cli/web.py:74

No docstring.

Calls: html.unescape, urllib.parse.parse_qs, urllib.parse.urlparse

### web_search

function nipux_cli/web.py:82

No docstring.

Calls: _duckduckgo_link, _request, html.unescape, len, match.group, pattern.finditer, re.compile, re.sub, results.append, strip, urllib.parse.urlencode

### web_extract

function nipux_cli/web.py:98

No docstring.

Calls: _request, _strip_html, anti_bot_reason, len, pages.append, str

### StepExecution

class nipux_cli/worker.py:122

No docstring.

Calls: dataclass

### build_messages

function nipux_cli/worker.py:170

No docstring.

Calls: _activity_stagnation_for_prompt, _candidate_file_discovery_for_prompt, _clip_text, _current_execution_focus_for_prompt, _deliverable_progress_guard_for_prompt, _durable_yield_for_prompt, _evidence_checkpoint_accounting_for_prompt, _experiment_stagnation_guard_for_prompt, _expe...

### _acknowledge_non_prompt_operator_context

function nipux_cli/worker.py:285

No docstring.

Calls: db.acknowledge_operator_messages, db.get_job, inactive_prompt_operator_ids, int, isinstance, job.get, metadata.get, result.get

### _measured_progress_guard_for_prompt

function nipux_cli/worker.py:300

No docstring.

Calls: _as_int, _candidate_file_discovery_context, _measured_progress_guard_context, context.get

### _deliverable_progress_guard_for_prompt

function nipux_cli/worker.py:330

No docstring.

Calls: _deliverable_progress_guard_context, context.get

### _experiment_stagnation_guard_for_prompt

function nipux_cli/worker.py:345

No docstring.

Calls: _experiment_stagnation_context, context.get

### _measurement_obligation_for_prompt

function nipux_cli/worker.py:360

No docstring.

Calls: _clip_text, isinstance, job.get, join, lines.append, metadata.get, obligation.get, str

### _recent_measurement_evidence_for_prompt

function nipux_cli/worker.py:382

No docstring.

Calls: _as_int, _clip_text, _completed_or_failed_recent_steps, _pending_measurement_obligation, _step_command, isinstance, join, len, lines.append, max, measurement_candidates, measurement_candidates_are_diagnostic_only, output.get, reversed, step.get, str

### _file_validation_obligation_for_prompt

function nipux_cli/worker.py:421

No docstring.

Calls: _pending_file_validation_obligation, join, lines.append, obligation.get, str, strip

### _current_execution_focus_for_prompt

function nipux_cli/worker.py:442

No docstring.

Calls: _clip_text, _current_execution_focus_context, backlog.get, focus.get, isinstance, join, lines.append, str, strip

### _current_execution_focus_context

function nipux_cli/worker.py:475

No docstring.

Calls: _auto_checkpoint_accounting_context, _candidate_file_discovery_context, _candidate_file_recently_validated, _latest_evidence_grounding_block, _latest_experiment_next_action_context, _milestone_validation_needed, _pending_file_validation_obligation, _pending_measurement_obligat...

### _task_backlog_pressure_context

function nipux_cli/worker.py:574

No docstring.

Calls: _metadata_list, isinstance, len, lower, str, sum, task.get

### _primary_execution_task

function nipux_cli/worker.py:586

No docstring.

Calls: _as_int, _metadata_list, isinstance, lower, sorted, status_rank.get, str, task.get

### _candidate_file_discovery_for_prompt

function nipux_cli/worker.py:602

No docstring.

Calls: _candidate_file_discovery_context, _candidate_file_recently_validated, _clip_text, context.get, isinstance, join, lines.append, str

### _shell_path_recovery_for_prompt

function nipux_cli/worker.py:639

No docstring.

Calls: _clip_text, _shell_path_recovery_context, candidate_executables.items, context.get, isinstance, join, lines.append, list, str

### _shell_path_recovery_context

function nipux_cli/worker.py:685

No docstring.

Calls: _candidate_executable_paths_for_missing_commands, _completed_or_failed_recent_steps, _missing_commands_from_shell_output, _missing_paths_from_shell_output, _observed_executable_paths_from_recent_shell, _shell_output_has_missing_command, isinstance, join, output.get, reversed, ...

### _shell_permission_recovery_for_prompt

function nipux_cli/worker.py:712

No docstring.

Calls: _clip_text, _recent_privileged_shell_failure_context, context.get, join, lines.append, str

### _shell_step_failure_text

function nipux_cli/worker.py:734

No docstring.

Calls: isinstance, join, output.get, step.get, str

### _shell_output_has_missing_command

function nipux_cli/worker.py:739

No docstring.

Calls: any, text.lower

### _missing_paths_from_shell_output

function nipux_cli/worker.py:744

No docstring.

Calls: get, len, match.groupdict, paths.append, re.finditer, seen.add, set, str, strip

### _missing_commands_from_shell_output

function nipux_cli/worker.py:764

No docstring.

Calls: commands.append, get, len, match.groupdict, re.finditer, seen.add, set, str, strip

### _candidate_executable_paths_for_missing_commands

function nipux_cli/worker.py:789

No docstring.

Calls: Path, _observed_executable_paths_from_recent_shell, append, len, lower, matches.get, matches.items, matches.setdefault, name.lower, path.lower, seen.add, set, str, strip

### _observed_executable_paths_from_recent_shell

function nipux_cli/worker.py:810

No docstring.

Calls: _completed_or_failed_recent_steps, _extract_candidate_executable_paths, _shell_line_reports_missing_candidate, isinstance, join, len, lower, output.get, path.lower, paths.append, seen.add, set, step.get, str, text.splitlines

### _shell_line_reports_missing_candidate

function nipux_cli/worker.py:840

No docstring.

Calls: any, lower, str

### _extract_candidate_executable_paths

function nipux_cli/worker.py:855

No docstring.

Calls: Path, _clean_candidate_file_path, _looks_like_candidate_executable_path, command.lower, match.group, name.lower, paths.append, raw.lower, re.finditer, seen.add, set

### _looks_like_candidate_executable_path

function nipux_cli/worker.py:874

No docstring.

Calls: Path, any, len, name.startswith, raw.startswith, str, strip

### _shell_command_looks_privileged_or_package_manager

function nipux_cli/worker.py:898

No docstring.

Calls: PACKAGE_MANAGER_WRITE_COMMAND_PATTERN.search, PRIVILEGED_COMMAND_PATTERN.search, bool, str, strip

### _shell_output_has_permission_failure

function nipux_cli/worker.py:905

No docstring.

Calls: any, lower, str

### _recent_privileged_shell_failure_context

function nipux_cli/worker.py:924

No docstring.

Calls: _as_int, _completed_or_failed_recent_steps, _shell_command_looks_privileged_or_package_manager, _shell_output_has_permission_failure, _shell_step_failure_text, _step_command, isinstance, max, output.get, reversed, step.get, str, text.strip

### _observed_candidate_recovery_required_context

function nipux_cli/worker.py:955

No docstring.

Calls: _shell_command_invokes_bare_executable, _shell_command_mentions_candidate_path, _shell_path_recovery_context, args.get, candidate_executables.items, command.strip, context.get, isinstance, str, strip

### _shell_command_invokes_bare_executable

function nipux_cli/worker.py:986

No docstring.

Calls: bool, re.escape, re.search, str, strip

### _shell_command_mentions_candidate_path

function nipux_cli/worker.py:993

No docstring.

Calls: Path, str, strip

### _candidate_file_discovery_context

function nipux_cli/worker.py:1007

No docstring.

Calls: _candidate_file_paths_from_durable_records, _candidate_file_paths_from_recent_grounding_blocks, _candidate_file_paths_from_recent_shell, _invalid_candidate_file_paths, _open_file_dependent_task_text, _rank_candidate_file_paths, path.lower, paths.append, seen.add, set

### _shell_exec_targets_candidate_file

function nipux_cli/worker.py:1039

No docstring.

Calls: _candidate_file_discovery_context, any, args.get, command.replace, command.strip, context.get, str

### _rank_candidate_file_paths

function nipux_cli/worker.py:1050

No docstring.

Calls: _candidate_context_tokens, _candidate_file_path_score, enumerate, list, sorted

### _candidate_context_tokens

function nipux_cli/worker.py:1072

No docstring.

Calls: job.get, join, len, re.findall, re.split, set, str, text.lower, token.strip, tokens.add

### _candidate_file_path_score

function nipux_cli/worker.py:1086

No docstring.

Calls: Path, _candidate_file_observation_score, any, len, min, name.lower, name.startswith, path.count, path.lower, path_tokens.add, path_tokens.update, re.findall, re.split, set, stem.lower, suffix.lower

### _invalid_candidate_file_paths

function nipux_cli/worker.py:1133

No docstring.

Calls: _candidate_file_observation_score, invalid.append

### _candidate_file_observation_score

function nipux_cli/worker.py:1141

No docstring.

Calls: _candidate_file_size_score_from_line, _completed_or_failed_recent_steps, _shell_line_reports_missing_candidate, any, isinstance, join, line.lower, output.get, path.lower, step.get, str, text.splitlines

### _candidate_file_recently_validated

function nipux_cli/worker.py:1163

No docstring.

Calls: _candidate_file_observation_score, _completed_or_failed_recent_steps, _shell_line_reports_missing_candidate, evidence_lines.append, isinstance, join, len, line.lower, line.split, output.get, path.lower, step.get, str, text.splitlines

### _candidate_file_size_score_from_line

function nipux_cli/worker.py:1184

No docstring.

Calls: any, int, lower, re.findall, re.search, str

### _open_file_dependent_task_text

function nipux_cli/worker.py:1198

No docstring.

Calls: _metadata_list, any, isinstance, join, len, lower, parts.append, str, task.get, text.lower, text.split

### _candidate_file_paths_from_recent_shell

function nipux_cli/worker.py:1219

No docstring.

Calls: _completed_or_failed_recent_steps, _extract_candidate_file_paths, isinstance, join, len, output.get, path.lower, paths.append, seen.add, set, step.get, str

### _candidate_file_paths_from_recent_grounding_blocks

function nipux_cli/worker.py:1240

No docstring.

Calls: _clean_candidate_file_path, _looks_like_exact_candidate_file_path, grounding.get, isinstance, len, output.get, path.lower, paths.append, seen.add, set, step.get, str

### _candidate_file_paths_from_durable_records

function nipux_cli/worker.py:1265

No docstring.

Calls: _extract_candidate_file_paths, _metadata_list, isinstance, job.get, json.dumps, len, metadata.get, path.lower, paths.append, record_groups.append, reversed, roadmap.get, seen.add, set, str

### _extract_candidate_file_paths

function nipux_cli/worker.py:1304

No docstring.

Calls: _clean_candidate_file_path, _looks_like_exact_candidate_file_path, match.group, paths.append, re.finditer

### _looks_like_exact_candidate_file_path

function nipux_cli/worker.py:1319

No docstring.

Calls: Path, any, ch.isalpha, len, name.startswith, raw.startswith, re.match, str, strip

### _clean_candidate_file_path

function nipux_cli/worker.py:1336

No docstring.

Calls: raw.split, raw.strip, rstrip, str, strip

### _progress_accounting_for_prompt

function nipux_cli/worker.py:1343

No docstring.

Calls: _artifact_accounting_context, context.get, join, str

### _activity_stagnation_for_prompt

function nipux_cli/worker.py:1357

No docstring.

Calls: _activity_stagnation_context, context.get

### _research_balance_guard_for_prompt

function nipux_cli/worker.py:1371

No docstring.

Calls: _research_balance_context, context.get

### _source_yield_guard_for_prompt

function nipux_cli/worker.py:1386

No docstring.

Calls: _source_yield_context, context.get, join, str

### _task_planning_guard_for_prompt

function nipux_cli/worker.py:1401

No docstring.

Calls: _task_planning_stagnation_context, context.get

### _task_queue_saturation_for_prompt

function nipux_cli/worker.py:1414

No docstring.

Calls: _current_task_backlog_pressure_context, _recent_task_queue_saturation_context, context.get, counts.append, current_pressure.get, guard_recovery.get, isinstance, job.get, join, json.dumps, metadata.get, pressure.get, str, strip, task_queue.get

### _current_task_backlog_pressure_context

function nipux_cli/worker.py:1468

No docstring.

Calls: _is_guard_recovery_task, _metadata_list, len, lower, replace, str, strip, task.get

### _memory_consolidation_guard_for_prompt

function nipux_cli/worker.py:1491

No docstring.

Calls: _memory_graph_consolidation_context, context.get

### _lesson_consolidation_guard_for_prompt

function nipux_cli/worker.py:1505

No docstring.

Calls: _lesson_sprawl_context, context.get

### _memory_graph_consolidation_context

function nipux_cli/worker.py:1519

No docstring.

Calls: _durable_memory_signal_count, any, len, memory_graph_from_job, step.get

### _lesson_sprawl_context

function nipux_cli/worker.py:1545

No docstring.

Calls: _memory_graph_consolidation_context, _metadata_list, len, lower, memory_context.get, step.get, str

### _durable_memory_signal_count

function nipux_cli/worker.py:1570

No docstring.

Calls: _metadata_list, isinstance, job.get, len, lower, metadata.get, milestone.get, roadmap.get, str, sum, task.get

### _durable_yield_for_prompt

function nipux_cli/worker.py:1596

No docstring.

Calls: _metadata_list, enumerate, isinstance, job.get, len, max, metadata.get, roadmap.get, step.get

### _reflections_for_prompt

function nipux_cli/worker.py:1637

No docstring.

Calls: _clip_text, _metadata_list, join, lines.append, reflection.get

### _next_action_constraint

function nipux_cli/worker.py:1648

No docstring.

Calls: _activity_stagnation_context, _artifact_accounting_context, _auto_checkpoint_accounting_context, _candidate_file_discovery_context, _candidate_file_recently_validated, _clean_candidate_file_path, _clip_text, _deliverable_progress_guard_context, _experiment_next_action_failure_...

### _latest_evidence_grounding_block

function nipux_cli/worker.py:1849

No docstring.

Calls: isinstance, output.get, reversed, step.get

### _milestone_validation_needed

function nipux_cli/worker.py:1870

No docstring.

Calls: all, feature.get, isinstance, job.get, metadata.get, milestone.get, roadmap.get, str

### _tool_call_matches_pending_milestone_need

function nipux_cli/worker.py:1890

No docstring.

Calls: _json_value_text, _text_matches_pending_milestone_need, lower, milestone.get, str, strip

### _text_matches_pending_milestone_need

function nipux_cli/worker.py:1898

No docstring.

Calls: _substantive_next_action_tokens, bool, feature.get, isinstance, join, milestone.get, parts.extend, str

### _milestone_validation_call_matches_current

function nipux_cli/worker.py:1927

No docstring.

Calls: _norm_task_key, args.get, milestone.get, str

### _normalize_milestone_validation_args_for_active_gate

function nipux_cli/worker.py:1943

No docstring.

Calls: _json_value_text, _milestone_validation_call_matches_current, _milestone_validation_needed, _text_matches_pending_milestone_need, args.get, dict, isinstance, milestone.get, normalized.get, str

### _latest_experiment_next_action_context

function nipux_cli/worker.py:1966

No docstring.

Calls: _metadata_list, experiment.get, isinstance, lower, reversed, str, strip

### _experiment_next_action_requires_delivery

function nipux_cli/worker.py:1986

No docstring.

Calls: bool, context.get, lower, re.findall, set, str

### _experiment_next_action_failure_context

function nipux_cli/worker.py:1998

No docstring.

Calls: _as_int, _completed_or_failed_recent_steps, _experiment_next_action_requires_delivery, _latest_experiment_next_action_context, _missing_commands_from_shell_output, _missing_paths_from_shell_output, _shell_command_matches_next_action, _shell_output_has_missing_command, _shell_s...

### _shell_command_looks_like_write

function nipux_cli/worker.py:2033

No docstring.

Calls: any, command.strip, re.match, re.search

### _shell_command_looks_read_only

function nipux_cli/worker.py:2057

No docstring.

Calls: READ_ONLY_SHELL_COMMAND_PATTERN.search, _shell_command_looks_like_write, bool, command.strip, re.match, re.search

### _shell_command_supports_experiment_next_action

function nipux_cli/worker.py:2071

No docstring.

Calls: EXPERIMENT_NEXT_ACTION_VERIFY_SHELL_PATTERN.search, _substantive_next_action_tokens, bool, command.strip, context.get, next_action.strip, str

### _shell_command_matches_next_action

function nipux_cli/worker.py:2087

No docstring.

Calls: _substantive_next_action_tokens, bool, command.strip, next_action.strip

### _substantive_next_action_tokens

function nipux_cli/worker.py:2095

No docstring.

Calls: len, re.findall, re.split, set, text.lower, token.strip, tokens.add

### _roadmap_staleness_context

function nipux_cli/worker.py:2110

No docstring.

Calls: any, isinstance, job.get, len, metadata.get, milestone.get, roadmap.get, step.get, str

### _roadmap_missing_for_broad_job

function nipux_cli/worker.py:2148

No docstring.

Calls: any, isinstance, job.get, len, metadata.get, objective.lower, re.findall, str

### _task_queue_exhausted

function nipux_cli/worker.py:2161

No docstring.

Calls: _metadata_list, any, lower, str, strip, task.get

### _task_queue_saturation_context

function nipux_cli/worker.py:2169

No docstring.

Calls: _is_guard_recovery_task, _metadata_list, _norm_task_key, args.get, bool, find_semantic_task_match, isinstance, len, lower, new_open_titles.append, new_titles.append, replace, semantic_match.get, semantic_matches.append, str, strip

### _recent_task_queue_saturation_context

function nipux_cli/worker.py:2250

No docstring.

Calls: isinstance, output.get, reversed, step.get, task_queue.get

### _record_task_backlog_pressure

function nipux_cli/worker.py:2268

No docstring.

Calls: datetime.now, db.append_agent_update, db.update_job_metadata, isinstance, isoformat, task_queue.get

### _clear_stale_task_backlog_pressure

function nipux_cli/worker.py:2301

No docstring.

Calls: _current_task_backlog_pressure_context, datetime.now, db.append_agent_update, db.update_job_metadata, dict, isinstance, isoformat, job.get, metadata.get

### _repeated_task_queue_saturation_context

function nipux_cli/worker.py:2320

No docstring.

Calls: isinstance, latest.get, len, matches.append, output.get, step.get, task_queue.get

### _task_planning_stagnation_context

function nipux_cli/worker.py:2340

No docstring.

Calls: _as_int, _metadata_list, isinstance, job.get, len, lower, metadata.get, replace, str, strip, task.get

### _is_guard_recovery_task

function nipux_cli/worker.py:2359

No docstring.

Calls: bool, isinstance, lower, metadata.get, startswith, str, strip, task.get

### _record_tasks_adds_new_open_work

function nipux_cli/worker.py:2364

No docstring.

Calls: _metadata_list, _norm_task_key, args.get, isinstance, lower, replace, str, strip, task.get

### _norm_task_key

function nipux_cli/worker.py:2386

No docstring.

Calls: task_key

### _parse_tool_result

function nipux_cli/worker.py:2390

No docstring.

Calls: isinstance, json.loads

### _load_program_text

function nipux_cli/worker.py:2398

No docstring.

Calls: path.exists, path.read_text

### _browser_warning_context

function nipux_cli/worker.py:2405

No docstring.

Calls: anti_bot_reason, data.get, isinstance, output.get, str

### _recent_anti_bot_context

function nipux_cli/worker.py:2416

No docstring.

Calls: _browser_warning_context, isinstance, reversed, step.get

### _artifact_args_acknowledge_block

function nipux_cli/worker.py:2427

No docstring.

Calls: any, args.get, join, lower, str

### _same_source_url

function nipux_cli/worker.py:2432

No docstring.

Calls: left.split, right.split, rstrip

### _normalized_source_url

function nipux_cli/worker.py:2438

No docstring.

Calls: str, strip

### _source_host

function nipux_cli/worker.py:2447

No docstring.

Calls: _normalized_source_url, parsed.netloc.lower, removeprefix, urlparse

### _source_matches

function nipux_cli/worker.py:2452

No docstring.

Calls: _same_source_url, _source_path_key, left_path.startswith, right_path.startswith

### _source_path_key

function nipux_cli/worker.py:2464

No docstring.

Calls: _normalized_source_url, parsed.netloc.lower, removeprefix, rstrip, urlparse

### _shell_source_matches

function nipux_cli/worker.py:2471

No docstring.

Calls: _same_source_url, _source_path_key, left_path.startswith, right_path.startswith

### _urls_from_text

function nipux_cli/worker.py:2483

No docstring.

Calls: match.group, re.finditer, rstrip, seen.add, set, str, url.lower, urls.append

### _source_url_has_path

function nipux_cli/worker.py:2496

No docstring.

Calls: _source_path_key

### _shell_guard_urls

function nipux_cli/worker.py:2501

No docstring.

Calls: _source_url_has_path, _urls_from_text, len

### _shell_placeholder_context

function nipux_cli/worker.py:2541

No docstring.

Calls: _urls_from_text, join, lower, match.group, re.escape, re.search, str, strip, urlparse

### _shell_syntax_preflight_context

function nipux_cli/worker.py:2585

No docstring.

Calls: shlex.split, str, strip

### _source_failure_family_url

function nipux_cli/worker.py:2600

No docstring.

Calls: _normalized_source_url, join, len, split, urlparse

### _known_bad_sources

function nipux_cli/worker.py:2615

No docstring.

Calls: _as_float, _as_int, _metadata_list, bad_sources.append, source.get

### _known_bad_source_for_call

function nipux_cli/worker.py:2627

No docstring.

Calls: _known_bad_sources, _shell_guard_urls, _shell_source_matches, _source_failure_family_url, _source_matches, args.get, isinstance, source.get, str, url.strip

### _tool_signature

function nipux_cli/worker.py:2661

No docstring.

Calls: json.dumps

### _duplicate_recent_tool_call

function nipux_cli/worker.py:2665

No docstring.

Calls: _tool_signature, input_data.get, isinstance, reversed, step.get

### _completed_recent_steps

function nipux_cli/worker.py:2685

No docstring.

Calls: step.get

### _completed_or_failed_recent_steps

function nipux_cli/worker.py:2689

No docstring.

Calls: step.get

### _is_browser_tool

function nipux_cli/worker.py:2716

No docstring.

Calls: bool, startswith, str

### _browser_runtime_unavailable_context

function nipux_cli/worker.py:2720

No docstring.

Calls: _clip_text, _is_browser_tool, any, int, isinstance, join, lower, max, output.get, reversed, step.get, str

### _self_defer_context

function nipux_cli/worker.py:2764

No docstring.

Calls: args.get, lower, next, next_action.strip, reason.strip, str

### _stale_claim_tokens_from_unsupported

function nipux_cli/worker.py:3050

No docstring.

Calls: _looks_like_generated_or_file_token, _normalize_claim_text, any, ch.isalpha, ch.isdigit, cleaned.isupper, cleaned.lower, len, seen.add, set, stale_tokens.append, str, strip

### _looks_like_generated_or_file_token

function nipux_cli/worker.py:3076

No docstring.

Calls: any, ch.isalpha, ch.isdigit, lowered.endswith, lowered.startswith, token.lower

### _normalize_claim_text

function nipux_cli/worker.py:3108

No docstring.

Calls: lower, re.sub, str

### _evidence_grounding_context

function nipux_cli/worker.py:3112

No docstring.

Calls: _active_stale_claim_token_set, _candidate_file_paths_from_recent_grounding_blocks, _cited_step_numbers, _concrete_evidence_tokens_for_grounding, _evidence_grounding_proposed_text, _evidence_steps_for_grounding, _grounding_token_in_reference_text, _high_risk_evidence_token, _js...

### _concrete_evidence_tokens_for_grounding

function nipux_cli/worker.py:3260

No docstring.

Calls: _concrete_evidence_tokens, _high_risk_evidence_token

### _grounding_token_in_reference_text

function nipux_cli/worker.py:3267

No docstring.

Calls: _normalize_claim_text

### _missing_candidate_paths_for_grounding

function nipux_cli/worker.py:3274

No docstring.

Calls: _candidate_file_paths_from_recent_grounding_blocks, _evidence_line_is_negative, _extract_candidate_file_paths, _path_mentioned_in_text, _rank_candidate_file_paths, any, distinctive_paths.append, join, len, line.lower, lower, path.lower, seen.add, set, splitlines, str

### _positive_path_claim_conflicts_for_grounding

function nipux_cli/worker.py:3330

No docstring.

Calls: _clip_text, _evidence_line_is_negative, _excerpt_around, _extract_candidate_executable_paths, _extract_candidate_file_paths, _path_near_positive_claim, conflicts.append, len, line.lower, line.strip, path.lower, proposed_combined.lower, seen.add, set, splitlines, str

### _path_near_positive_claim

function nipux_cli/worker.py:3373

No docstring.

Calls: _evidence_line_is_negative, _excerpts_around_all, any, excerpt.lower

### _excerpt_around

function nipux_cli/worker.py:3383

No docstring.

Calls: _excerpts_around_all

### _excerpts_around_all

function nipux_cli/worker.py:3388

No docstring.

Calls: excerpts.append, len, max, min, needle_text.lower, source.lower, source_lower.find, str

### _path_mentioned_in_text

function nipux_cli/worker.py:3408

No docstring.

Calls: Path, bool, name.lower, path.lower

### _refresh_contradicted_negative_claims

function nipux_cli/worker.py:3416

No docstring.

Calls: _concrete_evidence_tokens, _metadata_list, _negative_claim_conflicts_for_grounding, _negative_record_id, _negative_record_text, _negative_record_title, _recent_evidence_text, conflict.get, datetime.now, db.append_agent_update, db.update_job_metadata, fresh_evidence_text.strip,...

### _negative_record_text

function nipux_cli/worker.py:3498

No docstring.

Calls: join, record.get, str

### _negative_record_id

function nipux_cli/worker.py:3512

No docstring.

Calls: _negative_record_title, _normalize_claim_text, record.get, str, strip

### _negative_record_title

function nipux_cli/worker.py:3520

No docstring.

Calls: _clip_text, record.get, str

### _negative_claim_conflicts_for_grounding

function nipux_cli/worker.py:3526

No docstring.

Calls: _clip_text, _file_pattern_tokens_for_grounding, _high_risk_evidence_token, _positive_evidence_line_for_token, _token_near_negative_claim, any, conflicts.append, fresh_evidence_text.splitlines, line.strip, proposed_text.lower, seen.add, set, token.lower, token.startswith

### _file_pattern_tokens_for_grounding

function nipux_cli/worker.py:3560

No docstring.

Calls: len, lower, lstrip, match.end, match.group, match.start, raw.startswith, re.finditer, seen.add, set, strip, tokens.append

### _token_near_negative_claim

function nipux_cli/worker.py:3586

No docstring.

Calls: _nearby_negative_is_positive_validation, _nearby_negative_is_role_classification, any, len, max, text.lower, text_lower.find, token.lower

### _nearby_negative_is_role_classification

function nipux_cli/worker.py:3606

No docstring.

Calls: any

### _nearby_negative_is_positive_validation

function nipux_cli/worker.py:3610

No docstring.

Calls: bool, re.search

### _positive_evidence_line_for_token

function nipux_cli/worker.py:3614

No docstring.

Calls: _evidence_line_is_negative, line.lower, token.lower

### _evidence_line_is_negative

function nipux_cli/worker.py:3626

No docstring.

Calls: any, line_lower.startswith

### _evidence_grounding_proposed_text

function nipux_cli/worker.py:3632

No docstring.

Calls: _json_text, _json_value_text, args.get, edge.get, isinstance, join, node.get, parts.append

### _json_text

function nipux_cli/worker.py:3675

No docstring.

Calls: json.dumps, str

### _json_value_text

function nipux_cli/worker.py:3682

No docstring.

Calls: _json_value_text, isinstance, join, str, value.values

### _cited_step_numbers

function nipux_cli/worker.py:3690

No docstring.

Calls: int, match.group, numbers.add, re.finditer, set

### _evidence_steps_for_grounding

function nipux_cli/worker.py:3710

No docstring.

Calls: _completed_recent_steps, int, step.get

### _recent_evidence_text

function nipux_cli/worker.py:3735

No docstring.

Calls: _durable_records_for_grounding, _evidence_steps_for_grounding, _json_text, isinstance, item.get, job.get, join, output.get, page.get, parts.append, parts.extend, step.get, str

### _active_stale_claim_token_set

function nipux_cli/worker.py:3772

No docstring.

Calls: _stale_claim_tokens_from_unsupported, isinstance, job.get, join, lower, metadata.get, str, strip

### _durable_records_for_grounding

function nipux_cli/worker.py:3782

No docstring.

Calls: _json_text, _metadata_list, experiment.get, finding.get, graph.get, isinstance, job.get, join, metadata.get, node.get, parts.append, roadmap.get, source.get

### _concrete_evidence_tokens

function nipux_cli/worker.py:3835

No docstring.

Calls: _looks_like_generated_evidence_token, any, ch.isalpha, ch.isdigit, islower, isupper, len, lowered.endswith, lowered.startswith, raw.strip, re.findall, re.match, re.sub, replace, seen_numeric.add, set

### _high_risk_evidence_token

function nipux_cli/worker.py:3878

No docstring.

Calls: _looks_like_generated_evidence_token, any, ch.isalpha, ch.isdigit, len, lowered.endswith, lowered.startswith, token.isupper, token.lower

### _looks_like_generated_evidence_token

function nipux_cli/worker.py:3895

No docstring.

Calls: bool, re.match, strip, token.lower

### _step_has_evidence

function nipux_cli/worker.py:3905

No docstring.

Calls: anti_bot_reason, data.get, isinstance, join, len, output.get, page.get, snapshot.strip, step.get, str, strip, text.strip

### _unpersisted_evidence_step

function nipux_cli/worker.py:3927

No docstring.

Calls: _step_has_evidence, isinstance, output.get, reversed, step.get

### _evidence_checkpoint_accounting_for_prompt

function nipux_cli/worker.py:3941

No docstring.

Calls: _auto_checkpoint_accounting_context, context.get

### _pending_evidence_checkpoint

function nipux_cli/worker.py:3963

No docstring.

Calls: checkpoint.get, isinstance, job.get, metadata.get

### _step_created_auto_checkpoint

function nipux_cli/worker.py:3971

No docstring.

Calls: checkpoint.get, isinstance, output.get, step.get

### _auto_checkpoint_accounting_context

function nipux_cli/worker.py:3985

No docstring.

Calls: _pending_evidence_checkpoint, _read_artifact_args_match_checkpoint, _step_created_auto_checkpoint, any, bool, checkpoint.get, checkpoint_step.get, int, pending.get, reversed, step.get, str

### _evidence_checkpoint_blocks_tool

function nipux_cli/worker.py:4032

No docstring.

Calls: _read_artifact_call_matches_checkpoint, context.get, str

### _evidence_checkpoint_block_guidance

function nipux_cli/worker.py:4050

No docstring.

Calls: context.get

### _read_artifact_args_match_checkpoint

function nipux_cli/worker.py:4068

No docstring.

Calls: _read_artifact_call_matches_checkpoint, input_data.get, isinstance, step.get

### _read_artifact_call_matches_checkpoint

function nipux_cli/worker.py:4074

No docstring.

Calls: any, args.get, bool, str, strip

### _recent_search_streak

function nipux_cli/worker.py:4082

No docstring.

Calls: _recent_tool_streak

### _pending_measurement_obligation

function nipux_cli/worker.py:4086

No docstring.

Calls: isinstance, job.get, metadata.get, obligation.get

### _pending_file_validation_obligation

function nipux_cli/worker.py:4127

No docstring.

Calls: isinstance, job.get, metadata.get, obligation.get

### _file_output_needs_validation

function nipux_cli/worker.py:4135

No docstring.

Calls: Path, content.lstrip, content.strip, first_line.startswith, name.lower, splitlines, suffix.lower

### _suggested_file_validation

function nipux_cli/worker.py:4146

No docstring.

Calls: Path, shlex_quote, suffix.lower

### shlex_quote

function nipux_cli/worker.py:4160

No docstring.

Calls: replace, str

### _clear_invalid_measurement_obligation

function nipux_cli/worker.py:4164

No docstring.

Calls: _pending_measurement_obligation, db.append_agent_update, db.get_job, db.update_job_metadata, isinstance, measurement_candidates_are_diagnostic_only, obligation.get, str

### _progress_churn_context

function nipux_cli/worker.py:4185

No docstring.

Calls: any, get, len, step.get, sum

### _activity_stagnation_context

function nipux_cli/worker.py:4203

No docstring.

Calls: _as_int, counts.get, isinstance, job.get, metadata.get

### _research_balance_context

function nipux_cli/worker.py:4216

No docstring.

Calls: _metadata_list, any, bool, isinstance, job.get, len, lower, metadata.get, step.get, str, task.get

### _source_yield_context

function nipux_cli/worker.py:4276

No docstring.

Calls: _as_float, _as_int, _metadata_list, get, len, max, source.get, step.get, str, strip

### _artifact_accounting_context

function nipux_cli/worker.py:4335

No docstring.

Calls: _clip_text, args.get, get, input_data.get, isinstance, len, reversed, step.get, str, tail.append, tail.reverse, titles.append

### _job_requires_measured_progress

function nipux_cli/worker.py:4366

No docstring.

Calls: MEASURABLE_PROGRESS_PATTERN.search, _metadata_list, _task_text_requires_measurement, any, job.get, str, task.get, text_parts.extend

### _task_text_requires_measurement

function nipux_cli/worker.py:4389

No docstring.

Calls: MEASURABLE_PROGRESS_PATTERN.search, any, str, task.get

### _job_requires_deliverable_progress

function nipux_cli/worker.py:4396

No docstring.

Calls: _as_int, _metadata_list, any, bool, competing_execution_tasks.append, job.get, join, lower, max, re.findall, report_tasks.append, set, str, strip, task.get

### _step_is_deliverable_checkpoint

function nipux_cli/worker.py:4428

No docstring.

Calls: args.get, bool, input_data.get, isinstance, join, lower, re.findall, set, step.get, str

### _deliverable_progress_guard_context

function nipux_cli/worker.py:4451

No docstring.

Calls: _job_requires_deliverable_progress, _shell_command_looks_read_only, _step_command, _step_is_deliverable_checkpoint, any, enumerate, get, len, step.get

### _step_command

function nipux_cli/worker.py:4491

No docstring.

Calls: args.get, input_data.get, isinstance, step.get, str

### _read_only_shell_churn_context

function nipux_cli/worker.py:4497

No docstring.

Calls: _clip_text, _shell_command_looks_read_only, _step_command, get, len, step.get

### _experiment_metric_group_key

function nipux_cli/worker.py:4536

No docstring.

Calls: bool, experiment.get, lower, str, strip

### _experiment_metric_number

function nipux_cli/worker.py:4549

No docstring.

Calls: experiment.get, float

### _experiment_value_improves

function nipux_cli/worker.py:4556

No docstring.

Calls: none

### _experiment_stagnation_context

function nipux_cli/worker.py:4560

No docstring.

Calls: _as_int, _experiment_metric_group_key, _experiment_metric_number, _experiment_value_improves, _job_requires_measured_progress, _metadata_list, any, best.get, bool, enumerate, experiment.get, latest.get, len, lower, max, step.get

### _measured_progress_guard_context

function nipux_cli/worker.py:4625

No docstring.

Calls: _job_requires_measured_progress, _metadata_list, _pending_measurement_obligation, _step_accounts_for_measured_progress_guard, any, enumerate, get, len, step.get

### _step_accounts_for_measured_progress_guard

function nipux_cli/worker.py:4663

No docstring.

Calls: _task_text_requires_measurement, isinstance, lower, output.get, replace, step.get, str, strip, task.get

### _maybe_create_measurement_obligation

function nipux_cli/worker.py:4685

No docstring.

Calls: args.get, datetime.now, db.append_agent_update, db.get_job, db.update_job_metadata, existing.get, get, isinstance, isoformat, join, measurement_candidates, metadata.get, result.get, step.get, str

### _maybe_create_file_validation_obligation

function nipux_cli/worker.py:4723

No docstring.

Calls: _file_output_needs_validation, _suggested_file_validation, args.get, datetime.now, db.append_agent_update, db.get_job, db.update_job_metadata, existing.get, get, isinstance, isoformat, metadata.get, result.get, step.get, str, strip

### _command_references_path

function nipux_cli/worker.py:4758

No docstring.

Calls: Path, any, needles.add, path_obj.expanduser, resolve, str

### _resolve_file_validation_obligation

function nipux_cli/worker.py:4770

No docstring.

Calls: _pending_file_validation_obligation, datetime.now, db.append_agent_update, db.get_job, db.update_job_metadata, dict, isoformat, resolved.update, result.get

### _maybe_resolve_file_validation_obligation

function nipux_cli/worker.py:4811

No docstring.

Calls: _command_references_path, _pending_file_validation_obligation, _resolve_file_validation_obligation, args.get, db.get_job, obligation.get, result.get, str

### _step_by_id

function nipux_cli/worker.py:4843

No docstring.

Calls: db.list_steps, step.get, str

### _search_query

function nipux_cli/worker.py:4850

No docstring.

Calls: args.get, str, strip

### _query_tokens

function nipux_cli/worker.py:4854

No docstring.

Calls: len, query.lower, re.findall

### _text_tokens

function nipux_cli/worker.py:4862

No docstring.

Calls: len, lower, re.findall, str

### _similar_recent_search

function nipux_cli/worker.py:4870

No docstring.

Calls: _similar_recent_query_tool

### _similar_recent_query_tool

function nipux_cli/worker.py:4879

No docstring.

Calls: _completed_recent_steps, _query_tokens, _search_query, input_data.get, isinstance, len, max, reversed, step.get

### _recent_tool_streak

function nipux_cli/worker.py:4907

No docstring.

Calls: _completed_recent_steps, reversed, step.get

### _repeated_guard_block_context

function nipux_cli/worker.py:4919

No docstring.

Calls: _already_read_checkpoint_accounting_block, any, blocked_tools.append, int, isinstance, last_recovery.get, latest_blocked.get, max, next, output.get, recovery_context.get, recovery_output.get, reversed, step.get, step_output.get, str

### _already_read_checkpoint_accounting_block

function nipux_cli/worker.py:4993

No docstring.

Calls: bool, checkpoint.get, isinstance, output.get, step.get

### _step_error_text

function nipux_cli/worker.py:5002

No docstring.

Calls: isinstance, join, output.get, step.get, str

### _blocked_tool_call_result

function nipux_cli/worker.py:5015

No docstring.

Calls: _activity_stagnation_context, _artifact_accounting_context, _artifact_args_acknowledge_block, _as_int, _auto_checkpoint_accounting_context, _browser_runtime_unavailable_context, _deliverable_progress_guard_context, _duplicate_recent_tool_call, _evidence_checkpoint_block_guidan...

### _error_result

function nipux_cli/worker.py:5744

No docstring.

Calls: isinstance, str, type

### _hard_llm_provider_failure_note

function nipux_cli/worker.py:5755

No docstring.

Calls: provider_action_required_note

### _max_step_no

function nipux_cli/worker.py:5759

No docstring.

Calls: int, max, step.get

### _should_reflect

function nipux_cli/worker.py:5763

No docstring.

Calls: _max_step_no, _metadata_list, get, isinstance, metadata.get

### _lesson_already_recorded

function nipux_cli/worker.py:5781

No docstring.

Calls: _metadata_list, any, entry.get, join, lower, split, str, strip

### _reflection_strategy

function nipux_cli/worker.py:5791

No docstring.

Calls: _as_float, _as_int, isinstance, len, lower, max, source.get, str, task.get

### _claim_operator_queue

function nipux_cli/worker.py:5831

No docstring.

Calls: db.claim_operator_messages

### _emit_loop_start

function nipux_cli/worker.py:5838

No docstring.

Calls: db.append_event

### _emit_assistant_message_event

function nipux_cli/worker.py:5857

No docstring.

Calls: db.append_event, float, join, max, round, turn_usage_metadata

### _emit_loop_end

function nipux_cli/worker.py:5892

No docstring.

Calls: db.append_event

### _run_reflection_step

function nipux_cli/worker.py:5923

No docstring.

Calls: StepExecution, _as_float, _as_int, _emit_loop_end, _lesson_already_recorded, _max_step_no, _pending_measurement_obligation, _reflection_strategy, artifact.get, bool, db.add_step, db.append_agent_update, db.append_lesson, db.append_reflection, db.finish_run, db.finish_step

### _run_guard_recovery_step

function nipux_cli/worker.py:6042

No docstring.

Calls: StepExecution, _emit_loop_end, _resolve_evidence_checkpoint, _step_by_id, context.get, datetime.now, db.add_step, db.append_agent_update, db.append_lesson, db.append_task_record, db.finish_run, db.finish_step, db.update_job_metadata, isinstance, isoformat, join

### _usage_budget_limit_context

function nipux_cli/worker.py:6187

No docstring.

Calls: _as_float, _as_int, bool, float, usage.get

### _run_usage_budget_limit_step

function nipux_cli/worker.py:6204

No docstring.

Calls: StepExecution, _compact_usage_tokens, _emit_loop_end, context.get, datetime.now, db.add_step, db.append_agent_update, db.finish_run, db.finish_step, db.update_job_status, float, isoformat, refresh_memory_index

### _compact_usage_tokens

function nipux_cli/worker.py:6241

No docstring.

Calls: _as_int, str

### _evidence_checkpoint_content

function nipux_cli/worker.py:6250

No docstring.

Calls: _observation_for_prompt, evidence_step.get, input_data.get, isinstance, join, json.dumps

### _auto_persist_evidence

function nipux_cli/worker.py:6264

No docstring.

Calls: _evidence_checkpoint_content, artifacts.write_text, datetime.now, db.append_agent_update, db.append_lesson, db.update_job_metadata, evidence_step.get, isoformat, str

### _auto_record_grounding_block_lesson

function nipux_cli/worker.py:6319

No docstring.

Calls: _stale_claim_tokens_from_unsupported, combined.append, combined_seen.add, db.append_lesson, db.get_job, db.update_job_metadata, grounding.get, isinstance, job.get, join, metadata.get, result.get, set, str, strip, token.lower

### _mark_evidence_checkpoint_read

function nipux_cli/worker.py:6369

No docstring.

Calls: _pending_evidence_checkpoint, _read_artifact_call_matches_checkpoint, datetime.now, db.append_agent_update, db.get_job, db.update_job_metadata, dict, isoformat, pending.get, step.get, str

### _resolve_evidence_checkpoint

function nipux_cli/worker.py:6403

No docstring.

Calls: _pending_evidence_checkpoint, datetime.now, db.append_agent_update, db.get_job, db.update_job_metadata, dict, isoformat, pending.get, step.get

### _auto_record_blocked_source

function nipux_cli/worker.py:6431

No docstring.

Calls: context.get, db.append_agent_update, db.append_lesson, db.append_source_record, int, record.get, str

### _auto_record_tool_source_quality

function nipux_cli/worker.py:6468

No docstring.

Calls: _auto_record_blocked_source, _browser_warning_context, context.get, db.append_source_record, isinstance, item.get, len, page.get, result.get, str, strip, text.strip

### _auto_record_failed_shell_sources

function nipux_cli/worker.py:6541

No docstring.

Calls: _clip_text, _same_source_url, _shell_guard_urls, _source_failure_family_url, any, args.get, candidate.lower, candidates.append, db.append_source_record, error_text.lower, join, metadata.update, recorded.add, result.get, set, str

### _auto_reconcile_artifact_tasks

function nipux_cli/worker.py:6601

No docstring.

Calls: _artifact_can_reconcile_task, _as_int, _clip_text, _metadata_list, _text_tokens, args.get, db.append_agent_update, db.append_task_record, db.get_job, isinstance, join, len, lower, max, min, reconciled.append

### _auto_open_revision_task_for_deliverable

function nipux_cli/worker.py:6676

No docstring.

Calls: _artifact_can_reconcile_task, _as_int, _clip_text, _metadata_list, args.get, db.append_agent_update, db.append_task_record, db.get_job, isinstance, lower, metadata.get, result.get, str, strip, task.get

### _artifact_can_reconcile_task

function nipux_cli/worker.py:6744

No docstring.

Calls: any, contract.strip, lower, task_text.lower

### _auto_checkpoint_update

function nipux_cli/worker.py:6768

No docstring.

Calls: _as_int, any, args.get, build_progress_checkpoint, checkpoint.deltas.get, checkpoint.resolutions.get, checkpoint.updates.get, datetime.now, db.append_agent_update, db.get_job, db.update_job_metadata, isinstance, isoformat, job.get, join, lower

### _execute_tool_call

function nipux_cli/worker.py:6828

No docstring.

Calls: StepExecution, ToolContext, _auto_checkpoint_update, _auto_open_revision_task_for_deliverable, _auto_persist_evidence, _auto_reconcile_artifact_tasks, _auto_record_blocked_source, _auto_record_failed_shell_sources, _auto_record_grounding_block_lesson, _auto_record_tool_source_...

### _is_continuable_recoverable_input_block

function nipux_cli/worker.py:7070

No docstring.

Calls: bool, error.startswith, isinstance, result.get, str

### _ordered_tool_calls_for_execution

function nipux_cli/worker.py:7084

Run guard-unblocking calls before branch work when a model batches both.

Calls: _auto_checkpoint_accounting_context, _browser_runtime_unavailable_context, _is_browser_tool, _read_artifact_call_matches_checkpoint, _task_queue_saturation_context, any, bool, checkpoint.get, enumerate, len, priority, sorted, str

### priority

function nipux_cli/worker.py:7121

No docstring.

Calls: _read_artifact_call_matches_checkpoint, _task_queue_saturation_context

### _registry_tools

function nipux_cli/worker.py:7146

No docstring.

Calls: registry.openai_tools

### _registry_tools_for_step

function nipux_cli/worker.py:7153

No docstring.

Calls: _active_obligation_tool_names, _browser_runtime_unavailable_context, _is_browser_tool, _openai_tool_name, _registry_tools, _suppressed_tool_names

### _active_obligation_tool_names

function nipux_cli/worker.py:7174

No docstring.

Calls: _as_int, _auto_checkpoint_accounting_context, _experiment_next_action_failure_context, _measured_progress_guard_context, _pending_file_validation_obligation, _pending_measurement_obligation, allowed.add, allowed.update, checkpoint.get, measured_progress.get, set

### _suppressed_tool_names

function nipux_cli/worker.py:7197

No docstring.

Calls: _as_int, _auto_checkpoint_accounting_context, _has_acknowledgeable_operator_context, _pending_file_validation_obligation, _pending_measurement_obligation, _repeated_task_queue_saturation_context, _task_backlog_pressure_context, _task_queue_exhausted, backlog.get, set, suppress...

### _has_acknowledgeable_operator_context

function nipux_cli/worker.py:7217

No docstring.

Calls: entry.get, isinstance, job.get, lower, metadata.get, replace, str, strip

### _openai_tool_name

function nipux_cli/worker.py:7234

No docstring.

Calls: function.get, isinstance, str, tool.get

### _call_next_action_with_timeout

function nipux_cli/worker.py:7241

No docstring.

Calls: TimeoutError, float, llm.next_action, max, signal.getitimer, signal.getsignal, signal.setitimer, signal.signal, threading.current_thread, threading.main_thread, time.monotonic

### _raise_timeout

function nipux_cli/worker.py:7256

No docstring.

Calls: TimeoutError

### _tool_repair_messages

function nipux_cli/worker.py:7272

No docstring.

Calls: len, list, repaired.append, str, strip

### run_one_step

function nipux_cli/worker.py:7289

No docstring.

Calls: AgentDB, ArtifactStore, OpenAIChatLLM, StepExecution, _acknowledge_non_prompt_operator_context, _call_next_action_with_timeout, _claim_operator_queue, _clear_invalid_measurement_obligation, _clear_stale_task_backlog_pressure, _emit_assistant_message_event, _emit_loop_end, _emi...

### _memory_entries_for_prompt

function nipux_cli/worker_prompt_context.py:64

No docstring.

Calls: entry.get, isinstance, len, next, selected.append

### _render_worker_prompt

function nipux_cli/worker_prompt_context.py:79

No docstring.

Calls: PROMPT_SECTION_BUDGETS.get, _clip_text, _redact_stale_tokens_for_prompt, _stale_claim_tokens_for_prompt, int, isinstance, job.get, join, len, max, parts.append, str, suffix_sections.append

### _redact_stale_tokens_for_prompt

function nipux_cli/worker_prompt_context.py:118

No docstring.

Calls: _match_inside_path_like_span, match.end, match.group, match.start, re.escape, re.sub, sorted, str, strip

### _match_inside_path_like_span

function nipux_cli/worker_prompt_context.py:131

No docstring.

Calls: isspace, len, span.startswith

### _operator_messages_for_prompt

function nipux_cli/worker_prompt_context.py:142

No docstring.

Calls: _operator_message_line, _operator_message_visible_in_prompt, active.get, active_prompt_operator_entries, entry.get, isinstance, job.get, join, lines.append, metadata.get, operator_entry_is_prompt_relevant

### _operator_message_line

function nipux_cli/worker_prompt_context.py:177

No docstring.

Calls: _clip_text, entry.get, isinstance, join, split, states.append, str

### _lessons_for_prompt

function nipux_cli/worker_prompt_context.py:199

No docstring.

Calls: _clip_text, _lesson_prompt_text, _negative_lesson_conflict_tokens, _positive_durable_lines_for_lesson_conflicts, _record_id_for_staleness, _stale_negative_record_ids, entry.get, isinstance, job.get, join, lines.append, metadata.get, str

### _lesson_prompt_text

function nipux_cli/worker_prompt_context.py:230

No docstring.

Calls: _normalize_claim_text, _stale_token_is_distinctive, _unsupported_tokens_from_lesson, cleaned.lower, join, lesson.lower, seen.add, set, split, stale_tokens.append, str

### _positive_durable_lines_for_lesson_conflicts

function nipux_cli/worker_prompt_context.py:257

No docstring.

Calls: _dict_scalar_text, graph.get, isinstance, lines.append, metadata.get

### _negative_lesson_conflict_tokens

function nipux_cli/worker_prompt_context.py:276

No docstring.

Calls: _distinctive_claim_tokens, _positive_line_contains_token, _token_near_negative_marker, any, conflicts.append, join, lesson.lower, seen.add, set, split, str, token.lower

### _dict_scalar_text

function nipux_cli/worker_prompt_context.py:296

No docstring.

Calls: _dict_scalar_text, isinstance, join, parts.append, record.items, str

### _distinctive_claim_tokens

function nipux_cli/worker_prompt_context.py:308

No docstring.

Calls: _stale_token_is_distinctive, raw.strip, re.findall, tokens.append

### _token_near_negative_marker

function nipux_cli/worker_prompt_context.py:317

No docstring.

Calls: any, len, max, text.lower, text_lower.find, token.lower

### _positive_line_contains_token

function nipux_cli/worker_prompt_context.py:331

No docstring.

Calls: any, line.lower, token.lower

### _memory_graph_for_prompt

function nipux_cli/worker_prompt_context.py:343

No docstring.

Calls: _stale_claim_tokens_for_prompt, isinstance, job.get, join, memory_graph_for_prompt, str

### _roadmap_for_prompt

function nipux_cli/worker_prompt_context.py:352

No docstring.

Calls: _clip_text, feature.get, isinstance, job.get, join, len, lines.append, metadata.get, milestone.get, roadmap.get, sorted, status_counts.get, status_counts.items, str, sum, validation_counts.get

### _tasks_for_prompt

function nipux_cli/worker_prompt_context.py:416

No docstring.

Calls: _as_int, _clip_text, _metadata_list, _task_output_contract, bits.append, counts.get, counts.items, join, len, lines.append, selected.extend, sorted, status_rank.get, str, task.get

### _task_output_contract

function nipux_cli/worker_prompt_context.py:464

No docstring.

Calls: isinstance, metadata.get, str, task.get

### _timeline_for_prompt

function nipux_cli/worker_prompt_context.py:469

No docstring.

Calls: _clip_text, _timeline_event_for_prompt, counts.get, counts.items, join, lines.append, selected.append, sorted

### _outcomes_for_prompt

function nipux_cli/worker_prompt_context.py:491

Summarize durable outputs so the worker sees progress, not just activity.

Calls: _clip_text, hourly_outcome_summary, join, label.lower, len, lines.append, model_update_event_parts, outcome_counts, reversed, seen.add, set

### _ledgers_for_prompt

function nipux_cli/worker_prompt_context.py:519

No docstring.

Calls: _as_float, _as_int, _clip_text, _metadata_list, _record_contains_stale_token, _record_id_for_staleness, _stale_claim_tokens_for_prompt, _stale_negative_record_ids, _stale_negative_records_for_prompt, finding.get, isinstance, item.get, job.get, join, len, lines.append

### _stale_claim_tokens_for_prompt

function nipux_cli/worker_prompt_context.py:616

No docstring.

Calls: _normalize_claim_text, _stale_token_is_distinctive, _unsupported_tokens_from_lesson, candidates.append, candidates.extend, isinstance, join, lesson.get, metadata.get, record.get, seen.add, set, split, str, token.lower, tokens.append

### _unsupported_tokens_from_lesson

function nipux_cli/worker_prompt_context.py:651

No docstring.

Calls: lesson.lower, match.group, part.strip, re.search, split

### _stale_token_is_distinctive

function nipux_cli/worker_prompt_context.py:663

No docstring.

Calls: any, ch.isalpha, ch.isdigit, len, lowered.endswith, lowered.startswith, re.match, token.isupper, token.lower

### _normalize_claim_text

function nipux_cli/worker_prompt_context.py:746

No docstring.

Calls: lower, re.sub, str

### _record_contains_stale_token

function nipux_cli/worker_prompt_context.py:750

No docstring.

Calls: isinstance, join, metadata.values, re.escape, re.search, record.get, str

### _stale_negative_records_for_prompt

function nipux_cli/worker_prompt_context.py:766

No docstring.

Calls: isinstance, metadata.get, record.get, str

### _stale_negative_record_ids

function nipux_cli/worker_prompt_context.py:777

No docstring.

Calls: _stale_negative_records_for_prompt, ids.add, record.get, set, str, strip

### _record_id_for_staleness

function nipux_cli/worker_prompt_context.py:786

No docstring.

Calls: _normalize_claim_text, record.get, str, strip

### _experiments_for_prompt

function nipux_cli/worker_prompt_context.py:794

No docstring.

Calls: _clip_text, _metadata_list, bool, experiment.get, format_metric_value, join, len, lines.append, sorted, status_counts.get, status_counts.items, str

### _operator_message_visible_in_prompt

function nipux_cli/worker_prompt_context.py:868

No docstring.

Calls: entry.get, lower, replace, str, strip

### _metadata_list

function nipux_cli/worker_prompt_context.py:875

No docstring.

Calls: isinstance, job.get, metadata.get

### _timeline_event_for_prompt

function nipux_cli/worker_prompt_context.py:883

No docstring.

Calls: event.get, isinstance, join, lower, metadata.get, split, str, strip, title.lower

### _as_float

function nipux_cli/worker_prompt_context.py:910

No docstring.

Calls: float

### _as_int

function nipux_cli/worker_prompt_context.py:917

No docstring.

Calls: int

### compact

function nipux_cli/worker_prompt_format.py:15

No docstring.

Calls: isinstance, join, json.dumps, len, text.split

### clip_text

function nipux_cli/worker_prompt_format.py:21

No docstring.

Calls: join, len, max, rstrip, split, str

### format_step_for_prompt

function nipux_cli/worker_prompt_format.py:28

No docstring.

Calls: compact, input_data.get, isinstance, join, observation_for_prompt, pieces.append, step.get

### observation_for_prompt

function nipux_cli/worker_prompt_format.py:43

No docstring.

Calls: anti_bot_reason, artifact.get, browser_candidates_for_prompt, clean_prompt_candidate_path, clip_text, compact, data.get, evidence_grounding.get, experiment.get, format_metric_value, isinstance, join, len, lesson.get, node.get, output.get

### clean_prompt_candidate_path

function nipux_cli/worker_prompt_format.py:172

No docstring.

Calls: Path, any, ch.isalpha, name.startswith, raw.startswith, re.match, rstrip, str, strip

### browser_candidates_for_prompt

function nipux_cli/worker_prompt_format.py:185

No docstring.

Calls: _looks_like_metric_cell, _looks_like_service_description, candidates.append, data.get, isinstance, item.get, join, len, name.lower, output.get, refs.items, seen.add, set, split, str, strip

### _looks_like_metric_cell

function nipux_cli/worker_prompt_format.py:213

No docstring.

Calls: bool, name.strip, re.fullmatch

### _looks_like_service_description

function nipux_cli/worker_prompt_format.py:218

No docstring.

Calls: any, len, name.lower, text.split

### summarize_tool_result

function nipux_cli/worker_tool_summary.py:10

No docstring.

Calls: args.get, data.get, experiment.get, format_metric_value, isinstance, item.get, join, len, lesson.get, page.get, result.get, roadmap.get, source.get, str, update.get, validation.get

### turn_usage_metadata

function nipux_cli/worker_usage.py:11

No docstring.

Calls: _as_int, bool, dict, estimate_token_count, isinstance, json.dumps, len, max, round, usage.get, usage.setdefault

### estimate_token_count

function nipux_cli/worker_usage.py:38

No docstring.

Calls: len, max

### _as_int

function nipux_cli/worker_usage.py:44

No docstring.

Calls: float, int

### SourceFile

class scripts/generate_project_atlas.py:28

No docstring.

Calls: none

### Symbol

class scripts/generate_project_atlas.py:37

No docstring.

Calls: none

### Prompt

class scripts/generate_project_atlas.py:48

No docstring.

Calls: none

### main

function scripts/generate_project_atlas.py:56

No docstring.

Calls: OUT.parent.mkdir, OUT.relative_to, OUT.write_text, extract_prompts, extract_symbols, extract_tables, extract_tools, git, len, load_source_files, print, render

### load_source_files

function scripts/generate_project_atlas.py:69

No docstring.

Calls: SourceFile, ast.parse, files.append, full.is_file, full.read_text, str, text.splitlines, tracked_paths

### tracked_paths

function scripts/generate_project_atlas.py:90

No docstring.

Calls: git, line.strip, output.splitlines, sorted

### git

function scripts/generate_project_atlas.py:97

No docstring.

Calls: result.stdout.strip, subprocess.run

### extract_symbols

function scripts/generate_project_atlas.py:105

No docstring.

Calls: Symbol, ast.get_docstring, ast.walk, call_names, getattr, isinstance, sorted, symbols.append

### call_names

function scripts/generate_project_atlas.py:127

No docstring.

Calls: ast.walk, dotted_name, isinstance, names.add, set

### dotted_name

function scripts/generate_project_atlas.py:138

No docstring.

Calls: dotted_name, isinstance

### extract_prompts

function scripts/generate_project_atlas.py:147

No docstring.

Calls: Prompt, any, assignment_names, ast.walk, deduped.get, deduped.values, getattr, is_prompt_name, is_prompt_text, isinstance, join, literal_string, prompts.append, sorted

### assignment_names

function scripts/generate_project_atlas.py:188

No docstring.

Calls: isinstance, names.append

### literal_string

function scripts/generate_project_atlas.py:199

No docstring.

Calls: isinstance, join

### is_prompt_name

function scripts/generate_project_atlas.py:207

No docstring.

Calls: any, name.upper, upper.endswith

### is_prompt_text

function scripts/generate_project_atlas.py:212

No docstring.

Calls: clean.lower, join, len, text.split

### extract_tools

function scripts/generate_project_atlas.py:225

No docstring.

Calls: count, join, match.group, match.start, next, pattern.finditer, re.compile, split, str, tools.append

### extract_tables

function scripts/generate_project_atlas.py:235

No docstring.

Calls: count, line.strip, line.upper, match.group, match.start, next, re.finditer, rstrip, splitlines, startswith, tables.append

### render

function scripts/generate_project_atlas.py:245

No docstring.

Calls: architecture_nodes, esc, join, len, render_file_card, render_prompt, render_review_points, render_source_file, render_symbol, render_table, runtime_flow, source.path.endswith, sum, test_cards

### architecture_nodes

function scripts/generate_project_atlas.py:450

No docstring.

Calls: esc, join

### runtime_flow

function scripts/generate_project_atlas.py:469

No docstring.

Calls: esc, join

### render_file_card

function scripts/generate_project_atlas.py:483

No docstring.

Calls: esc, join, len, module_doc, module_imports, short, source.text.count

### module_doc

function scripts/generate_project_atlas.py:496

No docstring.

Calls: ast.get_docstring, line.strip, stripped.startswith

### module_imports

function scripts/generate_project_atlas.py:506

No docstring.

Calls: isinstance, names.append, names.extend

### render_source_file

function scripts/generate_project_atlas.py:519

No docstring.

Calls: enumerate, esc, join, len, redact_source_line

### redact_source_line

function scripts/generate_project_atlas.py:532

No docstring.

Calls: SENSITIVE_ASSIGNMENT_RE.match, match.groups

### render_symbol

function scripts/generate_project_atlas.py:540

No docstring.

Calls: esc, join, short

### render_prompt

function scripts/generate_project_atlas.py:551

No docstring.

Calls: esc, len

### render_table

function scripts/generate_project_atlas.py:560

No docstring.

Calls: esc, join

### test_cards

function scripts/generate_project_atlas.py:569

No docstring.

Calls: ast.walk, cards.append, esc, isinstance, join, len, node.name.startswith, short, source.path.startswith

### render_review_points

function scripts/generate_project_atlas.py:584

No docstring.

Calls: esc, join, len, sorted

### short

function scripts/generate_project_atlas.py:607

No docstring.

Calls: join, len, max, split, str

### esc

function scripts/generate_project_atlas.py:614

No docstring.

Calls: html.escape, str

### main

function scripts/live_memory_graph_smoke.py:32

No docstring.

Calls: AgentDB, AppConfig, ModelConfig, Path, RuntimeConfig, ToolAccessConfig, _execution_summary, _finish, _seed_metadata, argparse.ArgumentParser, args.base_url.rstrip, config.ensure_dirs, db.close, db.create_job, db.get_job, db.update_job_status

### _seed_metadata

function scripts/live_memory_graph_smoke.py:121

No docstring.

Calls: none

### _execution_summary

function scripts/live_memory_graph_smoke.py:187

No docstring.

Calls: isinstance, result.get

### _finish

function scripts/live_memory_graph_smoke.py:200

No docstring.

Calls: _human_summary, json.dumps, payload.get, print

### _human_summary

function scripts/live_memory_graph_smoke.py:208

No docstring.

Calls: bool, isinstance, item.get, join, lines.append, payload.get

### Cell

class scripts/render_nipux_ascii_video.py:156

No docstring.

Calls: dataclass

### TextGrid

class scripts/render_nipux_ascii_video.py:161

No docstring.

Calls: Cell, enumerate, len, range, self.put, self.set

### __init__

function scripts/render_nipux_ascii_video.py:162

No docstring.

Calls: Cell, range

### set

function scripts/render_nipux_ascii_video.py:165

No docstring.

Calls: Cell

### put

function scripts/render_nipux_ascii_video.py:169

No docstring.

Calls: enumerate, self.set

### center

function scripts/render_nipux_ascii_video.py:173

No docstring.

Calls: len, self.put

### box

function scripts/render_nipux_ascii_video.py:176

No docstring.

Calls: range, self.put, self.set

### clamp

function scripts/render_nipux_ascii_video.py:188

No docstring.

Calls: max, min

### ease

function scripts/render_nipux_ascii_video.py:192

No docstring.

Calls: clamp

### mix

function scripts/render_nipux_ascii_video.py:197

No docstring.

Calls: clamp, int, range, tuple

### logo_origin

function scripts/render_nipux_ascii_video.py:202

No docstring.

Calls: len, max

### put_logo

function scripts/render_nipux_ascii_video.py:207

No docstring.

Calls: enumerate, grid.set, logo_origin, mix, random.Random, rng.choice, rng.random

### put_collapsing_logo

function scripts/render_nipux_ascii_video.py:228

No docstring.

Calls: enumerate, grid.set, logo_origin, math.sin, mix, random.Random, rng.choice, rng.random, round

### put_progress_bar

function scripts/render_nipux_ascii_video.py:251

No docstring.

Calls: clamp, grid.put, int, round

### put_rain

function scripts/render_nipux_ascii_video.py:260

No docstring.

Calls: choice, grid.set, max, mix, random, random.Random, range, rng.randint

### put_boot_lines

function scripts/render_nipux_ascii_video.py:276

No docstring.

Calls: clamp, enumerate, grid.put, int, len, put_progress_bar

### put_cli

function scripts/render_nipux_ascii_video.py:291

No docstring.

Calls: clamp, grid.put, grid.set, int, len, put_progress_bar

### build_grid

function scripts/render_nipux_ascii_video.py:313

No docstring.

Calls: TextGrid, ease, grid.box, grid.center, grid.put, int, put_boot_lines, put_cli, put_collapsing_logo, put_logo, put_rain

### draw_rect

function scripts/render_nipux_ascii_video.py:358

No docstring.

Calls: bytes, len, max, min, range

### build_base_frame

function scripts/render_nipux_ascii_video.py:371

No docstring.

Calls: bytearray, bytes, draw_rect, len, range

### glyph_for

function scripts/render_nipux_ascii_video.py:398

No docstring.

Calls: GLYPHS.get, char.upper

### draw_glyph

function scripts/render_nipux_ascii_video.py:405

No docstring.

Calls: draw_rect, enumerate, glyph_for, int, max, tuple

### render_frame

function scripts/render_nipux_ascii_video.py:422

No docstring.

Calls: build_grid, bytearray, bytes, draw_glyph, draw_rect, enumerate, int, sum

### render_video

function scripts/render_nipux_ascii_video.py:441

No docstring.

Calls: SystemExit, decode, int, output.parent.mkdir, poster.parent.mkdir, print, process.stderr.read, process.stdin.close, process.stdin.write, process.wait, range, render_frame, shutil.which, str, subprocess.Popen, subprocess.run

### parse_args

function scripts/render_nipux_ascii_video.py:508

No docstring.

Calls: Path, argparse.ArgumentParser, parser.add_argument, parser.parse_args

### main

function scripts/render_nipux_ascii_video.py:515

No docstring.

Calls: parse_args, print, render_video

### test_artifact_store_writes_reads_and_searches

function tests/nipux_cli/test_artifacts.py:7

No docstring.

Calls: AgentDB, ArtifactStore, db.close, db.create_job, startswith, store.read_text, store.search_text, store.write_text

### test_artifact_store_rejects_paths_outside_home

function tests/nipux_cli/test_artifacts.py:28

No docstring.

Calls: ArtifactStore, outside.write_text, pytest.raises, store.read_text, str

### test_session_name_is_stable_and_safe

function tests/nipux_cli/test_browser_web.py:11

No docstring.

Calls: _session_name

### test_long_session_name_is_short_and_hashed

function tests/nipux_cli/test_browser_web.py:15

No docstring.

Calls: _session_name, _socket_dir, len, name.startswith, str

### test_strip_html_removes_scripts_and_keeps_text

function tests/nipux_cli/test_browser_web.py:25

No docstring.

Calls: _strip_html

### test_browser_marks_anti_bot_interstitial_as_warning

function tests/nipux_cli/test_browser_web.py:32

No docstring.

Calls: _annotate_source_quality

### test_browser_marks_captcha_block_as_warning

function tests/nipux_cli/test_browser_web.py:45

No docstring.

Calls: _annotate_source_quality

### test_web_extract_marks_anti_bot_pages_as_warning

function tests/nipux_cli/test_browser_web.py:56

No docstring.

Calls: monkeypatch.setattr, web.web_extract

### fake_request

function tests/nipux_cli/test_browser_web.py:59

No docstring.

Calls: none

### test_browser_tool_uses_native_wrapper

function tests/nipux_cli/test_browser_web.py:73

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.close, db.create_job, json.loads, monkeypatch.setattr

### fake_navigate

function tests/nipux_cli/test_browser_web.py:82

No docstring.

Calls: none

### test_browser_click_adds_recovery_snapshot_for_stale_ref

function tests/nipux_cli/test_browser_web.py:93

No docstring.

Calls: AppConfig, RuntimeConfig, browser.click, calls.append, monkeypatch.setattr

### fake_command

function tests/nipux_cli/test_browser_web.py:99

No docstring.

Calls: calls.append

### _mode

function tests/nipux_cli/test_cli.py:77

No docstring.

Calls: path.stat

### _mark_test_model_ready

function tests/nipux_cli/test_cli.py:81

No docstring.

Calls: _mark_model_setup_verified, ensure_dirs, load_config

### test_cli_has_operator_commands

function tests/nipux_cli/test_cli.py:86

No docstring.

Calls: build_parser, parser.parse_args

### test_cli_version_flag

function tests/nipux_cli/test_cli.py:146

No docstring.

Calls: capsys.readouterr, main

### test_main_catches_keyboard_interrupt_without_traceback

function tests/nipux_cli/test_cli.py:155

No docstring.

Calls: capsys.readouterr, main, monkeypatch.setattr

### interrupt

function tests/nipux_cli/test_cli.py:156

No docstring.

Calls: none

### test_python_module_entrypoint_uses_cli_main

function tests/nipux_cli/test_cli.py:166

No docstring.

Calls: none

### test_init_openrouter_writes_secret_free_config_and_env_template

function tests/nipux_cli/test_cli.py:172

No docstring.

Calls: _mode, capsys.readouterr, endswith, env_text.strip, main, monkeypatch.setenv, read_text, str

### test_init_defaults_to_local_endpoint

function tests/nipux_cli/test_cli.py:191

No docstring.

Calls: endswith, env_text.strip, main, monkeypatch.setenv, read_text, str

### test_init_openrouter_defaults_to_generic_route

function tests/nipux_cli/test_cli.py:204

No docstring.

Calls: main, monkeypatch.setenv, read_text, str

### test_shell_freeform_text_adds_operator_message

function tests/nipux_cli/test_cli.py:215

No docstring.

Calls: AgentDB, _run_shell_line, capsys.readouterr, db.close, db.create_job, db.get_job, monkeypatch.setenv, str

### test_main_no_args_enters_chat_first_home

function tests/nipux_cli/test_cli.py:238

No docstring.

Calls: AgentDB, _mark_test_model_ready, capsys.readouterr, db.append_agent_update, db.append_operator_message, db.close, db.create_job, main, monkeypatch.setattr, monkeypatch.setenv, str

### eof_input

function tests/nipux_cli/test_cli.py:249

No docstring.

Calls: none

### test_main_no_args_with_no_jobs_requires_setup_frame

function tests/nipux_cli/test_cli.py:262

No docstring.

Calls: capsys.readouterr, main, monkeypatch.setattr, monkeypatch.setenv, str

### eof_input

function tests/nipux_cli/test_cli.py:265

No docstring.

Calls: none

### test_main_no_args_with_old_setup_marker_still_requires_model_verification

function tests/nipux_cli/test_cli.py:282

No docstring.

Calls: capsys.readouterr, ensure_dirs, load_config, main, monkeypatch.setenv, str, write_shell_state

### test_main_no_args_after_setup_complete_does_not_reopen_setup

function tests/nipux_cli/test_cli.py:296

No docstring.

Calls: _mark_test_model_ready, capsys.readouterr, main, monkeypatch.setenv, str

### test_main_no_args_autoverifies_existing_model_config

function tests/nipux_cli/test_cli.py:308

No docstring.

Calls: Check, _model_setup_verified, capsys.readouterr, load_config, main, monkeypatch.setattr, monkeypatch.setenv, str, tmp_path.mkdir, write_text

### fake_doctor

function tests/nipux_cli/test_cli.py:320

No docstring.

Calls: Check

### test_main_no_args_enters_setup_when_existing_model_config_fails

function tests/nipux_cli/test_cli.py:341

No docstring.

Calls: Check, _model_setup_verified, capsys.readouterr, load_config, main, monkeypatch.setattr, monkeypatch.setenv, str, tmp_path.mkdir, write_text

### fake_doctor

function tests/nipux_cli/test_cli.py:352

No docstring.

Calls: Check

### test_main_no_args_keeps_workspace_locked_after_completed_setup_if_provider_fails

function tests/nipux_cli/test_cli.py:366

No docstring.

Calls: Check, _model_setup_verified, _write_shell_state, capsys.readouterr, load_config, main, monkeypatch.setattr, monkeypatch.setenv, str, tmp_path.mkdir, write_text

### fake_doctor

function tests/nipux_cli/test_cli.py:378

No docstring.

Calls: Check

### test_first_run_refuses_job_before_model_is_verified

function tests/nipux_cli/test_cli.py:392

No docstring.

Calls: AgentDB, _handle_first_run_frame_line, db.close, db.list_jobs, monkeypatch.setenv, str

### test_doctor_check_model_marks_model_setup_verified

function tests/nipux_cli/test_cli.py:412

No docstring.

Calls: Check, _model_setup_verified, args.func, build_parser, capsys.readouterr, load_config, monkeypatch.setattr, monkeypatch.setenv, parse_args, str

### fake_doctor

function tests/nipux_cli/test_cli.py:415

No docstring.

Calls: Check

### test_first_run_doctor_failure_shows_inline_fix_commands

function tests/nipux_cli/test_cli.py:434

No docstring.

Calls: Check, _verify_model_setup_from_first_run, join, monkeypatch.setattr, monkeypatch.setenv, str

### fake_doctor

function tests/nipux_cli/test_cli.py:437

No docstring.

Calls: Check

### test_setting_change_clears_model_setup_verification

function tests/nipux_cli/test_cli.py:453

No docstring.

Calls: _inline_setting_notice, _mark_test_model_ready, _model_setup_verified, load_config, monkeypatch.setenv, str

### test_first_run_menu_blocks_job_creation_until_workspace_chat

function tests/nipux_cli/test_cli.py:463

No docstring.

Calls: AgentDB, _handle_first_run_menu_line, _mark_test_model_ready, capsys.readouterr, db.close, db.list_jobs, monkeypatch.setenv, str

### test_first_run_plain_greeting_does_not_create_job

function tests/nipux_cli/test_cli.py:478

No docstring.

Calls: AgentDB, _handle_first_run_menu_line, capsys.readouterr, db.close, db.list_jobs, monkeypatch.setenv, str

### test_first_run_frame_uses_full_screen_ui_not_banner

function tests/nipux_cli/test_cli.py:492

No docstring.

Calls: _build_first_run_frame, frame.splitlines, lower, monkeypatch.setenv, str

### test_first_run_frame_hides_command_popup_during_setup

function tests/nipux_cli/test_cli.py:514

No docstring.

Calls: _build_first_run_frame, monkeypatch.setenv, str

### test_first_run_frame_walks_setup_screens

function tests/nipux_cli/test_cli.py:528

No docstring.

Calls: _build_first_run_frame, monkeypatch.setenv, str

### test_first_run_frame_does_not_use_command_palette_for_setup

function tests/nipux_cli/test_cli.py:557

No docstring.

Calls: _build_first_run_frame, monkeypatch.setenv, str

### test_settings_editor_persists_model_config

function tests/nipux_cli/test_cli.py:568

No docstring.

Calls: _config_field_value, _inline_setting_notice, _save_config_field, monkeypatch.setenv, read_text, str

### test_slash_autocomplete_filters_commands

function tests/nipux_cli/test_cli.py:583

No docstring.

Calls: _autocomplete_slash, _cycle_slash, _slash_completion_for_submit, _slash_suggestion_lines, join

### test_terminal_escape_decodes_arrows_and_mouse_click

function tests/nipux_cli/test_cli.py:636

No docstring.

Calls: _decode_terminal_escape

### test_first_run_click_maps_right_pane_actions

function tests/nipux_cli/test_cli.py:647

No docstring.

Calls: _first_run_click_action, monkeypatch.setattr

### test_first_run_arrow_navigation_changes_setup_screens

function tests/nipux_cli/test_cli.py:657

No docstring.

Calls: _directional_first_run_action

### test_frame_next_job_cycles_jobs

function tests/nipux_cli/test_cli.py:683

No docstring.

Calls: _frame_next_job_id

### test_frame_refresh_slows_background_updates_while_typing

function tests/nipux_cli/test_cli.py:691

No docstring.

Calls: _frame_refresh_interval

### test_first_run_empty_submit_without_actions_does_not_crash

function tests/nipux_cli/test_cli.py:695

No docstring.

Calls: _FirstRunRuntimeDeps, _submit_first_run_line

### test_first_run_required_edit_cancel_and_clear_stay_on_same_field

function tests/nipux_cli/test_cli.py:710

No docstring.

Calls: _handle_first_run_edit_input, join

### test_chat_settings_edit_supports_ctrl_u_clear

function tests/nipux_cli/test_cli.py:739

No docstring.

Calls: _handle_chat_edit_input

### test_first_run_render_failure_uses_safe_mode

function tests/nipux_cli/test_cli.py:753

No docstring.

Calls: RuntimeError, _FirstRunRuntimeDeps, _safe_first_run_render_frame, capsys.readouterr, join, throw

### test_chat_submit_failure_stays_in_frame

function tests/nipux_cli/test_cli.py:778

No docstring.

Calls: RuntimeError, _ChatFrameDeps, _drain_chat_async_notices, _handle_chat_submit, async_messages.get, async_messages.put, join, queue.Queue, throw

### test_chat_submit_plain_message_returns_without_waiting_for_model

function tests/nipux_cli/test_cli.py:819

No docstring.

Calls: _ChatFrameDeps, _handle_chat_submit, async_messages.get, join, queue.Queue, time.monotonic, time.sleep

### slow_chat

function tests/nipux_cli/test_cli.py:823

No docstring.

Calls: time.sleep

### test_chat_submit_plain_message_renders_thinking_notice_without_echoing_message

function tests/nipux_cli/test_cli.py:857

No docstring.

Calls: _ChatFrameDeps, _display_chat_notices, _handle_chat_submit, chat_pane_lines, join, queue.Queue

### test_chat_submit_waiting_command_output_becomes_animation

function tests/nipux_cli/test_cli.py:891

No docstring.

Calls: _ChatFrameDeps, _display_chat_notices, _handle_chat_submit, chat_pane_lines, join

### test_chat_submit_new_refreshes_focused_job_from_shell_state

function tests/nipux_cli/test_cli.py:926

No docstring.

Calls: _ChatFrameDeps, _handle_chat_submit, join, loaded.append

### load_snapshot

function tests/nipux_cli/test_cli.py:931

No docstring.

Calls: loaded.append

### test_workspace_chat_submit_new_keeps_workspace_chat_left_pane

function tests/nipux_cli/test_cli.py:962

No docstring.

Calls: _ChatFrameDeps, _handle_chat_submit, join, loaded.append

### load_snapshot

function tests/nipux_cli/test_cli.py:972

No docstring.

Calls: loaded.append

### test_chat_render_failure_uses_safe_mode

function tests/nipux_cli/test_cli.py:1004

No docstring.

Calls: RuntimeError, _ChatFrameDeps, _safe_chat_render_frame, capsys.readouterr, join, throw

### test_chat_help_has_config_slash_commands_without_settings_page

function tests/nipux_cli/test_cli.py:1034

No docstring.

Calls: AgentDB, _chat_handle_line, capsys.readouterr, db.close, db.create_job, monkeypatch.setenv, str

### test_chat_slash_palette_matches_public_chat_commands

function tests/nipux_cli/test_cli.py:1063

No docstring.

Calls: len

### test_workspace_help_is_minimal_and_actionable

function tests/nipux_cli/test_cli.py:1133

No docstring.

Calls: _capture_chat_command, _mark_test_model_ready, monkeypatch.setenv, str

### test_first_run_slash_palette_matches_setup_commands

function tests/nipux_cli/test_cli.py:1148

No docstring.

Calls: len

### test_chat_settings_slash_commands_persist_config

function tests/nipux_cli/test_cli.py:1185

No docstring.

Calls: AgentDB, _chat_handle_line, _config_field_value, _mode, capsys.readouterr, db.close, db.create_job, monkeypatch.setenv, read_text, str

### test_chat_init_slash_command_does_not_crash

function tests/nipux_cli/test_cli.py:1251

No docstring.

Calls: AgentDB, _chat_handle_line, capsys.readouterr, db.close, db.create_job, exists, monkeypatch.setenv, str

### test_chat_config_slash_command_summarizes_runtime_without_secret

function tests/nipux_cli/test_cli.py:1267

No docstring.

Calls: AgentDB, _chat_handle_line, capsys.readouterr, db.close, db.create_job, monkeypatch.setenv, str, write_text

### test_chat_usage_slash_command_reports_tokens

function tests/nipux_cli/test_cli.py:1308

No docstring.

Calls: AgentDB, _chat_handle_line, capsys.readouterr, db.append_event, db.close, db.create_job, monkeypatch.setenv, str

### test_chat_usage_estimates_cost_from_configured_rates

function tests/nipux_cli/test_cli.py:1337

No docstring.

Calls: AgentDB, _chat_handle_line, capsys.readouterr, db.append_event, db.close, db.create_job, monkeypatch.setenv, str, write_text

### test_chat_usage_shows_configured_job_cost_limit

function tests/nipux_cli/test_cli.py:1374

No docstring.

Calls: AgentDB, _chat_handle_line, capsys.readouterr, db.append_event, db.close, db.create_job, monkeypatch.setenv, str, write_text

### test_first_run_settings_slash_commands_persist_config

function tests/nipux_cli/test_cli.py:1404

No docstring.

Calls: _config_field_value, _handle_first_run_frame_line, any, isinstance, monkeypatch.setenv, str

### test_first_run_local_connector_action_sets_generic_local_endpoint

function tests/nipux_cli/test_cli.py:1415

No docstring.

Calls: _config_field_value, _handle_first_run_action, any, isinstance, monkeypatch.setenv, str

### test_first_run_access_action_toggles_generic_tools

function tests/nipux_cli/test_cli.py:1429

No docstring.

Calls: _config_field_value, _handle_first_run_action, any, isinstance, monkeypatch.setenv, str

### test_first_run_doctor_success_opens_workspace_chat

function tests/nipux_cli/test_cli.py:1440

No docstring.

Calls: _handle_first_run_action, _mark_test_model_ready, monkeypatch.setattr, monkeypatch.setenv, str

### fake_verify

function tests/nipux_cli/test_cli.py:1443

No docstring.

Calls: _mark_test_model_ready

### test_first_run_open_workspace_action_requires_verified_model

function tests/nipux_cli/test_cli.py:1455

No docstring.

Calls: _handle_first_run_action, monkeypatch.setenv, str

### test_first_run_open_workspace_action_opens_after_verified_model

function tests/nipux_cli/test_cli.py:1464

No docstring.

Calls: _handle_first_run_action, _mark_test_model_ready, monkeypatch.setenv, str

### test_workspace_frame_snapshot_exists_without_jobs

function tests/nipux_cli/test_cli.py:1474

No docstring.

Calls: _build_chat_frame, _load_frame_snapshot, _mark_test_model_ready, monkeypatch.setenv, str

### test_workspace_frame_right_pane_tracks_focused_worker

function tests/nipux_cli/test_cli.py:1490

No docstring.

Calls: AgentDB, _build_chat_frame, _load_frame_snapshot, _mark_test_model_ready, _write_shell_state, db.add_artifact, db.close, db.create_job, monkeypatch.setenv, str

### test_workspace_slash_new_creates_and_focuses_job

function tests/nipux_cli/test_cli.py:1518

No docstring.

Calls: AgentDB, _capture_chat_command, _mark_test_model_ready, _read_shell_state, db.close, db.list_jobs, get, len, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:1523

No docstring.

Calls: started.update

### test_workspace_chat_job_dossier_includes_progress_outputs_and_outcomes

function tests/nipux_cli/test_cli.py:1545

No docstring.

Calls: AgentDB, _workspace_chat_job_dossier, db.add_artifact, db.add_step, db.append_experiment_record, db.append_finding_record, db.append_source_record, db.append_task_record, db.close, db.create_job, db.finish_step, db.get_job, db.start_run

### test_workspace_run_with_objective_creates_worker_when_no_job_matches

function tests/nipux_cli/test_cli.py:1585

No docstring.

Calls: AgentDB, _capture_chat_command, _mark_test_model_ready, db.close, db.list_jobs, len, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:1590

No docstring.

Calls: started.update

### test_workspace_run_with_existing_job_does_not_create_duplicate

function tests/nipux_cli/test_cli.py:1613

No docstring.

Calls: AgentDB, _capture_chat_command, _mark_test_model_ready, _read_shell_state, db.close, db.create_job, db.list_jobs, get, len, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:1623

No docstring.

Calls: started.update

### test_workspace_start_with_existing_job_runs_without_parser_error

function tests/nipux_cli/test_cli.py:1642

No docstring.

Calls: AgentDB, _capture_chat_command, _mark_test_model_ready, _read_shell_state, db.close, db.create_job, db.list_jobs, get, len, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:1652

No docstring.

Calls: started.update

### test_workspace_slash_new_without_objective_is_minimal

function tests/nipux_cli/test_cli.py:1672

No docstring.

Calls: _capture_chat_command, _mark_test_model_ready, monkeypatch.setenv, output.lower, output.strip, str

### test_workspace_slash_new_hides_model_preflight_noise

function tests/nipux_cli/test_cli.py:1683

No docstring.

Calls: _capture_chat_command, _mark_test_model_ready, monkeypatch.setattr, monkeypatch.setenv, print, str

### fake_start

function tests/nipux_cli/test_cli.py:1687

No docstring.

Calls: print

### test_workspace_settings_slash_commands_persist_config

function tests/nipux_cli/test_cli.py:1703

No docstring.

Calls: AgentDB, _capture_chat_command, _config_field_value, _mark_test_model_ready, db.close, db.list_jobs, monkeypatch.setenv, str

### test_workspace_settings_slash_command_summarizes_config

function tests/nipux_cli/test_cli.py:1720

No docstring.

Calls: _capture_chat_command, _mark_test_model_ready, monkeypatch.setenv, str, write_text

### test_workspace_natural_control_phrase_uses_mapped_command

function tests/nipux_cli/test_cli.py:1746

No docstring.

Calls: AgentDB, _capture_chat_command, _mark_test_model_ready, db.close, db.list_jobs, monkeypatch.setenv, str

### test_workspace_natural_settings_phrase_opens_settings_summary

function tests/nipux_cli/test_cli.py:1763

No docstring.

Calls: _capture_chat_command, _mark_test_model_ready, monkeypatch.setenv, str, write_text

### test_workspace_how_to_start_job_question_uses_local_help

function tests/nipux_cli/test_cli.py:1787

No docstring.

Calls: AssertionError, _capture_chat_command, _mark_test_model_ready, monkeypatch.setattr, monkeypatch.setenv, str

### fail_model

function tests/nipux_cli/test_cli.py:1791

No docstring.

Calls: AssertionError

### test_workspace_chat_connection_error_is_operator_friendly

function tests/nipux_cli/test_cli.py:1803

No docstring.

Calls: RuntimeError, _handle_workspace_chat_message, _load_frame_snapshot, _mark_test_model_ready, event.get, join, monkeypatch.setattr, monkeypatch.setenv, str

### raise_connection

function tests/nipux_cli/test_cli.py:1807

No docstring.

Calls: RuntimeError

### test_chat_start_reports_model_provider_not_ready

function tests/nipux_cli/test_cli.py:1824

No docstring.

Calls: AgentDB, _chat_handle_line, _mark_test_model_ready, capsys.readouterr, db.close, db.create_job, monkeypatch.setattr, monkeypatch.setenv, print, str

### fake_start

function tests/nipux_cli/test_cli.py:1833

No docstring.

Calls: print

### test_chat_doctor_checks_configured_model

function tests/nipux_cli/test_cli.py:1847

No docstring.

Calls: AgentDB, Check, _chat_handle_line, _mark_test_model_ready, capsys.readouterr, db.close, db.create_job, monkeypatch.setattr, monkeypatch.setenv, str

### fake_doctor

function tests/nipux_cli/test_cli.py:1858

No docstring.

Calls: Check

### test_workspace_chat_control_phrase_runs_job_command

function tests/nipux_cli/test_cli.py:1870

No docstring.

Calls: AgentDB, _handle_workspace_chat_message, _mark_test_model_ready, _read_shell_state, any, db.close, db.create_job, db.get_job, get, monkeypatch.setenv, str

### test_shell_ls_alias_lists_jobs_instead_of_steering

function tests/nipux_cli/test_cli.py:1898

No docstring.

Calls: AgentDB, _run_shell_line, capsys.readouterr, db.close, db.create_job, monkeypatch.setenv, str

### test_roadmap_command_renders_roadmap

function tests/nipux_cli/test_cli.py:1913

No docstring.

Calls: AgentDB, capsys.readouterr, db.append_roadmap_record, db.close, db.create_job, main, monkeypatch.setenv, str

### test_shell_focus_controls_default_steering_job

function tests/nipux_cli/test_cli.py:1942

No docstring.

Calls: AgentDB, _run_shell_line, capsys.readouterr, db.close, db.create_job, db.get_job, get, monkeypatch.setenv, str

### test_shell_rename_updates_job_title_and_program

function tests/nipux_cli/test_cli.py:1966

No docstring.

Calls: AgentDB, _run_shell_line, capsys.readouterr, db.close, db.create_job, db.get_job, monkeypatch.setenv, program.parent.mkdir, program.read_text, program.write_text, startswith, str

### test_shell_delete_removes_job_and_artifact_dir

function tests/nipux_cli/test_cli.py:1990

No docstring.

Calls: AgentDB, ArtifactStore, AssertionError, _run_shell_line, artifact_path.exists, capsys.readouterr, db.add_step, db.close, db.create_job, db.get_job, db.start_run, exists, monkeypatch.setenv, store.write_text, str

### test_shell_help_has_no_examples_or_control_run_sections

function tests/nipux_cli/test_cli.py:2029

No docstring.

Calls: _print_shell_help, capsys.readouterr

### test_update_checkout_falls_back_to_tool_install_for_non_git_path

function tests/nipux_cli/test_cli.py:2043

No docstring.

Calls: _update_checkout, join, monkeypatch.setattr, subprocess.CompletedProcess

### runner

function tests/nipux_cli/test_cli.py:2046

No docstring.

Calls: subprocess.CompletedProcess

### test_update_checkout_upgrades_uv_tool_when_installed_package

function tests/nipux_cli/test_cli.py:2068

No docstring.

Calls: _update_checkout, calls.append, join, monkeypatch.setattr, subprocess.CompletedProcess, tuple

### runner

function tests/nipux_cli/test_cli.py:2073

No docstring.

Calls: calls.append, subprocess.CompletedProcess, tuple

### test_update_checkout_fast_forwards_git_checkout

function tests/nipux_cli/test_cli.py:2099

No docstring.

Calls: AssertionError, _update_checkout, calls.append, join, mkdir, repo.mkdir, str, subprocess.CompletedProcess, tuple

### runner

function tests/nipux_cli/test_cli.py:2106

No docstring.

Calls: AssertionError, calls.append, str, subprocess.CompletedProcess, tuple

### test_update_checkout_verifies_installed_command

function tests/nipux_cli/test_cli.py:2132

No docstring.

Calls: _update_checkout, calls.append, join, monkeypatch.setattr, subprocess.CompletedProcess, tuple

### which

function tests/nipux_cli/test_cli.py:2135

No docstring.

Calls: none

### runner

function tests/nipux_cli/test_cli.py:2145

No docstring.

Calls: calls.append, subprocess.CompletedProcess, tuple

### test_update_command_reports_no_restart_when_daemon_is_stopped

function tests/nipux_cli/test_cli.py:2158

No docstring.

Calls: args.func, build_parser, capsys.readouterr, monkeypatch.setattr, monkeypatch.setenv, parse_args, str

### test_update_command_restarts_running_daemon

function tests/nipux_cli/test_cli.py:2174

No docstring.

Calls: args.func, build_parser, capsys.readouterr, monkeypatch.setattr, monkeypatch.setenv, parse_args, print, str

### fake_restart

function tests/nipux_cli/test_cli.py:2183

No docstring.

Calls: print

### test_update_command_no_restart_flag_skips_running_daemon

function tests/nipux_cli/test_cli.py:2199

No docstring.

Calls: AssertionError, args.func, build_parser, capsys.readouterr, monkeypatch.setattr, monkeypatch.setenv, parse_args, str, throw

### test_uninstall_dry_run_removes_installed_tool_by_default

function tests/nipux_cli/test_cli.py:2214

No docstring.

Calls: args.func, build_parser, capsys.readouterr, monkeypatch.setenv, parse_args, str

### test_uninstall_keep_tool_skips_tool_removal

function tests/nipux_cli/test_cli.py:2226

No docstring.

Calls: args.func, build_parser, capsys.readouterr, monkeypatch.setenv, parse_args, str

### test_uninstall_runtime_skips_missing_systemd_service_without_runner_noise

function tests/nipux_cli/test_cli.py:2237

No docstring.

Calls: calls.append, join, monkeypatch.setattr, monkeypatch.setenv, runtime.mkdir, str, subprocess.CompletedProcess, uninstall_runtime

### which

function tests/nipux_cli/test_cli.py:2245

No docstring.

Calls: none

### runner

function tests/nipux_cli/test_cli.py:2250

No docstring.

Calls: calls.append, subprocess.CompletedProcess

### test_chat_clear_does_not_queue_operator_message

function tests/nipux_cli/test_cli.py:2265

No docstring.

Calls: AgentDB, _chat_handle_line, capsys.readouterr, db.close, db.create_job, db.get_job, get, monkeypatch.setenv, str

### test_minimal_live_event_line_summarizes_tool_steps

function tests/nipux_cli/test_cli.py:2285

No docstring.

Calls: _minimal_live_event_line

### test_chat_frame_is_bounded_and_has_composer

function tests/nipux_cli/test_cli.py:2298

No docstring.

Calls: _build_chat_frame, frame.splitlines, len, startswith, wide_frame.splitlines

### test_chat_frame_separates_chat_from_worker_activity

function tests/nipux_cli/test_cli.py:2412

No docstring.

Calls: _build_chat_frame, frame.split

### test_chat_frame_empty_state_is_minimal_and_actionable

function tests/nipux_cli/test_cli.py:2447

No docstring.

Calls: _build_chat_frame

### test_frame_emit_skips_unchanged_render

function tests/nipux_cli/test_cli.py:2478

No docstring.

Calls: _emit_frame_if_changed, capsys.readouterr, out.count

### test_chat_frame_does_not_cap_long_agent_messages

function tests/nipux_cli/test_cli.py:2493

No docstring.

Calls: _build_chat_frame

### test_plain_chat_control_intents_map_to_commands

function tests/nipux_cli/test_cli.py:2533

No docstring.

Calls: _chat_control_command

### test_plain_chat_classifier_keeps_natural_controls_out_of_model_path

function tests/nipux_cli/test_cli.py:2570

No docstring.

Calls: _is_plain_chat_line

### test_plain_chat_control_intent_does_not_queue_operator_context

function tests/nipux_cli/test_cli.py:2577

No docstring.

Calls: AgentDB, _handle_chat_message, _mark_test_model_ready, db.close, db.create_job, db.get_job, get, monkeypatch.setattr, monkeypatch.setenv, str

### fake_capture

function tests/nipux_cli/test_cli.py:2588

No docstring.

Calls: none

### test_plain_chat_reply_usage_is_recorded

function tests/nipux_cli/test_cli.py:2608

No docstring.

Calls: AgentDB, LLMResponse, _handle_chat_message, _mark_test_model_ready, db.close, db.create_job, db.job_token_usage, db.list_events, monkeypatch.setenv, str

### test_chat_frame_surfaces_actual_work_events

function tests/nipux_cli/test_cli.py:2645

No docstring.

Calls: _build_chat_frame

### test_chat_frame_has_model_updates_page

function tests/nipux_cli/test_cli.py:2695

No docstring.

Calls: _build_chat_frame

### test_workspace_status_page_does_not_render_fake_worker_when_no_jobs

function tests/nipux_cli/test_cli.py:2735

No docstring.

Calls: _build_chat_frame

### test_status_job_cards_show_durable_work_mix

function tests/nipux_cli/test_cli.py:2782

No docstring.

Calls: _build_chat_frame

### test_recent_outcome_lines_wrap_long_updates

function tests/nipux_cli/test_cli.py:2823

No docstring.

Calls: join, len, recent_model_update_lines

### test_recent_outcome_lines_do_not_pretruncate_actual_work

function tests/nipux_cli/test_cli.py:2845

No docstring.

Calls: join, recent_model_update_lines

### test_chat_updates_page_keeps_updates_to_one_line_each

function tests/nipux_cli/test_cli.py:2871

No docstring.

Calls: len, recent_model_update_lines

### test_chat_pane_marks_hidden_overflow

function tests/nipux_cli/test_cli.py:2892

No docstring.

Calls: chat_pane_lines, join, len, range

### test_chat_pane_groups_multiline_command_output_under_one_label

function tests/nipux_cli/test_cli.py:2911

No docstring.

Calls: chat_pane_lines, join, rendered.count

### test_chat_pane_suppresses_transient_duplicates_after_events_arrive

function tests/nipux_cli/test_cli.py:2931

No docstring.

Calls: chat_pane_lines, join, rendered.count

### test_chat_pane_hides_persisted_legacy_waiting_notice

function tests/nipux_cli/test_cli.py:2961

No docstring.

Calls: chat_pane_lines, join

### test_chat_pane_renders_waiting_notice_as_animation_only

function tests/nipux_cli/test_cli.py:2996

No docstring.

Calls: _display_chat_notices, chat_pane_lines, join

### test_chat_pane_hides_persisted_worker_waiting_text

function tests/nipux_cli/test_cli.py:3007

No docstring.

Calls: chat_pane_lines, join

### test_chat_pane_renders_stored_provider_errors_as_actions

function tests/nipux_cli/test_cli.py:3044

No docstring.

Calls: chat_pane_lines, join

### test_chat_updates_page_uses_deeper_summary_events

function tests/nipux_cli/test_cli.py:3066

No docstring.

Calls: _build_chat_frame

### test_hourly_outcomes_prioritize_durable_work_over_research_noise

function tests/nipux_cli/test_cli.py:3098

No docstring.

Calls: hourly_update_lines, join

### test_status_recent_outcomes_hide_research_noise

function tests/nipux_cli/test_cli.py:3140

No docstring.

Calls: join, recent_model_update_lines

### test_status_recent_outcomes_hide_plan_update_noise

function tests/nipux_cli/test_cli.py:3164

No docstring.

Calls: join, recent_model_update_lines

### test_status_recent_outcomes_show_durable_checkpoint_updates

function tests/nipux_cli/test_cli.py:3196

No docstring.

Calls: join, recent_model_update_lines

### test_status_recent_outcomes_compact_repeated_updates

function tests/nipux_cli/test_cli.py:3219

No docstring.

Calls: join, range, recent_model_update_lines, rendered.count

### test_hourly_outcomes_hide_plan_update_noise

function tests/nipux_cli/test_cli.py:3237

No docstring.

Calls: hourly_update_lines, join

### test_hourly_outcomes_count_durable_checkpoint_updates

function tests/nipux_cli/test_cli.py:3269

No docstring.

Calls: hourly_update_lines, join

### test_hourly_outcome_summary_uses_progress_order

function tests/nipux_cli/test_cli.py:3290

No docstring.

Calls: hourly_update_lines, join

### test_hourly_outcomes_wrap_long_durable_updates_without_pre_truncation

function tests/nipux_cli/test_cli.py:3320

No docstring.

Calls: hourly_update_lines, join

### test_hourly_outcomes_limit_visible_hours_without_losing_headers

function tests/nipux_cli/test_cli.py:3345

No docstring.

Calls: events.extend, hourly_update_lines, join, range

### test_chat_updates_page_includes_agent_error_updates

function tests/nipux_cli/test_cli.py:3376

No docstring.

Calls: _build_chat_frame

### test_chat_status_marks_provider_blocked_jobs_before_daemon_retry

function tests/nipux_cli/test_cli.py:3413

No docstring.

Calls: _build_chat_frame

### test_chat_status_page_surfaces_context_pressure

function tests/nipux_cli/test_cli.py:3443

No docstring.

Calls: _build_chat_frame

### test_chat_status_page_surfaces_low_durable_yield

function tests/nipux_cli/test_cli.py:3473

No docstring.

Calls: _build_chat_frame

### test_chat_status_page_shows_job_outputs

function tests/nipux_cli/test_cli.py:3501

No docstring.

Calls: _build_chat_frame

### test_frame_snapshot_keeps_summary_events_durable

function tests/nipux_cli/test_cli.py:3567

No docstring.

Calls: AgentDB, _load_frame_snapshot, db.append_event, db.close, db.create_job, event.get, join, monkeypatch.setenv, range, str

### test_frame_snapshot_respects_explicit_job_over_saved_focus

function tests/nipux_cli/test_cli.py:3610

No docstring.

Calls: AgentDB, _load_frame_snapshot, db.close, db.create_job, json.dumps, monkeypatch.setenv, str, write_text

### test_chat_status_page_marks_deferred_jobs_waiting

function tests/nipux_cli/test_cli.py:3626

No docstring.

Calls: _build_chat_frame

### test_chat_frame_collapses_repeated_failures_and_hides_memory_noise

function tests/nipux_cli/test_cli.py:3664

No docstring.

Calls: _build_chat_frame

### test_work_pane_uses_badges_without_duplicate_action_verbs

function tests/nipux_cli/test_cli.py:3708

No docstring.

Calls: _build_chat_frame

### test_run_reopens_completed_focused_job

function tests/nipux_cli/test_cli.py:3743

No docstring.

Calls: AgentDB, _mark_test_model_ready, args.func, build_parser, capsys.readouterr, db.close, db.create_job, db.get_job, db.update_job_status, monkeypatch.setattr, monkeypatch.setenv, parser.parse_args, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:3755

No docstring.

Calls: started.update

### test_run_delegates_unverified_provider_state_to_daemon_start

function tests/nipux_cli/test_cli.py:3775

No docstring.

Calls: AgentDB, args.func, build_parser, capsys.readouterr, db.close, db.create_job, monkeypatch.setattr, monkeypatch.setenv, parser.parse_args, print, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:3785

No docstring.

Calls: print, started.update

### test_run_marks_job_waiting_when_provider_recovery_is_needed

function tests/nipux_cli/test_cli.py:3801

No docstring.

Calls: AgentDB, args.func, build_parser, capsys.readouterr, db.close, db.create_job, db.get_job, monkeypatch.setattr, monkeypatch.setenv, parser.parse_args, print, str

### fake_start

function tests/nipux_cli/test_cli.py:3812

No docstring.

Calls: print

### test_run_does_not_reopen_already_provider_blocked_job

function tests/nipux_cli/test_cli.py:3832

No docstring.

Calls: AgentDB, all, args.func, build_parser, capsys.readouterr, db.close, db.create_job, db.get_job, db.list_events, db.update_job_status, event.get, len, monkeypatch.setattr, monkeypatch.setenv, parser.parse_args, print

### fake_start

function tests/nipux_cli/test_cli.py:3853

No docstring.

Calls: print

### test_run_does_not_reopen_job_when_provider_preflight_is_hard_failure

function tests/nipux_cli/test_cli.py:3876

No docstring.

Calls: AgentDB, args.func, build_parser, db.close, db.create_job, db.get_job, db.update_job_status, monkeypatch.setattr, monkeypatch.setenv, parser.parse_args, str

### test_create_sets_new_job_as_shell_focus

function tests/nipux_cli/test_cli.py:3901

No docstring.

Calls: AgentDB, _mark_test_model_ready, _run_shell_line, all, args.func, build_parser, capsys.readouterr, db.close, db.get_job, exists, monkeypatch.setenv, out.strip, parser.parse_args, str

### test_commands_accept_unquoted_job_titles_in_shell

function tests/nipux_cli/test_cli.py:3931

No docstring.

Calls: AgentDB, _run_shell_line, capsys.readouterr, db.close, db.create_job, monkeypatch.setenv, str

### test_shell_stop_job_title_pauses_job_instead_of_stopping_daemon

function tests/nipux_cli/test_cli.py:3947

No docstring.

Calls: AgentDB, _run_shell_line, capsys.readouterr, db.close, db.create_job, db.get_job, db.update_job_status, monkeypatch.setenv, str

### test_resume_clears_provider_block_before_retry

function tests/nipux_cli/test_cli.py:3969

No docstring.

Calls: AgentDB, capsys.readouterr, db.close, db.create_job, db.get_job, db.update_job_status, main, monkeypatch.setenv, str

### test_shell_cancel_prefers_multiword_job_title_over_note

function tests/nipux_cli/test_cli.py:4004

No docstring.

Calls: AgentDB, _run_shell_line, capsys.readouterr, db.close, db.create_job, db.get_job, db.update_job_status, monkeypatch.setenv, str

### test_shell_pause_splits_note_after_longest_matching_job_title

function tests/nipux_cli/test_cli.py:4027

No docstring.

Calls: AgentDB, _run_shell_line, capsys.readouterr, db.close, db.create_job, db.get_job, db.update_job_status, monkeypatch.setenv, str

### test_chat_handle_line_adds_operator_message

function tests/nipux_cli/test_cli.py:4049

No docstring.

Calls: AgentDB, _chat_handle_line, _mark_test_model_ready, capsys.readouterr, db.close, db.create_job, db.get_job, monkeypatch.setenv, str

### test_chat_can_spawn_new_job_from_plain_message

function tests/nipux_cli/test_cli.py:4078

No docstring.

Calls: AgentDB, _chat_handle_line, _mark_test_model_ready, capsys.readouterr, db.close, db.create_job, db.list_jobs, len, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:4088

No docstring.

Calls: started.update

### test_workspace_chat_can_create_refined_worker_job

function tests/nipux_cli/test_cli.py:4120

No docstring.

Calls: AgentDB, _handle_chat_message, _load_frame_snapshot, _mark_test_model_ready, db.close, db.list_jobs, event.get, join, len, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:4125

No docstring.

Calls: started.update

### test_workspace_chat_start_objective_creates_worker_without_model_reply

function tests/nipux_cli/test_cli.py:4156

No docstring.

Calls: AgentDB, _handle_workspace_chat_message, _mark_test_model_ready, _read_shell_state, db.close, db.list_jobs, get, len, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:4161

No docstring.

Calls: started.update

### test_workspace_chat_accepts_natural_worker_and_task_phrasing

function tests/nipux_cli/test_cli.py:4185

No docstring.

Calls: AgentDB, _handle_workspace_chat_message, _mark_test_model_ready, append, db.close, db.list_jobs, len, monkeypatch.setattr, monkeypatch.setenv, started.setdefault, str

### fake_start

function tests/nipux_cli/test_cli.py:4190

No docstring.

Calls: append, started.setdefault

### test_chat_can_queue_new_job_without_starting

function tests/nipux_cli/test_cli.py:4222

No docstring.

Calls: AgentDB, _chat_handle_line, _mark_test_model_ready, capsys.readouterr, db.close, db.create_job, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:4232

No docstring.

Calls: started.update

### test_chat_can_spawn_generic_deliverable_job_from_plain_message

function tests/nipux_cli/test_cli.py:4252

No docstring.

Calls: AgentDB, _chat_handle_line, _mark_test_model_ready, db.close, db.create_job, db.list_jobs, len, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:4262

No docstring.

Calls: started.update

### test_chat_start_job_message_starts_daemon

function tests/nipux_cli/test_cli.py:4287

No docstring.

Calls: AgentDB, _chat_handle_line, _mark_test_model_ready, capsys.readouterr, db.close, db.create_job, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:4297

No docstring.

Calls: started.update

### test_chat_create_job_and_run_it_starts_daemon

function tests/nipux_cli/test_cli.py:4317

No docstring.

Calls: AgentDB, _chat_handle_line, _mark_test_model_ready, capsys.readouterr, db.close, db.create_job, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:4327

No docstring.

Calls: started.update

### test_chat_jobs_command_lists_jobs_instead_of_steering

function tests/nipux_cli/test_cli.py:4347

No docstring.

Calls: AgentDB, _chat_handle_line, capsys.readouterr, db.close, db.create_job, db.get_job, get, monkeypatch.setenv, str

### test_chat_command_inside_chat_is_not_queued

function tests/nipux_cli/test_cli.py:4368

No docstring.

Calls: AgentDB, _chat_handle_line, capsys.readouterr, db.close, db.create_job, db.get_job, get, monkeypatch.setenv, str

### test_chat_run_accepts_initial_plan_before_starting

function tests/nipux_cli/test_cli.py:4392

No docstring.

Calls: AgentDB, _chat_handle_line, _mark_test_model_ready, args.func, build_parser, db.close, db.get_job, monkeypatch.setattr, monkeypatch.setenv, parser.parse_args, str

### fake_run

function tests/nipux_cli/test_cli.py:4401

No docstring.

Calls: none

### test_run_without_jobs_does_not_start_empty_daemon

function tests/nipux_cli/test_cli.py:4417

No docstring.

Calls: _mark_test_model_ready, _run_shell_line, capsys.readouterr, monkeypatch.setattr, monkeypatch.setenv, started.update, str

### fake_start

function tests/nipux_cli/test_cli.py:4422

No docstring.

Calls: started.update

### test_build_chat_messages_includes_recent_job_state

function tests/nipux_cli/test_cli.py:4434

No docstring.

Calls: AgentDB, _build_chat_messages, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.start_run

### test_build_chat_messages_includes_durable_outcome_summary

function tests/nipux_cli/test_cli.py:4457

No docstring.

Calls: AgentDB, _build_chat_messages, db.append_event, db.close, db.create_job, db.get_job

### test_build_chat_messages_does_not_include_local_machine_context

function tests/nipux_cli/test_cli.py:4484

No docstring.

Calls: AgentDB, _build_chat_messages, db.close, db.create_job, db.get_job, monkeypatch.setenv, ssh_dir.mkdir, str, write_text

### test_build_chat_messages_points_to_artifact_and_lessons

function tests/nipux_cli/test_cli.py:4504

No docstring.

Calls: AgentDB, ArtifactStore, _build_chat_messages, db.add_step, db.append_lesson, db.close, db.create_job, db.get_job, db.start_run, write_text

### test_build_chat_messages_clip_large_visible_state

function tests/nipux_cli/test_cli.py:4530

No docstring.

Calls: AgentDB, _build_chat_messages, db.append_event, db.close, db.create_job, db.get_job, len, range

### test_artifact_command_resolves_title_query

function tests/nipux_cli/test_cli.py:4554

No docstring.

Calls: AgentDB, ArtifactStore, args.func, build_parser, capsys.readouterr, db.add_step, db.close, db.create_job, db.start_run, monkeypatch.setenv, parser.parse_args, str, write_text

### test_artifacts_command_prints_compact_view_command

function tests/nipux_cli/test_cli.py:4581

No docstring.

Calls: AgentDB, ArtifactStore, args.func, build_parser, capsys.readouterr, db.add_step, db.close, db.create_job, db.start_run, monkeypatch.setenv, parser.parse_args, str, write_text

### test_artifact_command_opens_recent_output_by_number

function tests/nipux_cli/test_cli.py:4609

No docstring.

Calls: AgentDB, ArtifactStore, args.func, build_parser, capsys.readouterr, db.add_step, db.close, db.create_job, db.start_run, monkeypatch.setenv, parser.parse_args, str, write_text

### test_chat_work_defaults_to_compact_output

function tests/nipux_cli/test_cli.py:4636

No docstring.

Calls: AgentDB, _chat_handle_line, db.close, db.create_job, monkeypatch.setattr, monkeypatch.setenv, str

### fake_work

function tests/nipux_cli/test_cli.py:4645

No docstring.

Calls: none

### test_chat_learn_adds_lesson

function tests/nipux_cli/test_cli.py:4656

No docstring.

Calls: AgentDB, _chat_handle_line, capsys.readouterr, db.close, db.create_job, db.get_job, monkeypatch.setenv, str

### test_chat_follow_queues_follow_up_message

function tests/nipux_cli/test_cli.py:4676

No docstring.

Calls: AgentDB, _chat_handle_line, capsys.readouterr, db.close, db.create_job, db.get_job, monkeypatch.setenv, str

### test_findings_sources_memory_metrics_commands

function tests/nipux_cli/test_cli.py:4698

No docstring.

Calls: AgentDB, args.func, build_parser, capsys.readouterr, db.append_experiment_record, db.append_finding_record, db.append_lesson, db.append_memory_graph_records, db.append_reflection, db.append_source_record, db.append_task_record, db.close, db.create_job, monkeypatch.setenv, pars...

### test_memory_graph_html_command_writes_clickable_artifact

function tests/nipux_cli/test_cli.py:4743

No docstring.

Calls: AgentDB, Path, args.func, build_parser, capsys.readouterr, db.append_memory_graph_records, db.close, db.create_job, db.list_artifacts, monkeypatch.setenv, parse_args, read_text, str

### test_shell_natural_update_phrase_shows_updates

function tests/nipux_cli/test_cli.py:4793

No docstring.

Calls: AgentDB, _run_shell_line, capsys.readouterr, db.close, db.create_job, monkeypatch.setenv, str

### test_updates_command_summarizes_durable_outcomes

function tests/nipux_cli/test_cli.py:4809

No docstring.

Calls: AgentDB, args.func, artifact_path.write_text, build_parser, capsys.readouterr, db.add_artifact, db.append_event, db.append_finding_record, db.close, db.create_job, monkeypatch.setenv, parse_args, str

### test_updates_all_summarizes_durable_work_across_jobs

function tests/nipux_cli/test_cli.py:4846

No docstring.

Calls: AgentDB, args.func, build_parser, capsys.readouterr, db.add_artifact, db.append_experiment_record, db.append_finding_record, db.close, db.create_job, first_path.write_text, monkeypatch.setenv, parse_args, second_path.write_text, str

### test_history_and_events_commands_render_visible_timeline

function tests/nipux_cli/test_cli.py:4897

No docstring.

Calls: AgentDB, build_parser, capsys.readouterr, db.append_agent_update, db.append_operator_message, db.close, db.create_job, func, monkeypatch.setenv, parser.parse_args, str

### test_shell_natural_health_phrase_shows_health

function tests/nipux_cli/test_cli.py:4918

No docstring.

Calls: AgentDB, _run_shell_line, capsys.readouterr, db.close, db.create_job, monkeypatch.setenv, str

### test_health_prints_recent_daemon_events

function tests/nipux_cli/test_cli.py:4933

No docstring.

Calls: append_daemon_event, args.func, build_parser, capsys.readouterr, load_config, monkeypatch.setenv, parser.parse_args, str

### test_launch_agent_plist_contains_daemon_command

function tests/nipux_cli/test_cli.py:4949

No docstring.

Calls: _launch_agent_plist, monkeypatch.setenv, str

### test_systemd_service_text_contains_daemon_command

function tests/nipux_cli/test_cli.py:4961

No docstring.

Calls: _systemd_service_text, monkeypatch.setenv, str

### _config

function tests/nipux_cli/test_cli_model_preflight.py:7

No docstring.

Calls: SimpleNamespace

### test_remote_model_preflight_blocks_rejected_auth

function tests/nipux_cli/test_cli_model_preflight.py:18

No docstring.

Calls: Check, _config, _ensure_remote_model_ready_for_worker, capsys.readouterr, monkeypatch.setattr

### fake_doctor

function tests/nipux_cli/test_cli_model_preflight.py:19

No docstring.

Calls: Check

### test_remote_model_preflight_allows_recovery_monitor_for_quota

function tests/nipux_cli/test_cli_model_preflight.py:33

No docstring.

Calls: Check, _config, _ensure_remote_model_ready_for_worker, capsys.readouterr, monkeypatch.setattr

### fake_doctor

function tests/nipux_cli/test_cli_model_preflight.py:34

No docstring.

Calls: Check

### test_remote_model_preflight_skips_fake_runs

function tests/nipux_cli/test_cli_model_preflight.py:47

No docstring.

Calls: AssertionError, _config, _ensure_remote_model_ready_for_worker, monkeypatch.setattr

### fake_doctor

function tests/nipux_cli/test_cli_model_preflight.py:48

No docstring.

Calls: AssertionError

### test_model_preflight_checks_local_endpoints

function tests/nipux_cli/test_cli_model_preflight.py:56

No docstring.

Calls: _config, _ensure_remote_model_ready_for_worker, monkeypatch.setattr

### fake_doctor

function tests/nipux_cli/test_cli_model_preflight.py:59

No docstring.

Calls: none

### test_start_does_not_spawn_daemon_when_model_preflight_fails

function tests/nipux_cli/test_cli_model_preflight.py:69

No docstring.

Calls: AssertionError, args.func, build_parser, monkeypatch.setattr, monkeypatch.setenv, parse_args, str

### fake_ready

function tests/nipux_cli/test_cli_model_preflight.py:73

No docstring.

Calls: none

### fake_popen

function tests/nipux_cli/test_cli_model_preflight.py:77

No docstring.

Calls: AssertionError

### test_refresh_memory_index_includes_durable_progress_ledgers

function tests/nipux_cli/test_compression.py:5

No docstring.

Calls: AgentDB, db.append_event, db.close, db.create_job, db.list_memory, refresh_memory_index

### _mode

function tests/nipux_cli/test_config.py:6

No docstring.

Calls: path.stat

### test_load_config_defaults_to_local_endpoint

function tests/nipux_cli/test_config.py:10

No docstring.

Calls: load_config, monkeypatch.setenv, str

### test_load_config_from_yaml

function tests/nipux_cli/test_config.py:30

No docstring.

Calls: Path, cfg.write_text, load_config, monkeypatch.setenv, str

### test_load_config_reads_local_env_file

function tests/nipux_cli/test_config.py:79

No docstring.

Calls: load_config, monkeypatch.delenv, monkeypatch.setenv, str, write_text

### test_load_config_tightens_local_env_permissions

function tests/nipux_cli/test_config.py:98

No docstring.

Calls: _mode, env_path.chmod, env_path.write_text, load_config, monkeypatch.delenv, monkeypatch.setenv, str

### test_default_config_yaml_allows_provider_template_without_secret

function tests/nipux_cli/test_config.py:110

No docstring.

Calls: default_config_yaml

### test_config_example_matches_default_local_endpoint

function tests/nipux_cli/test_config.py:131

No docstring.

Calls: Path, read_text, resolve

### test_single_instance_lock_rejects_second_holder

function tests/nipux_cli/test_daemon.py:30

No docstring.

Calls: pytest.raises, single_instance_lock

### test_daemon_lock_status_reports_free_lock

function tests/nipux_cli/test_daemon.py:38

No docstring.

Calls: daemon_lock_status

### test_lock_metadata_can_be_updated_while_held

function tests/nipux_cli/test_daemon.py:45

No docstring.

Calls: daemon_lock_status, single_instance_lock, update_lock_metadata

### test_lock_metadata_update_restores_missing_process_fields

function tests/nipux_cli/test_daemon.py:58

No docstring.

Calls: daemon_lock_status, handle.flush, handle.seek, handle.truncate, handle.write, json.dumps, single_instance_lock, update_lock_metadata

### test_daemon_lock_heartbeat_updates_while_worker_turn_runs

function tests/nipux_cli/test_daemon.py:75

No docstring.

Calls: AgentDB, AppConfig, RuntimeConfig, SlowDaemon, daemon_lock_status, db.close, metadata.get, monkeypatch.setattr, status.get, thread.is_alive, thread.join, thread.start, threading.Thread, time.sleep, time.time

### SlowDaemon

class tests/nipux_cli/test_daemon.py:80

No docstring.

Calls: time.sleep

### run_once

function tests/nipux_cli/test_daemon.py:81

No docstring.

Calls: time.sleep

### test_stop_daemon_recovers_pidless_lock_from_process_list

function tests/nipux_cli/test_daemon.py:115

No docstring.

Calls: AppConfig, PsResult, RuntimeConfig, handle.flush, handle.seek, handle.truncate, handle.write, json.dumps, killed.append, monkeypatch.setattr, single_instance_lock, stop_daemon_process_impl

### PsResult

class tests/nipux_cli/test_daemon.py:120

No docstring.

Calls: none

### test_daemon_lock_status_detects_stale_runtime

function tests/nipux_cli/test_daemon.py:139

No docstring.

Calls: current_runtime_fingerprint, daemon_lock_status, runtime_stale, single_instance_lock, update_lock_metadata

### test_runtime_fingerprint_tracks_progress_code

function tests/nipux_cli/test_daemon.py:153

No docstring.

Calls: none

### test_rate_limit_backoff_uses_retry_after_header

function tests/nipux_cli/test_daemon.py:158

No docstring.

Calls: RateLimit, _exception_backoff, type

### RateLimit

class tests/nipux_cli/test_daemon.py:159

No docstring.

Calls: type

### test_rate_limit_backoff_has_conservative_fallback

function tests/nipux_cli/test_daemon.py:166

No docstring.

Calls: RateLimit, _exception_backoff

### RateLimit

class tests/nipux_cli/test_daemon.py:167

No docstring.

Calls: none

### test_failed_step_provider_config_error_uses_normal_backoff

function tests/nipux_cli/test_daemon.py:173

No docstring.

Calls: StepExecution, _step_failure_backoff

### test_failed_tool_auth_error_uses_normal_backoff

function tests/nipux_cli/test_daemon.py:189

No docstring.

Calls: StepExecution, _step_failure_backoff

### test_failed_step_rate_limit_uses_normal_backoff

function tests/nipux_cli/test_daemon.py:204

No docstring.

Calls: StepExecution, _step_failure_backoff

### test_failed_step_provider_timeout_uses_normal_backoff

function tests/nipux_cli/test_daemon.py:220

No docstring.

Calls: StepExecution, _step_failure_backoff

### test_retry_after_parses_epoch_milliseconds

function tests/nipux_cli/test_daemon.py:236

No docstring.

Calls: _parse_retry_after, int, str, time.time

### test_daemon_run_once_claims_next_job_with_fake_step

function tests/nipux_cli/test_daemon.py:245

No docstring.

Calls: AgentDB, AppConfig, Daemon, RuntimeConfig, daemon.run_once, db.close, db.create_job, db.list_artifacts

### test_daemon_ignores_ui_focus_for_worker_scheduling

function tests/nipux_cli/test_daemon.py:262

No docstring.

Calls: AgentDB, AppConfig, Daemon, RuntimeConfig, daemon.next_runnable_job, db.close, db.create_job, json.dumps, write_text

### test_daemon_skips_deferred_jobs_until_due

function tests/nipux_cli/test_daemon.py:280

No docstring.

Calls: AgentDB, AppConfig, Daemon, RuntimeConfig, daemon.next_runnable_job, datetime.now, db.close, db.create_job, db.update_job_status, isoformat, timedelta

### test_daemon_quarantines_provider_blocked_jobs

function tests/nipux_cli/test_daemon.py:301

No docstring.

Calls: AgentDB, AppConfig, Daemon, RuntimeConfig, any, daemon.next_runnable_job, datetime.now, db.close, db.create_job, db.get_job, db.list_events, db.update_job_status, get, isoformat, lower

### test_daemon_leaves_provider_blocked_job_paused_until_model_recovers

function tests/nipux_cli/test_daemon.py:327

No docstring.

Calls: AgentDB, AppConfig, Check, Daemon, RuntimeConfig, daemon.next_runnable_job, datetime.now, db.close, db.create_job, db.get_job, db.update_job_status, isoformat, monkeypatch.setattr, read_daemon_events, startswith, timedelta

### fake_doctor

function tests/nipux_cli/test_daemon.py:339

No docstring.

Calls: Check

### test_daemon_resumes_provider_blocked_job_when_model_recovers

function tests/nipux_cli/test_daemon.py:355

No docstring.

Calls: AgentDB, AppConfig, Daemon, RuntimeConfig, any, daemon.next_runnable_job, datetime.now, db.close, db.create_job, db.get_job, db.list_events, db.update_job_status, get, isoformat, monkeypatch.setattr, timedelta

### fake_doctor

function tests/nipux_cli/test_daemon.py:367

No docstring.

Calls: none

### test_daemon_idle_sleep_wakes_for_deferred_job

function tests/nipux_cli/test_daemon.py:387

No docstring.

Calls: AgentDB, AppConfig, Daemon, RuntimeConfig, daemon.idle_sleep_seconds, datetime.now, db.close, db.create_job, db.update_job_status, isoformat, timedelta

### test_daemon_idle_sleep_uses_poll_when_no_deferred_jobs

function tests/nipux_cli/test_daemon.py:407

No docstring.

Calls: AgentDB, AppConfig, Daemon, RuntimeConfig, daemon.idle_sleep_seconds, db.close, db.create_job

### test_daemon_advances_multiple_runnable_jobs_without_focus_starvation

function tests/nipux_cli/test_daemon.py:420

No docstring.

Calls: AgentDB, AppConfig, Daemon, RuntimeConfig, daemon.run_forever, db.close, db.create_job, db.list_steps, json.dumps, write_text

### test_daemon_writes_due_daily_digest_once

function tests/nipux_cli/test_daemon.py:437

No docstring.

Calls: AgentDB, AppConfig, Daemon, RuntimeConfig, daemon.send_due_daily_digest, datetime, db.close, db.create_job, exists

### test_daemon_event_log_round_trips_jsonl

function tests/nipux_cli/test_daemon.py:456

No docstring.

Calls: AppConfig, RuntimeConfig, append_daemon_event, read_daemon_events

### test_daemon_recovers_stale_running_steps_on_start

function tests/nipux_cli/test_daemon.py:467

No docstring.

Calls: AgentDB, AppConfig, Daemon, RuntimeConfig, any, daemon.run_forever, db.add_step, db.close, db.create_job, db.list_runs, db.list_steps, db.start_run, event.get, next, read_daemon_events

### test_daemon_survives_unexpected_step_exception

function tests/nipux_cli/test_daemon.py:489

No docstring.

Calls: AgentDB, AppConfig, ExplodingDaemon, RuntimeConfig, RuntimeError, any, daemon.run_forever, daemon_lock_status, db.close, event.get, read_daemon_events

### ExplodingDaemon

class tests/nipux_cli/test_daemon.py:490

No docstring.

Calls: RuntimeError

### run_once

function tests/nipux_cli/test_daemon.py:491

No docstring.

Calls: RuntimeError

### test_daemon_treats_blocked_steps_as_recoverable

function tests/nipux_cli/test_daemon.py:510

No docstring.

Calls: AgentDB, AppConfig, BlockedDaemon, RuntimeConfig, StepExecution, any, daemon.run_forever, daemon_lock_status, db.close, event.get, read_daemon_events, sum

### BlockedDaemon

class tests/nipux_cli/test_daemon.py:511

No docstring.

Calls: StepExecution

### run_once

function tests/nipux_cli/test_daemon.py:512

No docstring.

Calls: StepExecution

### test_fake_daemon_can_run_100_iterations_without_auto_stop

function tests/nipux_cli/test_daemon.py:538

No docstring.

Calls: AgentDB, AppConfig, Daemon, RuntimeConfig, any, daemon.run_forever, daemon_lock_status, db.close, db.create_job, db.get_job, db.list_artifacts, db.list_memory, db.list_steps, event.get, len, read_daemon_events

### test_dashboard_collects_jobs_steps_and_artifacts

function tests/nipux_cli/test_dashboard.py:9

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, RuntimeConfig, collect_dashboard_state, db.add_step, db.append_lesson, db.append_task_record, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, render_dashboard, render_overview, write_text

### test_overview_marks_idle_daemon_as_ready_for_work

function tests/nipux_cli/test_dashboard.py:55

No docstring.

Calls: AgentDB, AppConfig, RuntimeConfig, collect_dashboard_state, db.close, db.create_job, render_overview

### test_overview_marks_old_heartbeat_as_busy_for_running_step

function tests/nipux_cli/test_dashboard.py:68

No docstring.

Calls: AgentDB, AppConfig, RuntimeConfig, collect_dashboard_state, datetime.now, db.add_step, db.close, db.create_job, db.start_run, isoformat, render_overview, timedelta

### test_db_job_run_step_and_artifact_roundtrip

function tests/nipux_cli/test_db.py:4

No docstring.

Calls: AgentDB, db.add_artifact, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_artifact, db.get_job, db.list_artifacts, db.list_runs, db.list_steps, db.start_run

### test_create_job_uses_unique_readable_slug_ids

function tests/nipux_cli/test_db.py:42

No docstring.

Calls: AgentDB, db.close, db.create_job

### test_step_numbers_increment_across_runs_for_a_job

function tests/nipux_cli/test_db.py:54

No docstring.

Calls: AgentDB, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.list_steps, db.start_run

### test_job_token_usage_aggregates_message_usage

function tests/nipux_cli/test_db.py:73

No docstring.

Calls: AgentDB, db.append_event, db.close, db.create_job, db.job_token_usage

### test_append_operator_message_roundtrip

function tests/nipux_cli/test_db.py:125

No docstring.

Calls: AgentDB, db.append_operator_message, db.close, db.create_job, db.get_job, db.list_timeline_events

### test_claim_operator_messages_marks_one_message_at_a_time

function tests/nipux_cli/test_db.py:145

No docstring.

Calls: AgentDB, all, any, db.append_operator_message, db.claim_operator_messages, db.close, db.create_job, db.get_job, db.list_timeline_events, message.get

### test_acknowledge_operator_messages_marks_delivered_context

function tests/nipux_cli/test_db.py:168

No docstring.

Calls: AgentDB, any, db.acknowledge_operator_messages, db.append_operator_message, db.claim_operator_messages, db.close, db.create_job, db.get_job, db.list_timeline_events

### test_rename_job_updates_title_without_changing_id

function tests/nipux_cli/test_db.py:193

No docstring.

Calls: AgentDB, db.close, db.create_job, db.get_job, db.rename_job

### test_delete_job_removes_related_rows

function tests/nipux_cli/test_db.py:208

No docstring.

Calls: AgentDB, AssertionError, artifact_path.write_text, db.add_artifact, db.add_step, db.close, db.create_job, db.delete_job, db.get_job, db.list_artifacts, db.list_memory, db.list_steps, db.start_run, db.upsert_memory

### test_append_lesson_roundtrip

function tests/nipux_cli/test_db.py:247

No docstring.

Calls: AgentDB, db.append_lesson, db.close, db.create_job, db.get_job

### test_append_lesson_dedupes_repeated_memory

function tests/nipux_cli/test_db.py:267

No docstring.

Calls: AgentDB, db.append_lesson, db.close, db.create_job, db.get_job, db.list_events, len

### test_source_and_finding_ledgers_dedupe_and_update

function tests/nipux_cli/test_db.py:288

No docstring.

Calls: AgentDB, db.append_finding_record, db.append_reflection, db.append_source_record, db.close, db.create_job, db.get_job, len

### test_repeated_source_and_finding_records_mark_non_substantive_touches

function tests/nipux_cli/test_db.py:340

No docstring.

Calls: AgentDB, db.append_finding_record, db.append_source_record, db.close, db.create_job

### test_task_queue_dedupes_and_updates

function tests/nipux_cli/test_db.py:386

No docstring.

Calls: AgentDB, db.append_task_record, db.close, db.create_job, db.get_job, len

### test_repeated_task_and_experiment_records_mark_non_substantive_touches

function tests/nipux_cli/test_db.py:417

No docstring.

Calls: AgentDB, db.append_experiment_record, db.append_task_record, db.close, db.create_job

### test_non_substantive_ledger_touches_do_not_emit_visible_events

function tests/nipux_cli/test_db.py:458

No docstring.

Calls: AgentDB, db.append_experiment_record, db.append_finding_record, db.append_roadmap_record, db.append_source_record, db.append_task_record, db.close, db.create_job, db.list_timeline_events, sum

### test_roadmap_last_records_include_progress_accounting_metadata

function tests/nipux_cli/test_db.py:514

No docstring.

Calls: AgentDB, db.append_milestone_validation_record, db.append_roadmap_record, db.close, db.create_job, db.get_job

### test_repeated_roadmap_records_do_not_create_fake_milestone_updates

function tests/nipux_cli/test_db.py:548

No docstring.

Calls: AgentDB, db.append_roadmap_record, db.close, db.create_job, db.get_job

### test_timeline_events_cover_visible_activity

function tests/nipux_cli/test_db.py:567

No docstring.

Calls: AgentDB, any, db.add_artifact, db.add_step, db.append_agent_update, db.append_finding_record, db.append_lesson, db.append_operator_message, db.append_reflection, db.append_source_record, db.append_task_record, db.close, db.create_job, db.finish_step, db.list_timeline_events, d...

### test_daily_digest_includes_ledgers_lessons_sources_and_strategy

function tests/nipux_cli/test_digest.py:6

No docstring.

Calls: AgentDB, AppConfig, RuntimeConfig, db.append_event, db.append_finding_record, db.append_lesson, db.append_reflection, db.append_source_record, db.append_task_record, db.close, db.create_job, db.start_run, exists, render_daily_digest, render_job_digest, write_daily_digest

### FakeHTTPResponse

class tests/nipux_cli/test_doctor.py:9

No docstring.

Calls: encode, json.dumps

### __init__

function tests/nipux_cli/test_doctor.py:10

No docstring.

Calls: none

### __enter__

function tests/nipux_cli/test_doctor.py:13

No docstring.

Calls: none

### __exit__

function tests/nipux_cli/test_doctor.py:16

No docstring.

Calls: none

### read

function tests/nipux_cli/test_doctor.py:19

No docstring.

Calls: encode, json.dumps

### test_doctor_checks_local_runtime_without_model_call

function tests/nipux_cli/test_doctor.py:23

No docstring.

Calls: AppConfig, RuntimeConfig, all, run_doctor

### test_doctor_warns_when_remote_model_key_is_missing

function tests/nipux_cli/test_doctor.py:32

No docstring.

Calls: AppConfig, ModelConfig, RuntimeConfig, monkeypatch.delenv, next, run_doctor

### test_doctor_reports_openrouter_auth_failure

function tests/nipux_cli/test_doctor.py:51

No docstring.

Calls: AppConfig, ModelConfig, RuntimeConfig, monkeypatch.setattr, monkeypatch.setenv, run_doctor, urllib.error.HTTPError

### fake_urlopen

function tests/nipux_cli/test_doctor.py:62

No docstring.

Calls: urllib.error.HTTPError

### test_doctor_reports_generation_limit_after_model_listing

function tests/nipux_cli/test_doctor.py:81

No docstring.

Calls: AppConfig, AssertionError, FakeHTTPResponse, ModelConfig, RuntimeConfig, io.BytesIO, monkeypatch.setattr, monkeypatch.setenv, run_doctor, url.endswith, urllib.error.HTTPError

### fake_urlopen

function tests/nipux_cli/test_doctor.py:92

No docstring.

Calls: AssertionError, FakeHTTPResponse, io.BytesIO, url.endswith, urllib.error.HTTPError

### test_doctor_reports_nested_provider_generation_error

function tests/nipux_cli/test_doctor.py:113

No docstring.

Calls: AppConfig, AssertionError, FakeHTTPResponse, ModelConfig, RuntimeConfig, encode, io.BytesIO, json.dumps, monkeypatch.setattr, monkeypatch.setenv, run_doctor, url.endswith, urllib.error.HTTPError

### fake_urlopen

function tests/nipux_cli/test_doctor.py:124

No docstring.

Calls: AssertionError, FakeHTTPResponse, encode, io.BytesIO, json.dumps, url.endswith, urllib.error.HTTPError

### test_runtime_code_has_no_task_specific_literals

function tests/nipux_cli/test_generic_runtime_audit.py:26

No docstring.

Calls: Path, join, lower, path.read_text, resolve, root.glob, sorted

### _load_live_smoke

function tests/nipux_cli/test_live_memory_graph_smoke.py:8

No docstring.

Calls: Path, importlib.util.module_from_spec, importlib.util.spec_from_file_location, resolve, spec.loader.exec_module

### test_live_memory_graph_smoke_fails_cleanly_without_key

function tests/nipux_cli/test_live_memory_graph_smoke.py:19

No docstring.

Calls: _load_live_smoke, capsys.readouterr, monkeypatch.delenv, monkeypatch.setattr, out.lower, smoke.main

### test_live_memory_graph_smoke_seed_pushes_generic_consolidation

function tests/nipux_cli/test_live_memory_graph_smoke.py:32

No docstring.

Calls: _load_live_smoke, memory_graph_for_prompt, smoke._seed_metadata

### _FakeCompletions

class tests/nipux_cli/test_llm.py:7

No docstring.

Calls: SimpleNamespace, self.calls.append

### __init__

function tests/nipux_cli/test_llm.py:8

No docstring.

Calls: none

### create

function tests/nipux_cli/test_llm.py:12

No docstring.

Calls: SimpleNamespace, self.calls.append

### test_chat_llm_requires_tool_choice_for_worker_actions

function tests/nipux_cli/test_llm.py:21

No docstring.

Calls: ModelConfig, OpenAIChatLLM, SimpleNamespace, _FakeCompletions, llm.next_action, monkeypatch.setattr, monkeypatch.setenv

### FakeOpenAI

class tests/nipux_cli/test_llm.py:25

No docstring.

Calls: SimpleNamespace

### __init__

function tests/nipux_cli/test_llm.py:26

No docstring.

Calls: none

### test_chat_llm_retries_without_tool_choice_when_provider_rejects_it

function tests/nipux_cli/test_llm.py:46

No docstring.

Calls: ModelConfig, OpenAIChatLLM, RejectingCompletions, RuntimeError, SimpleNamespace, create, kwargs.get, llm.next_action, monkeypatch.setattr, monkeypatch.setenv, self.calls.append, super

### RejectingCompletions

class tests/nipux_cli/test_llm.py:49

No docstring.

Calls: RuntimeError, create, kwargs.get, self.calls.append, super

### create

function tests/nipux_cli/test_llm.py:50

No docstring.

Calls: RuntimeError, create, kwargs.get, self.calls.append, super

### FakeOpenAI

class tests/nipux_cli/test_llm.py:58

No docstring.

Calls: SimpleNamespace

### __init__

function tests/nipux_cli/test_llm.py:59

No docstring.

Calls: none

### test_chat_llm_complete_response_returns_usage

function tests/nipux_cli/test_llm.py:74

No docstring.

Calls: ModelConfig, OpenAIChatLLM, SimpleNamespace, _FakeCompletions, llm.complete_response, monkeypatch.setattr, monkeypatch.setenv

### FakeOpenAI

class tests/nipux_cli/test_llm.py:78

No docstring.

Calls: SimpleNamespace

### __init__

function tests/nipux_cli/test_llm.py:79

No docstring.

Calls: none

### test_chat_llm_disables_provider_sdk_retries

function tests/nipux_cli/test_llm.py:98

No docstring.

Calls: ModelConfig, OpenAIChatLLM, SimpleNamespace, _FakeCompletions, captured.update, monkeypatch.setattr, monkeypatch.setenv

### FakeOpenAI

class tests/nipux_cli/test_llm.py:102

No docstring.

Calls: SimpleNamespace, _FakeCompletions, captured.update

### __init__

function tests/nipux_cli/test_llm.py:103

No docstring.

Calls: captured.update

### test_openrouter_generation_usage_enriches_cost_and_tokens

function tests/nipux_cli/test_llm.py:116

No docstring.

Calls: FakeHTTPResponse, _enrich_openrouter_generation_usage, monkeypatch.setattr

### FakeHTTPResponse

class tests/nipux_cli/test_llm.py:117

No docstring.

Calls: none

### __enter__

function tests/nipux_cli/test_llm.py:118

No docstring.

Calls: none

### __exit__

function tests/nipux_cli/test_llm.py:121

No docstring.

Calls: none

### read

function tests/nipux_cli/test_llm.py:124

No docstring.

Calls: none

### fake_urlopen

function tests/nipux_cli/test_llm.py:132

No docstring.

Calls: FakeHTTPResponse

### test_measurement_candidates_extract_markdown_table_unit_columns

function tests/nipux_cli/test_measurement.py:4

No docstring.

Calls: measurement_candidates, measurement_candidates_are_diagnostic_only

### test_measurement_candidates_extract_generic_table_metrics

function tests/nipux_cli/test_measurement.py:21

No docstring.

Calls: measurement_candidates

### test_format_metric_value_spaces_named_units

function tests/nipux_cli/test_metric_format.py:4

No docstring.

Calls: format_metric_value

### test_format_metric_value_keeps_attached_symbol_units

function tests/nipux_cli/test_metric_format.py:9

No docstring.

Calls: format_metric_value

### _entry

function tests/nipux_cli/test_operator_context.py:4

No docstring.

Calls: none

### test_conversation_only_operator_messages_do_not_enter_worker_prompt

function tests/nipux_cli/test_operator_context.py:8

No docstring.

Calls: _entry, operator_entry_is_prompt_relevant

### test_actionable_operator_messages_remain_worker_constraints

function tests/nipux_cli/test_operator_context.py:13

No docstring.

Calls: _entry, operator_entry_is_prompt_relevant

### test_inactive_prompt_operator_ids_returns_only_conversation_active_messages

function tests/nipux_cli/test_operator_context.py:23

No docstring.

Calls: _entry, inactive_prompt_operator_ids

### test_initial_task_contracts_are_generic_and_complete

function tests/nipux_cli/test_planning.py:4

No docstring.

Calls: initial_task_contract

### test_initial_roadmap_uses_valid_generic_contracts

function tests/nipux_cli/test_planning.py:19

No docstring.

Calls: initial_roadmap_for_objective

### test_initial_plan_adapts_to_measurable_objectives

function tests/nipux_cli/test_planning.py:35

No docstring.

Calls: any, initial_plan_for_objective, initial_task_contract, question.lower, title.lower

### test_initial_plan_adapts_to_deliverable_objectives

function tests/nipux_cli/test_planning.py:45

No docstring.

Calls: any, initial_plan_for_objective, initial_task_contract, title.lower

### test_initial_plan_treats_generated_files_as_deliverables

function tests/nipux_cli/test_planning.py:55

No docstring.

Calls: any, initial_plan_for_objective, initial_task_contract, question.lower

### test_initial_plan_adapts_to_monitoring_objectives

function tests/nipux_cli/test_planning.py:64

No docstring.

Calls: any, initial_plan_for_objective, initial_task_contract, question.lower

### test_initial_plan_does_not_add_meta_progress_update_task

function tests/nipux_cli/test_planning.py:73

No docstring.

Calls: all, initial_plan_for_objective, title.lower

### test_objective_profiles_stay_generic

function tests/nipux_cli/test_planning.py:86

No docstring.

Calls: all, objective_profiles

### test_progress_checkpoint_reports_deltas_and_recent_durable_work

function tests/nipux_cli/test_progress.py:4

No docstring.

Calls: build_progress_checkpoint

### test_progress_checkpoint_for_saved_output_is_concise

function tests/nipux_cli/test_progress.py:45

No docstring.

Calls: build_progress_checkpoint, checkpoint.message.startswith

### test_progress_checkpoint_without_delta_is_activity_not_progress

function tests/nipux_cli/test_progress.py:61

No docstring.

Calls: build_progress_checkpoint

### test_progress_checkpoint_counts_existing_record_updates_as_progress

function tests/nipux_cli/test_progress.py:75

No docstring.

Calls: build_progress_checkpoint

### test_progress_checkpoint_ignores_non_substantive_record_touches

function tests/nipux_cli/test_progress.py:124

No docstring.

Calls: build_progress_checkpoint

### test_progress_checkpoint_counts_roadmap_updates_and_validations

function tests/nipux_cli/test_progress.py:167

No docstring.

Calls: build_progress_checkpoint

### test_progress_helpers_ignore_malformed_metadata

function tests/nipux_cli/test_progress.py:202

No docstring.

Calls: ledger_counts, recent_progress_bits

### _load_generator

function tests/nipux_cli/test_project_atlas.py:6

No docstring.

Calls: Path, importlib.util.module_from_spec, importlib.util.spec_from_file_location, resolve, spec.loader.exec_module

### test_project_atlas_generator_maps_prompts_tools_and_source_without_self_embedding

function tests/nipux_cli/test_project_atlas.py:17

No docstring.

Calls: _load_generator, any, generator.extract_prompts, generator.extract_tools, generator.load_source_files

### test_project_atlas_redacts_secret_assignments_from_rendered_source

function tests/nipux_cli/test_project_atlas.py:29

No docstring.

Calls: _load_generator, generator.SourceFile, generator.render_source_file

### ProviderPayloadError

class tests/nipux_cli/test_provider_errors.py:8

No docstring.

Calls: none

### test_provider_action_required_detects_payload_and_status_text

function tests/nipux_cli/test_provider_errors.py:12

No docstring.

Calls: ProviderPayloadError, provider_action_required, provider_action_required_note

### test_provider_rate_limited_detects_transient_rate_text

function tests/nipux_cli/test_provider_errors.py:18

No docstring.

Calls: provider_rate_limited

### test_generic_template_pushes_artifacts_and_updates

function tests/nipux_cli/test_templates.py:4

No docstring.

Calls: program_for_job

### test_static_tool_surface_is_focused

function tests/nipux_cli/test_tools.py:14

No docstring.

Calls: DEFAULT_REGISTRY.names, sorted, tuple

### test_tool_registry_validates_required_arguments

function tests/nipux_cli/test_tools.py:37

No docstring.

Calls: AppConfig, DEFAULT_REGISTRY.validate_arguments, RuntimeConfig

### test_tool_registry_blocks_truncated_reference_arguments

function tests/nipux_cli/test_tools.py:65

No docstring.

Calls: AppConfig, DEFAULT_REGISTRY.validate_arguments, RuntimeConfig

### test_tool_access_config_filters_worker_schema_and_blocks_calls

function tests/nipux_cli/test_tools.py:83

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, DEFAULT_REGISTRY.openai_tools, RuntimeConfig, ToolAccessConfig, ToolContext, db.close, db.create_job, json.loads

### test_artifact_tools_roundtrip

function tests/nipux_cli/test_tools.py:104

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_read_artifact_missing_ref_returns_valid_recent_refs

function tests/nipux_cli/test_tools.py:135

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, ctx.artifacts.write_text, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_defer_job_records_resume_time_without_pausing

function tests/nipux_cli/test_tools.py:158

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, any, db.add_step, db.close, db.create_job, db.get_job, db.list_events, db.start_run, json.loads

### test_shell_exec_tool_runs_bounded_command

function tests/nipux_cli/test_tools.py:186

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_shell_exec_flags_masked_auth_failure_output

function tests/nipux_cli/test_tools.py:205

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_write_file_tool_writes_and_appends_workspace_file

function tests/nipux_cli/test_tools.py:234

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads, monkeypatch.chdir, read_text

### test_shell_exec_timeout_kills_process_group

function tests/nipux_cli/test_tools.py:260

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_cleanup_registered_shell_processes_kills_orphaned_group

function tests/nipux_cli/test_tools.py:279

No docstring.

Calls: cleanup_registered_shell_processes, json.dumps, os.kill, os.killpg, process.poll, process.wait, range, registry.exists, registry.parent.mkdir, registry.write_text, subprocess.Popen, time.sleep

### test_shell_exec_does_not_attach_local_ssh_config

function tests/nipux_cli/test_tools.py:307

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_shell_exec_reports_nonzero_stderr_as_error

function tests/nipux_cli/test_tools.py:324

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_shell_exec_flags_sudo_password_hidden_by_success_status

function tests/nipux_cli/test_tools.py:346

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_shell_exec_flags_missing_command_hidden_by_success_status

function tests/nipux_cli/test_tools.py:375

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_shell_exec_flags_missing_absolute_executable_hidden_by_success_status

function tests/nipux_cli/test_tools.py:397

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_shell_exec_reports_empty_which_probe_as_missing_executable

function tests/nipux_cli/test_tools.py:421

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_shell_exec_flags_empty_successful_probe_as_no_observation

function tests/nipux_cli/test_tools.py:444

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_shell_exec_flags_missing_which_probe_hidden_by_true

function tests/nipux_cli/test_tools.py:468

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_shell_exec_flags_make_failure_hidden_by_pipe_status

function tests/nipux_cli/test_tools.py:491

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_update_job_state_keeps_terminal_statuses_operator_only

function tests/nipux_cli/test_tools.py:513

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_report_update_tool_records_operator_visible_note

function tests/nipux_cli/test_tools.py:548

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_lesson_tool_records_durable_learning

function tests/nipux_cli/test_tools.py:568

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_lesson_cannot_clear_measurement_obligation_with_vague_lesson

function tests/nipux_cli/test_tools.py:592

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_lesson_can_explain_invalid_measurement_obligation

function tests/nipux_cli/test_tools.py:627

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, get, json.loads

### test_memory_graph_tools_roundtrip

function tests/nipux_cli/test_tools.py:668

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.list_events, db.start_run, json.loads, len

### test_record_source_and_findings_tools_update_ledgers

function tests/nipux_cli/test_tools.py:728

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, any, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_source_requires_assessment

function tests/nipux_cli/test_tools.py:773

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, get, json.loads

### test_record_source_does_not_accept_type_without_assessment

function tests/nipux_cli/test_tools.py:792

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_record_findings_reports_unchanged_duplicates_without_agent_update_noise

function tests/nipux_cli/test_tools.py:814

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.list_events, db.start_run, json.loads, len

### test_record_findings_requires_evidence_anchor

function tests/nipux_cli/test_tools.py:849

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, get, json.loads

### test_record_findings_reports_rejected_unevidenced_items_in_mixed_batch

function tests/nipux_cli/test_tools.py:873

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_tasks_tool_updates_task_queue

function tests/nipux_cli/test_tools.py:904

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_tasks_dedupes_semantic_task_under_backlog_pressure

function tests/nipux_cli/test_tools.py:951

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads, len, range

### test_record_tasks_reports_unchanged_duplicates_without_agent_update_noise

function tests/nipux_cli/test_tools.py:1007

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.list_events, db.start_run, db.update_job_metadata, json.loads, len

### test_record_tasks_cannot_defer_measurement_with_unrelated_task

function tests/nipux_cli/test_tools.py:1047

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_tasks_can_defer_measurement_with_explicit_measurement_task

function tests/nipux_cli/test_tools.py:1081

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, get, json.loads

### test_record_roadmap_tool_updates_roadmap

function tests/nipux_cli/test_tools.py:1125

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_roadmap_dedupes_milestone_titles_even_when_keys_change

function tests/nipux_cli/test_tools.py:1171

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, len

### test_record_milestone_validation_creates_follow_up_tasks

function tests/nipux_cli/test_tools.py:1216

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_milestone_validation_requires_evidence_for_passed_status

function tests/nipux_cli/test_tools.py:1255

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, get, json.loads

### test_record_milestone_validation_allows_passed_status_with_metadata_evidence

function tests/nipux_cli/test_tools.py:1281

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_record_milestone_validation_requires_gap_for_failed_or_blocked_status

function tests/nipux_cli/test_tools.py:1307

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, get, json.loads

### test_record_experiment_tool_tracks_best_measured_result

function tests/nipux_cli/test_tools.py:1342

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_experiment_synthesizes_missing_title

function tests/nipux_cli/test_tools.py:1395

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_record_experiment_requires_next_action_for_closed_trials

function tests/nipux_cli/test_tools.py:1421

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, get, json.loads

### test_record_experiment_requires_context_for_closed_non_measured_trials

function tests/nipux_cli/test_tools.py:1449

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, get, json.loads

### test_record_experiment_accepts_blocked_trial_with_context

function tests/nipux_cli/test_tools.py:1477

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_record_experiment_requires_metric_for_measured_trials

function tests/nipux_cli/test_tools.py:1506

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, get, json.loads

### test_record_experiment_accepts_numeric_metric_strings

function tests/nipux_cli/test_tools.py:1547

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.start_run, json.loads

### test_acknowledge_operator_context_tool_marks_context

function tests/nipux_cli/test_tools.py:1577

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.append_operator_message, db.claim_operator_messages, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_acknowledge_operator_context_requires_active_context

function tests/nipux_cli/test_tools.py:1604

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_tasks_accepts_generic_output_contracts

function tests/nipux_cli/test_tools.py:1629

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_tasks_promotes_output_contract_from_metadata

function tests/nipux_cli/test_tools.py:1664

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_tasks_downgrades_done_artifact_without_delivery_evidence

function tests/nipux_cli/test_tools.py:1695

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_tasks_downgrades_done_without_result_evidence

function tests/nipux_cli/test_tools.py:1727

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_tasks_downgrades_done_research_without_durable_evidence

function tests/nipux_cli/test_tools.py:1757

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_tasks_allows_done_research_after_source_evidence

function tests/nipux_cli/test_tools.py:1789

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.start_run, json.loads, task.get

### test_record_tasks_allows_done_research_with_metadata_evidence

function tests/nipux_cli/test_tools.py:1822

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads, task.get

### test_record_tasks_downgrades_done_experiment_without_measurement_evidence

function tests/nipux_cli/test_tools.py:1852

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_tasks_allows_done_experiment_after_measurement_evidence

function tests/nipux_cli/test_tools.py:1882

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.start_run, json.loads, task.get

### test_record_tasks_downgrades_done_action_after_read_only_shell

function tests/nipux_cli/test_tools.py:1914

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.start_run, json.loads

### test_record_tasks_allows_done_action_after_action_shell

function tests/nipux_cli/test_tools.py:1952

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.start_run, json.loads, task.get

### test_record_tasks_downgrades_done_monitor_without_defer_evidence

function tests/nipux_cli/test_tools.py:1990

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.get_job, db.start_run, json.loads

### test_record_tasks_allows_done_monitor_after_defer_evidence

function tests/nipux_cli/test_tools.py:2020

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.start_run, json.loads, task.get

### test_record_tasks_allows_done_artifact_after_delivery_evidence

function tests/nipux_cli/test_tools.py:2052

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.start_run, json.loads, task.get

### test_record_tasks_does_not_treat_stderr_redirect_as_delivery_write

function tests/nipux_cli/test_tools.py:2091

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.start_run, json.loads

### test_record_tasks_rejects_checkpoint_as_delivery_evidence

function tests/nipux_cli/test_tools.py:2130

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, DEFAULT_REGISTRY.handle, RuntimeConfig, ToolContext, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.start_run, json.loads

### _completed

function tests/nipux_cli/test_uninstall.py:7

No docstring.

Calls: subprocess.CompletedProcess

### test_uninstall_plan_includes_runtime_and_legacy_state

function tests/nipux_cli/test_uninstall.py:11

No docstring.

Calls: build_uninstall_plan, monkeypatch.setenv, str

### test_uninstall_plan_includes_configured_runtime_home

function tests/nipux_cli/test_uninstall.py:26

No docstring.

Calls: build_uninstall_plan, monkeypatch.setenv, str

### test_uninstall_runtime_removes_state_and_service_files

function tests/nipux_cli/test_uninstall.py:39

No docstring.

Calls: any, exists, monkeypatch.setenv, path.mkdir, profile.exists, str, uninstall_runtime, write_text

### test_uninstall_runtime_dry_run_keeps_files

function tests/nipux_cli/test_uninstall.py:68

No docstring.

Calls: any, monkeypatch.setenv, profile.exists, profile.mkdir, str, uninstall_runtime

### test_uninstall_installed_tool_uses_uv_when_available

function tests/nipux_cli/test_uninstall.py:81

No docstring.

Calls: calls.append, join, monkeypatch.setattr, monkeypatch.setenv, str, subprocess.CompletedProcess, tuple, uninstall_installed_tool

### runner

function tests/nipux_cli/test_uninstall.py:87

No docstring.

Calls: calls.append, subprocess.CompletedProcess, tuple

### test_uninstall_installed_tool_falls_back_to_safe_uv_paths

function tests/nipux_cli/test_uninstall.py:98

No docstring.

Calls: join, monkeypatch.setattr, monkeypatch.setenv, shim.exists, shim.parent.mkdir, shim.symlink_to, str, target.write_text, tool_bin.mkdir, tool_dir.exists, uninstall_installed_tool

### which

function tests/nipux_cli/test_uninstall.py:110

No docstring.

Calls: str

### test_installed_tool_paths_ignore_non_user_tool

function tests/nipux_cli/test_uninstall.py:128

No docstring.

Calls: Path, installed_tool_paths, monkeypatch.setattr, monkeypatch.setenv, str

### SnapshotRegistry

class tests/nipux_cli/test_worker.py:22

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:23

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:26

No docstring.

Calls: json.dumps

### SuccessRegistry

class tests/nipux_cli/test_worker.py:31

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:32

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:35

No docstring.

Calls: json.dumps

### MeasuredShellRegistry

class tests/nipux_cli/test_worker.py:40

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:41

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:44

No docstring.

Calls: json.dumps

### DiagnosticShellRegistry

class tests/nipux_cli/test_worker.py:51

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:52

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:55

No docstring.

Calls: json.dumps

### TableBenchmarkShellRegistry

class tests/nipux_cli/test_worker.py:68

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:69

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:72

No docstring.

Calls: json.dumps

### FailedTableBenchmarkShellRegistry

class tests/nipux_cli/test_worker.py:90

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:91

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:94

No docstring.

Calls: json.dumps

### FailedUrlShellRegistry

class tests/nipux_cli/test_worker.py:112

No docstring.

Calls: args.get, json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:113

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:116

No docstring.

Calls: args.get, json.dumps

### HangingLLM

class tests/nipux_cli/test_worker.py:133

No docstring.

Calls: LLMResponse, ToolCall, time.sleep

### next_action

function tests/nipux_cli/test_worker.py:134

No docstring.

Calls: LLMResponse, ToolCall, time.sleep

### SlowLLM

class tests/nipux_cli/test_worker.py:142

No docstring.

Calls: LLMResponse, ToolCall, time.sleep

### __init__

function tests/nipux_cli/test_worker.py:143

No docstring.

Calls: none

### next_action

function tests/nipux_cli/test_worker.py:146

No docstring.

Calls: LLMResponse, ToolCall, time.sleep

### RepairableLLM

class tests/nipux_cli/test_worker.py:154

No docstring.

Calls: LLMResponse, list, self.messages.append, self.responses.pop, self.tools.append

### __init__

function tests/nipux_cli/test_worker.py:157

No docstring.

Calls: list

### next_action

function tests/nipux_cli/test_worker.py:162

No docstring.

Calls: LLMResponse, self.messages.append, self.responses.pop, self.tools.append

### SourceCodeShellRegistry

class tests/nipux_cli/test_worker.py:170

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:171

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:174

No docstring.

Calls: json.dumps

### LargeShellEvidenceRegistry

class tests/nipux_cli/test_worker.py:187

No docstring.

Calls: join, json.dumps, range

### openai_tools

function tests/nipux_cli/test_worker.py:188

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:191

No docstring.

Calls: join, json.dumps, range

### ExtractRegistry

class tests/nipux_cli/test_worker.py:204

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:205

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:208

No docstring.

Calls: json.dumps

### SearchRegistry

class tests/nipux_cli/test_worker.py:221

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:222

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:225

No docstring.

Calls: json.dumps

### BrowserAndWebRegistry

class tests/nipux_cli/test_worker.py:239

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:240

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:247

No docstring.

Calls: json.dumps

### CapturingLLM

class tests/nipux_cli/test_worker.py:252

No docstring.

Calls: none

### __init__

function tests/nipux_cli/test_worker.py:253

No docstring.

Calls: none

### next_action

function tests/nipux_cli/test_worker.py:258

No docstring.

Calls: none

### ExplodingLLM

class tests/nipux_cli/test_worker.py:264

No docstring.

Calls: AssertionError

### next_action

function tests/nipux_cli/test_worker.py:265

No docstring.

Calls: AssertionError

### AntiBotBrowserRegistry

class tests/nipux_cli/test_worker.py:270

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:271

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:274

No docstring.

Calls: json.dumps

### test_system_prompt_is_contract_first_not_research_first

function tests/nipux_cli/test_worker.py:287

No docstring.

Calls: none

### test_run_one_step_executes_scripted_tool_call

function tests/nipux_cli/test_worker.py:295

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.list_artifacts, db.list_memory, db.list_steps, run_one_step

### test_run_one_step_records_estimated_usage_for_scripted_model

function tests/nipux_cli/test_worker.py:329

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, db.close, db.create_job, db.job_token_usage, db.list_events, event.get, next, run_one_step

### test_run_one_step_blocks_content_only_worker_turn

function tests/nipux_cli/test_worker.py:360

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, build_messages, db.close, db.create_job, db.get_job, db.list_steps, db.list_timeline_events, run_one_step

### test_run_one_step_repairs_content_only_worker_turn_with_tool_retry

function tests/nipux_cli/test_worker.py:393

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RepairableLLM, RuntimeConfig, ToolCall, db.close, db.create_job, db.job_token_usage, db.list_steps, len, run_one_step

### test_run_one_step_recovers_repeated_content_only_worker_turns

function tests/nipux_cli/test_worker.py:418

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, LLMResponse, RuntimeConfig, ScriptedLLM, any, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_records_context_pressure_without_spam

function tests/nipux_cli/test_worker.py:443

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, ModelConfig, RuntimeConfig, ScriptedLLM, db.close, db.create_job, db.get_job, db.list_events, get, len, run_one_step

### test_run_one_step_executes_tool_call_batch_in_order

function tests/nipux_cli/test_worker.py:474

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, any, db.close, db.create_job, db.get_job, db.list_artifacts, db.list_runs, db.list_steps, run_one_step

### test_write_artifact_reconciles_matching_report_task

function tests/nipux_cli/test_worker.py:526

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, get, item.get, len, run_one_step

### test_evidence_artifact_does_not_complete_deliverable_task

function tests/nipux_cli/test_worker.py:579

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, get, item.get, run_one_step, task.get

### test_new_deliverable_supersedes_old_auto_revision_task

function tests/nipux_cli/test_worker.py:629

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, get, next, run_one_step

### test_audit_report_draft_counts_as_deliverable_output

function tests/nipux_cli/test_worker.py:679

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step

### test_checkpoint_artifact_does_not_complete_deliverable_task

function tests/nipux_cli/test_worker.py:724

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step, task.get

### test_evidence_artifact_can_complete_research_task

function tests/nipux_cli/test_worker.py:768

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_blocks_artifact_churn_until_progress_accounting

function tests/nipux_cli/test_worker.py:812

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_activity_checkpoint_streak_blocks_more_churn_until_ledger_update

function tests/nipux_cli/test_worker.py:866

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.update_job_metadata, run_one_step

### test_task_only_checkpoint_streak_blocks_new_task_sprawl

function tests/nipux_cli/test_worker.py:911

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.update_job_metadata, run_one_step

### test_task_only_checkpoint_updates_planning_streak

function tests/nipux_cli/test_worker.py:975

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.append_finding_record, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, range, run_one_step

### test_task_resolution_checkpoint_resets_planning_streak

function tests/nipux_cli/test_worker.py:1025

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.append_task_record, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, db.update_job_metadata, range

### test_run_one_step_blocks_similar_artifact_search

function tests/nipux_cli/test_worker.py:1085

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_blocks_artifact_review_when_tasks_are_exhausted

function tests/nipux_cli/test_worker.py:1124

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, run_one_step

### test_run_one_step_recovers_repeated_guard_blocks_without_llm

function tests/nipux_cli/test_worker.py:1151

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, RuntimeConfig, any, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, enumerate, run_one_step

### test_guard_recovery_does_not_add_task_for_queue_saturation

function tests/nipux_cli/test_worker.py:1185

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, RuntimeConfig, any, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, len, range, run_one_step, startswith

### test_run_one_step_recovers_repeated_evidence_grounding_blocks

function tests/nipux_cli/test_worker.py:1241

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, RuntimeConfig, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_run_one_step_recovers_repeated_known_bad_source_blocks

function tests/nipux_cli/test_worker.py:1272

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, RuntimeConfig, any, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, range, run_one_step

### test_guard_recovery_does_not_repeat_after_recovery_step

function tests/nipux_cli/test_worker.py:1305

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.list_steps, db.start_run, range, run_one_step

### test_guard_recovery_does_not_keep_reopening_same_guard

function tests/nipux_cli/test_worker.py:1348

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.list_steps, db.start_run, range, run_one_step

### test_guard_recovery_reopens_same_guard_after_progress

function tests/nipux_cli/test_worker.py:1381

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, RuntimeConfig, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.list_steps, db.start_run, range, run_one_step, sum

### test_guard_recovery_accounts_pending_evidence_checkpoint

function tests/nipux_cli/test_worker.py:1435

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, RuntimeConfig, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, db.update_job_metadata, range, run_one_step

### test_guard_recovery_immediately_recovers_already_read_checkpoint_reread

function tests/nipux_cli/test_worker.py:1479

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, RuntimeConfig, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_prompt_does_not_tell_worker_to_reread_checkpoint_after_it_was_read

function tests/nipux_cli/test_worker.py:1519

No docstring.

Calls: AgentDB, build_messages, db.close, db.create_job, db.get_job, db.list_steps, db.update_job_metadata

### test_checkpoint_reread_block_requires_accounting_not_more_reads

function tests/nipux_cli/test_worker.py:1544

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.update_job_metadata, run_one_step

### test_already_read_checkpoint_branch_block_recovers_immediately

function tests/nipux_cli/test_worker.py:1587

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, RuntimeConfig, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, db.update_job_metadata, run_one_step

### test_evidence_grounding_ignores_format_protocol_tokens

function tests/nipux_cli/test_worker.py:1640

No docstring.

Calls: _concrete_evidence_tokens

### test_evidence_grounding_ignores_lowercase_command_shorthand_tokens

function tests/nipux_cli/test_worker.py:1664

No docstring.

Calls: _concrete_evidence_tokens

### test_record_experiment_allows_not_stub_validation_for_observed_token

function tests/nipux_cli/test_worker.py:1671

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_record_findings_ignores_generated_step_labels_as_claims

function tests/nipux_cli/test_worker.py:1720

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_write_artifact_allows_plain_prose_headings_without_evidence

function tests/nipux_cli/test_worker.py:1769

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_write_artifact_blocks_unsupported_high_risk_identifier

function tests/nipux_cli/test_worker.py:1815

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_web_search_auto_records_source_quality

function tests/nipux_cli/test_worker.py:1860

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SearchRegistry, ToolCall, all, db.close, db.create_job, db.get_job, run_one_step

### test_web_extract_auto_records_source_quality

function tests/nipux_cli/test_worker.py:1887

No docstring.

Calls: AgentDB, AppConfig, ExtractRegistry, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, next, run_one_step

### test_worker_cannot_mark_job_completed_by_default

function tests/nipux_cli/test_worker.py:1914

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step

### test_report_update_completion_claim_is_rewritten_as_checkpoint

function tests/nipux_cli/test_worker.py:1939

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, next, run_one_step

### test_run_one_step_claims_one_message_but_keeps_all_steering_in_prompt

function tests/nipux_cli/test_worker.py:1974

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, any, db.append_operator_message, db.close, db.create_job, db.get_job, db.list_timeline_events, get, run_one_step

### FailingLLM

class tests/nipux_cli/test_worker.py:2000

No docstring.

Calls: RuntimeError

### next_action

function tests/nipux_cli/test_worker.py:2001

No docstring.

Calls: RuntimeError

### HardProviderFailingLLM

class tests/nipux_cli/test_worker.py:2006

No docstring.

Calls: LLMResponseError

### next_action

function tests/nipux_cli/test_worker.py:2007

No docstring.

Calls: LLMResponseError

### test_run_one_step_records_model_failures_instead_of_raising

function tests/nipux_cli/test_worker.py:2015

No docstring.

Calls: AgentDB, AppConfig, FailingLLM, RuntimeConfig, db.close, db.create_job, db.list_runs, db.list_steps, run_one_step

### test_run_one_step_blocks_missing_tool_arguments_as_recoverable

function tests/nipux_cli/test_worker.py:2036

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.list_steps, run_one_step

### test_run_one_step_continues_after_malformed_tool_arguments_when_batch_has_more_work

function tests/nipux_cli/test_worker.py:2056

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.list_runs, db.list_steps, run_one_step

### test_run_one_step_continues_after_missing_artifact_when_batch_has_more_work

function tests/nipux_cli/test_worker.py:2082

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.list_runs, db.list_steps, run_one_step

### test_run_one_step_continues_after_empty_operator_ack_when_batch_has_more_work

function tests/nipux_cli/test_worker.py:2109

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.list_runs, db.list_steps, run_one_step

### test_run_one_step_blocks_placeholder_tool_arguments_as_recoverable

function tests/nipux_cli/test_worker.py:2136

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.list_steps, run_one_step

### test_run_one_step_blocks_truncated_optional_reference_arguments

function tests/nipux_cli/test_worker.py:2156

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.list_steps, run_one_step

### test_run_one_step_blocks_placeholder_shell_command_before_execution

function tests/nipux_cli/test_worker.py:2187

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.list_steps, run_one_step

### test_run_one_step_blocks_tool_markup_shell_command_before_execution

function tests/nipux_cli/test_worker.py:2209

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.list_steps, run_one_step

### test_run_one_step_blocks_unbalanced_shell_quotes_before_execution

function tests/nipux_cli/test_worker.py:2234

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.list_steps, run_one_step

### test_run_one_step_blocks_markdown_fenced_shell_command_before_execution

function tests/nipux_cli/test_worker.py:2261

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.list_steps, run_one_step

### test_run_one_step_times_out_stalled_model_call

function tests/nipux_cli/test_worker.py:2295

No docstring.

Calls: AgentDB, AppConfig, HangingLLM, ModelConfig, RuntimeConfig, db.close, db.create_job, db.list_steps, run_one_step

### test_repeated_model_failures_do_not_create_automatic_defer

function tests/nipux_cli/test_worker.py:2317

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, ModelConfig, RuntimeConfig, all, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, get, range

### test_legacy_model_cooldown_metadata_is_ignored

function tests/nipux_cli/test_worker.py:2348

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, db.list_events, db.update_job_metadata, next, run_one_step

### test_run_one_step_pauses_job_on_hard_provider_failure

function tests/nipux_cli/test_worker.py:2371

No docstring.

Calls: AgentDB, AppConfig, HardProviderFailingLLM, RuntimeConfig, any, db.close, db.create_job, db.get_job, db.list_events, run_one_step

### test_prompt_includes_recent_tool_arguments_and_observations

function tests/nipux_cli/test_worker.py:2392

No docstring.

Calls: Path.cwd, build_messages, str

### test_prompt_recovers_from_missing_artifact_reference

function tests/nipux_cli/test_worker.py:2415

No docstring.

Calls: build_messages

### test_prompt_does_not_inject_local_ssh_alias_context

function tests/nipux_cli/test_worker.py:2440

No docstring.

Calls: build_messages, monkeypatch.setenv, ssh_dir.mkdir, str, write_text

### test_prompt_includes_operator_steering_messages

function tests/nipux_cli/test_worker.py:2455

No docstring.

Calls: build_messages

### test_prompt_keeps_claimed_operator_context_until_acknowledged

function tests/nipux_cli/test_worker.py:2475

No docstring.

Calls: AgentDB, build_messages, db.acknowledge_operator_messages, db.append_operator_message, db.claim_operator_messages, db.close, db.create_job, db.get_job

### test_prompt_keeps_unclaimed_steering_but_not_followup_until_claimed

function tests/nipux_cli/test_worker.py:2500

No docstring.

Calls: AgentDB, build_messages, db.append_operator_message, db.close, db.create_job, db.get_job

### test_prompt_includes_context_pressure_constraint

function tests/nipux_cli/test_worker.py:2516

No docstring.

Calls: build_messages

### test_prompt_includes_cumulative_usage_pressure

function tests/nipux_cli/test_worker.py:2539

No docstring.

Calls: build_messages

### test_prompt_renders_task_contract_from_metadata_for_existing_tasks

function tests/nipux_cli/test_worker.py:2575

No docstring.

Calls: build_messages

### test_prompt_keeps_persistent_task_backlog_pressure_visible

function tests/nipux_cli/test_worker.py:2600

No docstring.

Calls: build_messages, range

### test_prompt_shows_current_task_backlog_pressure_without_prior_block

function tests/nipux_cli/test_worker.py:2631

No docstring.

Calls: build_messages, range

### test_prompt_ignores_stale_task_backlog_pressure_after_queue_is_cleaned_up

function tests/nipux_cli/test_worker.py:2652

No docstring.

Calls: build_messages

### test_run_one_step_clears_stale_task_backlog_pressure

function tests/nipux_cli/test_worker.py:2677

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, any, db.close, db.create_job, db.get_job, db.list_events, run_one_step

### test_run_one_step_records_usage_pressure_without_spam

function tests/nipux_cli/test_worker.py:2715

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, ModelConfig, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, db.list_events, get, len, run_one_step

### test_critical_usage_does_not_create_automatic_defer

function tests/nipux_cli/test_worker.py:2750

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, ModelConfig, RuntimeConfig, ScriptedLLM, ToolCall, db.append_event, db.close, db.create_job, db.get_job, get, run_one_step

### test_prompt_ignores_legacy_usage_pressure_recovery_metadata

function tests/nipux_cli/test_worker.py:2776

No docstring.

Calls: build_messages

### test_run_one_step_pauses_when_configured_cost_limit_is_reached

function tests/nipux_cli/test_worker.py:2805

No docstring.

Calls: AgentDB, AppConfig, ExplodingLLM, ModelConfig, RuntimeConfig, db.append_event, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_ignores_cost_limit_without_provider_cost_metadata

function tests/nipux_cli/test_worker.py:2834

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, ModelConfig, RuntimeConfig, ScriptedLLM, ToolCall, db.append_event, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_does_not_defer_critical_usage_after_progress

function tests/nipux_cli/test_worker.py:2861

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, ModelConfig, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.append_event, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_drops_conversation_only_chat_from_worker_prompt

function tests/nipux_cli/test_worker.py:2912

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.append_operator_message, db.close, db.create_job, db.get_job, get, run_one_step

### test_build_messages_keeps_generic_context_under_budget

function tests/nipux_cli/test_worker.py:2937

No docstring.

Calls: build_messages, len, range

### test_prompt_timeline_filters_low_signal_tool_noise

function tests/nipux_cli/test_worker.py:3012

No docstring.

Calls: build_messages, range, timeline.extend

### test_prompt_includes_durable_outcome_summary

function tests/nipux_cli/test_worker.py:3063

No docstring.

Calls: build_messages, content.split, split

### test_emergency_prompt_clipping_repeats_operator_and_next_action

function tests/nipux_cli/test_worker.py:3107

No docstring.

Calls: _render_worker_prompt, content.split, len, range, sections.append, sections.insert

### test_build_messages_keeps_rolling_memory_when_not_first

function tests/nipux_cli/test_worker.py:3124

No docstring.

Calls: build_messages

### test_build_messages_surfaces_recent_measurement_evidence_outside_state_window

function tests/nipux_cli/test_worker.py:3139

No docstring.

Calls: build_messages, range

### test_measurement_obligation_blocks_research_until_recorded

function tests/nipux_cli/test_worker.py:3180

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, MeasuredShellRegistry, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, get, run_one_step

### test_measurement_obligation_preserves_table_metric_candidates

function tests/nipux_cli/test_worker.py:3235

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, TableBenchmarkShellRegistry, ToolCall, db.close, db.create_job, db.get_job, run_one_step

### test_failed_shell_measurement_output_still_requires_accounting

function tests/nipux_cli/test_worker.py:3258

No docstring.

Calls: AgentDB, AppConfig, FailedTableBenchmarkShellRegistry, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step

### test_measurement_obligation_blocks_operator_acknowledgement_churn

function tests/nipux_cli/test_worker.py:3280

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.update_job_metadata, run_one_step

### test_pending_measurement_narrows_available_tools

function tests/nipux_cli/test_worker.py:3318

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.close, db.create_job, db.update_job_metadata, issubset, run_one_step

### test_resolution_tools_survive_task_saturation_suppression

function tests/nipux_cli/test_worker.py:3361

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.add_step, db.close, db.create_job, db.finish_step, db.start_run, issubset, range, run_one_step

### test_pending_evidence_checkpoint_narrows_available_tools

function tests/nipux_cli/test_worker.py:3409

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.close, db.create_job, db.update_job_metadata, issubset, run_one_step

### test_acknowledge_operator_context_hidden_without_active_operator_context

function tests/nipux_cli/test_worker.py:3445

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.close, db.create_job, run_one_step

### test_acknowledge_operator_context_visible_with_active_operator_context

function tests/nipux_cli/test_worker.py:3460

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.append_operator_message, db.close, db.create_job, run_one_step

### test_diagnostic_shell_output_does_not_create_measurement_obligation

function tests/nipux_cli/test_worker.py:3476

No docstring.

Calls: AgentDB, AppConfig, DiagnosticShellRegistry, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, get, run_one_step

### test_source_code_shell_output_does_not_create_measurement_obligation

function tests/nipux_cli/test_worker.py:3497

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SourceCodeShellRegistry, ToolCall, db.close, db.create_job, db.get_job, get, run_one_step

### test_prose_from_timed_command_does_not_create_measurement_obligation

function tests/nipux_cli/test_worker.py:3520

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, ProseShellRegistry, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, get, json.dumps, run_one_step

### ProseShellRegistry

class tests/nipux_cli/test_worker.py:3521

No docstring.

Calls: json.dumps

### openai_tools

function tests/nipux_cli/test_worker.py:3522

No docstring.

Calls: none

### handle

function tests/nipux_cli/test_worker.py:3525

No docstring.

Calls: json.dumps

### test_large_shell_output_must_be_saved_before_more_shell_churn

function tests/nipux_cli/test_worker.py:3557

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, LargeShellEvidenceRegistry, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, run_one_step

### test_stale_diagnostic_measurement_obligation_is_cleared

function tests/nipux_cli/test_worker.py:3585

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, get, run_one_step

### test_measurable_objective_blocks_research_after_budget_but_allows_action

function tests/nipux_cli/test_worker.py:3627

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, MeasuredShellRegistry, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, range, run_one_step

### test_measurable_objective_blocks_shell_churn_without_experiment_accounting

function tests/nipux_cli/test_worker.py:3669

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, MeasuredShellRegistry, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_measured_progress_guard_narrows_available_tools_after_shell_budget

function tests/nipux_cli/test_worker.py:3700

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, issubset, range, run_one_step

### test_measured_progress_guard_keeps_shell_available_before_shell_budget

function tests/nipux_cli/test_worker.py:3736

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, issubset, range, run_one_step

### test_measured_progress_guard_ignores_non_measurement_task_updates

function tests/nipux_cli/test_worker.py:3772

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, MeasuredShellRegistry, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_measured_progress_guard_accepts_measurement_task_update

function tests/nipux_cli/test_worker.py:3814

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, MeasuredShellRegistry, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_measurable_objective_allows_candidate_file_validation_shell_after_budget

function tests/nipux_cli/test_worker.py:3856

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, MeasuredShellRegistry, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, db.update_job_metadata, range, run_one_step

### test_prompt_includes_durable_lessons

function tests/nipux_cli/test_worker.py:3911

No docstring.

Calls: build_messages

### test_prompt_suppresses_stale_negative_lessons_when_positive_durable_evidence_exists

function tests/nipux_cli/test_worker.py:3931

No docstring.

Calls: build_messages

### test_prompt_keeps_negative_lessons_when_durable_evidence_is_negative

function tests/nipux_cli/test_worker.py:3961

No docstring.

Calls: build_messages

### test_prompt_includes_memory_graph_slice

function tests/nipux_cli/test_worker.py:3988

No docstring.

Calls: build_messages

### test_prompt_suppresses_memory_graph_nodes_matching_stale_claim_tokens

function tests/nipux_cli/test_worker.py:4035

No docstring.

Calls: AgentDB, build_messages, db.append_lesson, db.append_memory_graph_records, db.close, db.create_job, db.get_job, db.list_steps

### test_prompt_suppresses_negative_memory_graph_nodes_matching_stale_file_type

function tests/nipux_cli/test_worker.py:4074

No docstring.

Calls: AgentDB, build_messages, db.append_memory_graph_records, db.close, db.create_job, db.get_job, db.list_steps, db.update_job_metadata

### test_prompt_pushes_memory_graph_consolidation_when_ledgers_exist_without_nodes

function tests/nipux_cli/test_worker.py:4121

No docstring.

Calls: build_messages

### test_prompt_adds_memory_consolidation_guard_when_graph_lags_ledgers

function tests/nipux_cli/test_worker.py:4140

No docstring.

Calls: build_messages

### test_prompt_adds_research_balance_guard_for_execution_without_sources

function tests/nipux_cli/test_worker.py:4163

No docstring.

Calls: build_messages, range

### test_prompt_research_balance_guard_clears_when_sources_exist

function tests/nipux_cli/test_worker.py:4191

No docstring.

Calls: build_messages, range

### _source_yield_metadata

function tests/nipux_cli/test_worker.py:4217

No docstring.

Calls: range

### _source_gathering_steps

function tests/nipux_cli/test_worker.py:4252

No docstring.

Calls: range

### test_prompt_adds_source_yield_guard_when_sources_are_not_synthesized

function tests/nipux_cli/test_worker.py:4265

No docstring.

Calls: _source_gathering_steps, _source_yield_metadata, build_messages

### test_prompt_source_yield_guard_clears_when_findings_cover_sources

function tests/nipux_cli/test_worker.py:4282

No docstring.

Calls: _source_gathering_steps, _source_yield_metadata, build_messages

### test_run_one_step_blocks_more_source_gathering_when_source_yield_is_missing

function tests/nipux_cli/test_worker.py:4296

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, _source_gathering_steps, _source_yield_metadata, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_source_yield_guard_takes_priority_over_memory_consolidation

function tests/nipux_cli/test_worker.py:4334

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, _source_gathering_steps, _source_yield_metadata, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_allows_source_yield_accounting

function tests/nipux_cli/test_worker.py:4371

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, _source_gathering_steps, _source_yield_metadata, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_blocks_execution_when_research_balance_is_missing

function tests/nipux_cli/test_worker.py:4418

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_run_one_step_blocks_lesson_churn_when_research_balance_is_missing

function tests/nipux_cli/test_worker.py:4457

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_run_one_step_blocks_durable_records_with_unsupported_concrete_claims

function tests/nipux_cli/test_worker.py:4504

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, any, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, run_one_step

### test_record_experiment_blocks_unsupported_proper_noun_hardware_claims

function tests/nipux_cli/test_worker.py:4555

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step, set

### test_record_experiment_allows_supported_proper_noun_hardware_claims

function tests/nipux_cli/test_worker.py:4606

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, run_one_step

### test_record_lesson_blocks_negative_claim_that_conflicts_with_positive_evidence

function tests/nipux_cli/test_worker.py:4657

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_record_lesson_ignores_plain_titlecase_negative_conflict_tokens

function tests/nipux_cli/test_worker.py:4707

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_record_lesson_allows_negative_claim_when_evidence_is_also_negative

function tests/nipux_cli/test_worker.py:4746

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, run_one_step

### test_shell_path_recovery_prompt_shows_missing_executable

function tests/nipux_cli/test_worker.py:4790

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_shell_path_recovery_prompt_prefers_observed_candidate_executable

function tests/nipux_cli/test_worker.py:4830

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run

### test_shell_path_recovery_prompt_preserves_partial_success_paths

function tests/nipux_cli/test_worker.py:4871

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run

### test_shell_exec_blocks_bare_retry_when_candidate_executable_observed

function tests/nipux_cli/test_worker.py:4909

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_permission_failure_prompt_blocks_package_manager_retry

function tests/nipux_cli/test_worker.py:4960

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_permission_failure_prompt_mentions_non_privileged_recovery

function tests/nipux_cli/test_worker.py:5003

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run

### test_record_findings_blocks_negative_file_pattern_that_conflicts_with_positive_evidence

function tests/nipux_cli/test_worker.py:5039

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_file_pattern_grounding_ignores_hidden_path_components

function tests/nipux_cli/test_worker.py:5092

No docstring.

Calls: _file_pattern_tokens_for_grounding

### test_record_experiment_allows_classifying_observed_files_as_non_primary

function tests/nipux_cli/test_worker.py:5106

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_record_findings_requires_exact_paths_when_file_candidates_exist

function tests/nipux_cli/test_worker.py:5164

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_missing_candidate_paths_are_ranked_before_grounding_guidance

function tests/nipux_cli/test_worker.py:5217

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, join, range, run_one_step

### test_record_findings_allows_exact_candidate_path_summary

function tests/nipux_cli/test_worker.py:5267

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_evidence_grounding_blocks_positive_claim_for_missing_path

function tests/nipux_cli/test_worker.py:5312

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_evidence_grounding_checks_later_positive_path_mentions

function tests/nipux_cli/test_worker.py:5366

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_record_findings_allows_negative_file_pattern_when_evidence_is_negative

function tests/nipux_cli/test_worker.py:5415

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, run_one_step

### test_run_one_step_marks_contradicted_negative_finding_stale

function tests/nipux_cli/test_worker.py:5462

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, _ledgers_for_prompt, db.add_step, db.append_finding_record, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, get

### test_run_one_step_marks_contradicted_negative_memory_node_stale

function tests/nipux_cli/test_worker.py:5523

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, any, build_messages, db.add_step, db.append_memory_graph_records, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps

### test_record_lesson_allows_generic_strategy_without_concrete_facts

function tests/nipux_cli/test_worker.py:5592

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_record_lesson_allows_positive_checkpoint_summary_with_new_concrete_terms

function tests/nipux_cli/test_worker.py:5624

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_record_findings_blocks_single_unsupported_identifier

function tests/nipux_cli/test_worker.py:5663

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_evidence_grounding_ignores_job_context_labels

function tests/nipux_cli/test_worker.py:5708

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_evidence_grounding_blocks_unsupported_numeric_measurements

function tests/nipux_cli/test_worker.py:5753

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_evidence_grounding_ignores_record_schema_keys

function tests/nipux_cli/test_worker.py:5794

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, run_one_step

### test_evidence_grounding_uses_durable_finding_location_and_metadata

function tests/nipux_cli/test_worker.py:5837

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.append_finding_record, db.close, db.create_job, run_one_step

### test_evidence_grounding_ignores_json_literals_even_when_stale_tokens_exist

function tests/nipux_cli/test_worker.py:5880

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, db.update_job_metadata, run_one_step

### test_evidence_grounding_ignores_planning_and_status_labels

function tests/nipux_cli/test_worker.py:5919

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_blocks_memory_graph_with_unsupported_claims

function tests/nipux_cli/test_worker.py:5961

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_allows_memory_graph_identifier_labels_without_evidence

function tests/nipux_cli/test_worker.py:6008

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_still_blocks_stale_memory_graph_key_claims

function tests/nipux_cli/test_worker.py:6061

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_allows_memory_graph_grounded_in_durable_records

function tests/nipux_cli/test_worker.py:6115

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.append_finding_record, db.close, db.create_job, run_one_step

### test_run_one_step_blocks_memory_graph_grounded_only_in_stale_records

function tests/nipux_cli/test_worker.py:6157

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.append_finding_record, db.close, db.create_job, run_one_step

### test_run_one_step_allows_stale_token_when_fresh_evidence_revalidates_it

function tests/nipux_cli/test_worker.py:6205

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_allows_durable_records_grounded_in_read_artifact

function tests/nipux_cli/test_worker.py:6253

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, run_one_step

### test_run_one_step_scopes_grounding_to_cited_step

function tests/nipux_cli/test_worker.py:6310

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, run_one_step

### test_cited_step_numbers_ignore_ordinal_hash_labels

function tests/nipux_cli/test_worker.py:6370

No docstring.

Calls: _cited_step_numbers

### test_prompt_shows_evidence_grounding_tokens_after_block

function tests/nipux_cli/test_worker.py:6380

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.list_steps, db.start_run

### test_prompt_shows_missing_candidate_paths_after_grounding_block

function tests/nipux_cli/test_worker.py:6405

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.list_steps, db.start_run, range

### test_prompt_adds_ranked_current_candidates_to_stale_grounding_block

function tests/nipux_cli/test_worker.py:6445

No docstring.

Calls: AgentDB, build_messages, content.index, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata, next_constraint.index, ranked_text.index

### test_prompt_does_not_resurface_grounding_block_after_durable_resolution

function tests/nipux_cli/test_worker.py:6503

No docstring.

Calls: AgentDB, build_messages, content.split, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, split

### test_prompt_suppresses_findings_matching_stale_claim_tokens

function tests/nipux_cli/test_worker.py:6543

No docstring.

Calls: AgentDB, build_messages, db.append_finding_record, db.append_lesson, db.close, db.create_job, db.get_job, db.list_steps

### test_prompt_prioritizes_validation_for_recent_candidate_file_paths

function tests/nipux_cli/test_worker.py:6566

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata

### test_prompt_deprioritizes_recent_stub_candidate_file_paths

function tests/nipux_cli/test_worker.py:6610

No docstring.

Calls: AgentDB, build_messages, content.index, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata

### test_prompt_isolates_current_execution_focus_for_candidate_validation

function tests/nipux_cli/test_worker.py:6656

No docstring.

Calls: AgentDB, build_messages, content.index, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata, range

### test_prompt_moves_from_candidate_validation_to_candidate_use_after_positive_evidence

function tests/nipux_cli/test_worker.py:6709

No docstring.

Calls: AgentDB, build_messages, content.index, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata

### test_prompt_ranks_context_matching_candidate_paths_before_auxiliary_files

function tests/nipux_cli/test_worker.py:6752

No docstring.

Calls: AgentDB, _rank_candidate_file_paths, build_messages, content.index, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata

### test_next_action_prioritizes_candidate_file_validation_over_download_retry

function tests/nipux_cli/test_worker.py:6808

No docstring.

Calls: AgentDB, build_messages, content.index, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata

### test_prompt_ranks_late_candidate_paths_from_large_shell_listing

function tests/nipux_cli/test_worker.py:6858

No docstring.

Calls: AgentDB, build_messages, content.index, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata, join, range

### test_prompt_prioritizes_structured_candidate_file_paths

function tests/nipux_cli/test_worker.py:6899

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata

### test_prompt_filters_truncated_and_url_like_candidate_file_paths

function tests/nipux_cli/test_worker.py:6941

No docstring.

Calls: AgentDB, build_messages, db.close, db.create_job, db.get_job, db.list_steps, db.update_job_metadata

### test_candidate_path_extraction_stops_at_escaped_newline_metadata

function tests/nipux_cli/test_worker.py:6977

No docstring.

Calls: _extract_candidate_file_paths, all

### test_candidate_path_extraction_skips_globs_and_truncated_fragments

function tests/nipux_cli/test_worker.py:6990

No docstring.

Calls: _extract_candidate_file_paths

### test_prompt_resurfaces_durable_candidate_file_paths

function tests/nipux_cli/test_worker.py:7008

No docstring.

Calls: AgentDB, build_messages, db.close, db.create_job, db.get_job, db.list_steps, db.update_job_metadata

### test_prompt_resurfaces_candidate_paths_from_recent_grounding_block

function tests/nipux_cli/test_worker.py:7044

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata

### test_grounding_uses_recent_missing_candidate_paths_after_raw_evidence_ages

function tests/nipux_cli/test_worker.py:7092

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, db.update_job_metadata, range, run_one_step

### test_prompt_filters_stale_generated_and_objective_tokens

function tests/nipux_cli/test_worker.py:7163

No docstring.

Calls: AgentDB, build_messages, db.append_finding_record, db.append_lesson, db.close, db.create_job, db.get_job, db.list_steps

### test_prompt_redacts_stale_tokens_from_recent_state

function tests/nipux_cli/test_worker.py:7191

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run

### test_prompt_does_not_redact_stale_tokens_inside_exact_paths

function tests/nipux_cli/test_worker.py:7224

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata

### test_prompt_redacts_older_stale_tokens_from_task_queue

function tests/nipux_cli/test_worker.py:7278

No docstring.

Calls: build_messages, range

### test_run_one_step_requires_accounting_after_auto_checkpoint_read

function tests/nipux_cli/test_worker.py:7304

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step, str

### test_run_one_step_reads_checkpoint_before_batched_branch_work

function tests/nipux_cli/test_worker.py:7352

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.close, db.create_job, db.get_job, db.list_steps, db.update_job_metadata, run_one_step, step.get

### test_run_one_step_allows_checkpoint_read_when_deliverable_guard_is_active

function tests/nipux_cli/test_worker.py:7394

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, db.update_job_metadata, range

### test_run_one_step_accounts_checkpoint_before_batched_branch_work

function tests/nipux_cli/test_worker.py:7441

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.close, db.create_job, db.get_job, db.list_steps, db.update_job_metadata, run_one_step, step.get

### test_run_one_step_treats_guard_recovery_as_checkpoint_accounting

function tests/nipux_cli/test_worker.py:7483

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step, str

### test_checkpoint_resolution_tool_bypasses_measured_progress_guard

function tests/nipux_cli/test_worker.py:7562

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, db.update_job_metadata, range, run_one_step

### test_run_one_step_persists_checkpoint_obligation_until_accounted

function tests/nipux_cli/test_worker.py:7611

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, db.update_job_metadata, run_one_step, str

### test_run_one_step_blocks_branch_work_when_memory_graph_needs_consolidation

function tests/nipux_cli/test_worker.py:7666

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.append_experiment_record, db.append_finding_record, db.append_lesson, db.append_source_record, db.close, db.create_job, run_one_step

### test_run_one_step_allows_memory_graph_consolidation_when_guard_is_active

function tests/nipux_cli/test_worker.py:7688

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.append_experiment_record, db.append_finding_record, db.append_lesson, db.append_source_record, db.close, db.create_job, db.get_job, run_one_step

### test_prompt_adds_lesson_consolidation_guard_when_raw_lessons_sprawl

function tests/nipux_cli/test_worker.py:7730

No docstring.

Calls: build_messages, range

### test_run_one_step_blocks_more_lessons_when_lesson_sprawl_needs_graph

function tests/nipux_cli/test_worker.py:7761

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.append_lesson, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_run_one_step_allows_memory_graph_when_lesson_sprawl_is_active

function tests/nipux_cli/test_worker.py:7791

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.append_lesson, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_prompt_includes_activity_stagnation_context

function tests/nipux_cli/test_worker.py:7835

No docstring.

Calls: build_messages

### test_prompt_includes_task_planning_guard_context

function tests/nipux_cli/test_worker.py:7860

No docstring.

Calls: build_messages

### test_prompt_includes_durable_yield_pressure

function tests/nipux_cli/test_worker.py:7881

No docstring.

Calls: build_messages, range

### test_prompt_includes_finding_source_ledgers_and_reflections

function tests/nipux_cli/test_worker.py:7906

No docstring.

Calls: build_messages

### test_prompt_includes_experiment_ledger_and_best_result

function tests/nipux_cli/test_worker.py:7929

No docstring.

Calls: build_messages

### _stagnant_experiments

function tests/nipux_cli/test_worker.py:7973

No docstring.

Calls: range

### _stagnant_experiment_metadata

function tests/nipux_cli/test_worker.py:8001

No docstring.

Calls: _stagnant_experiments

### test_prompt_includes_experiment_stagnation_guard

function tests/nipux_cli/test_worker.py:8013

No docstring.

Calls: _stagnant_experiment_metadata, build_messages

### test_prompt_infers_experiment_stagnation_from_metric_direction

function tests/nipux_cli/test_worker.py:8029

No docstring.

Calls: build_messages, range

### test_prompt_does_not_treat_unmarked_improvements_as_stagnation

function tests/nipux_cli/test_worker.py:8073

No docstring.

Calls: build_messages, float, range

### test_run_one_step_blocks_branch_work_after_experiment_stagnation

function tests/nipux_cli/test_worker.py:8106

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, _stagnant_experiment_metadata, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_run_one_step_allows_branch_decision_after_experiment_stagnation

function tests/nipux_cli/test_worker.py:8139

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, _stagnant_experiment_metadata, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_run_one_step_allows_blocked_experiment_after_experiment_stagnation

function tests/nipux_cli/test_worker.py:8184

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, _stagnant_experiment_metadata, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_delivery_experiment_next_action_blocks_unrelated_research

function tests/nipux_cli/test_worker.py:8227

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.close, db.create_job, db.update_job_metadata, run_one_step

### test_research_experiment_next_action_allows_research

function tests/nipux_cli/test_worker.py:8258

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.close, db.create_job, db.update_job_metadata, run_one_step

### test_delivery_experiment_next_action_blocks_read_only_shell

function tests/nipux_cli/test_worker.py:8288

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.close, db.create_job, db.update_job_metadata, run_one_step

### test_delivery_experiment_next_action_allows_bounded_verification_shell

function tests/nipux_cli/test_worker.py:8318

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.close, db.create_job, db.update_job_metadata, run_one_step

### test_failed_next_action_requires_accounting_before_more_shell

function tests/nipux_cli/test_worker.py:8350

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, db.update_job_metadata, run_one_step

### test_failed_next_action_prompt_prioritizes_accounting

function tests/nipux_cli/test_worker.py:8404

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, db.update_job_metadata

### test_failed_next_action_narrows_available_tools_to_accounting

function tests/nipux_cli/test_worker.py:8449

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, db.update_job_metadata, issubset, run_one_step

### test_accounted_next_action_failure_does_not_keep_blocking

function tests/nipux_cli/test_worker.py:8493

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, db.update_job_metadata, run_one_step

### test_delivery_experiment_next_action_allows_write_shell

function tests/nipux_cli/test_worker.py:8551

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.close, db.create_job, db.update_job_metadata, run_one_step

### test_write_file_can_consume_recent_shell_evidence

function tests/nipux_cli/test_worker.py:8581

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, LargeShellEvidenceRegistry, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.close, db.create_job, run_one_step

### test_write_file_creates_validation_obligation_for_code_outputs

function tests/nipux_cli/test_worker.py:8609

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step, str

### test_file_validation_obligation_blocks_research_until_validated

function tests/nipux_cli/test_worker.py:8637

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.close, db.create_job, db.get_job, get, run_one_step, str

### test_delivery_experiment_next_action_allows_internal_artifact_review

function tests/nipux_cli/test_worker.py:8684

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.close, db.create_job, db.update_job_metadata, run_one_step

### test_prompt_marks_recent_anti_bot_browser_source

function tests/nipux_cli/test_worker.py:8714

No docstring.

Calls: build_messages

### test_prompt_marks_recent_captcha_browser_block

function tests/nipux_cli/test_worker.py:8734

No docstring.

Calls: build_messages

### test_prompt_includes_browser_candidate_names

function tests/nipux_cli/test_worker.py:8756

No docstring.

Calls: build_messages

### test_prompt_includes_candidate_names_from_table_cells

function tests/nipux_cli/test_worker.py:8784

No docstring.

Calls: build_messages

### test_prompt_includes_recovery_candidates_after_stale_ref

function tests/nipux_cli/test_worker.py:8820

No docstring.

Calls: build_messages

### test_run_one_step_blocks_exact_duplicate_tool_call

function tests/nipux_cli/test_worker.py:8851

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, run_one_step

### test_duplicate_artifact_read_guidance_pushes_follow_up_work

function tests/nipux_cli/test_worker.py:8872

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, artifacts.write_text, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step, str

### test_fresh_evidence_guard_takes_priority_over_duplicate_read

function tests/nipux_cli/test_worker.py:8895

No docstring.

Calls: AgentDB, AppConfig, ArtifactStore, LLMResponse, LargeShellEvidenceRegistry, RuntimeConfig, ScriptedLLM, ToolCall, artifacts.write_text, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_allows_repeated_browser_snapshot

function tests/nipux_cli/test_worker.py:8930

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SnapshotRegistry, ToolCall, db.close, db.create_job, run_one_step

### test_run_one_step_blocks_browser_tools_after_runtime_missing

function tests/nipux_cli/test_worker.py:8956

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SnapshotRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_allows_non_browser_work_after_runtime_missing

function tests/nipux_cli/test_worker.py:8998

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_skips_batched_browser_call_when_runtime_missing_and_fallback_present

function tests/nipux_cli/test_worker.py:9035

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, all, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.list_steps, db.start_run, get

### test_run_one_step_removes_browser_tools_from_schema_after_runtime_missing

function tests/nipux_cli/test_worker.py:9082

No docstring.

Calls: AgentDB, AppConfig, BrowserAndWebRegistry, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_removes_browser_tools_after_older_runtime_missing

function tests/nipux_cli/test_worker.py:9118

No docstring.

Calls: AgentDB, AppConfig, BrowserAndWebRegistry, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_run_one_step_allows_repeated_defer_for_monitor_intervals

function tests/nipux_cli/test_worker.py:9162

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, run_one_step

### test_run_one_step_blocks_self_defer_for_next_worker_turn

function tests/nipux_cli/test_worker.py:9179

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, run_one_step

### test_run_one_step_blocks_defer_without_wait_reason

function tests/nipux_cli/test_worker.py:9211

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, run_one_step

### test_run_one_step_blocks_search_after_unpersisted_extract

function tests/nipux_cli/test_worker.py:9242

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.list_artifacts, db.start_run, run_one_step, startswith

### test_prompt_tells_model_to_save_unpersisted_evidence_before_more_research

function tests/nipux_cli/test_worker.py:9287

No docstring.

Calls: AgentDB, build_messages, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.list_steps, db.start_run

### test_run_one_step_blocks_research_after_unpersisted_browser_snapshot

function tests/nipux_cli/test_worker.py:9309

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_prompt_tells_model_to_open_new_branch_when_tasks_are_exhausted

function tests/nipux_cli/test_worker.py:9344

No docstring.

Calls: build_messages

### test_prompt_pushes_deliverable_checkpoint_after_long_research

function tests/nipux_cli/test_worker.py:9364

No docstring.

Calls: build_messages, range

### test_low_priority_report_task_does_not_block_execution_task_prompt

function tests/nipux_cli/test_worker.py:9398

No docstring.

Calls: build_messages, range

### test_run_one_step_blocks_more_research_when_deliverable_needs_checkpoint

function tests/nipux_cli/test_worker.py:9437

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

### test_prompt_includes_roadmap_and_validation_constraints

function tests/nipux_cli/test_worker.py:9495

No docstring.

Calls: build_messages

### test_prompt_suggests_roadmap_for_broad_jobs_without_one

function tests/nipux_cli/test_worker.py:9527

No docstring.

Calls: build_messages

### test_run_one_step_blocks_branch_work_when_milestone_needs_validation

function tests/nipux_cli/test_worker.py:9542

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, run_one_step

### test_run_one_step_allows_milestone_validation_when_gate_is_active

function tests/nipux_cli/test_worker.py:9580

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_allows_matching_pending_milestone_evidence_action

function tests/nipux_cli/test_worker.py:9622

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, run_one_step

### test_run_one_step_allows_matching_pending_milestone_validation_evidence_action

function tests/nipux_cli/test_worker.py:9662

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, run_one_step

### test_run_one_step_blocks_non_matching_pending_milestone_action

function tests/nipux_cli/test_worker.py:9701

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, run_one_step

### test_run_one_step_blocks_wrong_milestone_validation_when_gate_is_active

function tests/nipux_cli/test_worker.py:9741

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_normalizes_matching_validation_to_active_milestone

function tests/nipux_cli/test_worker.py:9781

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_blocks_task_churn_when_roadmap_stalls

function tests/nipux_cli/test_worker.py:9831

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_step, db.start_run, range, run_one_step

### test_run_one_step_allows_roadmap_update_when_roadmap_stalls

function tests/nipux_cli/test_worker.py:9874

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_step, db.get_job, db.start_run, range, run_one_step

### test_run_one_step_blocks_branch_work_when_tasks_are_exhausted

function tests/nipux_cli/test_worker.py:9927

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, run_one_step

### test_run_one_step_allows_record_tasks_when_tasks_are_exhausted

function tests/nipux_cli/test_worker.py:9954

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, any, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_blocks_new_tasks_when_queue_is_saturated

function tests/nipux_cli/test_worker.py:9983

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, range, run_one_step

### test_run_one_step_blocks_batch_that_would_saturate_task_queue

function tests/nipux_cli/test_worker.py:10022

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, len, range, run_one_step

### test_run_one_step_executes_accounting_before_saturated_record_tasks

function tests/nipux_cli/test_worker.py:10067

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, any, db.close, db.create_job, db.get_job, db.list_steps, get, lesson.get, range, run_one_step, step.get

### test_run_one_step_blocks_batch_that_would_saturate_open_tasks

function tests/nipux_cli/test_worker.py:10111

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, len, range, run_one_step

### test_run_one_step_ignores_guard_recovery_tasks_for_queue_saturation

function tests/nipux_cli/test_worker.py:10156

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, any, db.close, db.create_job, db.get_job, range, run_one_step

### test_run_one_step_ignores_guard_recovery_tasks_for_total_sprawl

function tests/nipux_cli/test_worker.py:10196

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, any, db.close, db.create_job, db.get_job, range, run_one_step

### test_run_one_step_blocks_read_only_shell_churn

function tests/nipux_cli/test_worker.py:10236

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_allows_action_after_read_only_shell_churn

function tests/nipux_cli/test_worker.py:10269

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_allows_read_only_shell_after_durable_decision

function tests/nipux_cli/test_worker.py:10302

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_allows_explicit_download_after_read_only_shell_churn

function tests/nipux_cli/test_worker.py:10344

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_blocks_new_tasks_when_queue_sprawls

function tests/nipux_cli/test_worker.py:10377

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, range, run_one_step

### test_recent_task_saturation_keeps_record_tasks_for_existing_updates

function tests/nipux_cli/test_worker.py:10412

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, range, run_one_step

### test_repeated_task_saturation_temporarily_suppresses_record_tasks

function tests/nipux_cli/test_worker.py:10456

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, range, run_one_step

### test_chronic_backlog_suppresses_new_task_planning_tool

function tests/nipux_cli/test_worker.py:10494

No docstring.

Calls: AgentDB, AppConfig, CapturingLLM, LLMResponse, RuntimeConfig, ToolCall, db.close, db.create_job, range, run_one_step

### test_run_one_step_allows_existing_task_update_when_queue_is_saturated

function tests/nipux_cli/test_worker.py:10524

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, range, run_one_step

### test_run_one_step_allows_semantic_task_update_when_queue_is_saturated

function tests/nipux_cli/test_worker.py:10559

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, range, run_one_step

### test_run_one_step_auto_records_anti_bot_browser_source

function tests/nipux_cli/test_worker.py:10613

No docstring.

Calls: AgentDB, AntiBotBrowserRegistry, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_blocks_misleading_artifact_after_anti_bot_snapshot

function tests/nipux_cli/test_worker.py:10641

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_artifacts, db.start_run, run_one_step

### test_run_one_step_allows_blocked_source_artifact_when_acknowledged

function tests/nipux_cli/test_worker.py:10688

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.list_artifacts, db.start_run, run_one_step

### test_run_one_step_blocks_browser_loop_after_anti_bot_snapshot

function tests/nipux_cli/test_worker.py:10730

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_blocks_known_bad_browser_source_from_ledger

function tests/nipux_cli/test_worker.py:10766

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.append_source_record, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_blocks_known_bad_extract_source_from_ledger

function tests/nipux_cli/test_worker.py:10799

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.append_source_record, db.close, db.create_job, run_one_step

### test_run_one_step_allows_child_url_when_bad_web_source_is_domain_root

function tests/nipux_cli/test_worker.py:10832

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.append_source_record, db.close, db.create_job, run_one_step

### test_run_one_step_records_failed_shell_url_source

function tests/nipux_cli/test_worker.py:10865

No docstring.

Calls: AgentDB, AppConfig, FailedUrlShellRegistry, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_records_pathful_failed_shell_urls_not_root_health_checks

function tests/nipux_cli/test_worker.py:10894

No docstring.

Calls: AgentDB, AppConfig, FailedUrlShellRegistry, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.close, db.create_job, db.get_job, run_one_step

### test_run_one_step_blocks_known_bad_shell_source_family

function tests/nipux_cli/test_worker.py:10921

No docstring.

Calls: AgentDB, AppConfig, FailedUrlShellRegistry, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.close, db.create_job, run_one_step

### test_run_one_step_derives_bad_shell_source_family_from_exact_failure

function tests/nipux_cli/test_worker.py:10972

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.append_source_record, db.close, db.create_job, endswith, run_one_step

### test_run_one_step_does_not_block_entire_host_after_auth_source_families

function tests/nipux_cli/test_worker.py:11007

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.append_source_record, db.close, db.create_job, run_one_step

### test_run_one_step_blocks_known_bad_shell_source_path

function tests/nipux_cli/test_worker.py:11059

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.append_source_record, db.close, db.create_job, run_one_step

### test_run_one_step_allows_mixed_shell_command_with_bad_root_health_check

function tests/nipux_cli/test_worker.py:11106

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, SuccessRegistry, ToolCall, db.append_source_record, db.close, db.create_job, run_one_step

### test_run_one_step_saves_unpersisted_evidence_before_known_bad_source_block

function tests/nipux_cli/test_worker.py:11140

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.append_source_record, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_blocks_search_streak

function tests/nipux_cli/test_worker.py:11184

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_blocks_similar_search_query

function tests/nipux_cli/test_worker.py:11211

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, run_one_step

### test_run_one_step_reflects_every_fixed_interval

function tests/nipux_cli/test_worker.py:11242

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, build_messages, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.list_steps, db.start_run, range

### test_reflection_does_not_repeat_existing_strategy_lesson

function tests/nipux_cli/test_worker.py:11272

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.append_lesson, db.close, db.create_job, db.finish_run, db.finish_step, db.get_job, db.start_run, get, len

### test_reflection_strategy_uses_current_operator_state

function tests/nipux_cli/test_worker.py:11303

No docstring.

Calls: AgentDB, AppConfig, LLMResponse, RuntimeConfig, ScriptedLLM, ToolCall, db.add_step, db.close, db.create_job, db.finish_run, db.finish_step, db.start_run, range, run_one_step

Line-by-line

## Source Browser

Collapsed raw tracked source so the backend can be inspected directly in this page.
AGENTS.md 32 lines

       1# Development Notes
       2
       3This repo is a focused long-running worker, not a broad assistant distribution.
       4
       5## Active Surface
       6
       7- Runtime package: `nipux_cli/`
       8- Tests: `tests/nipux_cli/`
       9- Entry point: `nipux`
      10- State home: `~/.nipux` or `NIPUX_HOME`
      11- Planning notes: `plans/nipux-runtime-notes.md`
      12
      13## Constraints
      14
      15- Keep the default tool surface small and explicit.
      16- Do not reintroduce broad upstream surfaces such as gateways, skills, plugins, web UI, ACP, RL environments, voice, image generation, or arbitrary terminal execution. The chat-first terminal UI is part of Nipux's active product surface; keep it generic, minimal, and backed by persisted worker state.
      17- Preserve restartability: every worker step should persist state before and after tool execution.
      18- Store exact evidence as artifacts. Summaries should point back to artifacts instead of replacing them.
      19- Keep `memory_index` entries compact and artifact-referenced; do not use raw transcript replay as the long-term state strategy.
      20- Prefer OpenAI-compatible model serving, configured through `~/.nipux/config.yaml`.
      21- Keep runtime behavior domain-neutral. Do not add task-specific or environment-specific guards, keyword lists, examples, prompts, tools, or tests and describe them as generic framework improvements.
      22
      23## Validation
      24
      25Use the focused suite:
      26
      27```bash
      28PYTEST_ADDOPTS='' uv run --extra dev python -m pytest -q
      29uv run --extra dev ruff check --isolated nipux_cli tests/nipux_cli
      30```
      31
      32Use `nipux daemon --once --fake` for a no-model smoke test. Use `nipux logs JOB_ID --verbose` or `nipux watch JOB_ID --verbose` when inspecting what a background job is actually doing.

README.md 378 lines

       1# Nipux CLI
       2
       3```text
       4 _   _ ___ ____  _   ___  __
       5| \ | |_ _|  _ \| | | \ \/ /
       6|  \| || || |_) | | | |>  <
       7| |\  || ||  __/| |_| /_/\_\
       8|_| \_|___|_|    \__,_|
       9```
      10
      11Nipux CLI is a small, restartable worker for long-running browser, web research,
      12and command-line jobs. It supports any OpenAI-compatible local or remote model
      13endpoint. It is maintained for Nipux and built around one practical idea: keep a
      14worker moving in bounded steps, save exact evidence, learn from each branch, and
      15recover cleanly when a process or model call fails.
      16
      17- Website: [Nipux.com](https://nipux.com)
      18- Source: [github.com/nipuxx/agent-cli](https://github.com/nipuxx/agent-cli)
      19- License: [MIT](LICENSE)
      20
      21## What It Does
      22
      23Nipux runs jobs that are too long or repetitive for a single chat turn. A job can
      24search the web, operate a persistent browser profile, write artifacts, inspect
      25local files with bounded shell commands, update source and finding ledgers, and
      26continue through a daemon loop until the operator pauses or cancels it.
      27
      28The default runtime is intentionally narrow:
      29
      30- one OpenAI-compatible model endpoint chosen during setup
      31- one SQLite state store under `~/.nipux`
      32- one restartable daemon with a single-instance lock
      33- per-job artifact files for exact evidence
      34- per-job browser profiles through `agent-browser`
      35- compact memory summaries that point back to artifacts
      36- visible event history for chat, tools, artifacts, progress, errors, and digests
      37- durable ledgers for lessons, sources, findings, tasks, roadmap, and experiments
      38
      39Nipux does not include a messaging gateway, plugin marketplace, skills manager,
      40RL environment, voice stack, image stack, or broad web application. The public
      41surface is the `nipux` CLI and the focused `nipux_cli/` Python package.
      42
      43## Install
      44
      45Requirements:
      46
      47- Python 3.11+
      48- an OpenAI-compatible chat completions endpoint, local or remote
      49- optional browser automation: `npm install -g agent-browser && agent-browser install`
      50
      51Install and open the full-screen setup wizard with one command:
      52
      53```bash
      54curl -fsSL https://raw.githubusercontent.com/nipuxx/agent-cli/main/scripts/install.sh | bash
      55```
      56
      57The first-run wizard asks for the provider/model, endpoint, API key location,
      58and tool access. After the model is verified, Nipux opens the workspace chat
      59where you can describe worker jobs in plain language. It stores
      60secrets outside the git repo and writes runtime state under `~/.nipux` unless
      61`NIPUX_HOME` is set.
      62
      63Install from a local checkout while developing:
      64
      65```bash
      66git clone https://github.com/nipuxx/agent-cli.git
      67cd agent-cli
      68uv tool install --editable .
      69nipux
      70```
      71
      72Install directly from git once the repository is public:
      73
      74```bash
      75uv tool install git+https://github.com/nipuxx/agent-cli.git
      76```
      77
      78## First Run
      79
      80Run `nipux`. If this is a fresh profile, the full-screen setup wizard opens
      81immediately and locks chat/job creation until the configured model passes a real
      82chat request. The wizard writes `config.yaml` and a local `.env` template under
      83`~/.nipux` unless `NIPUX_HOME` is set. Real API keys stay in the environment or
      84`~/.nipux/.env`, not in the git repo.
      85
      86```bash
      87nipux
      88```
      89
      90After setup, `nipux` opens the workspace chat. Type a plain-English goal to spin
      91up a worker, or use `/new OBJECTIVE`. Use `/settings` to edit model, endpoint,
      92tool access, runtime, and cost fields from inside the UI.
      93
      94Manual configuration is still available for scripts or headless environments:
      95
      96```bash
      97nipux init --model local-model --base-url http://localhost:8000/v1 --api-key-env OPENAI_API_KEY
      98nipux doctor --check-model
      99```
     100
     101`nipux init` creates `~/.nipux/config.yaml` and `~/.nipux/.env` with private
     102file permissions. Later `/api-key` edits keep the secret in `~/.nipux/.env`
     103instead of writing it to config.
     104
     105Update an installed tool or source checkout from anywhere:
     106
     107```bash
     108nipux update
     109```
     110
     111When installed as a `uv tool`, `nipux update` force-refreshes the command from
     112the source repository and verifies the installed command afterward. When run
     113inside a git checkout, it fast-forwards the checkout. If a daemon is running,
     114update restarts it automatically unless `--no-restart` is used. Set
     115`NIPUX_UPDATE_SPEC` only when you need to update from a different package source.
     116
     117Inspect progress from the terminal:
     118
     119```bash
     120nipux status
     121nipux activity --follow
     122```
     123
     124On macOS, install launchd autostart:
     125
     126```bash
     127nipux autostart install --poll-seconds 0
     128nipux autostart status
     129```
     130
     131On Linux, install a user service:
     132
     133```bash
     134nipux service install
     135nipux service status
     136```
     137
     138Fully remove local runtime state when you want a fresh user install:
     139
     140```bash
     141nipux uninstall --yes
     142```
     143
     144This stops the daemon, removes launchd/systemd service files, deletes
     145`~/.nipux`, removes legacy `~/.kneepucks` state if it exists, and removes the
     146installed `nipux` command with `uv tool uninstall nipux`. Add `--keep-tool` only
     147when you intentionally want to keep the command installed.
     148
     149## Secrets
     150
     151Nipux never needs an API key in `config.yaml`. The config stores only the name
     152of the environment variable to read:
     153
     154```yaml
     155model:
     156  name: provider/model
     157  base_url: https://provider.example/v1
     158  api_key_env: PROVIDER_API_KEY
     159  # Optional fallback pricing when the provider does not return cost metadata.
     160  input_cost_per_million: null
     161  output_cost_per_million: null
     162```
     163
     164Put secrets in your shell, your process manager, or `~/.nipux/.env`:
     165
     166```bash
     167# ~/.nipux/.env
     168PROVIDER_API_KEY = <redacted>
     169```
     170
     171The repository includes `.env.example` and `config.example.yaml` as templates.
     172Do not commit real `.env`, state databases, logs, artifacts, or browser
     173profiles. The default `.gitignore` excludes those local runtime files.
     174
     175## Tool Access
     176
     177The first-run wizard and config slash commands control which generic tool groups
     178the worker can use:
     179
     180```yaml
     181tools:
     182  browser: true
     183  web: true
     184  shell: true
     185  files: true
     186```
     187
     188Use `/browser`, `/web`, `/cli-access`, and `/file-access` in the terminal UI to
     189change those switches later. Disabled tools are removed from the worker tool
     190schema and blocked if an old daemon tries to call them.
     191
     192## Local Model Examples
     193
     194Nipux talks to OpenAI-compatible `/v1/chat/completions` and `/v1/models`
     195servers. Use any serving stack that supports the model and tool-calling behavior
     196you want.
     197
     198SGLang example:
     199
     200```bash
     201python -m sglang.launch_server \
     202  --model-path "$MODEL_NAME" \
     203  --port 8000 \
     204  --context-length 262144 \
     205  --reasoning-parser auto \
     206  --tool-call-parser auto
     207```
     208
     209vLLM example:
     210
     211```bash
     212vllm serve "$MODEL_NAME" \
     213  --port 8000 \
     214  --max-model-len 262144 \
     215  --enable-auto-tool-choice \
     216  --tool-call-parser auto
     217```
     218
     219## Operator Workflow
     220
     221The no-argument CLI opens the focused job directly. Plain text becomes operator
     222steering for the next worker step. The terminal UI keeps conversation/output on
     223the left and status, jobs, saved outputs, updates, and worker activity on the
     224right. Configuration is handled through slash commands such as `/model`,
     225`/api-key`, `/base-url`, and `/context`, not a separate settings page.
     226
     227```text
     228nipux > what are you working on?
     229nipux > prioritize measured progress over notes
     230```
     231
     232For direct command use:
     233
     234```bash
     235uv run nipux status "nightly research" --full
     236uv run nipux history "nightly research"
     237uv run nipux events "nightly research" --follow
     238uv run nipux activity "nightly research" --follow
     239uv run nipux outcomes "nightly research"
     240uv run nipux outcomes --all
     241uv run nipux findings "nightly research"
     242uv run nipux tasks "nightly research"
     243uv run nipux roadmap "nightly research"
     244uv run nipux experiments "nightly research"
     245uv run nipux sources "nightly research"
     246uv run nipux memory "nightly research"
     247uv run nipux metrics "nightly research"
     248uv run nipux usage "nightly research"
     249uv run nipux artifacts "nightly research" --paths
     250```
     251
     252Use `nipux health` for daemon truth without opening the dashboard. It reports
     253the lock state, heartbeat, recent failures, log paths, autostart state, focused
     254job, and latest daemon events.
     255
     256### Seeing What It Actually Did
     257
     258Use these views when a job has been running unattended:
     259
     260- `nipux outcomes JOB` or the **Outcomes** pane: durable work grouped by time,
     261  including saved outputs, findings, measurements, decisions, lessons, and file
     262  changes.
     263- `nipux outcomes --all`: latest durable work and saved outputs for every job,
     264  useful when several agents have been running in the background.
     265- `nipux activity JOB --follow` or the **Work** pane: the raw live tool stream
     266  for debugging what the worker is doing right now.
     267- `nipux usage JOB`: model calls, context pressure, output tokens, and cost when
     268  the provider returns cost metadata. If the provider does not return cost,
     269  configure `/input-cost` and `/output-cost` to estimate it from token counts.
     270- `nipux digest JOB` and `nipux daily-digest`: durable summary reports that
     271  include progress counts, active operator context, experiments, artifacts, and
     272  token/cost usage.
     273
     274## Tool Surface
     275
     276The worker exposes a deliberately small tool registry:
     277
     278- `browser_navigate`
     279- `browser_snapshot`
     280- `browser_click`
     281- `browser_type`
     282- `browser_scroll`
     283- `browser_back`
     284- `browser_press`
     285- `browser_console`
     286- `web_search`
     287- `web_extract`
     288- `shell_exec`
     289- `write_file`
     290- `write_artifact`
     291- `read_artifact`
     292- `search_artifacts`
     293- `update_job_state`
     294- `defer_job`
     295- `report_update`
     296- `record_lesson`
     297- `acknowledge_operator_context`
     298- `record_source`
     299- `record_findings`
     300- `record_tasks`
     301- `record_roadmap`
     302- `record_milestone_validation`
     303- `record_experiment`
     304- `send_digest_email`
     305
     306`shell_exec` is bounded with timeouts and output capture. Browser sessions use
     307per-job profiles under `~/.nipux/browser-profiles/`. Anti-bot, CAPTCHA, login,
     308and paywall pages are recorded as visible source-quality warnings; Nipux does
     309not bypass protections.
     310
     311Workers can use `defer_job` for scheduled follow-up, monitor intervals, or long
     312external processes that are actually waiting on time to pass. Deferred jobs stay
     313runnable but show as waiting until their next check time, so the daemon can keep
     314other work moving without burning model calls on repeated polling.
     315
     316## Command Reference
     317
     318```bash
     319nipux init [--force] [--openrouter] [--model MODEL] [--base-url URL] [--api-key-env ENV]
     320nipux update [--path PATH] [--allow-dirty] [--no-restart]
     321nipux uninstall [--yes] [--dry-run] [--keep-legacy] [--keep-tool]
     322nipux doctor [--check-model]
     323nipux shell [--status]
     324nipux create "objective" [--title TITLE] [--kind KIND] [--cadence CADENCE]
     325nipux jobs
     326nipux ls
     327nipux focus [JOB_TITLE]
     328nipux rename JOB_TITLE --title NEW_TITLE
     329nipux delete JOB_TITLE [--keep-files]
     330nipux chat [JOB_TITLE] [--no-history]
     331nipux steer [--job JOB_TITLE] MESSAGE
     332nipux pause [JOB_TITLE] [note...]
     333nipux resume [JOB_TITLE]
     334nipux cancel [JOB_TITLE] [note...]
     335nipux start [--poll-seconds N]
     336nipux stop
     337nipux autostart install|status|uninstall [--poll-seconds N]
     338nipux service install|status|uninstall [--poll-seconds N]
     339nipux browser-dashboard [--port N] [--foreground] [--stop]
     340nipux health
     341nipux status [JOB_TITLE] [--full] [--json]
     342nipux history [JOB_TITLE] [--full] [--json]
     343nipux events [JOB_TITLE] [--follow] [--json]
     344nipux activity [JOB_TITLE] [--follow] [--verbose]
     345nipux updates [JOB_TITLE]
     346nipux outcomes [JOB_TITLE] [--all]
     347nipux dashboard [JOB_TITLE]
     348nipux findings [JOB_TITLE] [--limit N] [--json]
     349nipux tasks [JOB_TITLE] [--limit N] [--status STATUS] [--json]
     350nipux roadmap [JOB_TITLE] [--limit N] [--json]
     351nipux experiments [JOB_TITLE] [--limit N] [--status STATUS] [--json]
     352nipux sources [JOB_TITLE] [--limit N] [--json]
     353nipux memory [JOB_TITLE]
     354nipux metrics [JOB_TITLE]
     355nipux usage [JOB_TITLE] [--json]
     356nipux artifacts [JOB_TITLE] [--paths]
     357nipux artifact QUERY_OR_TITLE [--job JOB_TITLE]
     358nipux lessons [JOB_TITLE]
     359nipux learn [--job JOB_TITLE] [--category CATEGORY] LESSON
     360nipux logs [JOB_TITLE] [--limit N] [--verbose]
     361nipux outputs [JOB_TITLE] [--limit N] [--verbose]
     362nipux watch JOB_TITLE [--verbose]
     363nipux run-one JOB_TITLE [--fake]
     364nipux work [JOB_TITLE] [--steps N] [--verbose] [--dashboard]
     365nipux run [JOB_TITLE] [--poll-seconds N] [--no-follow]
     366nipux daemon [--once] [--fake] [--verbose] [--poll-seconds N]
     367nipux digest JOB_TITLE
     368nipux daily-digest [--day YYYY-MM-DD]
     369```
     370
     371## Development
     372
     373```bash
     374PYTEST_ADDOPTS='' uv run --extra dev python -m pytest -q
     375uv run --extra dev ruff check --isolated nipux_cli tests/nipux_cli
     376```
     377
     378The active implementation notes live in `plans/nipux-runtime-notes.md`.

RELEASE_CHECKLIST.md 38 lines

       1# Release Checklist
       2
       3Use this before sharing the repository with outside users.
       4
       5## Secrets
       6
       7- No real API keys in git.
       8- `config.yaml` examples use `model.api_key_env`, not literal keys.
       9- `.env`, `.env.*`, state databases, logs, artifacts, and browser profiles are ignored.
      10- `nipux doctor` reports missing remote API-key environment variables without printing key values.
      11
      12## Install
      13
      14- `uv tool install --editable .` works from a checkout.
      15- `uv run nipux --help` works without installing.
      16- `NIPUX_HOME=$(mktemp -d) uv run nipux` opens the first-run terminal UI, not argparse help or an ASCII-only prompt.
      17- `nipux init` writes the default Qwen/OpenRouter `~/.nipux/config.yaml` and a blank `~/.nipux/.env` template.
      18- `nipux doctor` passes for local runtime checks after initialization.
      19- `nipux daemon --once --fake` runs without a model key.
      20
      21## Runtime
      22
      23- `nipux start`, `nipux stop`, and `nipux restart` recover stale daemon state.
      24- `nipux status`, `nipux activity`, `nipux history`, and `nipux artifacts` expose enough state to debug jobs.
      25- Worker prompts stay bounded and do not replay raw transcript history.
      26- Operator chat that is only conversational stays in history but does not remain active worker context.
      27- Measurable jobs record experiments instead of treating notes as progress.
      28- Status, outcomes, and work panes show different layers clearly: jobs and latest outputs, durable progress by hour, and raw tool/console events.
      29
      30## Validation
      31
      32```bash
      33python -m compileall nipux_cli tests/nipux_cli
      34uv run --extra dev python -m pytest tests/nipux_cli -q
      35uv run --extra dev ruff check nipux_cli tests/nipux_cli
      36rg -n --hidden -S "(sk-[A-Za-z0-9_-]{20,}|OPENROUTER_API_KEY[=].+|OPENAI_API_KEY[=].+|Bearer\\s+[A-Za-z0-9._-]{20,})" . \
      37  -g '!uv.lock' -g '!**/__pycache__/**' -g '!*.db' -g '!*.log' -g '!*.pyc'
      38```

config.example.yaml 27 lines

       1model:
       2  name: local-model
       3  base_url: http://localhost:8000/v1
       4  api_key_env: OPENAI_API_KEY
       5  context_length: 262144
       6  input_cost_per_million: null
       7  output_cost_per_million: null
       8runtime:
       9  max_step_seconds: 600
      10  max_steps_per_run: 1
      11  artifact_inline_char_limit: 12000
      12  daily_digest_enabled: true
      13  daily_digest_time: "08:00"
      14  max_job_cost_usd: null
      15tools:
      16  browser: true
      17  web: true
      18  shell: true
      19  files: true
      20email:
      21  enabled: false
      22  smtp_host: ""
      23  smtp_port: 587
      24  username: ""
      25  password_env: NIPUX_EMAIL_PASSWORD
      26  from_addr: ""
      27  to_addr: ""

docs/long-running-memory-graph-design.md 62 lines

       1# Long-Running Memory Graph Design
       2
       3Nipux needs long-running workers that keep improving instead of flattening into repeated search, notes, or shallow checkpoints. The backend now treats each job as having a small durable "brain": a job-local memory graph made of connected nodes and links. It is not task-specific and does not require embeddings or a new service to be useful.
       4
       5## Research Takeaways
       6
       7- **Complementary learning systems:** human memory separates fast episodic capture from slower semantic consolidation. The hippocampus rapidly stores separated episodes while cortex gradually extracts structure. Nipux mirrors this with recent events/steps as fast episodic traces and `memory_graph` nodes as consolidated reusable knowledge. Source: [O'Reilly and Norman, 2002](https://collaborate.princeton.edu/en/publications/hippocampal-and-neocortical-contributions-to-memory-advances-in-t) and [McClelland et al., 1995](https://colab.ws/articles/10.1037/0033-295x.102.3.419).
       8- **Sleep/consolidation:** memory consolidation strengthens relevant traces and reorganizes them into associations that support later inference. Nipux should periodically turn raw work into compact graph nodes and edges instead of replaying full history. Source: [Born and Wilhelm, 2012](https://link.springer.com/article/10.1007/s00426-011-0335-6) and [Diekelmann and Born, 2010](https://www.nature.com/articles/nrn2762).
       9- **Reflexion:** agents improve without weight updates by writing verbal reflections into episodic memory after feedback. Nipux already has lessons and reflection; the graph adds structure so reflections can connect to facts, decisions, tasks, and evidence. Source: [Reflexion](https://huggingface.co/papers/2303.11366).
      10- **Generative Agents:** believable long-lived agents combine memory stream, retrieval, reflection, and planning. Nipux should keep the event stream, but retrieve distilled context through durable ledgers and graph nodes. Source: [Generative Agents](https://huggingface.co/papers/2304.03442).
      11- **MemGPT:** OS-style memory tiers let fixed-context models use long histories by paging between prompt context and archival memory. Nipux's prompt now gets only a ranked slice of graph memory, with `search_memory_graph` for deeper recall. Source: [MemGPT](https://huggingface.co/papers/2310.08560).
      12- **Voyager:** long-horizon improvement comes from an automatic curriculum, a growing reusable skill library, and iterative self-verification. Nipux's graph supports this by representing skills, strategies, open questions, decisions, and evidence links as reusable nodes. Source: [Voyager](https://voyager.minedojo.org/).
      13- **Agent memory surveys and graph memory work:** recent surveys and systems emphasize memory operations: write, retrieve, update, consolidate, forget/deprecate, and evaluate. Graph memory helps preserve relationships and temporal change better than a flat note list. Sources: [LLM Agent Memory Survey](https://huggingface.co/papers/2404.13501), [AriGraph](https://huggingface.co/papers/2407.04363), [Zep](https://huggingface.co/papers/2501.13956).
      14
      15## Backend Shape
      16
      17Each job can now maintain metadata under `memory_graph`:
      18
      19- `nodes`: connected notes with `kind`, `status`, `summary`, `salience`, `confidence`, `tags`, `parent_key`, `links`, and `evidence_refs`.
      20- `edges`: typed links between nodes such as `supports`, `replaces`, `raises`, `blocks`, or `depends_on`.
      21- Nodes are generic: `episode`, `fact`, `strategy`, `skill`, `question`, `decision`, `constraint`, `artifact`, `source`, `task`, `experiment`, and `milestone`.
      22
      23The worker gets a compact `Memory graph` prompt section that ranks active, salient, recent, and procedural nodes. It can call `search_memory_graph` when it needs deeper recall. It can call `record_memory_graph` whenever new work should become reusable knowledge.
      24
      25Operators can inspect the same graph with `nipux memory --graph`, which writes a self-contained clickable HTML artifact. The view uses a local canvas renderer, needs no external network assets, and lets the operator rotate, zoom, search, and click nodes to inspect summaries, evidence refs, tags, and links.
      26
      27The worker also has a generic consolidation guard: once findings, sources, experiments, lessons, resolved tasks, or roadmap milestones accumulate faster than graph nodes and links, more branch churn is blocked until the worker calls `record_memory_graph` or records why the current branch has no reusable memory value.
      28
      29## Live Model Smoke
      30
      31Use `scripts/live_memory_graph_smoke.py` to verify a real OpenAI-compatible model can follow the graph-consolidation contract. The script creates a temporary Nipux home, disables side-effect tools, seeds generic durable job state, and runs a few worker turns. It succeeds only after the model calls `record_memory_graph` and creates at least one node.
      32
      33Example:
      34
      35```bash
      36OPENROUTER_API_KEY = <redacted>
      37```
      38
      39The key is read from the configured environment variable and is never printed. If no key is present, the script exits before making a network request.
      40
      41Latest smoke result:
      42
      43- Model: `qwen/qwen3.6-27b`
      44- Provider path: OpenAI-compatible chat completions through OpenRouter
      45- Isolation: temporary Nipux home with browser, web, shell, and file tools disabled
      46- Result: first worker step called `record_memory_graph`
      47- Graph written: 7 nodes and 8 edges
      48
      49## Why This Should Improve Long Runs
      50
      51- Raw history stays available in events/artifacts, but the model sees a compact graph slice.
      52- Bad or small models get explicit, typed memory instead of relying on implicit recap.
      53- Repeated branches can be deprecated instead of merely summarized.
      54- Useful strategies and skills can compound across hundreds or thousands of actions.
      55- Open questions remain visible as first-class nodes, making it harder for the worker to drift away from unresolved blockers.
      56
      57## Next Backend Slices
      58
      59- Add periodic deterministic consolidation that proposes graph nodes from recent events when the model fails to do it.
      60- Tune graph-aware stagnation checks from real runs: if a branch has no new node, edge, validation, experiment, or deliverable after a budget, force consolidation or branch rejection.
      61- Add better retrieval scoring using local embeddings when available, while keeping lexical fallback mandatory.
      62- Add live UI/status counters for memory graph growth: new nodes, active questions, deprecated paths, and current strategy.

docs/pi-agent-core-port-plan.md 267 lines

       1# Pi Agent Core Port Plan
       2
       3Research date: 2026-04-30
       4
       5Sources:
       6- https://github.com/badlogic/pi-mono
       7- https://github.com/badlogic/pi-mono/tree/main/packages/agent
       8- https://github.com/badlogic/pi-mono/tree/main/packages/coding-agent
       9- https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/session.md
      10- https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/sdk.md
      11- https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/rpc.md
      12
      13Pi is MIT licensed, so direct adaptation is allowed if we preserve attribution
      14where substantial code is ported. The right move is not to copy the full
      15TypeScript app into Nipux. The right move is to port the small generic runtime
      16ideas from `packages/agent` and keep Nipux's SQLite daemon, tools, and
      17multi-job persistence.
      18
      19## What Pi Does Better
      20
      21Pi's core is a stateful agent loop, not a "one next action" prompt wrapper.
      22The important files are:
      23
      24- `packages/agent/src/agent-loop.ts`
      25- `packages/agent/src/agent.ts`
      26- `packages/agent/src/types.ts`
      27- `packages/coding-agent/src/core/session-manager.ts`
      28- `packages/coding-agent/src/core/agent-session.ts`
      29- `packages/coding-agent/src/core/compaction/*`
      30
      31Key behaviors to port:
      32
      331. Evented loop as the runtime contract.
      34   Pi emits `agent_start`, `turn_start`, `message_start`, `message_update`,
      35   `message_end`, `tool_execution_start`, `tool_execution_update`,
      36   `tool_execution_end`, `turn_end`, and `agent_end`. Nipux has events, but
      37   currently treats a daemon step as the main unit. We should make these events
      38   first-class and derive UI/status from them.
      39
      402. Real transcript state.
      41   Pi keeps `AgentMessage[]` as state and converts it to LLM messages only at
      42   the model boundary. Nipux currently rebuilds each prompt from job metadata,
      43   memory, recent steps, and ledgers. That works, but it loses the clean
      44   distinction between visible transcript, UI-only records, and model context.
      45
      463. Context transform boundary.
      47   Pi uses `transformContext(messages)` before `convertToLlm(messages)`. This is
      48   the exact place Nipux should inject durable operator context, compact memory,
      49   task contracts, ledgers, and active constraints without polluting raw history.
      50
      514. Steering and follow-up queues.
      52   Pi splits queued user input into:
      53   - `steer`: delivered after current tool execution and before the next model turn.
      54   - `followUp`: delivered only when the agent would otherwise stop.
      55   Nipux already has `steer` and `follow_up` metadata, but delivery is bolted
      56   onto single steps. This should move into the agent core.
      57
      585. Hookable tool preflight and postprocessing.
      59   Pi has `beforeToolCall` and `afterToolCall`. Nipux has good generic guards,
      60   but they live inside `worker.py`. They should become hooks:
      61   - duplicate/repetition guard
      62   - artifact obligation guard
      63   - measurement obligation guard
      64   - source quality guard
      65   - experiment accounting after measurable shell output
      66
      676. Tool batch semantics.
      68   Pi can prepare tool calls sequentially, execute safe tools in parallel, and
      69   still persist tool-result messages in assistant order. Nipux currently
      70   executes only the first tool call. That leaves useful model intent unused.
      71
      727. Compaction as session structure, not just memory refresh.
      73   Pi stores compaction entries in the session tree. Full history remains, but
      74   future model context sees a summary plus kept recent messages. Nipux has
      75   `memory_index`, but it should add explicit compaction entries tied to the
      76   transcript path.
      77
      788. Continue semantics.
      79   Pi has `continue()` for retries after errors or compaction. Nipux currently
      80   creates a new run every step. Continue semantics would make recovery cleaner
      81   after model errors, context overflow, daemon restart, and queued messages.
      82
      83## Nipux Mapping
      84
      85Current Nipux files:
      86
      87- `nipux_cli/worker.py`: prompt building, step execution, guards, reflection.
      88- `nipux_cli/daemon.py`: forever loop, lock, heartbeat, multi-job scheduling.
      89- `nipux_cli/db.py`: SQLite state, events, job metadata, ledgers.
      90- `nipux_cli/operator_context.py`: durable operator message filtering.
      91- `nipux_cli/tools.py`: tool registry and tool execution.
      92- `nipux_cli/compression.py`: compact memory refresh.
      93- `nipux_cli/cli.py`: chat/TUI/status/output rendering.
      94
      95Target files:
      96
      97- Add `nipux_cli/agent_core.py`
      98  - Python port of Pi's small `Agent`, `PendingMessageQueue`, event types,
      99    tool result types, and loop control.
     100  - Keep attribution header because it is directly inspired by Pi's MIT code.
     101  - Support non-streaming model responses first, then streaming later.
     102
     103- Add `nipux_cli/session.py`
     104  - Load/save transcript entries for a job.
     105  - Build current session context from entries plus compaction records.
     106  - Keep SQLite as source of truth instead of JSONL files, but use Pi's entry
     107    shape: `message`, `compaction`, `branch_summary`, `custom`,
     108    `custom_message`, `model_change`, and `label`.
     109
     110- Refactor `nipux_cli/worker.py`
     111  - Move prompt assembly into `transform_context`.
     112  - Move `_blocked_tool_call_result` into `before_tool_call`.
     113  - Move measurement/source/artifact side effects into `after_tool_call`.
     114  - Replace "only first tool call" execution with core loop tool execution.
     115  - Preserve one bounded heartbeat by limiting wall-clock/tool budget per daemon
     116    tick, not by discarding the agent loop structure.
     117
     118- Extend `nipux_cli/db.py`
     119  - Add a `session_entries` table:
     120    - `id`
     121    - `job_id`
     122    - `parent_id`
     123    - `entry_type`
     124    - `created_at`
     125    - `payload_json`
     126  - Add `job_session_state` metadata for current leaf and compaction stats.
     127  - Backfill existing `events`/`steps` into session view lazily.
     128
     129- Keep `nipux_cli/daemon.py`
     130  - Do not replace the daemon. Pi is mostly single-session interactive; Nipux
     131    needs multi-job background scheduling.
     132  - The daemon should call `AgentSession.continue_or_prompt()` for whichever job
     133    is runnable, then keep heartbeating while the agent loop emits events.
     134
     135## Implementation Sequence
     136
     137### Commit 1: Agent Core Skeleton
     138
     139Create `agent_core.py` with:
     140
     141- `AgentMessage`
     142- `AgentToolCall`
     143- `AgentToolResult`
     144- `AgentEvent`
     145- `PendingMessageQueue`
     146- `AgentState`
     147- `Agent`
     148
     149Support:
     150
     151- `prompt(messages)`
     152- `continue_()`
     153- `steer(message)`
     154- `follow_up(message)`
     155- `abort()`
     156- `wait_for_idle()`
     157- event subscription
     158- sequential tool execution only
     159- `before_tool_call`
     160- `after_tool_call`
     161- `transform_context`
     162- `convert_to_llm`
     163
     164Tests:
     165
     166- event order matches Pi's documented event order
     167- steering is delivered after tool execution
     168- follow-up waits until no tool calls remain
     169- prompt/continue reject concurrent runs
     170- tool errors become tool result messages instead of crashing the loop
     171
     172### Commit 2: Session Entries
     173
     174Add SQLite session entries and a `SessionManager` equivalent.
     175
     176Tests:
     177
     178- append messages with parent IDs
     179- build context from current leaf
     180- compaction summary appears before kept messages
     181- full raw history remains queryable
     182- branch summaries can be represented even if UI does not expose branching yet
     183
     184### Commit 3: Worker Integration
     185
     186Make `run_one_step` use the Pi-style agent loop.
     187
     188Important constraint:
     189
     190Nipux should still be generic and background-safe. Do not encode any objective,
     191host, model, source, or task domain. The old guards stay generic and move into
     192hooks.
     193
     194Tests:
     195
     196- model can call multiple tools and all are persisted in order
     197- duplicate/measurement/artifact guards block through `before_tool_call`
     198- measurable output creates obligations through `after_tool_call`
     199- operator steer persists until acknowledged and is injected through the queue
     200- follow-up waits behind active branch work
     201
     202### Commit 4: Compaction
     203
     204Replace fixed memory refresh as the main context strategy with session
     205compaction:
     206
     207- estimate context from last usage when available
     208- truncate tool results for summarization
     209- store compaction entries
     210- rebuild model context from compaction plus recent path
     211- continue after overflow or threshold compaction
     212
     213Tests:
     214
     215- long transcript compacts without losing recent messages
     216- compaction failure does not crash daemon
     217- queued messages survive compaction
     218- context overflow retry uses `continue_()`
     219
     220### Commit 5: UI/Event Stream Cleanup
     221
     222Make CLI/TUI read the event stream and session entries, not ad hoc step text.
     223
     224Tests:
     225
     226- chat shows user/assistant transcript
     227- right pane shows job/daemon/session stats
     228- activity shows tool start/update/end events
     229- history can show full transcript, compacted transcript, and raw events
     230
     231## Why This Should Fix The Current Failure Mode
     232
     233Nipux currently has many good guards, but the model is still treated like a
     234stateless planner that gets one tool call per daemon step. That encourages
     235research churn because the loop boundary is outside the model's natural
     236tool-result feedback cycle.
     237
     238Pi's design keeps the model inside a coherent turn loop:
     239
     2401. User/operator/context enters as messages.
     2412. Assistant proposes tool calls.
     2423. Tools execute and return tool-result messages.
     2434. The assistant immediately sees those results.
     2445. Steering and follow-up are delivered at well-defined boundaries.
     2456. Compaction preserves the useful path instead of stuffing every summary into
     246   every future prompt.
     247
     248Porting that structure should make Nipux feel less like a step counter and more
     249like an actual long-running agent runtime.
     250
     251## What Not To Copy
     252
     253Do not copy Pi's task-specific extension examples into Nipux core.
     254
     255Do not make the harness depend on Node, Bun, or the Pi TUI.
     256
     257Do not encode any SSH, model, inference, lead-finding, browser-source, or local
     258machine assumptions. Everything here must stay generic:
     259
     260- transcript
     261- events
     262- queues
     263- tool hooks
     264- compaction
     265- session state
     266- UI rendering over events
     267

nipux_cli/__init__.py 11 lines

       1"""Minimal daemon-first Nipux runtime.
       2
       3This package owns the daemon, state store, model adapter, artifact store, and
       4fixed tool surface.
       5"""
       6
       7__all__ = [
       8    "__version__",
       9]
      10
      11__version__ = "0.1.0"

nipux_cli/__main__.py 9 lines

       1"""Run the Nipux CLI with ``python -m nipux_cli``."""
       2
       3from __future__ import annotations
       4
       5from nipux_cli.cli import main
       6
       7
       8if __name__ == "__main__":
       9    main()

nipux_cli/artifacts.py 138 lines

       1"""Artifact file storage for long-running jobs."""
       2
       3from __future__ import annotations
       4
       5import hashlib
       6import re
       7from dataclasses import dataclass
       8from pathlib import Path
       9from typing import Any
      10
      11from nipux_cli.db import AgentDB, new_id, utc_now
      12
      13_SAFE_NAME_RE = re.compile(r"[^A-Za-z0-9._-]+")
      14
      15
      16def safe_filename(value: str, *, default: str = "artifact") -> str:
      17    cleaned = _SAFE_NAME_RE.sub("-", value.strip()).strip(".-")
      18    return (cleaned or default)[:96]
      19
      20
      21def sha256_text(text: str) -> str:
      22    return hashlib.sha256(text.encode("utf-8")).hexdigest()
      23
      24
      25@dataclass(frozen=True)
      26class StoredArtifact:
      27    id: str
      28    path: Path
      29    sha256: str
      30    title: str | None = None
      31    summary: str | None = None
      32
      33
      34class ArtifactStore:
      35    def __init__(self, home: str | Path, db: AgentDB | None = None):
      36        self.home = Path(home)
      37        self.db = db
      38        self.home.mkdir(parents=True, exist_ok=True)
      39
      40    def job_dir(self, job_id: str) -> Path:
      41        path = self.home / "jobs" / job_id / "artifacts"
      42        path.mkdir(parents=True, exist_ok=True)
      43        return path
      44
      45    def _assert_inside_home(self, path: Path) -> Path:
      46        resolved = path.resolve()
      47        root = self.home.resolve()
      48        try:
      49            resolved.relative_to(root)
      50        except ValueError as exc:
      51            raise ValueError(f"Refusing to read outside agent home: {path}") from exc
      52        return resolved
      53
      54    def write_text(
      55        self,
      56        *,
      57        job_id: str,
      58        content: str,
      59        title: str | None = None,
      60        summary: str | None = None,
      61        artifact_type: str = "text",
      62        run_id: str | None = None,
      63        step_id: str | None = None,
      64        metadata: dict[str, Any] | None = None,
      65    ) -> StoredArtifact:
      66        suffix = "html" if artifact_type == "html" else "md" if artifact_type in {"digest", "markdown", "text"} else "txt"
      67        stem = safe_filename(title or artifact_type)
      68        timestamp = utc_now().replace("+00:00", "Z").replace(":", "")
      69        filename = f"{timestamp}-{stem}-{new_id('file')}.{suffix}"
      70        path = self.job_dir(job_id) / filename
      71        path.write_text(content, encoding="utf-8")
      72        digest = sha256_text(content)
      73        artifact_id = new_id("art")
      74        if self.db is not None:
      75            artifact_id = self.db.add_artifact(
      76                job_id=job_id,
      77                run_id=run_id,
      78                step_id=step_id,
      79                path=path,
      80                sha256=digest,
      81                artifact_type=artifact_type,
      82                title=title,
      83                summary=summary,
      84                metadata=metadata,
      85            )
      86        return StoredArtifact(id=artifact_id, path=path, sha256=digest, title=title, summary=summary)
      87
      88    def read_text(self, artifact_id_or_path: str) -> str:
      89        path = Path(artifact_id_or_path)
      90        if self.db is not None and not path.exists():
      91            path = Path(self.db.get_artifact(artifact_id_or_path)["path"])
      92        safe_path = self._assert_inside_home(path)
      93        return safe_path.read_text(encoding="utf-8")
      94
      95    def search_text(self, *, job_id: str, query: str, limit: int = 10) -> list[dict[str, Any]]:
      96        if self.db is None:
      97            return []
      98        query_lower = query.lower().strip()
      99        results: list[dict[str, Any]] = []
     100        for artifact in self.db.list_artifacts(job_id, limit=250):
     101            haystack = " ".join(
     102                str(artifact.get(key) or "") for key in ("title", "summary", "type")
     103            ).lower()
     104            content = ""
     105            if query_lower and query_lower not in haystack:
     106                try:
     107                    content = self.read_text(artifact["id"])
     108                except OSError:
     109                    content = ""
     110                if query_lower not in content.lower():
     111                    continue
     112            elif not query_lower:
     113                try:
     114                    content = self.read_text(artifact["id"])
     115                except OSError:
     116                    content = ""
     117            if not content:
     118                try:
     119                    content = self.read_text(artifact["id"])
     120                except OSError:
     121                    content = ""
     122            excerpt = content[:500]
     123            if query_lower:
     124                idx = content.lower().find(query_lower)
     125                if idx >= 0:
     126                    start = max(0, idx - 160)
     127                    excerpt = content[start:start + 500]
     128            results.append({
     129                "id": artifact["id"],
     130                "title": artifact.get("title"),
     131                "type": artifact.get("type"),
     132                "path": artifact.get("path"),
     133                "summary": artifact.get("summary"),
     134                "excerpt": excerpt,
     135            })
     136            if len(results) >= limit:
     137                break
     138        return results

nipux_cli/browser.py 189 lines

       1"""Small `agent-browser` wrapper for the Nipux runtime."""
       2
       3from __future__ import annotations
       4
       5import json
       6import os
       7import shutil
       8import subprocess
       9import tempfile
      10import hashlib
      11from pathlib import Path
      12from typing import Any
      13
      14from nipux_cli.config import AppConfig
      15from nipux_cli.source_quality import anti_bot_reason
      16
      17
      18def _find_agent_browser() -> list[str]:
      19    direct = shutil.which("agent-browser")
      20    if direct:
      21        return [direct]
      22    if shutil.which("npx"):
      23        return ["npx", "--yes", "agent-browser"]
      24    raise FileNotFoundError("agent-browser CLI not found. Install with: npm install -g agent-browser && agent-browser install")
      25
      26
      27def _session_name(task_id: str) -> str:
      28    safe = "".join(ch if ch.isalnum() or ch in "-_" else "_" for ch in task_id)
      29    if len(safe) <= 32:
      30        return f"nipux_{safe}"
      31    digest = hashlib.sha1(task_id.encode("utf-8")).hexdigest()[:10]
      32    return f"nipux_{safe[:20]}_{digest}"
      33
      34
      35def _profile_dir(config: AppConfig, task_id: str) -> Path:
      36    return config.runtime.home / "browser-profiles" / _session_name(task_id)
      37
      38
      39def _socket_dir(task_id: str) -> Path:
      40    root = Path(os.environ.get("NIPUX_BROWSER_SOCKET_ROOT") or "/tmp")
      41    return root / "nipux-ab" / _session_name(task_id)
      42
      43
      44def run_browser_command(
      45    config: AppConfig,
      46    *,
      47    task_id: str,
      48    command: str,
      49    args: list[str] | None = None,
      50    timeout: int = 60,
      51) -> dict[str, Any]:
      52    args = args or []
      53    profile_dir = _profile_dir(config, task_id)
      54    profile_dir.mkdir(parents=True, exist_ok=True)
      55    cmd = [
      56        *_find_agent_browser(),
      57        "--session",
      58        _session_name(task_id),
      59        "--session-name",
      60        _session_name(task_id),
      61        "--profile",
      62        str(profile_dir),
      63        "--json",
      64        command,
      65        *args,
      66    ]
      67    socket_dir = _socket_dir(task_id)
      68    socket_dir.mkdir(parents=True, exist_ok=True)
      69    env = {
      70        **os.environ,
      71        "AGENT_BROWSER_SOCKET_DIR": str(socket_dir),
      72        "AGENT_BROWSER_SESSION_NAME": _session_name(task_id),
      73        "AGENT_BROWSER_PROFILE": str(profile_dir),
      74    }
      75
      76    with tempfile.TemporaryDirectory(dir=str(socket_dir)) as tmp:
      77        stdout_path = Path(tmp) / "stdout"
      78        stderr_path = Path(tmp) / "stderr"
      79        with stdout_path.open("w", encoding="utf-8") as stdout, stderr_path.open("w", encoding="utf-8") as stderr:
      80            proc = subprocess.Popen(cmd, stdin=subprocess.DEVNULL, stdout=stdout, stderr=stderr, env=env)
      81            try:
      82                proc.wait(timeout=timeout)
      83            except subprocess.TimeoutExpired:
      84                proc.kill()
      85                proc.wait()
      86                return {"success": False, "error": f"browser command timed out after {timeout}s"}
      87        stdout_text = stdout_path.read_text(encoding="utf-8").strip()
      88        stderr_text = stderr_path.read_text(encoding="utf-8").strip()
      89
      90    if stdout_text:
      91        try:
      92            result = json.loads(stdout_text)
      93        except json.JSONDecodeError:
      94            return {"success": False, "error": f"agent-browser returned non-JSON output: {stdout_text[:1000]}"}
      95        if isinstance(result, dict):
      96            result.setdefault("browser_session", _session_name(task_id))
      97            result.setdefault("browser_profile", str(profile_dir))
      98            return result
      99        return {"success": True, "data": result, "browser_session": _session_name(task_id), "browser_profile": str(profile_dir)}
     100    if proc.returncode != 0:
     101        return {
     102            "success": False,
     103            "error": stderr_text or f"agent-browser exited {proc.returncode}",
     104            "browser_session": _session_name(task_id),
     105            "browser_profile": str(profile_dir),
     106        }
     107    return {"success": True, "data": {}, "browser_session": _session_name(task_id), "browser_profile": str(profile_dir)}
     108
     109
     110def navigate(config: AppConfig, *, task_id: str, url: str) -> dict[str, Any]:
     111    result = run_browser_command(config, task_id=task_id, command="open", args=[url], timeout=90)
     112    if not result.get("success"):
     113        return result
     114    snapshot = run_browser_command(config, task_id=task_id, command="snapshot", args=["-c"], timeout=30)
     115    if snapshot.get("success"):
     116        result["snapshot"] = snapshot.get("data", {}).get("snapshot", "")
     117        result["refs"] = snapshot.get("data", {}).get("refs", {})
     118    return _annotate_source_quality(result)
     119
     120
     121def snapshot(config: AppConfig, *, task_id: str, full: bool = False) -> dict[str, Any]:
     122    return _annotate_source_quality(run_browser_command(config, task_id=task_id, command="snapshot", args=[] if full else ["-c"]))
     123
     124
     125def click(config: AppConfig, *, task_id: str, ref: str) -> dict[str, Any]:
     126    result = run_browser_command(config, task_id=task_id, command="click", args=[ref if ref.startswith("@") else f"@{ref}"])
     127    return _with_recovery_snapshot(config, task_id=task_id, result=result)
     128
     129
     130def fill(config: AppConfig, *, task_id: str, ref: str, text: str) -> dict[str, Any]:
     131    result = run_browser_command(config, task_id=task_id, command="fill", args=[ref if ref.startswith("@") else f"@{ref}", text])
     132    return _with_recovery_snapshot(config, task_id=task_id, result=result)
     133
     134
     135def scroll(config: AppConfig, *, task_id: str, direction: str) -> dict[str, Any]:
     136    return run_browser_command(config, task_id=task_id, command="scroll", args=[direction, "500"])
     137
     138
     139def back(config: AppConfig, *, task_id: str) -> dict[str, Any]:
     140    return run_browser_command(config, task_id=task_id, command="back")
     141
     142
     143def press(config: AppConfig, *, task_id: str, key: str) -> dict[str, Any]:
     144    return run_browser_command(config, task_id=task_id, command="press", args=[key])
     145
     146
     147def console(config: AppConfig, *, task_id: str, clear: bool = False, expression: str | None = None) -> dict[str, Any]:
     148    if expression is not None:
     149        return run_browser_command(config, task_id=task_id, command="eval", args=[expression])
     150    args = ["--clear"] if clear else []
     151    console_result = run_browser_command(config, task_id=task_id, command="console", args=args)
     152    errors_result = run_browser_command(config, task_id=task_id, command="errors", args=args)
     153    return {
     154        "success": bool(console_result.get("success") or errors_result.get("success")),
     155        "console": console_result,
     156        "errors": errors_result,
     157    }
     158
     159
     160def _annotate_source_quality(result: dict[str, Any]) -> dict[str, Any]:
     161    data = result.get("data") if isinstance(result.get("data"), dict) else {}
     162    reason = anti_bot_reason(
     163        str(data.get("title") or ""),
     164        str(data.get("url") or data.get("origin") or ""),
     165        str(result.get("snapshot") or data.get("snapshot") or ""),
     166    )
     167    if not reason:
     168        return result
     169    result["source_warning"] = reason
     170    warnings = result.get("warnings") if isinstance(result.get("warnings"), list) else []
     171    warnings.append({
     172        "type": "anti_bot",
     173        "message": reason,
     174        "guidance": "This page may require normal human browser verification. Do not bypass protections; continue only with visible browser actions or choose another source if stuck.",
     175    })
     176    result["warnings"] = warnings
     177    return result
     178
     179
     180def _with_recovery_snapshot(config: AppConfig, *, task_id: str, result: dict[str, Any]) -> dict[str, Any]:
     181    if result.get("success", True):
     182        return result
     183    error = str(result.get("error") or "")
     184    if "unknown ref" not in error.lower():
     185        return result
     186    recovery = run_browser_command(config, task_id=task_id, command="snapshot", args=["-c"], timeout=30)
     187    result["recovery_guidance"] = "The ref was stale or missing. Use refs from recovery_snapshot before clicking or typing again."
     188    result["recovery_snapshot"] = _annotate_source_quality(recovery)
     189    return result

nipux_cli/chat_commands.py 286 lines

       1"""Slash-command dispatch for focused chat sessions."""
       2
       3from __future__ import annotations
       4
       5import argparse
       6from dataclasses import dataclass
       7from typing import Any, Callable
       8
       9from nipux_cli.config import DEFAULT_CONTEXT_LENGTH
      10from nipux_cli.tui_style import _one_line
      11
      12
      13@dataclass(frozen=True)
      14class ChatCommandDeps:
      15    db_factory: Callable[[], tuple[Any, Any]]
      16    jobs: Callable[[argparse.Namespace], None]
      17    history: Callable[[argparse.Namespace], None]
      18    events: Callable[[argparse.Namespace], None]
      19    logs: Callable[[argparse.Namespace], None]
      20    updates: Callable[[argparse.Namespace], None]
      21    artifacts: Callable[[argparse.Namespace], None]
      22    artifact: Callable[[argparse.Namespace], None]
      23    lessons: Callable[[argparse.Namespace], None]
      24    findings: Callable[[argparse.Namespace], None]
      25    tasks: Callable[[argparse.Namespace], None]
      26    roadmap: Callable[[argparse.Namespace], None]
      27    experiments: Callable[[argparse.Namespace], None]
      28    sources: Callable[[argparse.Namespace], None]
      29    memory: Callable[[argparse.Namespace], None]
      30    metrics: Callable[[argparse.Namespace], None]
      31    activity: Callable[[argparse.Namespace], None]
      32    digest: Callable[[argparse.Namespace], None]
      33    status: Callable[[argparse.Namespace], None]
      34    usage: Callable[[argparse.Namespace], None]
      35    handle_setting: Callable[[str, list[str]], bool]
      36    doctor: Callable[[argparse.Namespace], None]
      37    init: Callable[[argparse.Namespace], None]
      38    health: Callable[[argparse.Namespace], None]
      39    start: Callable[[argparse.Namespace], None]
      40    ensure_job_runnable: Callable[[Any, str], None]
      41    run: Callable[[argparse.Namespace], None]
      42    restart: Callable[[argparse.Namespace], None]
      43    work: Callable[[argparse.Namespace], None]
      44    pause: Callable[[argparse.Namespace], None]
      45    resume: Callable[[argparse.Namespace], None]
      46    cancel: Callable[[argparse.Namespace], None]
      47    queue_note: Callable[..., None]
      48    create_job: Callable[..., tuple[str, str]]
      49    focus: Callable[[argparse.Namespace], None]
      50    delete: Callable[[argparse.Namespace], None]
      51
      52
      53def handle_chat_slash_command(job_id: str, command: str, rest: list[str], *, deps: ChatCommandDeps) -> bool:
      54    if command in {"jobs", "ls"}:
      55        deps.jobs(argparse.Namespace())
      56        return True
      57    if command == "history":
      58        deps.history(
      59            argparse.Namespace(
      60                job_id=job_id,
      61                limit=_optional_int(rest, default=40),
      62                chars=220,
      63                full=False,
      64                json=False,
      65            )
      66        )
      67        return True
      68    if command == "events":
      69        deps.events(
      70            argparse.Namespace(
      71                job_id=job_id,
      72                limit=_optional_int(rest, default=40),
      73                chars=220,
      74                full=False,
      75                json=False,
      76                follow=False,
      77                interval=2.0,
      78            )
      79        )
      80        return True
      81    if command == "outputs":
      82        deps.logs(
      83            argparse.Namespace(
      84                job_id=[job_id],
      85                limit=_optional_int(rest, default=25),
      86                verbose=False,
      87                chars=260,
      88            )
      89        )
      90        return True
      91    if command in {"updates", "outcomes", "outcome"}:
      92        all_jobs = bool(rest and rest[0].lower() == "all")
      93        deps.updates(argparse.Namespace(job_id=job_id, all=all_jobs, limit=5, chars=180, paths=False))
      94        return True
      95    if command == "artifacts":
      96        deps.artifacts(argparse.Namespace(job_id=job_id, limit=10, chars=220, paths=False))
      97        return True
      98    if command == "artifact":
      99        query = " ".join(rest).strip()
     100        if not query:
     101            print("usage: /artifact QUERY_OR_ID")
     102            return True
     103        deps.artifact(argparse.Namespace(artifact_id_or_path=[query], job_id=job_id, chars=12000))
     104        return True
     105    if command == "lessons":
     106        deps.lessons(argparse.Namespace(job_id=job_id, limit=10, chars=220))
     107        return True
     108    if command == "findings":
     109        deps.findings(argparse.Namespace(job_id=job_id, limit=20, chars=220, json=False))
     110        return True
     111    if command == "tasks":
     112        deps.tasks(argparse.Namespace(job_id=job_id, limit=20, chars=220, status=None, json=False))
     113        return True
     114    if command == "roadmap":
     115        deps.roadmap(argparse.Namespace(job_id=job_id, limit=20, features=3, chars=220, json=False))
     116        return True
     117    if command == "experiments":
     118        deps.experiments(argparse.Namespace(job_id=job_id, limit=20, chars=220, status=None, json=False))
     119        return True
     120    if command == "sources":
     121        deps.sources(argparse.Namespace(job_id=job_id, limit=20, chars=220, json=False))
     122        return True
     123    if command == "memory":
     124        deps.memory(
     125            argparse.Namespace(
     126                job_id=job_id,
     127                limit=10,
     128                chars=220,
     129                json=False,
     130                graph=bool(rest and rest[0].lower() in {"graph", "view", "html"}),
     131                output=None,
     132            )
     133        )
     134        return True
     135    if command == "metrics":
     136        deps.metrics(argparse.Namespace(job_id=job_id, chars=220))
     137        return True
     138    if command == "learn":
     139        lesson = " ".join(rest).strip()
     140        if not lesson:
     141            print("usage: /learn LESSON")
     142            return True
     143        db, _config = deps.db_factory()
     144        try:
     145            entry = db.append_lesson(job_id, lesson, category="operator_preference", metadata={"source": "chat"})
     146            job = db.get_job(job_id)
     147            print(f"learned for {job['title']}: {_one_line(entry['lesson'], 220)}")
     148        finally:
     149            db.close()
     150        return True
     151    if command == "activity":
     152        deps.activity(
     153            argparse.Namespace(job_id=job_id, limit=20, chars=180, follow=False, interval=2.0, verbose=False, paths=False)
     154        )
     155        return True
     156    if command == "digest":
     157        deps.digest(argparse.Namespace(job_id=[job_id]))
     158        return True
     159    if command == "status":
     160        deps.status(argparse.Namespace(job_id=job_id, limit=8, chars=180, full=False, json=False))
     161        return True
     162    if command == "usage":
     163        deps.usage(argparse.Namespace(job_id=job_id, json=False))
     164        return True
     165    if command == "settings":
     166        deps.handle_setting("config", [])
     167        return True
     168    if deps.handle_setting(command, rest):
     169        return True
     170    if command == "doctor":
     171        try:
     172            deps.doctor(argparse.Namespace(check_model=True))
     173        except SystemExit:
     174            pass
     175        return True
     176    if command == "init":
     177        deps.init(
     178            argparse.Namespace(
     179                path=None,
     180                force=False,
     181                model=None,
     182                base_url=None,
     183                api_key_env=None,
     184                openrouter=False,
     185                context_length=DEFAULT_CONTEXT_LENGTH,
     186            )
     187        )
     188        return True
     189    if command == "health":
     190        deps.health(argparse.Namespace(limit=8, chars=180))
     191        return True
     192    if command == "start":
     193        deps.start(argparse.Namespace(poll_seconds=0.0, fake=False, quiet=False, log_file=None))
     194        return True
     195    if command == "run":
     196        db, _config = deps.db_factory()
     197        try:
     198            deps.ensure_job_runnable(db, job_id)
     199        finally:
     200            db.close()
     201        deps.run(
     202            argparse.Namespace(
     203                job_id=job_id,
     204                poll_seconds=0.0,
     205                interval=2.0,
     206                limit=20,
     207                chars=180,
     208                verbose=False,
     209                paths=False,
     210                fake=False,
     211                quiet=False,
     212                log_file=None,
     213                no_follow=True,
     214            )
     215        )
     216        return True
     217    if command == "restart":
     218        deps.restart(argparse.Namespace(poll_seconds=0.0, wait=5.0, fake=False, quiet=False, log_file=None))
     219        return True
     220    if command in {"work", "work-verbose"}:
     221        deps.work(
     222            argparse.Namespace(
     223                job_id=job_id,
     224                steps=_optional_int(rest, default=1),
     225                poll_seconds=0.5,
     226                fake=False,
     227                verbose=command == "work-verbose",
     228                dashboard=False,
     229                limit=12,
     230                chars=260 if command == "work" else 4000,
     231                continue_on_error=False,
     232            )
     233        )
     234        return True
     235    if command in {"pause", "stop"}:
     236        deps.pause(argparse.Namespace(job_id=job_id, note=rest))
     237        return True
     238    if command == "resume":
     239        deps.resume(argparse.Namespace(job_id=job_id))
     240        return True
     241    if command == "cancel":
     242        deps.cancel(argparse.Namespace(job_id=job_id, note=rest))
     243        return True
     244    if command == "note":
     245        message = " ".join(rest).strip()
     246        if not message:
     247            print("usage: /note MESSAGE")
     248            return True
     249        deps.queue_note(job_id, message, mode="note")
     250        return True
     251    if command == "follow":
     252        message = " ".join(rest).strip()
     253        if not message:
     254            print("usage: /follow MESSAGE")
     255            return True
     256        deps.queue_note(job_id, message, mode="follow_up")
     257        return True
     258    if command == "new":
     259        objective = " ".join(rest).strip()
     260        if not objective:
     261            print("usage: /new OBJECTIVE")
     262            return True
     263        _created_id, title = deps.create_job(objective=objective, title=None, kind="generic", cadence=None)
     264        print(f"created {title}")
     265        started = deps.start(argparse.Namespace(poll_seconds=0.0, fake=False, quiet=True, log_file=None))
     266        if started is False:
     267            print(f"focus set to {title}; worker is waiting for a working model.")
     268        else:
     269            print(f"focus set to {title}; initial plan accepted and worker started.")
     270        return True
     271    if command in {"focus", "switch"}:
     272        if not " ".join(rest).strip():
     273            deps.focus(argparse.Namespace(query=[]))
     274            return True
     275        deps.focus(argparse.Namespace(query=rest))
     276        return True
     277    if command == "delete":
     278        target = rest if rest else [job_id]
     279        deps.delete(argparse.Namespace(job_id=target, keep_files=False))
     280        return bool(rest)
     281    print(f"unknown chat command: /{command}")
     282    return True
     283
     284
     285def _optional_int(values: list[str], *, default: int) -> int:
     286    return int(values[0]) if values and values[0].isdigit() else default

nipux_cli/chat_context.py 223 lines

       1"""Prompt context builder for the Nipux chat-side controller model."""
       2
       3from __future__ import annotations
       4
       5from typing import Any
       6
       7from nipux_cli.db import AgentDB
       8from nipux_cli.event_render import event_line
       9from nipux_cli.metric_format import format_metric_value
      10from nipux_cli.operator_context import active_prompt_operator_entries
      11from nipux_cli.tui_event_format import clean_step_summary
      12from nipux_cli.tui_outcomes import (
      13    SUMMARY_EVENT_TYPES,
      14    SUMMARY_TOOL_EVENT_TYPES,
      15    hourly_outcome_summary,
      16    is_summary_event_candidate,
      17    model_update_event_parts,
      18    outcome_counts,
      19)
      20
      21
      22def build_chat_messages(db: AgentDB, job: dict[str, Any], message: str) -> list[dict[str, str]]:
      23    """Build bounded visible-state context for conversational job control."""
      24
      25    steps = db.list_steps(job_id=job["id"])[-10:]
      26    jobs = db.list_jobs()[:12]
      27    artifacts = db.list_artifacts(job["id"], limit=5)
      28    timeline_events = db.list_timeline_events(job["id"], limit=18)
      29    outcome_events = _durable_outcome_events(db, job["id"])
      30    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
      31    operator_messages = metadata.get("operator_messages") if isinstance(metadata.get("operator_messages"), list) else []
      32    agent_updates = metadata.get("agent_updates") if isinstance(metadata.get("agent_updates"), list) else []
      33    lessons = metadata.get("lessons") if isinstance(metadata.get("lessons"), list) else []
      34    findings = metadata.get("finding_ledger") if isinstance(metadata.get("finding_ledger"), list) else []
      35    sources = metadata.get("source_ledger") if isinstance(metadata.get("source_ledger"), list) else []
      36    tasks = metadata.get("task_queue") if isinstance(metadata.get("task_queue"), list) else []
      37    experiments = metadata.get("experiment_ledger") if isinstance(metadata.get("experiment_ledger"), list) else []
      38    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
      39
      40    step_lines = "\n".join(
      41        f"- #{step['step_no']} {step['status']} {step.get('tool_name') or step['kind']}: "
      42        f"{clean_step_summary(step.get('summary') or step.get('error') or '')}"
      43        for step in steps
      44    )
      45    artifact_lines = "\n".join(
      46        f"- #{index} {artifact.get('title') or artifact['id']}: {artifact.get('summary') or ''} "
      47        f"(view with /artifact {index})"
      48        for index, artifact in enumerate(artifacts, start=1)
      49    )
      50    steering_lines = "\n".join(
      51        f"- {entry.get('source', 'operator')} {entry.get('mode', 'steer')}: {entry.get('message', '')}"
      52        for entry in active_prompt_operator_entries(operator_messages)[-6:]
      53        if isinstance(entry, dict)
      54    )
      55    update_lines = "\n".join(
      56        f"- {entry.get('category', 'progress')}: {entry.get('message', '')}"
      57        for entry in agent_updates[-5:]
      58        if isinstance(entry, dict)
      59    )
      60    lesson_lines = "\n".join(
      61        f"- {entry.get('category', 'memory')}: {entry.get('lesson', '')}"
      62        for entry in lessons[-8:]
      63        if isinstance(entry, dict)
      64    )
      65    finding_lines = "\n".join(
      66        f"- {entry.get('name')}: {entry.get('category') or ''} {entry.get('location') or ''} score={entry.get('score')}"
      67        for entry in findings[-8:]
      68        if isinstance(entry, dict)
      69    )
      70    task_lines = "\n".join(
      71        f"- {entry.get('status') or 'open'} p={entry.get('priority') or 0}: {entry.get('title')}"
      72        for entry in tasks[-10:]
      73        if isinstance(entry, dict)
      74    )
      75    milestone_lines = _roadmap_lines(roadmap)
      76    experiment_lines = "\n".join(_experiment_line(entry) for entry in experiments[-10:] if isinstance(entry, dict))
      77    source_lines = "\n".join(
      78        f"- {entry.get('source')}: score={entry.get('usefulness_score')} "
      79        f"findings={entry.get('yield_count') or 0} outcome={entry.get('last_outcome') or ''}"
      80        for entry in sources[-8:]
      81        if isinstance(entry, dict)
      82    )
      83    timeline_lines = "\n".join(event_line(event, chars=700) for event in timeline_events[-12:])
      84
      85    sections = {
      86        "Jobs": _clip_chat_context(_job_list_lines(jobs, focused_job_id=job["id"]), 1_300),
      87        "Durable outcomes": _clip_chat_context(_durable_outcome_lines(outcome_events), 1_600),
      88        "Recent tool calls": _clip_chat_context(step_lines, 1_800),
      89        "Latest artifacts": _clip_chat_context(artifact_lines, 1_200),
      90        "Finding ledger": _clip_chat_context(finding_lines, 1_200),
      91        "Task queue": _clip_chat_context(task_lines, 1_300),
      92        "Roadmap": _clip_chat_context(milestone_lines, 1_200),
      93        "Experiment ledger": _clip_chat_context(experiment_lines, 1_300),
      94        "Source ledger": _clip_chat_context(source_lines, 1_100),
      95        "Lessons learned": _clip_chat_context(lesson_lines, 1_000),
      96        "Recent operator steering": _clip_chat_context(steering_lines, 1_200),
      97        "Recent agent notes": _clip_chat_context(update_lines, 1_200),
      98        "Recent visible timeline": _clip_chat_context(timeline_lines, 1_800),
      99    }
     100    section_text = "\n\n".join(f"{title}:\n{body or _empty_section_text(title)}" for title, body in sections.items())
     101    return [
     102        {
     103            "role": "system",
     104            "content": (
     105                "You are Nipux, the chat model that controls a generic long-running agent workspace. "
     106                "You know the visible CLI state, focused job, job list, task queue, artifacts, memory, metrics, and recent activity. "
     107                "Answer directly from the visible job state. Do not claim hidden chain-of-thought. "
     108                "If the operator asks for work to be done, explain the concrete job/control action Nipux will take or how to run it from the Jobs/Status panel. "
     109                "If the operator asks where saved work is, explain that artifacts and history are visible from the Jobs/Status panel or direct CLI commands. "
     110                "Do not start replies with an introduction. Keep replies concise and useful."
     111            ),
     112        },
     113        {
     114            "role": "user",
     115            "content": (
     116                f"Job title: {job['title']}\n"
     117                f"Job status: {job['status']}\n"
     118                f"Kind: {job['kind']}\n"
     119                f"Objective: {job['objective']}\n\n"
     120                f"{section_text}\n\n"
     121                f"Operator message:\n{message}"
     122            ),
     123        },
     124    ]
     125
     126
     127def _durable_outcome_events(db: AgentDB, job_id: str) -> list[dict[str, Any]]:
     128    durable_events = db.list_events(job_id=job_id, limit=160, event_types=SUMMARY_EVENT_TYPES)
     129    tool_events = [
     130        event
     131        for event in db.list_events(job_id=job_id, limit=80, event_types=SUMMARY_TOOL_EVENT_TYPES)
     132        if is_summary_event_candidate(event)
     133    ]
     134    merged: dict[str, dict[str, Any]] = {}
     135    for event in [*durable_events, *tool_events]:
     136        event_id = str(event.get("id") or "")
     137        key = event_id or f"{event.get('created_at')}-{event.get('event_type')}-{event.get('title')}-{len(merged)}"
     138        merged[key] = event
     139    return sorted(merged.values(), key=lambda event: (str(event.get("created_at") or ""), str(event.get("id") or "")))
     140
     141
     142def _durable_outcome_lines(events: list[dict[str, Any]]) -> str:
     143    if not events:
     144        return ""
     145    counts = outcome_counts(events, include_research=True, include_failures=True)
     146    lines = [f"- summary: {hourly_outcome_summary(counts)}"]
     147    seen: set[str] = set()
     148    for event in reversed(events):
     149        parsed = model_update_event_parts(event, width=240, compact=False)
     150        if not parsed:
     151            continue
     152        label, text, _clock = parsed
     153        if label in {"DONE", "PLAN", "UPDATE"}:
     154            continue
     155        key = f"{label}:{text}"
     156        if key in seen:
     157            continue
     158        seen.add(key)
     159        lines.append(f"- {label.lower()}: {text}")
     160        if len(lines) >= 9:
     161            break
     162    return "\n".join(lines)
     163
     164
     165def _job_list_lines(jobs: list[dict[str, Any]], *, focused_job_id: str) -> str:
     166    lines: list[str] = []
     167    for index, entry in enumerate(jobs, start=1):
     168        marker = "*" if str(entry.get("id") or "") == focused_job_id else "-"
     169        title = entry.get("title") or entry.get("id") or "untitled"
     170        objective = " ".join(str(entry.get("objective") or "").split())
     171        if len(objective) > 120:
     172            objective = objective[:119].rstrip() + "..."
     173        lines.append(
     174            f"{marker} {index}. {title} status={entry.get('status') or 'unknown'} "
     175            f"kind={entry.get('kind') or 'generic'} objective={objective}"
     176        )
     177    return "\n".join(lines)
     178
     179
     180def _roadmap_lines(roadmap: dict[str, Any]) -> str:
     181    if not roadmap:
     182        return ""
     183    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
     184    body = "\n".join(
     185        (
     186            f"- {entry.get('status') or 'planned'} validation={entry.get('validation_status') or 'not_started'} "
     187            f"p={entry.get('priority') or 0}: {entry.get('title')}"
     188        )
     189        for entry in milestones[-8:]
     190        if isinstance(entry, dict)
     191    )
     192    header = (
     193        f"{roadmap.get('status') or 'planned'}: {roadmap.get('title') or 'Roadmap'}"
     194        + (f" current={roadmap.get('current_milestone')}" if roadmap.get("current_milestone") else "")
     195    )
     196    return f"{header}\n{body}".strip()
     197
     198
     199def _experiment_line(entry: dict[str, Any]) -> str:
     200    if entry.get("metric_value") is None:
     201        return f"- {entry.get('status') or 'planned'}: {entry.get('title')}"
     202    metric = format_metric_value(
     203        entry.get("metric_name") or "metric",
     204        entry.get("metric_value"),
     205        entry.get("metric_unit") or "",
     206    )
     207    return (
     208        f"- {entry.get('status') or 'planned'}: {entry.get('title')}"
     209        f" {metric}"
     210        f"{' best' if entry.get('best_observed') else ''}"
     211    )
     212
     213
     214def _empty_section_text(title: str) -> str:
     215    return "None." if title.startswith("Recent operator") else "None yet."
     216
     217
     218def _clip_chat_context(value: str, limit: int) -> str:
     219    text = str(value or "")
     220    if len(text) <= limit:
     221        return text
     222    marker = f"\n... clipped {len(text) - limit} chars from this visible state section ..."
     223    return text[: max(0, limit - len(marker))].rstrip() + marker

nipux_cli/chat_controller.py 173 lines

       1"""Chat-controller behavior shared by the interactive CLI."""
       2
       3from __future__ import annotations
       4
       5from dataclasses import dataclass
       6from typing import Any, Callable
       7
       8from nipux_cli.chat_intent import (
       9    chat_control_command,
      10    extract_job_objective_from_message,
      11    message_requests_immediate_run,
      12    message_requests_queued_job,
      13)
      14
      15
      16@dataclass(frozen=True)
      17class ChatControllerDeps:
      18    db_factory: Callable[[], tuple[Any, Any]]
      19    reply_fn: Callable[[str, str], Any]
      20    create_job: Callable[..., tuple[str, str]]
      21    write_shell_state: Callable[[dict[str, Any]], None]
      22    start_daemon: Callable[..., Any]
      23    capture_command: Callable[[str, str], tuple[bool, str]]
      24    compact_command_output: Callable[[str], list[str]]
      25    friendly_error_text: Callable[[str], str]
      26
      27
      28def handle_chat_message(
      29    job_id: str,
      30    line: str,
      31    *,
      32    deps: ChatControllerDeps,
      33    reply_fn: Callable[[str, str], Any] | None = None,
      34    quiet: bool = False,
      35) -> tuple[bool, str]:
      36    reply_callable = reply_fn or deps.reply_fn
      37    spawned = maybe_spawn_job_from_chat(job_id, line, deps=deps, quiet=quiet)
      38    if spawned:
      39        return True, spawned
      40    controlled = handle_chat_control_intent(job_id, line, deps=deps, quiet=quiet)
      41    if controlled is not None:
      42        return controlled
      43    queue_chat_note(job_id, line, deps=deps, mode="steer", quiet=quiet)
      44    try:
      45        reply = reply_callable(job_id, line)
      46    except Exception as exc:
      47        detail = deps.friendly_error_text(f"{type(exc).__name__}: {exc}")
      48        message = f"{detail}; message saved for the worker"
      49        if not quiet:
      50            print(detail)
      51            print("Your message is still saved for the next worker step.")
      52        return True, message
      53    reply_text, reply_metadata = chat_reply_text_and_metadata(reply)
      54    if reply_text.strip():
      55        db, _config = deps.db_factory()
      56        try:
      57            if reply_metadata:
      58                db.append_event(
      59                    job_id,
      60                    event_type="loop",
      61                    title="message_end",
      62                    body=reply_text[:1000],
      63                    metadata={"source": "chat", "tool_calls": [], **reply_metadata},
      64                )
      65            db.append_agent_update(job_id, reply_text.strip(), category="chat")
      66        finally:
      67            db.close()
      68        if not quiet:
      69            print()
      70            print(reply_text.strip())
      71            print()
      72        return True, ""
      73    message = "model returned an empty reply; message is queued"
      74    if not quiet:
      75        print("model returned an empty reply; your message is still queued.")
      76    return True, message
      77
      78
      79def chat_reply_text_and_metadata(reply: Any) -> tuple[str, dict[str, Any]]:
      80    content = getattr(reply, "content", None)
      81    if content is None:
      82        return str(reply), {}
      83    metadata: dict[str, Any] = {}
      84    usage = getattr(reply, "usage", None)
      85    if isinstance(usage, dict) and usage:
      86        metadata["usage"] = usage
      87    model = getattr(reply, "model", "")
      88    if model:
      89        metadata["model"] = model
      90    response_id = getattr(reply, "response_id", "")
      91    if response_id:
      92        metadata["response_id"] = response_id
      93    return str(content), metadata
      94
      95
      96def handle_chat_control_intent(
      97    job_id: str,
      98    line: str,
      99    *,
     100    deps: ChatControllerDeps,
     101    quiet: bool = False,
     102) -> tuple[bool, str] | None:
     103    command = chat_control_command(line)
     104    if not command:
     105        return None
     106    keep_running, output = deps.capture_command(job_id, command)
     107    compact = deps.compact_command_output(output)
     108    message = " | ".join(compact[-4:]) if compact else f"{command.lstrip('/')} done"
     109    if not quiet:
     110        print(message)
     111    return keep_running, message
     112
     113
     114def maybe_spawn_job_from_chat(
     115    job_id: str,
     116    message: str,
     117    *,
     118    deps: ChatControllerDeps,
     119    quiet: bool = False,
     120) -> str:
     121    objective = extract_job_objective_from_message(message)
     122    if not objective:
     123        return ""
     124    created_id, title = deps.create_job(objective=objective, title=None, kind="generic", cadence=None)
     125    deps.write_shell_state({"focus_job_id": created_id})
     126    db, _config = deps.db_factory()
     127    try:
     128        db.append_operator_message(created_id, message, source="chat", mode="steer")
     129        run_now = not message_requests_queued_job(message) or message_requests_immediate_run(message)
     130        update = "Created this job from chat and drafted its initial plan."
     131        if run_now:
     132            update += " Starting the daemon so it can begin work."
     133        else:
     134            update += " Use the right pane to run it."
     135        db.append_agent_update(created_id, update, category="chat")
     136        db.append_agent_update(
     137            job_id,
     138            f"Created job '{title}' from your chat request and switched focus to it.",
     139            category="chat",
     140        )
     141    finally:
     142        db.close()
     143    run_now = not message_requests_queued_job(message) or message_requests_immediate_run(message)
     144    text = f"Created job: {title}. Focus switched to it."
     145    if run_now:
     146        started = deps.start_daemon(poll_seconds=0.0, quiet=True)
     147        if started is False:
     148            text += " Worker is waiting for a working model."
     149        else:
     150            text += " Started worker."
     151    if not quiet:
     152        print(text)
     153    return text
     154
     155
     156def queue_chat_note(
     157    job_id: str,
     158    message: str,
     159    *,
     160    deps: ChatControllerDeps,
     161    mode: str = "steer",
     162    quiet: bool = False,
     163) -> None:
     164    db, _config = deps.db_factory()
     165    try:
     166        entry = db.append_operator_message(job_id, message, source="chat", mode=mode)
     167        if not quiet:
     168            if entry.get("mode") == "follow_up":
     169                print(f"waiting after current branch: {entry['message']}")
     170            else:
     171                print(f"waiting: {entry['message']}")
     172    finally:
     173        db.close()

nipux_cli/chat_frame_runtime.py 571 lines

       1"""Terminal chat-frame runtime helpers."""
       2
       3from __future__ import annotations
       4
       5import queue
       6import select
       7import shutil
       8import sys
       9import termios
      10import threading
      11import time
      12import tty
      13from dataclasses import dataclass
      14from typing import Callable
      15from typing import Any
      16
      17from nipux_cli.settings import inline_setting_notice
      18from nipux_cli.tui_commands import CHAT_SLASH_COMMANDS, autocomplete_slash, cycle_slash, slash_completion_for_submit
      19from nipux_cli.tui_input import (
      20    decode_terminal_escape,
      21    drain_pending_input,
      22    read_escape_sequence,
      23    read_terminal_char,
      24)
      25from nipux_cli.tui_outcomes import CHAT_RIGHT_PAGES
      26from nipux_cli.tui_style import _frame_enter_sequence, _frame_exit_sequence, _one_line, _strip_ansi
      27
      28
      29IDLE_REFRESH_SECONDS = 0.75
      30ACTIVE_INPUT_REFRESH_SECONDS = 2.0
      31THINKING_REFRESH_SECONDS = 0.18
      32WORKSPACE_CHAT_ID = "__workspace__"
      33THINKING_NOTICE = "__nipux_thinking__"
      34THINKING_FRAMES = ("◐ thinking", "◓ thinking", "◑ thinking", "◒ thinking")
      35WAITING_NOTICE = "__nipux_waiting__"
      36WAITING_FRAMES = ("∙ waiting", "· waiting", "• waiting", "· waiting")
      37
      38
      39@dataclass(frozen=True)
      40class ChatFrameDeps:
      41    load_snapshot: Callable[[str, int], dict[str, Any]]
      42    render_frame: Callable[[dict[str, Any], str, list[str], str, int, str | None, str | None, str], str]
      43    handle_chat_message: Callable[[str, str], tuple[bool, str]]
      44    capture_chat_command: Callable[[str, str], tuple[bool, str]]
      45    write_shell_state: Callable[[dict[str, str]], None]
      46    is_plain_chat_line: Callable[[str], bool]
      47    page_click: Callable[[int, int, str], str | None]
      48
      49
      50def compact_command_output(output: str) -> list[str]:
      51    lines = [" ".join(line.split()) for line in output.splitlines() if line.strip()]
      52    compacted: list[str] = []
      53    for line in lines:
      54        if line.startswith("\033[2J"):
      55            continue
      56        compacted.append(_one_line(line, 120))
      57    return compacted[-8:]
      58
      59
      60def frame_next_job_id(snapshot: dict[str, Any], current_job_id: str, *, direction: int) -> str | None:
      61    jobs = snapshot.get("jobs")
      62    if not isinstance(jobs, list) or not jobs:
      63        return None
      64    ids = [str(job.get("id")) for job in jobs if job.get("id")]
      65    if not ids:
      66        return None
      67    try:
      68        index = ids.index(str(current_job_id))
      69    except ValueError:
      70        index = 0
      71    return ids[(index + direction) % len(ids)]
      72
      73
      74def next_chat_right_view(current: str, direction: int) -> str:
      75    keys = [key for key, _label in CHAT_RIGHT_PAGES]
      76    try:
      77        index = keys.index(current)
      78    except ValueError:
      79        index = 0
      80    return keys[(index + direction) % len(keys)]
      81
      82
      83def frame_refresh_interval(input_buffer: str, *, thinking: bool = False) -> float:
      84    if thinking:
      85        return THINKING_REFRESH_SECONDS
      86    return ACTIVE_INPUT_REFRESH_SECONDS if input_buffer else IDLE_REFRESH_SECONDS
      87
      88
      89def run_chat_frame(job_id: str, *, history_limit: int, deps: ChatFrameDeps) -> None:
      90    if job_id != WORKSPACE_CHAT_ID:
      91        deps.write_shell_state({"focus_job_id": job_id})
      92    buffer = ""
      93    notices: list[str] = []
      94    right_view = "updates"
      95    modal_view: str | None = None
      96    selected_control = 0
      97    editing_field: str | None = None
      98    async_messages: queue.Queue[str] = queue.Queue()
      99    snapshot = deps.load_snapshot(job_id, history_limit)
     100    job_id = str(snapshot["job_id"])
     101    old_attrs = termios.tcgetattr(sys.stdin)
     102    print(_frame_enter_sequence(), end="", flush=True)
     103    try:
     104        stdin_fd = sys.stdin.fileno()
     105        tty.setcbreak(stdin_fd)
     106        last_snapshot = 0.0
     107        needs_render = True
     108        last_frame = ""
     109        while True:
     110            now = time.monotonic()
     111            if _drain_async_notices(async_messages, notices):
     112                last_snapshot = 0.0
     113                needs_render = True
     114            if now - last_snapshot >= frame_refresh_interval(buffer, thinking=_has_active_state_notice(notices)):
     115                try:
     116                    snapshot = deps.load_snapshot(job_id, history_limit)
     117                    job_id = str(snapshot["job_id"])
     118                    last_snapshot = now
     119                    needs_render = True
     120                except Exception as exc:
     121                    _append_notice(notices, f"frame refresh failed: {type(exc).__name__}")
     122            if needs_render:
     123                selected_control = 0
     124                last_frame = _safe_render_frame(
     125                    deps,
     126                    snapshot=snapshot,
     127                    buffer=buffer,
     128                    notices=notices,
     129                    right_view=right_view,
     130                    selected_control=selected_control,
     131                    editing_field=editing_field,
     132                    modal_view=modal_view,
     133                    previous_frame=last_frame,
     134                )
     135                needs_render = False
     136            try:
     137                readable, _, _ = select.select([stdin_fd], [], [], 0.05)
     138            except OSError as exc:
     139                _append_notice(notices, f"terminal read failed: {type(exc).__name__}: {_one_line(exc, 90)}")
     140                needs_render = True
     141                continue
     142            if not readable:
     143                continue
     144            try:
     145                char = read_terminal_char(stdin_fd)
     146            except OSError as exc:
     147                _append_notice(notices, f"terminal input failed: {type(exc).__name__}: {_one_line(exc, 90)}")
     148                needs_render = True
     149                continue
     150            if editing_field is not None:
     151                try:
     152                    buffer, editing_field, should_exit = _handle_edit_input(
     153                        char,
     154                        buffer=buffer,
     155                        editing_field=editing_field,
     156                        notices=notices,
     157                        stdin_fd=stdin_fd,
     158                    )
     159                except Exception as exc:
     160                    buffer = ""
     161                    editing_field = None
     162                    _append_notice(notices, f"edit failed: {type(exc).__name__}: {_one_line(exc, 90)}")
     163                    needs_render = True
     164                    continue
     165                if should_exit:
     166                    return
     167                needs_render = True
     168                continue
     169            if char in {"\r", "\n"}:
     170                buffer, should_submit = slash_completion_for_submit(buffer, CHAT_SLASH_COMMANDS)
     171                if not should_submit:
     172                    needs_render = True
     173                    continue
     174                try:
     175                    keep_running, snapshot, job_id, notices, right_view, modal_view = _handle_chat_submit(
     176                        buffer,
     177                        job_id=job_id,
     178                        history_limit=history_limit,
     179                        snapshot=snapshot,
     180                        notices=notices,
     181                        right_view=right_view,
     182                        modal_view=modal_view,
     183                        deps=deps,
     184                        async_messages=async_messages,
     185                    )
     186                except Exception as exc:
     187                    keep_running = True
     188                    _append_notice(notices, f"submit failed: {type(exc).__name__}: {_one_line(exc, 100)}")
     189                buffer = ""
     190                needs_render = True
     191                if not keep_running:
     192                    return
     193                continue
     194            if char in {"\x04"}:
     195                return
     196            if char == "\x03":
     197                buffer = ""
     198                _append_notice(notices, "cancelled input")
     199                needs_render = True
     200                continue
     201            if char == "\x15":
     202                buffer = ""
     203                needs_render = True
     204                continue
     205            if char in {"\x7f", "\b"}:
     206                buffer = buffer[:-1]
     207                needs_render = True
     208                continue
     209            if char == "\t":
     210                try:
     211                    buffer = autocomplete_slash(buffer, CHAT_SLASH_COMMANDS)
     212                except Exception as exc:
     213                    _append_notice(notices, f"autocomplete failed: {type(exc).__name__}: {_one_line(exc, 90)}")
     214                needs_render = True
     215                continue
     216            if char == "\x1b":
     217                try:
     218                    snapshot, job_id, right_view, modal_view, buffer = _handle_chat_escape(
     219                        stdin_fd,
     220                        snapshot=snapshot,
     221                        job_id=job_id,
     222                        history_limit=history_limit,
     223                        right_view=right_view,
     224                        modal_view=modal_view,
     225                        buffer=buffer,
     226                        notices=notices,
     227                        deps=deps,
     228                    )
     229                except Exception as exc:
     230                    modal_view = None
     231                    _append_notice(notices, f"navigation failed: {type(exc).__name__}: {_one_line(exc, 90)}")
     232                needs_render = True
     233                continue
     234            if char.isprintable():
     235                buffer += char
     236                needs_render = True
     237    except KeyboardInterrupt:
     238        return
     239    finally:
     240        termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_attrs)
     241        print(_frame_exit_sequence(), flush=True)
     242
     243
     244def emit_frame_if_changed(frame: str, previous_frame: str = "") -> str:
     245    if frame != previous_frame:
     246        if not previous_frame:
     247            print("\033[H" + frame, end="", flush=True)
     248        else:
     249            print(_diff_frame_update(frame, previous_frame), end="", flush=True)
     250    return frame
     251
     252
     253def _safe_render_frame(
     254    deps: ChatFrameDeps,
     255    *,
     256    snapshot: dict[str, Any],
     257    buffer: str,
     258    notices: list[str],
     259    right_view: str,
     260    selected_control: int,
     261    editing_field: str | None,
     262    modal_view: str | None,
     263    previous_frame: str,
     264) -> str:
     265    try:
     266        return deps.render_frame(
     267            snapshot,
     268            buffer,
     269            _display_notices(notices),
     270            right_view,
     271            selected_control,
     272            editing_field,
     273            modal_view,
     274            previous_frame,
     275        )
     276    except Exception as exc:
     277        _append_notice(notices, f"render failed: {type(exc).__name__}: {_one_line(exc, 100)}")
     278        frame = _fallback_chat_frame(snapshot=snapshot, buffer=buffer, notices=notices)
     279        print("\033[H" + frame, end="", flush=True)
     280        return frame
     281
     282
     283def _fallback_chat_frame(*, snapshot: dict[str, Any], buffer: str, notices: list[str]) -> str:
     284    width, height = shutil.get_terminal_size((100, 30))
     285    width = max(60, width)
     286    job = snapshot.get("job") if isinstance(snapshot.get("job"), dict) else {}
     287    title = str(job.get("title") or snapshot.get("job_id") or "Nipux")
     288    lines = [
     289        _fit_plain("NIPUX - safe mode", width),
     290        _fit_plain("=" * width, width),
     291        _fit_plain(f"Job: {title}", width),
     292        _fit_plain("A UI render error was caught. You can keep typing; /exit leaves.", width),
     293        "",
     294        "Recent notices:",
     295    ]
     296    lines.extend(f"- {_one_line(notice, width - 3)}" for notice in notices[-8:])
     297    lines.extend(["", f"> {_one_line(buffer, width - 3)}"])
     298    return "\n".join(_fit_plain(line, width) for line in lines[:height])
     299
     300
     301def _diff_frame_update(frame: str, previous_frame: str) -> str:
     302    current_lines = frame.splitlines()
     303    previous_lines = previous_frame.splitlines()
     304    output: list[str] = []
     305    max_lines = max(len(current_lines), len(previous_lines))
     306    for index in range(max_lines):
     307        current = current_lines[index] if index < len(current_lines) else ""
     308        previous = previous_lines[index] if index < len(previous_lines) else ""
     309        if current == previous:
     310            continue
     311        output.append(f"\033[{index + 1};1H\033[2K{current}")
     312    return "".join(output)
     313
     314
     315def _fit_plain(text: Any, width: int) -> str:
     316    content = _strip_ansi(str(text))
     317    if len(content) > width:
     318        content = _one_line(content, width)
     319    return content + " " * max(0, width - len(content))
     320
     321
     322def _append_notice(notices: list[str], message: str, *, limit: int = 12) -> None:
     323    notices.append(message)
     324    notices[:] = notices[-limit:]
     325
     326
     327def _append_thinking_notice(notices: list[str]) -> None:
     328    if not _has_thinking_notice(notices):
     329        _append_notice(notices, THINKING_NOTICE)
     330
     331
     332def _append_waiting_notice(notices: list[str]) -> None:
     333    if not _has_waiting_notice(notices):
     334        _append_notice(notices, WAITING_NOTICE)
     335
     336
     337def _has_thinking_notice(notices: list[str]) -> bool:
     338    return any(notice == THINKING_NOTICE or notice.startswith(f"{THINKING_NOTICE}:") for notice in notices)
     339
     340
     341def _has_waiting_notice(notices: list[str]) -> bool:
     342    return any(notice == WAITING_NOTICE or notice.startswith(f"{WAITING_NOTICE}:") for notice in notices)
     343
     344
     345def _has_active_state_notice(notices: list[str]) -> bool:
     346    return _has_thinking_notice(notices) or _has_waiting_notice(notices)
     347
     348
     349def _clear_thinking_notices(notices: list[str]) -> None:
     350    notices[:] = [
     351        notice
     352        for notice in notices
     353        if notice != THINKING_NOTICE and not notice.startswith(f"{THINKING_NOTICE}:")
     354    ]
     355
     356
     357def _display_notices(notices: list[str]) -> list[str]:
     358    if not notices:
     359        return []
     360    index = int(time.monotonic() / THINKING_REFRESH_SECONDS)
     361    thinking_frame = THINKING_FRAMES[index % len(THINKING_FRAMES)]
     362    waiting_frame = WAITING_FRAMES[index % len(WAITING_FRAMES)]
     363    rendered = []
     364    for notice in notices:
     365        if notice == THINKING_NOTICE:
     366            rendered.append(f"{THINKING_NOTICE}:{thinking_frame}")
     367        elif notice == WAITING_NOTICE:
     368            rendered.append(f"{WAITING_NOTICE}:{waiting_frame}")
     369        else:
     370            rendered.append(notice)
     371    return rendered
     372
     373
     374def _handle_edit_input(
     375    char: str,
     376    *,
     377    buffer: str,
     378    editing_field: str,
     379    notices: list[str],
     380    stdin_fd: int,
     381) -> tuple[str, str | None, bool]:
     382    if char in {"\r", "\n"}:
     383        _append_notice(notices, inline_setting_notice(editing_field, buffer))
     384        return "", None, False
     385    if char in {"\x04"}:
     386        return buffer, editing_field, True
     387    if char == "\x03":
     388        _append_notice(notices, "cancelled edit")
     389        return "", None, False
     390    if char == "\x15":
     391        return "", editing_field, False
     392    if char in {"\x7f", "\b"}:
     393        return buffer[:-1], editing_field, False
     394    if char == "\x1b":
     395        key, _payload = decode_terminal_escape(read_escape_sequence(char, fd=stdin_fd))
     396        if key == "unknown":
     397            _append_notice(notices, "cancelled edit")
     398            return "", None, False
     399        return buffer, editing_field, False
     400    if char.isprintable():
     401        return buffer + char, editing_field, False
     402    return buffer, editing_field, False
     403
     404
     405def _handle_chat_submit(
     406    buffer: str,
     407    *,
     408    job_id: str,
     409    history_limit: int,
     410    snapshot: dict[str, Any],
     411    notices: list[str],
     412    right_view: str,
     413    modal_view: str | None,
     414    deps: ChatFrameDeps,
     415    async_messages: queue.Queue[str] | None = None,
     416) -> tuple[bool, dict[str, Any], str, list[str], str, str | None]:
     417    line = buffer.strip()
     418    if not line:
     419        return True, snapshot, job_id, notices, right_view, modal_view
     420    if line in {"clear", "/clear"}:
     421        notices.clear()
     422        return True, snapshot, job_id, notices, right_view, None
     423    if line in {"settings", "/settings"}:
     424        _append_notice(notices, "opened settings")
     425        return True, snapshot, job_id, notices, right_view, "settings"
     426    if line in {"jobs", "/jobs", "status", "/status"}:
     427        _append_notice(notices, "opened jobs")
     428        return True, snapshot, job_id, notices, "status", None
     429    if line in {"outcomes", "/outcomes", "updates", "/updates"}:
     430        _append_notice(notices, "opened outcomes")
     431        return True, snapshot, job_id, notices, "updates", None
     432    keep_running = True
     433    try:
     434        if deps.is_plain_chat_line(line):
     435            _append_thinking_notice(notices)
     436            _start_chat_message_worker(
     437                job_id,
     438                line,
     439                deps=deps,
     440                async_messages=async_messages,
     441            )
     442            modal_view = None
     443        else:
     444            _append_notice(notices, f"> {line}")
     445            keep_running, output = deps.capture_chat_command(job_id, line)
     446            output_lines = compact_command_output(output)
     447            if output_lines:
     448                output_text = "\n".join(output_lines)
     449                if _looks_like_waiting_output(output_text):
     450                    _append_waiting_notice(notices)
     451                else:
     452                    _append_notice(notices, output_text)
     453            if line.startswith(("/model", "/base-url", "/api-key", "/api-key-env", "/context", "/input-cost", "/output-cost", "/timeout", "/home", "/step-limit", "/output-chars", "/daily-digest", "/digest-time", "/config")):
     454                modal_view = "settings"
     455            else:
     456                modal_view = None
     457    except Exception as exc:
     458        _append_notice(notices, f"message failed: {type(exc).__name__}: {_one_line(exc, 120)}")
     459    try:
     460        refresh_job_id = _post_submit_snapshot_job_id(line, job_id)
     461        snapshot = deps.load_snapshot(refresh_job_id, history_limit)
     462        job_id = str(snapshot["job_id"])
     463    except Exception as exc:
     464        _append_notice(notices, f"refresh failed after message: {type(exc).__name__}: {_one_line(exc, 100)}")
     465    return keep_running, snapshot, job_id, notices, right_view, modal_view
     466
     467
     468def _post_submit_snapshot_job_id(line: str, current_job_id: str) -> str:
     469    """Return the job id to refresh after a submitted command or message."""
     470
     471    text = line.strip()
     472    if not text.startswith("/"):
     473        return current_job_id
     474    command = text[1:].split(maxsplit=1)[0].lower()
     475    if command in {"new", "focus", "switch"}:
     476        return WORKSPACE_CHAT_ID if current_job_id == WORKSPACE_CHAT_ID else ""
     477    return current_job_id
     478
     479
     480def _start_chat_message_worker(
     481    job_id: str,
     482    line: str,
     483    *,
     484    deps: ChatFrameDeps,
     485    async_messages: queue.Queue[str] | None,
     486) -> None:
     487    def run() -> None:
     488        try:
     489            deps.handle_chat_message(job_id, line)
     490            if async_messages is not None:
     491                async_messages.put("__refresh__")
     492        except Exception as exc:
     493            if async_messages is not None:
     494                async_messages.put(f"message failed: {type(exc).__name__}: {_one_line(exc, 120)}")
     495
     496    thread = threading.Thread(target=run, name="nipux-chat-submit", daemon=True)
     497    thread.start()
     498
     499
     500def _drain_async_notices(async_messages: queue.Queue[str], notices: list[str]) -> bool:
     501    changed = False
     502    while True:
     503        try:
     504            message = async_messages.get_nowait()
     505        except queue.Empty:
     506            return changed
     507        if message:
     508            if message == "__refresh__":
     509                _clear_thinking_notices(notices)
     510                changed = True
     511                continue
     512            _clear_thinking_notices(notices)
     513            if _looks_like_waiting_output(message):
     514                _append_waiting_notice(notices)
     515            else:
     516                _append_notice(notices, message)
     517            changed = True
     518
     519
     520def _looks_like_waiting_output(message: str) -> bool:
     521    normalized = " ".join(str(message or "").lower().split())
     522    if not normalized:
     523        return False
     524    return (
     525        normalized.startswith("waiting:")
     526        or normalized.startswith("waiting for ")
     527        or "waiting for model" in normalized
     528        or "waiting for the next worker step" in normalized
     529        or "message saved for the worker" in normalized
     530    )
     531
     532
     533def _handle_chat_escape(
     534    stdin_fd: int,
     535    *,
     536    snapshot: dict[str, Any],
     537    job_id: str,
     538    history_limit: int,
     539    right_view: str,
     540    modal_view: str | None,
     541    buffer: str,
     542    notices: list[str],
     543    deps: ChatFrameDeps,
     544) -> tuple[dict[str, Any], str, str, str | None, str]:
     545    key, payload = decode_terminal_escape(read_escape_sequence("\x1b", fd=stdin_fd))
     546    if modal_view:
     547        _append_notice(notices, "closed settings")
     548        drain_pending_input(stdin_fd)
     549        return snapshot, job_id, right_view, None, buffer
     550    if key in {"up", "down"} and buffer.startswith("/"):
     551        buffer = cycle_slash(buffer, CHAT_SLASH_COMMANDS, direction=-1 if key == "up" else 1)
     552        return snapshot, job_id, right_view, modal_view, buffer
     553    if key == "right" and not buffer:
     554        return snapshot, job_id, next_chat_right_view(right_view, 1), modal_view, buffer
     555    if key == "left" and not buffer:
     556        return snapshot, job_id, next_chat_right_view(right_view, -1), modal_view, buffer
     557    if key in {"up", "down"} and not buffer:
     558        next_focus = frame_next_job_id(snapshot, job_id, direction=-1 if key == "up" else 1)
     559        if next_focus and next_focus != job_id:
     560            job_id = next_focus
     561            deps.write_shell_state({"focus_job_id": job_id})
     562            snapshot = deps.load_snapshot(job_id, history_limit)
     563            title = snapshot["job"].get("title") or job_id
     564            _append_notice(notices, f"focus {title}")
     565        return snapshot, job_id, right_view, modal_view, buffer
     566    if key == "click" and isinstance(payload, tuple):
     567        clicked_view = deps.page_click(payload[0], payload[1], right_view)
     568        if clicked_view:
     569            return snapshot, job_id, clicked_view, modal_view, buffer
     570    drain_pending_input(stdin_fd)
     571    return snapshot, job_id, right_view, modal_view, buffer

nipux_cli/chat_intent.py 349 lines

       1"""Natural-language intent parsing for Nipux chat and shell control."""
       2
       3from __future__ import annotations
       4
       5import re
       6
       7
       8NATURAL_COMMANDS = {
       9    "tell me updates": "updates",
      10    "show updates": "updates",
      11    "show outcomes": "outcomes",
      12    "show all outcomes": "outcomes all",
      13    "show all accomplishments": "outcomes all",
      14    "show accomplishments": "outcomes",
      15    "what have all jobs done": "outcomes all",
      16    "what has everything done": "outcomes all",
      17    "what did all jobs do": "outcomes all",
      18    "what did it accomplish": "outcomes",
      19    "what has it done": "outcomes",
      20    "what has it done so far": "outcomes",
      21    "what have you done": "outcomes",
      22    "what have you done so far": "outcomes",
      23    "what did it actually do": "outcomes",
      24    "what did the model do": "outcomes",
      25    "show me what it did": "outcomes",
      26    "show history": "history",
      27    "what happened": "history",
      28    "show events": "events",
      29    "what did it find": "updates",
      30    "what did you find": "updates",
      31    "what has it found": "updates",
      32    "findings": "findings",
      33    "tasks": "tasks",
      34    "roadmap": "roadmap",
      35    "show roadmap": "roadmap",
      36    "show artifacts": "artifacts",
      37    "where are artifacts": "artifacts",
      38    "show lessons": "lessons",
      39    "what did it learn": "lessons",
      40    "show findings": "findings",
      41    "show tasks": "tasks",
      42    "show experiments": "experiments",
      43    "show sources": "sources",
      44    "show memory": "memory",
      45    "show metrics": "metrics",
      46    "show usage": "usage",
      47    "show cost": "usage",
      48    "show tokens": "usage",
      49    "show token usage": "usage",
      50    "context usage": "usage",
      51    "token usage": "usage",
      52    "how much did it cost": "usage",
      53    "how many tokens did it use": "usage",
      54    "status": "status",
      55    "check status": "status",
      56    "job status": "status",
      57    "what is going on": "status",
      58    "whats going on": "status",
      59    "what's going on": "status",
      60    "what is happening": "status",
      61    "whats happening": "status",
      62    "what's happening": "status",
      63    "what are you doing": "status",
      64    "what is it doing": "status",
      65    "how is it going": "status",
      66    "how are things going": "status",
      67    "check up on things": "status",
      68    "what is blocking it": "status",
      69    "what's blocking it": "status",
      70    "why is it stuck": "status",
      71    "is it stuck": "status",
      72    "is it running": "health",
      73    "is the daemon running": "health",
      74    "daemon health": "health",
      75    "show health": "health",
      76    "how do i start a job": "help",
      77    "how do i create a job": "help",
      78    "how do i make a job": "help",
      79    "how do i run a job": "help",
      80    "how do i start work": "help",
      81    "how do i use this": "help",
      82    "what can i do": "help",
      83    "show activity": "activity",
      84    "show tool calls": "activity",
      85    "show worker activity": "activity",
      86    "show worker output": "activity",
      87    "show raw work": "outputs",
      88    "show console output": "outputs",
      89    "show logs": "outputs",
      90    "show saved files": "artifacts",
      91    "what did it save": "artifacts",
      92    "what files did it create": "artifacts",
      93    "what outputs did it save": "artifacts",
      94    "what tasks are open": "tasks",
      95    "what is the current task": "tasks",
      96    "show measurements": "experiments",
      97    "show benchmarks": "experiments",
      98    "show milestones": "roadmap",
      99    "show plan": "roadmap",
     100    "show daemon": "health",
     101    "start daemon": "start",
     102    "restart daemon": "restart",
     103}
     104
     105
     106def natural_command_for(text: str) -> str:
     107    return NATURAL_COMMANDS.get(" ".join(text.strip().lower().split()), "")
     108
     109
     110def chat_control_command(line: str) -> str:
     111    text = " ".join(line.strip().split())
     112    if not text:
     113        return ""
     114    lowered = text.lower().rstrip("?.!")
     115    natural = NATURAL_COMMANDS.get(lowered)
     116    if natural:
     117        return f"/{natural}"
     118    control_phrase = _looks_like_control_phrase(lowered)
     119    if control_phrase and _mentions_any(lowered, ("token", "cost", "usage", "context window", "context budget")):
     120        return "/usage"
     121    if control_phrase and _mentions_any(lowered, ("tool call", "tool calls", "worker activity", "worker output", "right pane")):
     122        return "/activity"
     123    if control_phrase and _mentions_any(lowered, ("console output", "raw output", "raw run", "raw runs", "log", "logs")):
     124        return "/outputs"
     125    if control_phrase and _mentions_any(lowered, ("saved file", "saved files", "artifact", "artifacts")):
     126        return "/artifacts"
     127    if (
     128        _mentions_any(lowered, ("what did", "what has", "what have", "show me"))
     129        and _mentions_any(lowered, ("made", "created", "saved", "produced", "done", "accomplished"))
     130    ):
     131        return "/outcomes"
     132    if control_phrase and _mentions_any(lowered, ("measurement", "measurements", "experiment", "experiments", "benchmark", "benchmarks")):
     133        return "/experiments"
     134    if control_phrase and _mentions_any(lowered, ("roadmap", "milestone", "milestones", "plan")):
     135        return "/roadmap"
     136    if control_phrase and _mentions_any(lowered, ("task", "tasks", "todo", "to do", "queue")):
     137        return "/tasks"
     138    if control_phrase and _mentions_any(lowered, ("finding", "findings")):
     139        return "/findings"
     140    if control_phrase and _mentions_any(lowered, ("source", "sources")):
     141        return "/sources"
     142    if control_phrase and _mentions_any(lowered, ("lesson", "lessons", "learned")):
     143        return "/lessons"
     144    if control_phrase and _mentions_any(lowered, ("memory", "remembered", "learning state")):
     145        return "/memory"
     146    if lowered in {"start daemon", "launch daemon"}:
     147        return "/start"
     148    if lowered in {"restart daemon", "reload daemon"}:
     149        return "/restart"
     150    if lowered in {"jobs", "show jobs", "list jobs", "switch jobs", "change jobs"}:
     151        return "/jobs"
     152    if lowered in {"settings", "show settings"}:
     153        return "/settings"
     154    if lowered in {"model settings", "change model", "edit settings"}:
     155        return "/model"
     156    if lowered in {
     157        "run",
     158        "start",
     159        "run it",
     160        "start it",
     161        "run job",
     162        "run worker",
     163        "start job",
     164        "start worker",
     165        "start working",
     166        "start work",
     167        "run this",
     168        "run the job",
     169        "run this job",
     170        "start the job",
     171        "start this job",
     172        "continue",
     173        "continue it",
     174        "keep going",
     175        "keep working",
     176        "resume work",
     177    }:
     178        return "/run"
     179    if lowered in {
     180        "pause",
     181        "pause it",
     182        "pause job",
     183        "pause worker",
     184        "pause the job",
     185        "pause work",
     186        "pause this job",
     187        "stop",
     188        "stop it",
     189        "stop job",
     190        "stop worker",
     191        "stop the job",
     192        "stop work",
     193        "stop working",
     194        "stop this job",
     195        "halt",
     196        "halt job",
     197        "halt the job",
     198    }:
     199        return "/pause"
     200    if lowered in {
     201        "resume",
     202        "resume it",
     203        "resume job",
     204        "resume worker",
     205        "resume the job",
     206        "resume this job",
     207        "reopen this job",
     208    }:
     209        return "/resume"
     210    if lowered in {"history", "show history", "timeline", "show timeline"}:
     211        return "/history"
     212    if lowered in {
     213        "all outcomes",
     214        "show all outcomes",
     215        "show all accomplishments",
     216        "what have all jobs done",
     217        "what has everything done",
     218        "what did all jobs do",
     219    }:
     220        return "/outcomes all"
     221    if lowered in {
     222        "outcomes",
     223        "show outcomes",
     224        "accomplishments",
     225        "show accomplishments",
     226        "what has it done",
     227        "what has it done so far",
     228        "what have you done",
     229        "what have you done so far",
     230        "what did it actually do",
     231        "what did the model do",
     232        "show me what it did",
     233    }:
     234        return "/outcomes"
     235    if lowered in {"artifacts", "outputs", "saved outputs", "show artifacts", "show outputs"}:
     236        return "/artifacts"
     237    if lowered in {"memory", "show memory", "learning", "show learning"}:
     238        return "/memory"
     239    return ""
     240
     241
     242def _mentions_any(text: str, needles: tuple[str, ...]) -> bool:
     243    for needle in needles:
     244        if " " in needle:
     245            if needle in text:
     246                return True
     247            continue
     248        if re.search(rf"\b{re.escape(needle)}\b", text):
     249            return True
     250    return False
     251
     252
     253def _looks_like_control_phrase(text: str) -> bool:
     254    return text.startswith(
     255        (
     256            "show ",
     257            "view ",
     258            "open ",
     259            "list ",
     260            "display ",
     261            "give me ",
     262            "where ",
     263            "what ",
     264            "how ",
     265            "is ",
     266            "are ",
     267            "check ",
     268        )
     269    )
     270
     271
     272def message_requests_immediate_run(message: str) -> bool:
     273    lowered = " ".join(message.strip().lower().split())
     274    if message_requests_queued_job(message):
     275        return False
     276    if re.match(r"^(?:please\s+)?(?:start|launch|run|spin\s+off|spin\s+up)\b", lowered):
     277        return True
     278    return bool(re.search(r"\b(?:and|then)\s+(?:start|launch|run|resume)\s+(?:it|the\s+job|work)?\b", lowered))
     279
     280
     281def message_requests_queued_job(message: str) -> bool:
     282    lowered = " ".join(message.strip().lower().split())
     283    return bool(
     284        re.search(
     285            r"\b(?:queue only|plan only|create only|do not start|don't start|do not run|don't run|without starting)\b",
     286            lowered,
     287        )
     288    )
     289
     290
     291def extract_job_objective_from_message(message: str) -> str:
     292    text = " ".join(message.strip().split())
     293    if not text:
     294        return ""
     295    lowered = text.lower()
     296    patterns = [
     297        r"^(?:please\s+)?(?:create|start|spin\s+off|make|launch)\s+(?:a\s+)?(?:new\s+)?job\s+(?:to|for|that|which)?\s*(.+)$",
     298        r"^(?:please\s+)?(?:create|start|spin\s+off|spin\s+up|make|launch|run)\s+(?:a\s+|an\s+)?(?:new\s+)?(?:worker|agent|task)\s+(?:to|for|that|which)?\s*(.+)$",
     299        r"^(?:please\s+)?(?:send|queue)\s+(?:off\s+)?(?:a\s+)?(?:new\s+)?job\s+(?:to|for|that|which)?\s*(.+)$",
     300        r"^(?:please\s+)?(?:new|job)\s+(.+)$",
     301        r"^(?:please\s+)?(?:start|run|launch)\s+(?!daemon\b|it\b|this\b|that\b|the\s+job\b|the\s+worker\b|job\b|worker\b|work\b)(.+)$",
     302        r"^(?:please\s+)?(?:can\s+you|could\s+you|i\s+need\s+you\s+to|i\s+want\s+you\s+to)\s+(.+)$",
     303    ]
     304    for pattern in patterns:
     305        match = re.match(pattern, text, flags=re.IGNORECASE)
     306        if match:
     307            objective = match.group(1).strip(" .")
     308            return objective if looks_like_job_objective(objective) else ""
     309    if looks_like_job_objective(text) and not looks_like_smalltalk(lowered):
     310        return text
     311    return ""
     312
     313
     314def looks_like_smalltalk(lowered: str) -> bool:
     315    return lowered in {"hi", "hello", "hey", "yo", "sup", "thanks", "thank you"} or lowered.endswith("?")
     316
     317
     318def looks_like_job_objective(text: str) -> bool:
     319    lowered = text.lower()
     320    if len(text.split()) < 3:
     321        return False
     322    action_words = {
     323        "research",
     324        "monitor",
     325        "optimize",
     326        "build",
     327        "find",
     328        "test",
     329        "deploy",
     330        "fix",
     331        "write",
     332        "analyze",
     333        "audit",
     334        "track",
     335        "benchmark",
     336        "create",
     337        "document",
     338        "draft",
     339        "generate",
     340        "scrape",
     341        "produce",
     342        "watch",
     343        "automate",
     344        "summarize",
     345        "compare",
     346        "investigate",
     347        "improve",
     348    }
     349    return any(re.search(rf"\b{re.escape(word)}\b", lowered) for word in action_words)

nipux_cli/chat_tui.py 277 lines

       1"""Chat workspace terminal frame rendering."""
       2
       3from __future__ import annotations
       4
       5from typing import Any
       6
       7from nipux_cli.config import load_config
       8from nipux_cli.first_run_tui import first_run_themed_lines
       9from nipux_cli.settings import edit_target_hint, edit_target_label, edit_target_masks_input
      10from nipux_cli.tui_commands import CHAT_SLASH_COMMANDS, slash_suggestion_lines
      11from nipux_cli.tui_event_format import clean_step_summary
      12from nipux_cli.tui_events import chat_pane_lines
      13from nipux_cli.tui_layout import _compose_bar, _top_bar
      14from nipux_cli.tui_outcomes import chat_updates_pane_lines
      15from nipux_cli.tui_status import (
      16    job_display_state,
      17    right_pane_lines,
      18    worker_label,
      19)
      20from nipux_cli.tui_style import _accent, _bold, _fit_ansi, _muted, _one_line, _strip_ansi
      21
      22
      23def build_chat_frame(
      24    snapshot: dict[str, Any],
      25    input_buffer: str,
      26    notices: list[str],
      27    *,
      28    width: int,
      29    height: int,
      30    right_view: str = "updates",
      31    selected_control: int = 0,
      32    editing_field: str | None = None,
      33    modal_view: str | None = None,
      34) -> str:
      35    del selected_control
      36    if right_view == "work":
      37        right_view = "updates"
      38    width = max(92, width)
      39    height = max(22, height)
      40    job = snapshot["job"]
      41    right_job = snapshot.get("right_job") if isinstance(snapshot.get("right_job"), dict) else job
      42    jobs = snapshot["jobs"]
      43    steps = snapshot["steps"]
      44    artifacts = snapshot["artifacts"]
      45    job_id = str(snapshot["job_id"])
      46    right_job_id = str(snapshot.get("right_job_id") or job_id)
      47    job_artifacts = snapshot.get("job_artifacts") if isinstance(snapshot.get("job_artifacts"), dict) else {}
      48    if artifacts:
      49        job_artifacts.setdefault(job_id, artifacts)
      50    job_summary_events = snapshot.get("job_summary_events") if isinstance(snapshot.get("job_summary_events"), dict) else {}
      51    job_counts = snapshot.get("job_counts") if isinstance(snapshot.get("job_counts"), dict) else {}
      52    memory_entries = snapshot["memory_entries"]
      53    events = snapshot["events"]
      54    summary_events = snapshot.get("summary_events") if isinstance(snapshot.get("summary_events"), list) else events
      55    daemon = snapshot["daemon"]
      56    model = str(snapshot["model"])
      57    base_url = str(snapshot.get("base_url") or "")
      58    token_usage = snapshot.get("token_usage") if isinstance(snapshot.get("token_usage"), dict) else {}
      59    context_length = int(snapshot.get("context_length") or 0)
      60    counts = snapshot.get("counts") if isinstance(snapshot.get("counts"), dict) else {}
      61    findings = _metadata_records(right_job, "finding_ledger")
      62    sources = _metadata_records(right_job, "source_ledger")
      63    tasks = _metadata_records(right_job, "task_queue")
      64    experiments = _metadata_records(right_job, "experiment_ledger")
      65    lessons = _metadata_records(right_job, "lessons")
      66    roadmap = right_job.get("metadata", {}).get("roadmap") if isinstance(right_job.get("metadata"), dict) else {}
      67    milestones = roadmap.get("milestones") if isinstance(roadmap, dict) and isinstance(roadmap.get("milestones"), list) else []
      68    open_tasks = sum(1 for task in tasks if str(task.get("status") or "open") in {"open", "active"})
      69    state = job_display_state(right_job, bool(daemon["running"]))
      70    worker = worker_label(right_job, bool(daemon["running"]))
      71    latest_step = steps[-1] if steps else None
      72    right_width = min(max(52, int(width * 0.36)), 72)
      73    left_width = max(48, width - right_width - 3)
      74    if left_width < 48:
      75        left_width = 48
      76        right_width = max(34, width - left_width - 3)
      77    latest_text = _step_line(latest_step, chars=right_width - 6) if latest_step else "no worker steps yet"
      78    daemon_text = _daemon_state_line(daemon)
      79    goal_text = " ".join(str(right_job.get("objective") or "").split())
      80    metrics = [
      81        ("actions", counts.get("steps", _step_count(steps))),
      82        ("outputs", counts.get("artifacts", len(artifacts))),
      83        ("findings", len(findings)),
      84        ("sources", len(sources)),
      85        ("tasks", f"{len(tasks)}/{open_tasks} open"),
      86        ("roadmap", len(milestones)),
      87        ("experiments", len(experiments)),
      88        ("lessons", len(lessons)),
      89        ("memory", counts.get("memory", len(memory_entries))),
      90    ]
      91
      92    header = _top_bar(
      93        width,
      94        state=state,
      95        daemon=daemon_text,
      96        model=model,
      97        token_usage=token_usage,
      98        context_length=context_length,
      99        base_url=base_url,
     100    )
     101    if editing_field:
     102        hint = edit_target_hint(editing_field)
     103        prompt_label = edit_target_label(editing_field)
     104    elif not jobs:
     105        hint = "Type a goal to create the first worker  ·  / opens commands  ·  /settings configures"
     106        prompt_label = "❯"
     107    else:
     108        hint = "Enter sends  ·  / opens commands  ·  /settings configures  ·  ←→ updates/jobs"
     109        prompt_label = "❯"
     110    suggestions = [] if editing_field else slash_suggestion_lines(input_buffer, CHAT_SLASH_COMMANDS, width=width)
     111    compose_lines = _compose_bar(
     112        input_buffer,
     113        width=width,
     114        hint=hint,
     115        suggestions=suggestions,
     116        prompt_label=prompt_label,
     117        mask_input=edit_target_masks_input(editing_field),
     118    )
     119    footer_rows = len(compose_lines)
     120    body_rows = max(10, height - len(header) - 1 - footer_rows)
     121    chat_lines = chat_pane_lines(events, notices, width=left_width, rows=body_rows)
     122    if right_view == "updates":
     123        right_lines = chat_updates_pane_lines(
     124            job=right_job,
     125            events=summary_events,
     126            width=right_width,
     127            rows=body_rows,
     128        )
     129        right_title = "Model updates"
     130    else:
     131        right_lines = right_pane_lines(
     132            job=right_job,
     133            jobs=jobs,
     134            job_artifacts=job_artifacts,
     135            job_summary_events=job_summary_events,
     136            job_counts=job_counts,
     137            job_id=right_job_id,
     138            daemon_running=bool(daemon["running"]),
     139            state=state,
     140            worker=worker,
     141            daemon_text=daemon_text,
     142            model=model,
     143            goal_text=goal_text,
     144            latest_text=latest_text,
     145            metrics=metrics,
     146            events=summary_events,
     147            token_usage=token_usage,
     148            context_length=context_length,
     149            width=right_width,
     150            rows=body_rows,
     151            right_view=right_view,
     152        )
     153        right_title = "Jobs"
     154    lines = [*header, _two_col_title(left_width, right_width, "Chat", right_title)]
     155    for index in range(body_rows):
     156        left = chat_lines[index] if index < len(chat_lines) else ""
     157        right = right_lines[index] if index < len(right_lines) else ""
     158        lines.append(_two_col_line(left, right, left_width=left_width, right_width=right_width))
     159    lines.extend(compose_lines)
     160    if len(lines) > height:
     161        keep_top = min(4, len(header) + 1)
     162        keep_bottom = footer_rows
     163        middle_budget = max(0, height - keep_top - keep_bottom)
     164        lines = lines[:keep_top] + lines[-(middle_budget + keep_bottom) : -keep_bottom] + lines[-keep_bottom:]
     165    if modal_view == "settings":
     166        lines = _overlay_settings_modal(lines[:height], width=width, height=height)
     167    return "\n".join(first_run_themed_lines(lines[:height], width=width))
     168
     169
     170def _two_col_title(left_width: int, right_width: int, left: str, right: str) -> str:
     171    return _fit_ansi(_bold(left.upper()), left_width) + _muted(" │ ") + _fit_ansi(_bold(right.upper()), right_width)
     172
     173
     174def _two_col_line(left: str, right: str, *, left_width: int, right_width: int) -> str:
     175    return _fit_ansi(left, left_width) + _muted(" │ ") + _fit_ansi(right, right_width)
     176
     177
     178def _overlay_settings_modal(lines: list[str], *, width: int, height: int) -> list[str]:
     179    config = load_config()
     180    key_state = "set" if config.model.api_key else "missing"
     181    input_cost = _rate_text(config.model.input_cost_per_million)
     182    output_cost = _rate_text(config.model.output_cost_per_million)
     183    cost_limit = "none" if config.runtime.max_job_cost_usd is None else f"${config.runtime.max_job_cost_usd:g}"
     184    content = [
     185        _bold("Model"),
     186        _settings_row("id", config.model.model, "/model MODEL"),
     187        _settings_row("endpoint", config.model.base_url, "/base-url URL"),
     188        _settings_row("key", f"{key_state} in {config.model.api_key_env}", "/api-key KEY"),
     189        _settings_row(
     190            "limits",
     191            f"context {config.model.context_length}, timeout {config.model.request_timeout_seconds:g}s",
     192            "/context TOKENS /timeout SECONDS",
     193        ),
     194        "",
     195        _bold("Runtime"),
     196        _settings_row("home", str(config.runtime.home), "/home PATH"),
     197        _settings_row(
     198            "steps",
     199            f"tool {config.runtime.max_step_seconds}s, preview {config.runtime.artifact_inline_char_limit} chars",
     200            "/step-limit SECONDS /output-chars CHARS",
     201        ),
     202        _settings_row(
     203            "digest",
     204            f"{config.runtime.daily_digest_enabled} at {config.runtime.daily_digest_time}",
     205            "/daily-digest BOOL /digest-time HH:MM",
     206        ),
     207        "",
     208        _bold("Cost"),
     209        _settings_row("rates", f"input {input_cost}, output {output_cost}", "/input-cost DOLLARS /output-cost DOLLARS"),
     210        _settings_row("limit", cost_limit, "/max-cost DOLLARS"),
     211        "",
     212        _muted("Edit with slash commands in the composer. Esc closes."),
     213    ]
     214    box_width = min(max(64, int(width * 0.58)), width - 8)
     215    box_height = min(len(content) + 4, height - 6)
     216    inner = max(20, box_width - 4)
     217    title = f" Settings {_accent('●')} "
     218    rule_width = max(2, box_width - len(_strip_ansi(title)) - 2)
     219    left_rule = max(1, rule_width // 2)
     220    right_rule = max(1, rule_width - left_rule)
     221    top = "╭" + "─" * left_rule + title + "─" * right_rule + "╮"
     222    box = [top]
     223    for item in content[: box_height - 3]:
     224        if item:
     225            box.append("│ " + _fit_ansi(item, inner) + " │")
     226        else:
     227            box.append("│ " + " " * inner + " │")
     228    while len(box) < box_height - 1:
     229        box.append("│ " + " " * inner + " │")
     230    box.append("╰" + "─" * (box_width - 2) + "╯")
     231    output = [_fit_ansi(line, width) for line in lines]
     232    start_y = max(2, (height - len(box)) // 2)
     233    start_x = max(0, (width - box_width) // 2)
     234    for offset, modal_line in enumerate(box):
     235        target = start_y + offset
     236        if target >= len(output):
     237            break
     238        output[target] = _fit_ansi(" " * start_x + modal_line, width)
     239    return output
     240
     241
     242def _settings_row(label: str, value: Any, command: str) -> str:
     243    value_text = _one_line(value, 42)
     244    return f"{_muted(label.ljust(9))} {_bold(value_text)}  {_muted(command)}"
     245
     246
     247def _rate_text(value: float | None) -> str:
     248    return "provider-reported" if value is None else f"${value:g}/1M"
     249
     250
     251def _metadata_records(job: dict[str, Any], key: str) -> list[dict[str, Any]]:
     252    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     253    values = metadata.get(key)
     254    if not isinstance(values, list):
     255        return []
     256    return [value for value in values if isinstance(value, dict)]
     257
     258
     259def _step_count(steps: list[dict[str, Any]]) -> int:
     260    numbers = [int(step.get("step_no") or 0) for step in steps]
     261    return max(numbers, default=0)
     262
     263
     264def _step_line(step: dict[str, Any], *, chars: int = 180) -> str:
     265    tool = step.get("tool_name") or step.get("kind") or "-"
     266    summary = clean_step_summary(step.get("summary") or step.get("error") or "-")
     267    error = " ERROR" if step.get("error") else ""
     268    return f"#{step['step_no']:<4} {step['status']:<9} {tool:<18} {_one_line(summary, chars)}{error}"
     269
     270
     271def _daemon_state_line(lock: dict[str, Any]) -> str:
     272    metadata = lock.get("metadata") if isinstance(lock.get("metadata"), dict) else {}
     273    if lock.get("running"):
     274        pid = metadata.get("pid") or "unknown"
     275        stale = " stale-runtime" if lock.get("stale") else ""
     276        return f"running pid={pid}{stale}"
     277    return "ready when work starts"

nipux_cli/cli.py 3188 lines

       1"""Thin CLI for the Nipux agent runtime."""
       2
       3from __future__ import annotations
       4
       5import argparse
       6import json
       7import os
       8import shlex
       9import shutil
      10import subprocess
      11import sys
      12import threading
      13import time
      14from contextlib import redirect_stdout
      15from io import StringIO
      16from pathlib import Path
      17from typing import Any
      18
      19from nipux_cli import __version__
      20from nipux_cli.artifacts import ArtifactStore
      21from nipux_cli.chat_intent import (
      22    chat_control_command,
      23    extract_job_objective_from_message as _extract_job_objective_from_message,
      24    message_requests_immediate_run,
      25    message_requests_queued_job,
      26    natural_command_for,
      27)
      28from nipux_cli.cli_state import (
      29    configured_focus_job_id as _configured_focus_job_id,
      30    default_job_id as _default_job_id,
      31    find_job as _find_job,
      32    clear_model_setup_verified as _clear_model_setup_verified,
      33    mark_model_setup_verified as _mark_model_setup_verified,
      34    model_setup_verified as _model_setup_verified,
      35    read_shell_state as _read_shell_state,
      36    write_shell_state as _write_shell_state,
      37)
      38from nipux_cli.cli_render import (
      39    daemon_event_line as _daemon_event_line,
      40    daemon_state_line as _daemon_state_line,
      41    important_startup_events as _important_startup_events,
      42    job_ref_text as _job_ref_text,
      43    json_default as _json_default,
      44    next_operator_action as _next_operator_action,
      45    note_text as _note_text,
      46    print_artifact as _print_artifact,
      47    print_event_card as _print_event_card,
      48    print_event_details as _print_event_details,
      49    print_jobs_panel as _print_jobs_panel,
      50    print_metric_grid as _print_metric_grid,
      51    print_run as _print_run,
      52    print_step as _print_step,
      53    print_wrapped as _print_wrapped,
      54    public_event as _public_event,
      55    rule as _rule,
      56    section_title as _section_title,
      57    short_path as _short_path,
      58    step_line as _step_line,
      59    terminal_width as _terminal_width,
      60)
      61from nipux_cli.chat_context import build_chat_messages as _build_chat_messages
      62from nipux_cli.chat_commands import ChatCommandDeps, handle_chat_slash_command as _handle_chat_slash_command
      63from nipux_cli.chat_controller import (
      64    ChatControllerDeps,
      65    chat_reply_text_and_metadata as _controller_reply_text_and_metadata,
      66    handle_chat_control_intent as _controller_handle_chat_control_intent,
      67    handle_chat_message as _controller_handle_chat_message,
      68    maybe_spawn_job_from_chat as _controller_maybe_spawn_job_from_chat,
      69    queue_chat_note as _controller_queue_chat_note,
      70)
      71from nipux_cli.chat_frame_runtime import (
      72    ChatFrameDeps,
      73    compact_command_output as _compact_command_output,
      74    emit_frame_if_changed as _emit_frame_if_changed,
      75    run_chat_frame as _run_chat_frame,
      76)
      77from nipux_cli.chat_tui import build_chat_frame as _build_chat_tui_frame
      78from nipux_cli.cli_help import NIPUX_BANNER, print_shell_help as _render_shell_help
      79from nipux_cli.config import (
      80    DEFAULT_BASE_URL,
      81    DEFAULT_API_KEY_ENV,
      82    DEFAULT_CONTEXT_LENGTH,
      83    DEFAULT_MODEL,
      84    DEFAULT_OPENROUTER_API_KEY_ENV,
      85    DEFAULT_OPENROUTER_MODEL,
      86    default_config_yaml,
      87    load_config,
      88    write_private_text,
      89)
      90from nipux_cli.daemon_control import cmd_restart_impl as _cmd_restart_impl
      91from nipux_cli.daemon_control import cmd_start_impl as _cmd_start_impl
      92from nipux_cli.daemon_control import ensure_remote_model_ready_for_worker as _daemon_ensure_remote_model_ready
      93from nipux_cli.daemon_control import provider_preflight_is_recoverable as _daemon_provider_preflight_is_recoverable
      94from nipux_cli.daemon_control import recoverable_remote_model_preflight_failures as _daemon_recoverable_remote_model_preflight_failures
      95from nipux_cli.daemon_control import remote_model_preflight_failures as _daemon_remote_model_preflight_failures
      96from nipux_cli.daemon_control import start_daemon_if_needed_impl as _start_daemon_if_needed_impl
      97from nipux_cli.daemon_control import stop_daemon_process_impl as _stop_daemon_process_impl
      98from nipux_cli.daemon import Daemon, DaemonAlreadyRunning, daemon_lock_status, read_daemon_events
      99from nipux_cli.dashboard import collect_dashboard_state, render_dashboard, render_overview
     100from nipux_cli.db import AgentDB, utc_now
     101from nipux_cli.digest import render_job_digest, write_daily_digest
     102from nipux_cli.doctor import run_doctor
     103from nipux_cli.first_run_tui import (
     104    build_first_run_frame as _build_first_run_tui_frame,
     105    first_run_actions as _first_run_tui_actions,
     106    first_run_columns as _first_run_columns,
     107)
     108from nipux_cli.first_run_controller import (
     109    FirstRunFrameDeps,
     110    capture_first_run_command as _controller_capture_first_run_command,
     111    create_first_run_job as _controller_create_first_run_job,
     112    first_run_chat_reply as _controller_first_run_chat_reply,
     113    first_token as _controller_first_token,
     114    handle_first_run_action as _controller_handle_first_run_action,
     115    handle_first_run_frame_line as _controller_handle_first_run_frame_line,
     116)
     117from nipux_cli.first_run_frame_runtime import (
     118    FirstRunRuntimeDeps,
     119    clamp_selection as _clamp_first_run_runtime_selection,
     120    run_first_run_frame as _run_first_run_frame,
     121)
     122from nipux_cli.event_render import event_line as _event_line
     123from nipux_cli.frame_snapshot import WORKSPACE_CHAT_ID, load_frame_snapshot
     124from nipux_cli.parser_builder import build_arg_parser
     125from nipux_cli.planning import (
     126    format_initial_plan,
     127    initial_plan_for_objective,
     128    initial_roadmap_for_objective,
     129    initial_task_contract,
     130)
     131from nipux_cli.scheduling import job_provider_blocked, operator_resume_metadata
     132from nipux_cli.record_commands import (
     133    RecordCommandDeps,
     134    cmd_experiments_impl,
     135    cmd_findings_impl,
     136    cmd_memory_impl,
     137    cmd_metrics_impl,
     138    cmd_roadmap_impl,
     139    cmd_sources_impl,
     140    cmd_tasks_impl,
     141    cmd_usage_impl,
     142)
     143from nipux_cli.service_install import cmd_autostart, cmd_service
     144from nipux_cli.service_install import launch_agent_path as _launch_agent_path
     145from nipux_cli.service_install import launch_agent_plist as _service_launch_agent_plist
     146from nipux_cli.service_install import systemd_service_text as _service_systemd_service_text
     147from nipux_cli.templates import program_for_job
     148from nipux_cli.tui_commands import CHAT_SETTING_COMMANDS, slash_suggestion_lines
     149from nipux_cli.settings import (
     150    config_field_value,
     151    save_config_field,
     152)
     153from nipux_cli.settings_commands import (
     154    capture_setting_command as _capture_setting_command,
     155    handle_chat_setting_command as _handle_chat_setting_command,
     156)
     157from nipux_cli.tui_event_format import (
     158    clean_step_summary as _clean_step_summary,
     159    friendly_error_text as _friendly_error_text,
     160    generic_display_text as _generic_display_text,
     161)
     162from nipux_cli.tui_events import (
     163    live_badge as _live_badge,
     164    minimal_live_event_line as _minimal_live_event_line,
     165)
     166from nipux_cli.tui_outcomes import model_update_event_parts as _model_update_event_parts
     167from nipux_cli.tui_status import (
     168    job_display_state as _job_display_state,
     169    worker_label as _worker_label,
     170)
     171from nipux_cli.tui_style import (
     172    _accent,
     173    _fancy_ui,
     174    _one_line,
     175    _status_badge,
     176)
     177from nipux_cli.uninstall import build_uninstall_plan, uninstall_installed_tool, uninstall_runtime
     178from nipux_cli.updater import update_checkout
     179from nipux_cli.updates import render_all_updates_report, render_updates_report
     180
     181_save_config_field = save_config_field
     182_config_field_value = config_field_value
     183_slash_suggestion_lines = slash_suggestion_lines
     184_chat_control_command = chat_control_command
     185
     186
     187def _launch_agent_plist(*, poll_seconds: float, quiet: bool) -> str:
     188    return _service_launch_agent_plist(poll_seconds=poll_seconds, quiet=quiet)
     189
     190
     191def _systemd_service_text(*, poll_seconds: float, quiet: bool) -> str:
     192    return _service_systemd_service_text(poll_seconds=poll_seconds, quiet=quiet)
     193
     194
     195SHELL_BUILTINS = {"help", "?", "commands", "exit", "quit", ":q", "clear"}
     196SHELL_COMMAND_NAMES = {
     197    "init",
     198    "uninstall",
     199    "create",
     200    "jobs",
     201    "ls",
     202    "focus",
     203    "rename",
     204    "delete",
     205    "rm",
     206    "chat",
     207    "shell",
     208    "status",
     209    "health",
     210    "history",
     211    "events",
     212    "activity",
     213    "feed",
     214    "tail",
     215    "updates",
     216    "findings",
     217    "tasks",
     218    "roadmap",
     219    "experiments",
     220    "update",
     221    "dashboard",
     222    "dash",
     223    "start",
     224    "stop",
     225    "restart",
     226    "browser-dashboard",
     227    "artifacts",
     228    "artifact",
     229    "lessons",
     230    "learn",
     231    "findings",
     232    "sources",
     233    "memory",
     234    "metrics",
     235    "usage",
     236    "logs",
     237    "outputs",
     238    "output",
     239    "watch",
     240    "run-one",
     241    "run",
     242    "work",
     243    "steer",
     244    "say",
     245    "pause",
     246    "resume",
     247    "cancel",
     248    "digest",
     249    "daily-digest",
     250    "daemon",
     251    "doctor",
     252    "autostart",
     253    "service",
     254}
     255
     256def _db() -> tuple[AgentDB, object]:
     257    config = load_config()
     258    config.ensure_dirs()
     259    return AgentDB(config.runtime.state_db_path), config
     260
     261
     262def _record_command_deps() -> RecordCommandDeps:
     263    return RecordCommandDeps(
     264        db_factory=_db,
     265        resolve_job_id=_resolve_job_id,
     266        job_ref_text=_job_ref_text,
     267    )
     268
     269
     270def cmd_init(args: argparse.Namespace) -> None:
     271    config = load_config()
     272    config.ensure_dirs()
     273    path = Path(args.path).expanduser() if args.path else config.runtime.home / "config.yaml"
     274    if path.exists() and not args.force:
     275        print(f"Config already exists: {path}")
     276        return
     277    path.parent.mkdir(parents=True, exist_ok=True)
     278    model = args.model or DEFAULT_MODEL
     279    base_url = args.base_url or DEFAULT_BASE_URL
     280    api_key_env = args.api_key_env or DEFAULT_API_KEY_ENV
     281    if args.openrouter:
     282        base_url = args.base_url or "https://openrouter.ai/api/v1"
     283        api_key_env = args.api_key_env or DEFAULT_OPENROUTER_API_KEY_ENV
     284        model = args.model or DEFAULT_OPENROUTER_MODEL
     285    write_private_text(
     286        path,
     287        default_config_yaml(
     288            model=model,
     289            base_url=base_url,
     290            api_key_env=api_key_env,
     291            context_length=getattr(args, "context_length", DEFAULT_CONTEXT_LENGTH),
     292        ),
     293    )
     294    print(f"Wrote {path}")
     295    env_path = config.runtime.home / ".env"
     296    if not env_path.exists():
     297        write_private_text(
     298            env_path,
     299            f"# Optional local secrets for Nipux. This file stays outside the git repo.\n{api_key_env}=\n",
     300        )
     301        print(f"Wrote {env_path} (fill {api_key_env}; do not commit secrets)")
     302
     303
     304def cmd_update(args: argparse.Namespace) -> None:
     305    config = load_config()
     306    config.ensure_dirs()
     307    daemon_before = daemon_lock_status(config.runtime.home / "agentd.lock")
     308    code, lines = update_checkout(path=args.path, allow_dirty=args.allow_dirty)
     309    for line in lines:
     310        print(line)
     311    if code:
     312        raise SystemExit(code)
     313    if getattr(args, "no_restart", False):
     314        print("Daemon restart skipped by --no-restart.")
     315        return
     316    daemon_after = daemon_lock_status(config.runtime.home / "agentd.lock")
     317    if not daemon_before.get("running") and not daemon_after.get("running"):
     318        print("No daemon is running; no restart needed.")
     319        return
     320    print("Restarting running daemon so it uses the updated code.")
     321    try:
     322        cmd_restart(
     323            argparse.Namespace(
     324                poll_seconds=0.0,
     325                wait=5.0,
     326                fake=False,
     327                quiet=True,
     328                log_file=None,
     329            )
     330        )
     331    except SystemExit as exc:
     332        detail = str(exc) if str(exc) else "restart failed"
     333        print(f"Update succeeded, but daemon restart failed: {_one_line(detail, 160)}")
     334
     335
     336def cmd_uninstall(args: argparse.Namespace) -> None:
     337    config = load_config()
     338    plan = build_uninstall_plan(runtime_home=config.runtime.home, include_legacy=not args.keep_legacy)
     339    remove_tool = bool(getattr(args, "remove_tool", False)) or not bool(getattr(args, "keep_tool", False))
     340    if not args.yes and not args.dry_run:
     341        print("This will stop Nipux and remove local runtime state:")
     342        for path in (*plan.service_paths, *plan.paths):
     343            print(f"  {path.expanduser()}")
     344        if remove_tool:
     345            print("It will also remove the installed `nipux` command with: uv tool uninstall nipux")
     346        try:
     347            answer = input("Type 'uninstall' to continue: ").strip().lower()
     348        except (EOFError, KeyboardInterrupt):
     349            print()
     350            print("uninstall aborted")
     351            return
     352        if answer != "uninstall":
     353            print("uninstall aborted")
     354            return
     355    if not args.dry_run:
     356        try:
     357            _stop_daemon_process(config, wait=float(args.wait), quiet=True)
     358        except (OSError, SystemExit) as exc:
     359            print(f"daemon stop skipped: {exc}")
     360    for line in uninstall_runtime(
     361        runtime_home=config.runtime.home,
     362        dry_run=bool(args.dry_run),
     363        include_legacy=not args.keep_legacy,
     364    ):
     365        print(line)
     366    if remove_tool:
     367        code, lines = uninstall_installed_tool(dry_run=bool(args.dry_run))
     368        for line in lines:
     369            print(line)
     370        if code:
     371            print("installed command removal failed; runtime state was still removed")
     372    elif not args.dry_run:
     373        print("runtime removed. Installed `nipux` command kept by --keep-tool.")
     374
     375
     376def cmd_create(args: argparse.Namespace) -> None:
     377    if not _ensure_model_setup_verified_for_workspace():
     378        raise SystemExit(1)
     379    job_id, title = _create_job(
     380        objective=args.objective,
     381        title=args.title,
     382        kind=args.kind,
     383        cadence=args.cadence,
     384    )
     385    print(f"created {title}")
     386
     387
     388def _ensure_model_setup_verified_for_workspace() -> bool:
     389    config = load_config()
     390    if _model_setup_verified(config):
     391        return True
     392    if _workspace_has_model_config(config) and _auto_verify_model_setup(config):
     393        return True
     394    print("Model setup is not verified.")
     395    print("Run `nipux` and finish setup, or run `nipux doctor --check-model` after configuring a provider.")
     396    print("Jobs and chat stay locked until the configured model accepts a chat request.")
     397    return False
     398
     399
     400def _workspace_has_model_config(config: Any) -> bool:
     401    return bool(_read_shell_state().get("setup_completed")) or (config.runtime.home / "config.yaml").exists()
     402
     403
     404def _auto_verify_model_setup(config: Any) -> bool:
     405    checks = run_doctor(config=config, check_model=True)
     406    ok = all(check.ok for check in checks)
     407    if ok:
     408        _mark_model_setup_verified(config)
     409        return True
     410    _clear_model_setup_verified()
     411    return False
     412
     413
     414def _create_job(
     415    *, objective: str, title: str | None = None, kind: str = "generic", cadence: str | None = None
     416) -> tuple[str, str]:
     417    db, config = _db()
     418    try:
     419        title = title or objective.strip().splitlines()[0][:80] or "Untitled job"
     420        plan = initial_plan_for_objective(objective)
     421        job_id = db.create_job(
     422            objective,
     423            title=title,
     424            kind=kind,
     425            cadence=cadence,
     426            metadata={"planning": plan},
     427        )
     428        db.update_job_status(job_id, "queued", metadata_patch={"planning": plan, "planning_status": "auto_accepted"})
     429        db.append_agent_update(job_id, format_initial_plan(plan), category="plan", metadata={"planning": plan})
     430        db.append_agent_update(job_id, "Plan accepted automatically. I will start working from the planned tasks.", category="plan")
     431        db.append_roadmap_record(job_id, **initial_roadmap_for_objective(title=title, objective=objective))
     432        for index, task in enumerate(plan["tasks"], start=1):
     433            task_contract = initial_task_contract(str(task))
     434            db.append_task_record(
     435                job_id,
     436                title=str(task),
     437                status="open",
     438                priority=max(0, 10 - index),
     439                goal=objective,
     440                output_contract=task_contract["output_contract"],
     441                acceptance_criteria=task_contract["acceptance_criteria"],
     442                evidence_needed=task_contract["evidence_needed"],
     443                stall_behavior=task_contract["stall_behavior"],
     444                metadata={"phase": "initial_plan"},
     445            )
     446        program = config.runtime.jobs_dir / job_id / "program.md"
     447        program.parent.mkdir(parents=True, exist_ok=True)
     448        program.write_text(
     449            program_for_job(kind=kind, title=title, objective=objective),
     450            encoding="utf-8",
     451        )
     452        _write_shell_state({"focus_job_id": job_id})
     453        return job_id, title
     454    finally:
     455        db.close()
     456
     457
     458def cmd_jobs(args: argparse.Namespace) -> None:
     459    db, _ = _db()
     460    try:
     461        jobs = db.list_jobs()
     462        if not jobs:
     463            print('No jobs yet. Create one with: nipux create "objective"')
     464            return
     465        focused = _configured_focus_job_id(db)
     466        daemon_running = daemon_lock_status(load_config().runtime.home / "agentd.lock")["running"]
     467        _print_jobs_panel(jobs, focused_job_id=str(focused or ""), daemon_running=bool(daemon_running))
     468    finally:
     469        db.close()
     470
     471
     472def cmd_focus(args: argparse.Namespace) -> None:
     473    db, _ = _db()
     474    try:
     475        if not args.query:
     476            job_id = _default_job_id(db)
     477            if not job_id:
     478                print("No focused job. Create one first.")
     479                return
     480            job = db.get_job(job_id)
     481            daemon = daemon_lock_status(load_config().runtime.home / "agentd.lock")
     482            print(f"focus: {job['title']} | job {_job_display_state(job, bool(daemon['running']))}")
     483            return
     484        job = _find_job(db, " ".join(args.query))
     485        if not job:
     486            print(f"No job matched: {' '.join(args.query)}")
     487            return
     488        _write_shell_state({"focus_job_id": job["id"]})
     489        daemon = daemon_lock_status(load_config().runtime.home / "agentd.lock")
     490        print(f"focus set: {job['title']} | job {_job_display_state(job, bool(daemon['running']))}")
     491    finally:
     492        db.close()
     493
     494
     495def cmd_rename(args: argparse.Namespace) -> None:
     496    db, config = _db()
     497    try:
     498        job_id = _resolve_job_id(db, args.job_id)
     499        if not job_id:
     500            ref = _job_ref_text(args.job_id)
     501            print(f"No job matched: {ref}" if ref else "No jobs found.")
     502            return
     503        old = db.get_job(job_id)
     504        renamed = db.rename_job(job_id, _job_ref_text(args.title))
     505        program = config.runtime.jobs_dir / job_id / "program.md"
     506        if program.exists():
     507            try:
     508                content = program.read_text(encoding="utf-8")
     509                lines = content.splitlines()
     510                if lines and lines[0].startswith("# "):
     511                    lines[0] = f"# {renamed['title']}"
     512                    program.write_text("\n".join(lines) + ("\n" if content.endswith("\n") else ""), encoding="utf-8")
     513            except OSError:
     514                pass
     515        _write_shell_state({"focus_job_id": job_id})
     516        print(f"renamed {old['title']} -> {renamed['title']}")
     517    finally:
     518        db.close()
     519
     520
     521def cmd_delete(args: argparse.Namespace) -> None:
     522    db, config = _db()
     523    try:
     524        job_id = _resolve_job_id(db, args.job_id)
     525        if not job_id:
     526            ref = _job_ref_text(args.job_id)
     527            print(f"No job matched: {ref}" if ref else "usage: delete JOB_TITLE")
     528            return
     529        result = db.delete_job(job_id)
     530        job = result["job"]
     531    finally:
     532        db.close()
     533
     534    removed_files = 0
     535    if not args.keep_files:
     536        job_dir = config.runtime.jobs_dir / job_id
     537        for path_text in result.get("artifact_paths") or []:
     538            path = Path(path_text)
     539            try:
     540                if path.exists() and job_dir in path.parents:
     541                    path.unlink()
     542                    removed_files += 1
     543            except OSError:
     544                pass
     545        try:
     546            if job_dir.exists():
     547                shutil.rmtree(job_dir)
     548        except OSError:
     549            pass
     550    state = _read_shell_state()
     551    if state.get("focus_job_id") == job_id:
     552        _write_shell_state({"focus_job_id": ""})
     553    counts = result.get("counts") or {}
     554    file_text = "kept files" if args.keep_files else f"removed files={removed_files}"
     555    print(
     556        f"deleted {job['title']} | steps={counts.get('steps', 0)} "
     557        f"artifacts={counts.get('artifacts', 0)} runs={counts.get('runs', 0)} | {file_text}"
     558    )
     559
     560
     561def cmd_chat(args: argparse.Namespace) -> None:
     562    if not _ensure_model_setup_verified_for_workspace():
     563        return
     564    db, _ = _db()
     565    try:
     566        job_id = _resolve_job_id(db, args.job_id)
     567        if not job_id:
     568            ref = _job_ref_text(args.job_id)
     569            print(f"No job matched: {ref}" if ref else "No jobs found. Create one first.")
     570            return
     571        _write_shell_state({"focus_job_id": job_id})
     572    finally:
     573        db.close()
     574
     575    _enter_chat(job_id, show_history=not args.no_history, history_limit=args.history_limit)
     576
     577
     578def cmd_home(args: argparse.Namespace) -> None:
     579    _install_readline_history()
     580    config = load_config()
     581    if not _model_setup_verified(config) and _workspace_has_model_config(config):
     582        _auto_verify_model_setup(config)
     583    if not _model_setup_verified(load_config()):
     584        _enter_first_run_setup(history_limit=args.history_limit)
     585        return
     586
     587    if _has_saved_jobs():
     588        _start_interactive_daemon_if_possible()
     589    _enter_workspace_chat(history_limit=args.history_limit)
     590
     591
     592def _enter_first_run_setup(*, history_limit: int = 12) -> None:
     593    if _frame_chat_enabled():
     594        _enter_first_run_frame(history_limit=history_limit)
     595        return
     596
     597    print("Nipux setup requires an interactive terminal.")
     598    print("Run `nipux` in a terminal window to choose model, endpoint, and tool access.")
     599
     600
     601def _enter_empty_workspace(*, history_limit: int = 12) -> None:
     602    del history_limit
     603    db, _ = _db()
     604    try:
     605        jobs = db.list_jobs()[:12]
     606    finally:
     607        db.close()
     608    print("NIPUX WORKSPACE")
     609    print(_rule("="))
     610    if jobs:
     611        print("Jobs")
     612        for job in jobs:
     613            marker = "*" if str(job.get("id") or "") == _read_shell_state().get("focus_job_id") else " "
     614            title = _one_line(job.get("title") or job.get("id") or "untitled", 44)
     615            print(f"{marker} {title:44} {job.get('status') or 'unknown'}")
     616        print()
     617        print("Run in a terminal to use the full-screen workspace, or use: nipux chat JOB_TITLE")
     618    else:
     619        print("No jobs are saved in this profile.")
     620        print("Create a job with: nipux create \"objective\"")
     621    print("Settings: nipux init --force | Check setup: nipux doctor --check-model")
     622
     623
     624def _has_saved_jobs() -> bool:
     625    db, _ = _db()
     626    try:
     627        return bool(db.list_jobs()[:1])
     628    finally:
     629        db.close()
     630
     631
     632def _enter_workspace_chat(*, history_limit: int = 12) -> None:
     633    if _frame_chat_enabled():
     634        _enter_chat_frame(WORKSPACE_CHAT_ID, history_limit=history_limit)
     635        return
     636    _enter_empty_workspace(history_limit=history_limit)
     637
     638
     639def _print_first_run_menu() -> None:
     640    config = load_config()
     641    print("Start")
     642    print(f"  model   {config.model.model}")
     643    print("  status  ready when work starts")
     644    print(f"  home    {_short_path(config.runtime.home)}")
     645    print()
     646    print("Commands")
     647    print("  1  doctor    verify provider/model")
     648    print("  2  init      write config/env template")
     649    print("  3  exit      leave")
     650    print()
     651    print("Finish setup before chat or job creation is available.")
     652
     653
     654def _handle_first_run_menu_line(line: str, *, history_limit: int = 12) -> bool:
     655    line = line.strip()
     656    if not line:
     657        _print_first_run_menu()
     658        return True
     659    if line.startswith("/"):
     660        line = line[1:].strip()
     661    lowered = line.lower()
     662    if lowered in {"exit", "quit", ":q", "3", "5"}:
     663        return False
     664    if lowered in {"help", "?", "commands"}:
     665        _print_first_run_menu()
     666        return True
     667    if lowered in {"new"} or lowered.startswith("new "):
     668        print("Finish setup first. Then describe worker jobs in the chat workspace.")
     669        return True
     670    if lowered in {"jobs", "ls"}:
     671        cmd_jobs(argparse.Namespace())
     672        return True
     673    if lowered in {"1", "doctor"}:
     674        try:
     675            cmd_doctor(argparse.Namespace(check_model=False))
     676        except SystemExit:
     677            pass
     678        return True
     679    if lowered in {"2", "init"}:
     680        cmd_init(argparse.Namespace(path=None, force=False))
     681        return True
     682    first = _first_token(line)
     683    if first in {"create", "new"}:
     684        print("Finish setup first. Then describe worker jobs in the chat workspace.")
     685        return True
     686    if first in SHELL_COMMAND_NAMES:
     687        _run_shell_line(line)
     688        return True
     689    objective = _extract_job_objective_from_message(line)
     690    if objective:
     691        print("Finish setup first. Then describe worker jobs in the chat workspace.")
     692        return True
     693    print(_first_run_chat_reply(line))
     694    return True
     695
     696
     697def _prompt_first_run_value(label: str) -> str:
     698    try:
     699        return input(f"{label} > ").strip()
     700    except (EOFError, KeyboardInterrupt):
     701        print()
     702        return ""
     703
     704
     705def _first_run_create_and_open(objective: str, *, history_limit: int = 12) -> None:
     706    if not _ensure_model_setup_verified_for_workspace():
     707        return
     708    job_id, title = _create_job(objective=objective, title=None, kind="generic", cadence=None)
     709    _write_shell_state({"focus_job_id": job_id})
     710    print(f"created {title}")
     711    _start_interactive_daemon_if_possible()
     712    print("Opening workspace.")
     713    _enter_workspace_chat(history_limit=history_limit)
     714
     715
     716def _first_token(line: str) -> str:
     717    return _controller_first_token(line)
     718
     719
     720def _enter_first_run_frame(*, history_limit: int = 12) -> None:
     721    next_job_id = _run_first_run_frame(deps=_first_run_runtime_deps())
     722    if next_job_id == WORKSPACE_CHAT_ID:
     723        _enter_workspace_chat(history_limit=history_limit)
     724    elif next_job_id:
     725        _start_interactive_daemon_if_possible()
     726        _write_shell_state({"focus_job_id": next_job_id})
     727        _enter_workspace_chat(history_limit=history_limit)
     728
     729
     730def _first_run_runtime_deps() -> FirstRunRuntimeDeps:
     731    return FirstRunRuntimeDeps(
     732        render_frame=lambda buffer, notices, selected, view, editing_field, previous: _render_first_run_frame(
     733            buffer,
     734            notices,
     735            selected=selected,
     736            view=view,
     737            editing_field=editing_field,
     738            previous_frame=previous,
     739        ),
     740        actions=_first_run_actions,
     741        handle_action=_handle_first_run_action,
     742        handle_line=_handle_first_run_frame_line,
     743        click_action=lambda x, y, view: _first_run_click_action(x, y, view=view),
     744    )
     745
     746
     747def _first_run_actions(view: str) -> list[tuple[str, str, str]]:
     748    return _first_run_tui_actions(view)
     749
     750
     751def _clamp_first_run_selection(selected: int, view: str) -> int:
     752    return _clamp_first_run_runtime_selection(selected, _first_run_actions(view))
     753
     754
     755def _handle_first_run_action(action: str) -> tuple[str, str | list[str] | None]:
     756    return _controller_handle_first_run_action(action, deps=_first_run_frame_deps())
     757
     758
     759def _first_run_click_action(x: int, y: int, *, view: str) -> int | str | None:
     760    width, height = shutil.get_terminal_size((100, 30))
     761    width = max(92, width)
     762    actions = _first_run_actions(view)
     763    if not actions or y < 10 or y > max(10, height - 4):
     764        return None
     765    gap = 2
     766    card_width = max(18, min(34, (width - (len(actions) - 1) * gap - 4) // len(actions)))
     767    total_width = len(actions) * card_width + (len(actions) - 1) * gap
     768    start_x = max(1, (width - total_width) // 2 + 1)
     769    relative = x - start_x
     770    if relative < 0 or relative >= total_width:
     771        return None
     772    span = card_width + gap
     773    index = relative // span
     774    within_card = relative % span < card_width
     775    if not within_card:
     776        return None
     777    return index if 0 <= index < len(actions) else None
     778
     779
     780def _chat_page_click(x: int, y: int, *, right_view: str) -> str | None:
     781    del right_view
     782    width, _height = shutil.get_terminal_size((100, 30))
     783    width = max(92, width)
     784    right_width = min(max(52, int(width * 0.36)), 72)
     785    left_width = max(48, width - right_width - 3)
     786    if left_width < 48:
     787        left_width = 48
     788        right_width = max(34, width - left_width - 3)
     789    right_start = left_width + 4
     790    if x < right_start or y > 8:
     791        return None
     792    relative = max(0, x - right_start)
     793    third = max(1, right_width // 3)
     794    return ["updates", "status", "work"][min(2, relative // third)]
     795
     796
     797def _handle_first_run_frame_line(line: str) -> tuple[str, str | list[str] | None]:
     798    return _controller_handle_first_run_frame_line(line, deps=_first_run_frame_deps())
     799
     800
     801def _first_run_chat_reply(message: str) -> str:
     802    return _controller_first_run_chat_reply(message)
     803
     804
     805def _create_first_run_job(objective: str) -> str | list[str]:
     806    return _controller_create_first_run_job(objective, deps=_first_run_frame_deps())
     807
     808
     809def _capture_first_run_command(line: str) -> list[str]:
     810    return _controller_capture_first_run_command(line, _run_shell_line)
     811
     812
     813def _first_run_frame_deps() -> FirstRunFrameDeps:
     814    return FirstRunFrameDeps(
     815        capture_command=_capture_first_run_command,
     816        capture_setting_command=_capture_setting_command,
     817        create_job=_create_job,
     818        current_default_job_id=_current_default_job_id,
     819        extract_objective=_extract_job_objective_from_message,
     820        model_setup_verified=lambda: _model_setup_verified(load_config()),
     821        verify_model_setup=_verify_model_setup_from_first_run,
     822        shell_command_names=SHELL_COMMAND_NAMES,
     823    )
     824
     825
     826def _current_default_job_id() -> str | None:
     827    db, _ = _db()
     828    try:
     829        return _default_job_id(db)
     830    finally:
     831        db.close()
     832
     833
     834def _render_first_run_frame(
     835    input_buffer: str,
     836    notices: list[str],
     837    *,
     838    selected: int = 0,
     839    view: str = "start",
     840    editing_field: str | None = None,
     841    previous_frame: str = "",
     842) -> str:
     843    width, height = shutil.get_terminal_size((100, 30))
     844    frame = _build_first_run_frame(
     845        input_buffer,
     846        notices,
     847        width=width,
     848        height=height,
     849        selected=selected,
     850        view=view,
     851        editing_field=editing_field,
     852    )
     853    return _emit_frame_if_changed(frame, previous_frame)
     854
     855
     856def _build_first_run_frame(
     857    input_buffer: str,
     858    notices: list[str],
     859    *,
     860    width: int,
     861    height: int,
     862    selected: int = 0,
     863    view: str = "start",
     864    editing_field: str | None = None,
     865) -> str:
     866    width = max(92, width)
     867    height = max(22, height)
     868    config = load_config()
     869    daemon = daemon_lock_status(config.runtime.home / "agentd.lock")
     870    jobs: list[dict[str, Any]] = []
     871    db, _ = _db()
     872    try:
     873        jobs = db.list_jobs()
     874    finally:
     875        db.close()
     876    right_width = _first_run_columns(width)[1]
     877    return _build_first_run_tui_frame(
     878        input_buffer,
     879        notices,
     880        width=width,
     881        height=height,
     882        selected=selected,
     883        view=view,
     884        editing_field=editing_field,
     885        config=config,
     886        jobs=jobs,
     887        daemon_text=_daemon_state_line(daemon),
     888        home=_short_path(config.runtime.home, max_width=max(20, right_width - 8)),
     889        config_path=_short_path(config.runtime.home / "config.yaml", max_width=max(20, right_width - 8)),
     890    )
     891
     892
     893def _enter_chat(job_id: str, *, show_history: bool, history_limit: int = 12) -> None:
     894    if not _ensure_model_setup_verified_for_workspace():
     895        return
     896    _install_readline_history()
     897    startup_note = _start_interactive_daemon_if_possible()
     898    if _frame_chat_enabled():
     899        _enter_chat_frame(job_id, history_limit=history_limit)
     900        return
     901    db, _ = _db()
     902    try:
     903        job = db.get_job(job_id)
     904        _write_shell_state({"focus_job_id": job_id})
     905    finally:
     906        db.close()
     907
     908    if _fancy_ui():
     909        print("\033[2J\033[H", end="")
     910    print(NIPUX_BANNER)
     911    print(_rule("="))
     912    print(_shell_summary())
     913    print(_rule("="))
     914    if show_history:
     915        _print_startup_history(job_id, limit=history_limit, chars=180)
     916        print()
     917    if startup_note:
     918        print(_one_line(startup_note, 180))
     919    _print_chat_composer(job)
     920    live_stop, live_thread = _start_chat_live_feed(job_id)
     921    try:
     922        while True:
     923            db, _ = _db()
     924            try:
     925                refreshed = _default_job_id(db)
     926                if refreshed:
     927                    job_id = refreshed
     928                    job = db.get_job(job_id)
     929            finally:
     930                db.close()
     931            try:
     932                line = input(_chat_prompt(job))
     933            except EOFError:
     934                print()
     935                return
     936            except KeyboardInterrupt:
     937                print()
     938                continue
     939            if not _chat_handle_line(job_id, line):
     940                return
     941    finally:
     942        if live_stop is not None:
     943            live_stop.set()
     944        if live_thread is not None:
     945            live_thread.join(timeout=1.0)
     946
     947
     948def _frame_chat_enabled() -> bool:
     949    return (
     950        sys.stdin.isatty()
     951        and sys.stdout.isatty()
     952        and not os.environ.get("NIPUX_APPEND_LIVE")
     953        and not os.environ.get("NIPUX_NO_FRAME")
     954    )
     955
     956
     957def _enter_chat_frame(job_id: str, *, history_limit: int = 12) -> None:
     958    _run_chat_frame(job_id, history_limit=history_limit, deps=_chat_frame_deps())
     959
     960
     961def _chat_frame_deps() -> ChatFrameDeps:
     962    return ChatFrameDeps(
     963        load_snapshot=lambda job_id, history_limit: _load_frame_snapshot(job_id, history_limit=history_limit),
     964        render_frame=lambda snapshot, buffer, notices, right_view, selected, editing_field, modal_view, previous: _render_chat_frame(
     965            snapshot,
     966            buffer,
     967            notices,
     968            right_view=right_view,
     969            selected_control=selected,
     970            editing_field=editing_field,
     971            modal_view=modal_view,
     972            previous_frame=previous,
     973        ),
     974        handle_chat_message=lambda job_id, line: _handle_chat_message(job_id, line, quiet=True),
     975        capture_chat_command=_capture_chat_command,
     976        write_shell_state=_write_shell_state,
     977        is_plain_chat_line=_is_plain_chat_line,
     978        page_click=lambda x, y, right_view: _chat_page_click(x, y, right_view=right_view),
     979    )
     980
     981
     982def _capture_chat_command(job_id: str, line: str) -> tuple[bool, str]:
     983    stream = StringIO()
     984    with redirect_stdout(stream):
     985        if job_id == WORKSPACE_CHAT_ID:
     986            raw = line.strip()
     987            command = raw[1:].strip() if raw.startswith("/") else (chat_control_command(raw).lstrip("/") or raw)
     988            keep_running = _run_workspace_command_line(command) if command else True
     989        else:
     990            keep_running = _chat_handle_line(job_id, line)
     991    return keep_running, stream.getvalue()
     992
     993
     994def _run_workspace_command_line(command: str) -> bool:
     995    try:
     996        tokens = shlex.split(command)
     997    except ValueError as exc:
     998        print(f"parse error: {exc}")
     999        return True
    1000    if tokens and tokens[0] == "help":
    1001        _print_workspace_chat_help()
    1002        return True
    1003    if tokens and _run_workspace_setting_command(tokens[0], tokens[1:]):
    1004        return True
    1005    if tokens and tokens[0] == "new":
    1006        objective = command[len("new") :].strip()
    1007        if not objective:
    1008            print("usage: /new OBJECTIVE")
    1009            return True
    1010        operator_line = f"/new {objective}"
    1011        _append_workspace_chat_event("operator_message", "command", operator_line, {"source": "workspace"})
    1012        message = _create_workspace_job_from_chat(operator_line, objective)
    1013        _append_workspace_chat_event("agent_message", "chat", message, {"source": "workspace"})
    1014        print(message)
    1015        return True
    1016    if tokens and tokens[0] == "run":
    1017        if len(tokens) > 1 and not tokens[1].startswith("-") and _workspace_command_should_create_worker(command, " ".join(tokens[1:])):
    1018            objective = _extract_job_objective_from_message(command)
    1019            operator_line = f"/{tokens[0]} {objective}"
    1020            _append_workspace_chat_event("operator_message", "command", operator_line, {"source": "workspace"})
    1021            message = _create_workspace_job_from_chat(operator_line, objective)
    1022            _append_workspace_chat_event("agent_message", "chat", message, {"source": "workspace"})
    1023            print(message)
    1024            return True
    1025        return _run_workspace_run_command(tokens)
    1026    if tokens and tokens[0] in {"start", "launch"} and len(tokens) > 1 and not tokens[1].startswith("-"):
    1027        target_text = " ".join(tokens[1:])
    1028        if _workspace_command_should_create_worker(command, target_text):
    1029            objective = _extract_job_objective_from_message(command)
    1030            operator_line = f"/{tokens[0]} {objective}"
    1031            _append_workspace_chat_event("operator_message", "command", operator_line, {"source": "workspace"})
    1032            message = _create_workspace_job_from_chat(operator_line, objective)
    1033            _append_workspace_chat_event("agent_message", "chat", message, {"source": "workspace"})
    1034            print(message)
    1035            return True
    1036        return _run_workspace_run_command(["run", target_text])
    1037    return _run_shell_line(command)
    1038
    1039
    1040def _run_workspace_setting_command(command: str, rest: list[str]) -> bool:
    1041    if command == "settings":
    1042        command = "config"
    1043    if command in {"config", "key", "api-key"} or command in CHAT_SETTING_COMMANDS:
    1044        return _handle_chat_setting_command(command, rest)
    1045    return False
    1046
    1047
    1048def _workspace_command_should_create_worker(command: str, target_text: str) -> bool:
    1049    objective = _extract_job_objective_from_message(command)
    1050    if not objective:
    1051        return False
    1052    db, _config = _db()
    1053    try:
    1054        return _find_job(db, target_text) is None
    1055    finally:
    1056        db.close()
    1057
    1058
    1059def _run_workspace_run_command(tokens: list[str]) -> bool:
    1060    try:
    1061        parsed = build_parser().parse_args(tokens)
    1062    except SystemExit as exc:
    1063        if exc.code:
    1064            print(f"command exited with status {exc.code}")
    1065        return True
    1066    parsed.no_follow = True
    1067    parsed.quiet = True
    1068    parsed.func(parsed)
    1069    return True
    1070
    1071
    1072def _print_workspace_chat_help() -> None:
    1073    print("Create: type a goal, or /new OBJECTIVE.")
    1074    print("Run: /run, /pause, /resume. Inspect: /jobs, /outcomes, /artifacts, /activity.")
    1075    print("Config: /settings, /model, /base-url, /api-key. Navigate: ←→ pages, ↑↓ jobs.")
    1076
    1077
    1078def _start_worker_from_chat_context(
    1079    *,
    1080    poll_seconds: float = 0.0,
    1081    fake: bool = False,
    1082    quiet: bool = True,
    1083    log_file: str | None = None,
    1084) -> bool:
    1085    """Start the daemon from the TUI without dumping preflight internals into chat."""
    1086
    1087    def report(message: str) -> None:
    1088        if not quiet:
    1089            print(message)
    1090
    1091    stream = StringIO()
    1092    try:
    1093        with redirect_stdout(stream):
    1094            _start_daemon_if_needed(poll_seconds=poll_seconds, fake=fake, quiet=True, log_file=log_file)
    1095    except SystemExit as exc:
    1096        detail = _one_line(str(exc) or "daemon start failed", 120)
    1097        report(f"worker not started: {detail}")
    1098        return False
    1099    except Exception as exc:
    1100        detail = _one_line(f"{type(exc).__name__}: {exc}", 120)
    1101        report(f"worker not started: {detail}")
    1102        return False
    1103    output = stream.getvalue()
    1104    lowered = output.lower()
    1105    if (
    1106        "model is not ready" in lowered
    1107        or "model setup is not verified" in lowered
    1108        or "model_generation:" in lowered
    1109        or "model_endpoint:" in lowered
    1110        or "model_auth:" in lowered
    1111        or "model_config:" in lowered
    1112    ):
    1113        report("worker not started: model provider is not ready. Use /settings, then /doctor.")
    1114        return False
    1115    return True
    1116
    1117
    1118def _start_worker_from_chat_namespace(args: argparse.Namespace) -> bool:
    1119    return _start_worker_from_chat_context(
    1120        poll_seconds=float(getattr(args, "poll_seconds", 0.0) or 0.0),
    1121        fake=bool(getattr(args, "fake", False)),
    1122        quiet=bool(getattr(args, "quiet", True)),
    1123        log_file=getattr(args, "log_file", None),
    1124    )
    1125
    1126
    1127def _is_plain_chat_line(line: str) -> bool:
    1128    stripped = line.strip()
    1129    if not stripped or stripped.startswith("/"):
    1130        return False
    1131    lowered = stripped.lower()
    1132    if lowered in {"help", "jobs", "ls", "clear", "exit", "quit"}:
    1133        return False
    1134    if chat_control_command(stripped):
    1135        return False
    1136    try:
    1137        first = shlex.split(stripped)[0].lower()
    1138    except (IndexError, ValueError):
    1139        first = lowered.split(maxsplit=1)[0]
    1140    return first not in {"chat", "focus", "switch", "jobs", "ls", "help", "clear", "exit", "quit"}
    1141
    1142
    1143def _load_frame_snapshot(job_id: str, *, history_limit: int = 12) -> dict[str, Any]:
    1144    db, config = _db()
    1145    try:
    1146        return load_frame_snapshot(
    1147            db,
    1148            config,
    1149            job_id,
    1150            default_job_id=_default_job_id(db),
    1151            history_limit=history_limit,
    1152            workspace_events=_workspace_chat_events() if job_id == WORKSPACE_CHAT_ID else None,
    1153        )
    1154    finally:
    1155        db.close()
    1156
    1157
    1158def _render_chat_frame(
    1159    snapshot: dict[str, Any],
    1160    input_buffer: str,
    1161    notices: list[str],
    1162    *,
    1163    right_view: str = "updates",
    1164    selected_control: int = 0,
    1165    editing_field: str | None = None,
    1166    modal_view: str | None = None,
    1167    previous_frame: str = "",
    1168) -> str:
    1169    width, height = shutil.get_terminal_size((100, 30))
    1170    frame = _build_chat_frame(
    1171        snapshot,
    1172        input_buffer,
    1173        notices,
    1174        width=width,
    1175        height=height,
    1176        right_view=right_view,
    1177        selected_control=selected_control,
    1178        editing_field=editing_field,
    1179        modal_view=modal_view,
    1180    )
    1181    return _emit_frame_if_changed(frame, previous_frame)
    1182
    1183
    1184def _build_chat_frame(
    1185    snapshot: dict[str, Any],
    1186    input_buffer: str,
    1187    notices: list[str],
    1188    *,
    1189    width: int,
    1190    height: int,
    1191    right_view: str = "updates",
    1192    selected_control: int = 0,
    1193    editing_field: str | None = None,
    1194    modal_view: str | None = None,
    1195) -> str:
    1196    return _build_chat_tui_frame(
    1197        snapshot,
    1198        input_buffer,
    1199        notices,
    1200        width=width,
    1201        height=height,
    1202        right_view=right_view,
    1203        selected_control=selected_control,
    1204        editing_field=editing_field,
    1205        modal_view=modal_view,
    1206    )
    1207
    1208
    1209def _resolve_job_id(db: AgentDB, requested: Any = None) -> str | None:
    1210    requested = _job_ref_text(requested)
    1211    if requested:
    1212        job = _find_job(db, requested)
    1213        return str(job["id"]) if job else None
    1214    return _default_job_id(db)
    1215
    1216
    1217def _activate_job_if_planning(db: AgentDB, job_id: str) -> bool:
    1218    job = db.get_job(job_id)
    1219    if job.get("status") != "planning":
    1220        return False
    1221    db.update_job_status(job_id, "queued", metadata_patch={"planning_status": "accepted"})
    1222    db.append_agent_update(job_id, "Plan accepted. I will start working from the planned tasks.", category="plan")
    1223    return True
    1224
    1225
    1226def _ensure_job_runnable(db: AgentDB, job_id: str) -> None:
    1227    if _activate_job_if_planning(db, job_id):
    1228        return
    1229    job = db.get_job(job_id)
    1230    status = str(job.get("status") or "")
    1231    if status in {"completed", "paused", "cancelled", "failed"} or job_provider_blocked(job):
    1232        patch = operator_resume_metadata()
    1233        patch["last_note"] = f"reopened from {status} by operator run command"
    1234        db.update_job_status(
    1235            job_id,
    1236            "queued",
    1237            metadata_patch=patch,
    1238        )
    1239        db.append_agent_update(
    1240            job_id,
    1241            f"Reopened from {status}; continuing as a long-running job.",
    1242            category="progress",
    1243            metadata={"previous_status": status},
    1244        )
    1245
    1246
    1247def cmd_steer(args: argparse.Namespace) -> None:
    1248    message = " ".join(args.message).strip()
    1249    if not message:
    1250        print("No steering message provided.")
    1251        return
    1252    db, _ = _db()
    1253    try:
    1254        job_id = _resolve_job_id(db, args.job_id)
    1255        if not job_id:
    1256            ref = _job_ref_text(args.job_id)
    1257            print(f"No job matched: {ref}" if ref else "No jobs found. Create one first, then send steering.")
    1258            return
    1259        entry = db.append_operator_message(job_id, message, source="operator")
    1260        job = db.get_job(job_id)
    1261        print(f"waiting for {job['title']}: {entry['message']}")
    1262        print("The next worker step will include this in model-visible context.")
    1263    finally:
    1264        db.close()
    1265
    1266
    1267def cmd_pause(args: argparse.Namespace) -> None:
    1268    db, _ = _db()
    1269    try:
    1270        job_id, note, ref = _resolve_control_job_and_note(db, args)
    1271        if not job_id:
    1272            print(f"No job matched: {ref}" if ref else "No jobs found.")
    1273            return
    1274        patch = {"last_note": note} if note else None
    1275        db.update_job_status(job_id, "paused", metadata_patch=patch)
    1276        job = db.get_job(job_id)
    1277        print(f"paused {job['title']}" + (f": {note}" if note else ""))
    1278    finally:
    1279        db.close()
    1280
    1281
    1282def cmd_resume(args: argparse.Namespace) -> None:
    1283    db, _ = _db()
    1284    try:
    1285        job_id = _resolve_job_id(db, args.job_id)
    1286        if not job_id:
    1287            ref = _job_ref_text(args.job_id)
    1288            print(f"No job matched: {ref}" if ref else "No jobs found.")
    1289            return
    1290        db.update_job_status(job_id, "queued", metadata_patch=operator_resume_metadata())
    1291        job = db.get_job(job_id)
    1292        print(f"resumed {job['title']}")
    1293    finally:
    1294        db.close()
    1295
    1296
    1297def cmd_cancel(args: argparse.Namespace) -> None:
    1298    db, _ = _db()
    1299    try:
    1300        job_id, note, ref = _resolve_control_job_and_note(db, args)
    1301        if not job_id:
    1302            print(f"No job matched: {ref}" if ref else "No jobs found.")
    1303            return
    1304        patch = {"last_note": note} if note else None
    1305        db.update_job_status(job_id, "cancelled", metadata_patch=patch)
    1306        job = db.get_job(job_id)
    1307        print(f"cancelled {job['title']}" + (f": {note}" if note else ""))
    1308    finally:
    1309        db.close()
    1310
    1311
    1312def cmd_status(args: argparse.Namespace) -> None:
    1313    db, config = _db()
    1314    try:
    1315        job_id = _resolve_job_id(db, args.job_id)
    1316        if _job_ref_text(args.job_id) and not job_id:
    1317            print(f"No job matched: {_job_ref_text(args.job_id)}")
    1318            return
    1319        state = collect_dashboard_state(db, config, job_id=job_id, limit=args.limit)
    1320        if args.json:
    1321            print(json.dumps(state, ensure_ascii=False, indent=2, default=_json_default))
    1322            return
    1323        if args.full:
    1324            print(render_dashboard(state, width=_terminal_width(), chars=args.chars), end="")
    1325        else:
    1326            print(render_overview(state, width=_terminal_width()), end="")
    1327    finally:
    1328        db.close()
    1329
    1330
    1331def cmd_health(args: argparse.Namespace) -> None:
    1332    db, config = _db()
    1333    try:
    1334        config.ensure_dirs()
    1335        lock = daemon_lock_status(config.runtime.home / "agentd.lock")
    1336        metadata = lock.get("metadata") if isinstance(lock.get("metadata"), dict) else {}
    1337        events = read_daemon_events(config, limit=args.limit)
    1338        job_id = _default_job_id(db)
    1339        print("Nipux Health")
    1340        print(_rule("="))
    1341        print(f"daemon: {_daemon_state_line(lock)}")
    1342        if metadata.get("last_heartbeat"):
    1343            print(f"heartbeat: {metadata['last_heartbeat']}")
    1344        if metadata.get("last_state"):
    1345            print(f"state: {metadata['last_state']}")
    1346        if metadata.get("last_status") or metadata.get("last_tool"):
    1347            print(f"last step: {metadata.get('last_status') or '?'} {metadata.get('last_tool') or '-'}")
    1348        if metadata.get("consecutive_failures"):
    1349            print(f"consecutive failures: {metadata['consecutive_failures']}")
    1350        if metadata.get("last_error"):
    1351            print(
    1352                f"last error: {metadata.get('last_error_type') or 'error'}: {_one_line(metadata['last_error'], args.chars)}"
    1353            )
    1354        print(f"model: {config.model.model}")
    1355        print(f"state db: {config.runtime.state_db_path}")
    1356        print(f"daemon log: {config.runtime.logs_dir / 'daemon.log'}")
    1357        print(f"event log: {config.runtime.logs_dir / 'daemon-events.jsonl'}")
    1358        print(f"autostart: {'installed' if _launch_agent_path().exists() else 'not installed'}")
    1359        if job_id:
    1360            job = db.get_job(job_id)
    1361            steps = db.list_steps(job_id=job_id)
    1362            artifacts = db.list_artifacts(job_id, limit=1)
    1363            print()
    1364            print(f"focus: {job['title']}")
    1365            state = _job_display_state(job, bool(lock["running"]))
    1366            print(
    1367                f"state: {state} | worker: {_worker_label(job, bool(lock['running']))} | "
    1368                f"steps: {_step_count(steps)} | latest artifacts: {len(artifacts)}"
    1369            )
    1370            if steps:
    1371                print(f"latest: {_step_line(steps[-1], chars=args.chars)}")
    1372        else:
    1373            print()
    1374            print("focus: no jobs")
    1375        if events:
    1376            print()
    1377            print("recent daemon events:")
    1378            job_titles = {job["id"]: job["title"] for job in db.list_jobs()}
    1379            for event in events[-args.limit :]:
    1380                print(f"  {_daemon_event_line(event, chars=args.chars, job_titles=job_titles)}")
    1381        else:
    1382            print()
    1383            print("recent daemon events: none")
    1384    finally:
    1385        db.close()
    1386
    1387
    1388def cmd_history(args: argparse.Namespace) -> None:
    1389    db, _ = _db()
    1390    try:
    1391        job_id = _resolve_job_id(db, args.job_id)
    1392        if not job_id:
    1393            ref = _job_ref_text(args.job_id)
    1394            print(f"No job matched: {ref}" if ref else "No jobs found.")
    1395            return
    1396        job = db.get_job(job_id)
    1397        events = db.list_timeline_events(job_id, limit=args.limit)
    1398        if args.json:
    1399            print(
    1400                json.dumps(
    1401                    [_public_event(event) for event in events], ensure_ascii=False, indent=2, default=_json_default
    1402                )
    1403            )
    1404            return
    1405        print(f"history {job['title']}")
    1406        print(_rule("="))
    1407        if not events:
    1408            print("No visible history yet.")
    1409            return
    1410        for event in events:
    1411            if args.full:
    1412                print(_event_line(event, chars=max(args.chars, 1200), full=True))
    1413            else:
    1414                _print_event_card(event, chars=args.chars)
    1415    finally:
    1416        db.close()
    1417
    1418
    1419def cmd_events(args: argparse.Namespace) -> None:
    1420    db, _ = _db()
    1421    seen: set[str] = set()
    1422    try:
    1423        job_id = _resolve_job_id(db, args.job_id)
    1424        if not job_id:
    1425            ref = _job_ref_text(args.job_id)
    1426            print(f"No job matched: {ref}" if ref else "No jobs found.")
    1427            return
    1428        job = db.get_job(job_id)
    1429        if not args.json:
    1430            print(f"events {job['title']}")
    1431            print(_rule("="))
    1432
    1433        def emit() -> None:
    1434            events = db.list_timeline_events(job_id, limit=args.limit)
    1435            printed = False
    1436            for event in events:
    1437                event_id = str(event.get("id") or "")
    1438                if event_id in seen:
    1439                    continue
    1440                seen.add(event_id)
    1441                if args.json:
    1442                    print(json.dumps(_public_event(event), ensure_ascii=False, default=_json_default), flush=True)
    1443                else:
    1444                    if args.full:
    1445                        print(_event_line(event, chars=args.chars, full=True), flush=True)
    1446                    else:
    1447                        _print_event_card(event, chars=args.chars)
    1448                printed = True
    1449            if printed and not args.json:
    1450                print(_rule("-"), flush=True)
    1451
    1452        emit()
    1453        while args.follow:
    1454            time.sleep(args.interval)
    1455            emit()
    1456    except KeyboardInterrupt:
    1457        print("\nevents stopped")
    1458    finally:
    1459        db.close()
    1460
    1461
    1462def cmd_dashboard(args: argparse.Namespace) -> None:
    1463    db, config = _db()
    1464    try:
    1465        while True:
    1466            job_id = _resolve_job_id(db, args.job_id)
    1467            if _job_ref_text(args.job_id) and not job_id:
    1468                print(f"No job matched: {_job_ref_text(args.job_id)}")
    1469                return
    1470            state = collect_dashboard_state(db, config, job_id=job_id, limit=args.limit)
    1471            if args.clear:
    1472                print("\033[2J\033[H", end="")
    1473            print(render_dashboard(state, width=_terminal_width(), chars=args.chars), end="", flush=True)
    1474            if not args.follow:
    1475                return
    1476            time.sleep(args.interval)
    1477    except KeyboardInterrupt:
    1478        print("\ndashboard stopped")
    1479    finally:
    1480        db.close()
    1481
    1482
    1483def cmd_artifacts(args: argparse.Namespace) -> None:
    1484    db, _ = _db()
    1485    try:
    1486        job_id = _resolve_job_id(db, args.job_id)
    1487        if not job_id:
    1488            ref = _job_ref_text(args.job_id)
    1489            print(f"No job matched: {ref}" if ref else "No jobs found.")
    1490            return
    1491        job = db.get_job(job_id)
    1492        artifacts = db.list_artifacts(job_id, limit=args.limit)
    1493        if not artifacts:
    1494            print(f"No saved outputs recorded for {job['title']}.")
    1495            return
    1496        print(f"saved outputs {job['title']} (newest first)")
    1497        print(_rule("-"))
    1498        print("Open one with: artifact NUMBER, artifact latest, or artifact TITLE")
    1499        for index, artifact in enumerate(artifacts, start=1):
    1500            title = artifact.get("title") or artifact["id"]
    1501            print(f"{index:>2}. {_one_line(title, 72)}")
    1502            meta = f"{artifact['created_at']} | {artifact['type']} | id {artifact['id']}"
    1503            print(f"    {meta}")
    1504            if artifact.get("summary"):
    1505                print(f"    {_one_line(_generic_display_text(artifact['summary']), args.chars)}")
    1506            print(f"    view: artifact {index}")
    1507            if args.paths:
    1508                print(f"    path: {artifact['path']}")
    1509    finally:
    1510        db.close()
    1511
    1512
    1513def cmd_artifact(args: argparse.Namespace) -> None:
    1514    db, config = _db()
    1515    try:
    1516        store = ArtifactStore(config.runtime.home, db=db)
    1517        ref = _job_ref_text(args.artifact_id_or_path)
    1518        resolved = _resolve_artifact_ref(db, config, ref, job_id=_resolve_job_id(db, getattr(args, "job_id", None)))
    1519        if not resolved:
    1520            print(f"No artifact matched: {ref}")
    1521            return
    1522        content = store.read_text(resolved["id"] if resolved.get("id") else resolved["path"])
    1523        if resolved.get("title"):
    1524            print(f"artifact: {resolved['title']}")
    1525            if resolved.get("summary"):
    1526                print(f"summary: {resolved['summary']}")
    1527            print(_rule("-"))
    1528        if args.chars and len(content) > args.chars:
    1529            content = content[: args.chars] + f"\n... truncated {len(content) - args.chars} chars\n"
    1530        print(content, end="" if content.endswith("\n") else "\n")
    1531    finally:
    1532        db.close()
    1533
    1534
    1535def cmd_lessons(args: argparse.Namespace) -> None:
    1536    db, _ = _db()
    1537    try:
    1538        job_id = _resolve_job_id(db, args.job_id)
    1539        if not job_id:
    1540            ref = _job_ref_text(args.job_id)
    1541            print(f"No job matched: {ref}" if ref else "No jobs found.")
    1542            return
    1543        job = db.get_job(job_id)
    1544        _print_lessons(job, limit=args.limit, chars=args.chars)
    1545    finally:
    1546        db.close()
    1547
    1548
    1549def cmd_learn(args: argparse.Namespace) -> None:
    1550    lesson = " ".join(args.lesson).strip()
    1551    if not lesson:
    1552        print("usage: learn [--job JOB_TITLE] [--category CATEGORY] LESSON")
    1553        return
    1554    db, _ = _db()
    1555    try:
    1556        job_id = _resolve_job_id(db, args.job_id)
    1557        if not job_id:
    1558            ref = _job_ref_text(args.job_id)
    1559            print(f"No job matched: {ref}" if ref else "No jobs found.")
    1560            return
    1561        entry = db.append_lesson(
    1562            job_id, lesson, category=args.category or "operator_preference", metadata={"source": "operator"}
    1563        )
    1564        job = db.get_job(job_id)
    1565        print(f"learned for {job['title']}: {_one_line(entry['lesson'], args.chars)}")
    1566    finally:
    1567        db.close()
    1568
    1569
    1570def cmd_findings(args: argparse.Namespace) -> None:
    1571    return cmd_findings_impl(args, _record_command_deps())
    1572
    1573
    1574def cmd_tasks(args: argparse.Namespace) -> None:
    1575    return cmd_tasks_impl(args, _record_command_deps())
    1576
    1577
    1578def cmd_roadmap(args: argparse.Namespace) -> None:
    1579    return cmd_roadmap_impl(args, _record_command_deps())
    1580
    1581
    1582def cmd_experiments(args: argparse.Namespace) -> None:
    1583    return cmd_experiments_impl(args, _record_command_deps())
    1584
    1585
    1586def cmd_sources(args: argparse.Namespace) -> None:
    1587    return cmd_sources_impl(args, _record_command_deps())
    1588
    1589
    1590def cmd_memory(args: argparse.Namespace) -> None:
    1591    return cmd_memory_impl(args, _record_command_deps())
    1592
    1593
    1594def cmd_metrics(args: argparse.Namespace) -> None:
    1595    return cmd_metrics_impl(args, _record_command_deps())
    1596
    1597
    1598def cmd_usage(args: argparse.Namespace) -> None:
    1599    return cmd_usage_impl(args, _record_command_deps())
    1600
    1601
    1602def _remote_model_preflight_failures(config) -> list[str]:
    1603    return _daemon_remote_model_preflight_failures(config, doctor_fn=run_doctor)
    1604
    1605
    1606def _recoverable_remote_model_preflight_failures(config) -> list[str]:
    1607    return _daemon_recoverable_remote_model_preflight_failures(config, doctor_fn=run_doctor)
    1608
    1609
    1610def _provider_preflight_is_recoverable(failures: list[str]) -> bool:
    1611    return _daemon_provider_preflight_is_recoverable(failures)
    1612
    1613
    1614def _ensure_remote_model_ready_for_worker(config, *, fake: bool) -> bool:
    1615    return _daemon_ensure_remote_model_ready(config, fake=fake, doctor_fn=run_doctor)
    1616
    1617
    1618def cmd_start(args: argparse.Namespace) -> None:
    1619    return _cmd_start_impl(
    1620        args,
    1621        ready_fn=lambda config, fake: _ensure_remote_model_ready_for_worker(config, fake=fake),
    1622        stop_fn=lambda config, wait, quiet: _stop_daemon_process(config, wait=wait, quiet=quiet),
    1623    )
    1624
    1625
    1626def _start_daemon_if_needed(
    1627    *, poll_seconds: float, fake: bool = False, quiet: bool = False, log_file: str | None = None
    1628) -> None:
    1629    return _start_daemon_if_needed_impl(
    1630        poll_seconds=poll_seconds,
    1631        fake=fake,
    1632        quiet=quiet,
    1633        log_file=log_file,
    1634        start_fn=cmd_start,
    1635        stop_fn=lambda config, wait, quiet: _stop_daemon_process(config, wait=wait, quiet=quiet),
    1636    )
    1637
    1638
    1639def _start_interactive_daemon_if_possible() -> str:
    1640    """Best-effort daemon start for the full-screen UI without printing over the frame."""
    1641
    1642    stream = StringIO()
    1643    with redirect_stdout(stream):
    1644        try:
    1645            _start_daemon_if_needed(poll_seconds=0.0, quiet=True)
    1646        except SystemExit:
    1647            pass
    1648    return stream.getvalue()
    1649
    1650
    1651def cmd_restart(args: argparse.Namespace) -> None:
    1652    return _cmd_restart_impl(
    1653        args,
    1654        start_fn=cmd_start,
    1655        stop_fn=lambda config, wait, quiet: _stop_daemon_process(config, wait=wait, quiet=quiet),
    1656    )
    1657
    1658
    1659def _stop_daemon_process(config, *, wait: float, quiet: bool) -> bool:
    1660    return _stop_daemon_process_impl(config, wait=wait, quiet=quiet, pid_alive=_pid_is_alive)
    1661
    1662
    1663def cmd_stop(args: argparse.Namespace) -> None:
    1664    requested_job = _job_ref_text(getattr(args, "job_id", None))
    1665    if requested_job:
    1666        db, _ = _db()
    1667        try:
    1668            job_id = _resolve_job_id(db, requested_job)
    1669            if not job_id:
    1670                print(f"No job matched: {requested_job}")
    1671                return
    1672            db.update_job_status(job_id, "paused", metadata_patch={"last_note": "stopped by operator"})
    1673            job = db.get_job(job_id)
    1674            print(f"stopped {job['title']} (paused job)")
    1675            print("Use resume/run to start it again. Plain 'stop' still stops the daemon.")
    1676            return
    1677        finally:
    1678            db.close()
    1679
    1680    config = load_config()
    1681    _stop_daemon_process(config, wait=args.wait, quiet=False)
    1682
    1683
    1684def cmd_browser_dashboard(args: argparse.Namespace) -> None:
    1685    from nipux_cli.browser import _find_agent_browser
    1686
    1687    config = load_config()
    1688    config.ensure_dirs()
    1689    if args.stop:
    1690        result = subprocess.run([*_find_agent_browser(), "dashboard", "stop"], check=False)
    1691        if result.returncode:
    1692            raise SystemExit(result.returncode)
    1693        print("agent-browser dashboard stopped")
    1694        return
    1695
    1696    command = [*_find_agent_browser(), "dashboard", "start", "--port", str(args.port)]
    1697    if args.foreground:
    1698        raise SystemExit(subprocess.call(command))
    1699
    1700    log_path = Path(args.log_file).expanduser() if args.log_file else config.runtime.logs_dir / "browser-dashboard.log"
    1701    log_path.parent.mkdir(parents=True, exist_ok=True)
    1702    with log_path.open("a", encoding="utf-8") as log_file:
    1703        process = subprocess.Popen(
    1704            command,
    1705            cwd=str(Path.cwd()),
    1706            stdout=log_file,
    1707            stderr=subprocess.STDOUT,
    1708            start_new_session=True,
    1709        )
    1710    print(f"agent-browser dashboard started pid={process.pid}")
    1711    print(f"url: http://127.0.0.1:{args.port}")
    1712    print(f"log: {log_path}")
    1713
    1714
    1715def _print_startup_history(job_id: str, *, limit: int, chars: int) -> None:
    1716    db, config = _db()
    1717    try:
    1718        job = db.get_job(job_id)
    1719        jobs = db.list_jobs()
    1720        steps = db.list_steps(job_id=job_id)
    1721        artifacts = db.list_artifacts(job_id, limit=1000)
    1722        memory_entries = db.list_memory(job_id)
    1723        events = db.list_timeline_events(job_id, limit=limit)
    1724        daemon = daemon_lock_status(config.runtime.home / "agentd.lock")
    1725    finally:
    1726        db.close()
    1727    print()
    1728    _print_session_overview(
    1729        job,
    1730        steps=steps,
    1731        artifacts=artifacts,
    1732        memory_entries=memory_entries,
    1733        daemon_running=bool(daemon["running"]),
    1734        model=config.model.model,
    1735        artifacts_dir=config.runtime.jobs_dir / job_id / "artifacts",
    1736        jobs=jobs,
    1737        chars=chars,
    1738    )
    1739    print()
    1740    print(_section_title("Recent activity", f"{job['title']}"))
    1741    if not events:
    1742        print("  No visible history yet.")
    1743        return
    1744    display_events = _important_startup_events(events, limit=min(limit, 8))
    1745    artifact_indexes = {str(artifact["id"]): index for index, artifact in enumerate(artifacts, start=1)}
    1746    for event in display_events:
    1747        _print_event_card(event, chars=min(chars, 140), artifact_indexes=artifact_indexes)
    1748    if len(events) > len(display_events):
    1749        print(f"  ... {len(events) - len(display_events)} older events hidden. Use /history for the full timeline.")
    1750
    1751
    1752def _print_session_overview(
    1753    job: dict[str, Any],
    1754    *,
    1755    steps: list[dict[str, Any]],
    1756    artifacts: list[dict[str, Any]],
    1757    memory_entries: list[dict[str, Any]],
    1758    daemon_running: bool,
    1759    model: str,
    1760    artifacts_dir: Path,
    1761    jobs: list[dict[str, Any]],
    1762    chars: int,
    1763) -> None:
    1764    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    1765    findings = _metadata_records(job, "finding_ledger")
    1766    sources = _metadata_records(job, "source_ledger")
    1767    tasks = _metadata_records(job, "task_queue")
    1768    experiments = _metadata_records(job, "experiment_ledger")
    1769    lessons = _metadata_records(job, "lessons")
    1770    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
    1771    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
    1772    open_tasks = sum(1 for task in tasks if str(task.get("status") or "open") in {"open", "active"})
    1773    state = _job_display_state(job, daemon_running)
    1774    worker = _worker_label(job, daemon_running)
    1775    print(_section_title("Workspace"))
    1776    print(f"  model      {model}")
    1777    print(f"  focus      {job['title']}")
    1778    print(f"  state      {_status_badge(state)}   worker {_status_badge(worker)}   kind {job['kind']}")
    1779    next_action = _next_operator_action(job, daemon_running)
    1780    if next_action:
    1781        print(f"  next       {next_action}")
    1782
    1783    print()
    1784    _print_jobs_panel(jobs, focused_job_id=str(job["id"]), daemon_running=daemon_running)
    1785
    1786    print()
    1787    print(_section_title("Focus"))
    1788    _print_wrapped(
    1789        "  goal       ", job.get("objective") or "", width=_terminal_width(), subsequent_indent="             "
    1790    )
    1791    planning = metadata.get("planning") if isinstance(metadata.get("planning"), dict) else {}
    1792    if job.get("status") == "planning" and planning:
    1793        print("  plan       waiting for your answers or /run")
    1794        questions = planning.get("questions") if isinstance(planning.get("questions"), list) else []
    1795        for question in questions[:3]:
    1796            _print_wrapped("  question   ", question, width=_terminal_width(), subsequent_indent="             ")
    1797
    1798    print()
    1799    print(_section_title("Progress"))
    1800    _print_metric_grid(
    1801        [
    1802            ("actions", _step_count(steps)),
    1803            ("outputs", len(artifacts)),
    1804            ("findings", len(findings)),
    1805            ("sources", len(sources)),
    1806            ("tasks", f"{len(tasks)} ({open_tasks} open)"),
    1807            ("roadmap", len(milestones)),
    1808            ("experiments", len(experiments)),
    1809            ("lessons", len(lessons)),
    1810            ("memory", len(memory_entries)),
    1811        ]
    1812    )
    1813    print(f"  output dir {_short_path(artifacts_dir, max_width=min(_terminal_width() - 13, 84))}")
    1814
    1815
    1816def _print_chat_composer(job: dict[str, Any]) -> None:
    1817    width = min(_terminal_width(), 96)
    1818    if _fancy_ui():
    1819        print(_accent("╭─ Message " + "─" * max(0, width - 11)))
    1820        print("│ Type normally to chat. Live steps stream above. /jobs switches workspaces. /help shows commands.")
    1821        print("╰─" + "─" * max(0, width - 2))
    1822        return
    1823    print(_section_title("Message"))
    1824    print("  Type normally to chat. Live steps stream above. /jobs switches workspaces. /help shows commands.")
    1825
    1826
    1827def _chat_prompt(job: dict[str, Any]) -> str:
    1828    return f"{_accent('nipux')} > "
    1829
    1830
    1831def _start_chat_live_feed(job_id: str) -> tuple[threading.Event | None, threading.Thread | None]:
    1832    if (
    1833        not sys.stdin.isatty()
    1834        or not sys.stdout.isatty()
    1835        or os.environ.get("NIPUX_NO_LIVE")
    1836        or os.environ.get("NIPUX_PLAIN")
    1837    ):
    1838        return None, None
    1839    stop = threading.Event()
    1840    thread = threading.Thread(target=_chat_live_feed_loop, args=(job_id, stop), daemon=True)
    1841    thread.start()
    1842    return stop, thread
    1843
    1844
    1845def _chat_live_feed_loop(initial_job_id: str, stop: threading.Event) -> None:
    1846    seen_by_job: dict[str, set[str]] = {}
    1847    initialized_jobs: set[str] = set()
    1848    active_job_id = initial_job_id
    1849    while not stop.wait(1.0):
    1850        try:
    1851            db, _ = _db()
    1852            try:
    1853                focused = _default_job_id(db) or active_job_id
    1854                active_job_id = focused
    1855                seen = seen_by_job.setdefault(focused, set())
    1856                events = db.list_events(job_id=focused, limit=40)
    1857                if focused not in initialized_jobs:
    1858                    initialized_jobs.add(focused)
    1859                    seen.update(str(event.get("id") or "") for event in events)
    1860                    continue
    1861                for event in events:
    1862                    event_id = str(event.get("id") or "")
    1863                    if not event_id or event_id in seen:
    1864                        continue
    1865                    seen.add(event_id)
    1866                    line = _minimal_live_event_line(event)
    1867                    if line:
    1868                        _print_live_line(line)
    1869            finally:
    1870                db.close()
    1871        except Exception:
    1872            continue
    1873
    1874
    1875def _print_live_line(line: str) -> None:
    1876    try:
    1877        if _fancy_ui():
    1878            print(f"\r\033[K{_live_badge(line)} {line}\n{_chat_prompt({})}", end="", flush=True)
    1879        else:
    1880            print(f"\n· {line}", flush=True)
    1881    except Exception:
    1882        return
    1883
    1884
    1885def _resolve_control_job_and_note(db: AgentDB, args: argparse.Namespace) -> tuple[str | None, str, str | None]:
    1886    if hasattr(args, "parts"):
    1887        parts = [str(part) for part in getattr(args, "parts") or []]
    1888        if not parts:
    1889            return _default_job_id(db), "", None
    1890        for end in range(len(parts), 0, -1):
    1891            ref = " ".join(parts[:end])
    1892            job = _find_job(db, ref)
    1893            if job:
    1894                return str(job["id"]), " ".join(parts[end:]).strip(), ref
    1895        return None, "", " ".join(parts)
    1896    job_ref = _job_ref_text(getattr(args, "job_id", None))
    1897    return _resolve_job_id(db, job_ref), _note_text(getattr(args, "note", None)), job_ref
    1898
    1899
    1900def _pid_is_alive(pid: int) -> bool:
    1901    try:
    1902        os.kill(pid, 0)
    1903    except OSError:
    1904        return False
    1905    return True
    1906
    1907
    1908def _step_by_id(db: AgentDB, job_id: str, step_id: str) -> dict[str, Any] | None:
    1909    for step in db.list_steps(job_id=job_id):
    1910        if step["id"] == step_id:
    1911            return step
    1912    return None
    1913
    1914
    1915def _step_count(steps: list[dict[str, Any]]) -> int:
    1916    numbers = [int(step.get("step_no") or 0) for step in steps]
    1917    return max(numbers, default=0)
    1918
    1919
    1920def _job_lessons(job: dict[str, Any]) -> list[dict[str, Any]]:
    1921    return _metadata_records(job, "lessons")
    1922
    1923
    1924def _metadata_records(job: dict[str, Any], key: str) -> list[dict[str, Any]]:
    1925    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    1926    values = metadata.get(key) if isinstance(metadata.get(key), list) else []
    1927    return [entry for entry in values if isinstance(entry, dict)]
    1928
    1929
    1930def _print_lessons(job: dict[str, Any], *, limit: int, chars: int) -> None:
    1931    lessons = _job_lessons(job)
    1932    print(f"lessons {job['title']}")
    1933    print(_rule("="))
    1934    if not lessons:
    1935        print("none yet")
    1936        print("add one with: learn this source is not useful for the current objective")
    1937        return
    1938    for index, lesson in enumerate(lessons[-limit:], start=max(1, len(lessons) - limit + 1)):
    1939        category = lesson.get("category") or "memory"
    1940        confidence = lesson.get("confidence")
    1941        suffix = f" | confidence {confidence:g}" if isinstance(confidence, (int, float)) else ""
    1942        print(f"{index:>2}. {category}{suffix}")
    1943        print(f"    {_one_line(lesson.get('lesson') or '', chars)}")
    1944
    1945
    1946def _resolve_artifact_ref(
    1947    db: AgentDB,
    1948    config: Any,
    1949    query: str | None,
    1950    *,
    1951    job_id: str | None = None,
    1952) -> dict[str, Any] | None:
    1953    if not query:
    1954        return None
    1955    ref = query.strip()
    1956    path = Path(ref).expanduser()
    1957    if path.exists():
    1958        return {"path": str(path), "title": path.name, "summary": ""}
    1959
    1960    ref_lower = ref.lower()
    1961    focused_artifacts = db.list_artifacts(job_id, limit=250) if job_id else []
    1962    if focused_artifacts and ref_lower in {"latest", "last", "newest"}:
    1963        return focused_artifacts[0]
    1964    index_ref = ref_lower[1:] if ref_lower.startswith("#") else ref_lower
    1965    if focused_artifacts and index_ref.isdigit():
    1966        index = int(index_ref)
    1967        if 1 <= index <= len(focused_artifacts):
    1968            return focused_artifacts[index - 1]
    1969
    1970    jobs = db.list_jobs()
    1971    ordered_jobs = []
    1972    if job_id:
    1973        try:
    1974            selected = db.get_job(job_id)
    1975            ordered_jobs.append(selected)
    1976        except KeyError:
    1977            pass
    1978    ordered_jobs.extend(job for job in jobs if not job_id or job["id"] != job_id)
    1979    artifacts: list[dict[str, Any]] = []
    1980    for job in ordered_jobs:
    1981        artifacts.extend(db.list_artifacts(job["id"], limit=250))
    1982
    1983    for artifact in artifacts:
    1984        if str(artifact["id"]).lower() == ref_lower:
    1985            return artifact
    1986    for artifact in artifacts:
    1987        title = str(artifact.get("title") or "")
    1988        if title.lower() == ref_lower:
    1989            return artifact
    1990    for artifact in artifacts:
    1991        haystack = " ".join(str(artifact.get(key) or "") for key in ("title", "summary", "type")).lower()
    1992        if ref_lower in haystack:
    1993            return artifact
    1994
    1995    store = ArtifactStore(config.runtime.home, db=db)
    1996    search_job_ids = [job_id] if job_id else [str(job["id"]) for job in ordered_jobs]
    1997    for candidate_job_id in search_job_ids:
    1998        if not candidate_job_id:
    1999            continue
    2000        for result in store.search_text(job_id=candidate_job_id, query=ref, limit=1):
    2001            try:
    2002                return db.get_artifact(str(result["id"]))
    2003            except KeyError:
    2004                continue
    2005    return None
    2006
    2007
    2008def cmd_logs(args: argparse.Namespace) -> None:
    2009    db, _ = _db()
    2010    try:
    2011        job_id = _resolve_job_id(db, args.job_id)
    2012        if not job_id:
    2013            ref = _job_ref_text(args.job_id)
    2014            print(f"No job matched: {ref}" if ref else "No jobs found.")
    2015            return
    2016        job = db.get_job(job_id)
    2017        daemon = daemon_lock_status(load_config().runtime.home / "agentd.lock")
    2018        print(f"{job['title']}\tstate {_job_display_state(job, bool(daemon['running']))}\t{job['kind']}")
    2019        print()
    2020        print("Runs")
    2021        for run in db.list_runs(job_id, limit=args.limit):
    2022            error = f"\tERROR {run['error']}" if run.get("error") else ""
    2023            print(f"{run['started_at']}\t{run['status']}\t{run['id']}\t{run.get('model') or ''}{error}")
    2024        print()
    2025        print("Steps")
    2026        steps = db.list_steps(job_id=job_id)[-args.limit :]
    2027        if not steps:
    2028            print("No steps recorded.")
    2029        for step in steps:
    2030            if args.verbose:
    2031                _print_step(step, verbose=True, chars=args.chars)
    2032            else:
    2033                tool = step.get("tool_name") or "-"
    2034                summary = _one_line(_clean_step_summary(step.get("summary") or ""), args.chars)
    2035                error = f"\tERROR {step['error']}" if step.get("error") else ""
    2036                print(
    2037                    f"#{step['step_no']}\t{step['started_at']}\t{step['status']}\t{step['kind']}\t{tool}\t{summary}{error}"
    2038                )
    2039        print()
    2040        print("Artifacts")
    2041        artifacts = db.list_artifacts(job_id, limit=args.limit)
    2042        if not artifacts:
    2043            print("No artifacts recorded.")
    2044        for artifact in artifacts:
    2045            print(
    2046                f"{artifact['created_at']}\t{artifact['type']}\t{artifact.get('title') or artifact['id']}\t{artifact['path']}"
    2047            )
    2048    finally:
    2049        db.close()
    2050
    2051
    2052def cmd_activity(args: argparse.Namespace) -> None:
    2053    db, _ = _db()
    2054    seen_events: set[str] = set()
    2055    try:
    2056        job_id = _resolve_job_id(db, args.job_id)
    2057        if not job_id:
    2058            print("No jobs found.")
    2059            return
    2060        job = db.get_job(job_id)
    2061        daemon = daemon_lock_status(load_config().runtime.home / "agentd.lock")
    2062        print(f"activity {job['title']} | state {_job_display_state(job, bool(daemon['running']))}")
    2063        print("tool calls, artifacts, learning, and messages, oldest to newest")
    2064        print(_rule("-"))
    2065
    2066        def emit() -> None:
    2067            events = db.list_timeline_events(job_id, limit=args.limit)
    2068            printed = False
    2069            for event in events:
    2070                event_id = str(event.get("id") or "")
    2071                if event_id in seen_events:
    2072                    continue
    2073                print(_event_line(event, chars=args.chars, full=args.verbose))
    2074                if args.verbose:
    2075                    _print_event_details(event, chars=args.chars)
    2076                if args.paths and event.get("event_type") == "artifact":
    2077                    metadata = event.get("metadata") if isinstance(event.get("metadata"), dict) else {}
    2078                    if metadata.get("path"):
    2079                        print(f"     path: {metadata['path']}")
    2080                seen_events.add(event_id)
    2081                printed = True
    2082            if printed:
    2083                print(_rule("-"))
    2084
    2085        emit()
    2086        while args.follow:
    2087            time.sleep(args.interval)
    2088            emit()
    2089    except KeyboardInterrupt:
    2090        print("\nactivity stopped")
    2091    finally:
    2092        db.close()
    2093
    2094
    2095def cmd_updates(args: argparse.Namespace) -> None:
    2096    db, config = _db()
    2097    try:
    2098        if getattr(args, "all", False):
    2099            print(
    2100                "\n".join(
    2101                    render_all_updates_report(
    2102                        db,
    2103                        config,
    2104                        limit=args.limit,
    2105                        chars=args.chars,
    2106                        paths=args.paths,
    2107                    )
    2108                )
    2109            )
    2110            return
    2111        job_id = _resolve_job_id(db, args.job_id)
    2112        if not job_id:
    2113            print("No jobs found.")
    2114            return
    2115        print(
    2116            "\n".join(
    2117                render_updates_report(
    2118                    db,
    2119                    config,
    2120                    job_id,
    2121                    limit=args.limit,
    2122                    chars=args.chars,
    2123                    paths=args.paths,
    2124                )
    2125            )
    2126        )
    2127    finally:
    2128        db.close()
    2129
    2130
    2131def cmd_watch(args: argparse.Namespace) -> None:
    2132    db, _ = _db()
    2133    seen_runs: set[str] = set()
    2134    seen_steps: set[str] = set()
    2135    seen_artifacts: set[str] = set()
    2136    try:
    2137        job_id = _resolve_job_id(db, args.job_id)
    2138        if not job_id:
    2139            print(f"No job matched: {_job_ref_text(args.job_id)}")
    2140            return
    2141        job = db.get_job(job_id)
    2142        daemon = daemon_lock_status(load_config().runtime.home / "agentd.lock")
    2143        print(f"watching {job['title']} | state {_job_display_state(job, bool(daemon['running']))} | {job['kind']}")
    2144        print(f"objective: {job['objective']}")
    2145        print(
    2146            "Note: this shows model-visible state, tool calls, outputs, and errors. It does not expose hidden chain-of-thought."
    2147        )
    2148        print()
    2149
    2150        def emit_snapshot(*, initial: bool = False) -> None:
    2151            nonlocal job
    2152            job = db.get_job(job_id)
    2153            runs = list(reversed(db.list_runs(job_id, limit=args.limit)))
    2154            steps = db.list_steps(job_id=job_id)[-args.limit :]
    2155            artifacts = list(reversed(db.list_artifacts(job_id, limit=args.limit)))
    2156            printed = False
    2157            for run in runs:
    2158                if run["id"] in seen_runs:
    2159                    continue
    2160                if not initial:
    2161                    print()
    2162                _print_run(run)
    2163                seen_runs.add(run["id"])
    2164                printed = True
    2165            for step in steps:
    2166                if step["id"] in seen_steps:
    2167                    continue
    2168                if not initial and not printed:
    2169                    print()
    2170                _print_step(step, verbose=args.verbose, chars=args.chars)
    2171                seen_steps.add(step["id"])
    2172                printed = True
    2173            for artifact in artifacts:
    2174                if artifact["id"] in seen_artifacts:
    2175                    continue
    2176                if not initial and not printed:
    2177                    print()
    2178                _print_artifact(artifact)
    2179                seen_artifacts.add(artifact["id"])
    2180                printed = True
    2181            if printed:
    2182                print(f"status: {job['status']}")
    2183
    2184        emit_snapshot(initial=True)
    2185        while args.follow:
    2186            time.sleep(args.interval)
    2187            emit_snapshot()
    2188    except KeyboardInterrupt:
    2189        print("\nwatch stopped")
    2190    finally:
    2191        db.close()
    2192
    2193
    2194def cmd_run_one(args: argparse.Namespace) -> None:
    2195    from nipux_cli.worker import run_one_step
    2196
    2197    db, config = _db()
    2198    try:
    2199        job_id = _resolve_job_id(db, args.job_id)
    2200        if not job_id:
    2201            print(f"No job matched: {_job_ref_text(args.job_id)}")
    2202            return
    2203        if not args.fake and not _model_setup_verified(config):
    2204            _ensure_model_setup_verified_for_workspace()
    2205            return
    2206        _activate_job_if_planning(db, job_id)
    2207        llm = None
    2208        if args.fake:
    2209            from nipux_cli.llm import LLMResponse, ScriptedLLM, ToolCall
    2210
    2211            llm = ScriptedLLM(
    2212                [
    2213                    LLMResponse(
    2214                        tool_calls=[
    2215                            ToolCall(
    2216                                name="write_artifact",
    2217                                arguments={
    2218                                    "title": "fake-step",
    2219                                    "type": "text",
    2220                                    "summary": "Fake one-step smoke artifact",
    2221                                    "content": "This is a fake bounded worker step.",
    2222                                },
    2223                            )
    2224                        ]
    2225                    )
    2226                ]
    2227            )
    2228        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2229        print(json.dumps(result.__dict__, ensure_ascii=False, indent=2))
    2230    finally:
    2231        db.close()
    2232
    2233
    2234def cmd_work(args: argparse.Namespace) -> None:
    2235    from nipux_cli.worker import run_one_step
    2236
    2237    db, config = _db()
    2238    try:
    2239        job_id = _resolve_job_id(db, args.job_id)
    2240        if not job_id:
    2241            print('No jobs found. Create one with: nipux create "objective"')
    2242            return
    2243        if not args.fake and not _model_setup_verified(config):
    2244            _ensure_model_setup_verified_for_workspace()
    2245            return
    2246        _activate_job_if_planning(db, job_id)
    2247        job = db.get_job(job_id)
    2248        print(f"working {job['title']} | state foreground | {job['kind']}")
    2249        print(
    2250            "Note: this shows model-visible state, tool calls, outputs, and errors. It does not expose hidden chain-of-thought."
    2251        )
    2252        print()
    2253        for index in range(1, args.steps + 1):
    2254            llm = None
    2255            if args.fake:
    2256                from nipux_cli.llm import LLMResponse, ScriptedLLM, ToolCall
    2257
    2258                llm = ScriptedLLM(
    2259                    [
    2260                        LLMResponse(
    2261                            tool_calls=[
    2262                                ToolCall(
    2263                                    name="write_artifact",
    2264                                    arguments={
    2265                                        "title": f"fake-work-step-{index}",
    2266                                        "type": "text",
    2267                                        "summary": "Fake foreground work step",
    2268                                        "content": f"This is fake foreground work step {index}.",
    2269                                    },
    2270                                )
    2271                            ]
    2272                        )
    2273                    ]
    2274                )
    2275            print(f"work step {index}/{args.steps}", flush=True)
    2276            result = run_one_step(job_id, config=config, db=db, llm=llm)
    2277            step = _step_by_id(db, job_id, result.step_id)
    2278            if step:
    2279                _print_step(step, verbose=args.verbose, chars=args.chars)
    2280            else:
    2281                print(json.dumps(result.__dict__, ensure_ascii=False, indent=2, default=_json_default))
    2282            if args.dashboard:
    2283                state = collect_dashboard_state(db, config, job_id=job_id, limit=args.limit)
    2284                print()
    2285                print(render_dashboard(state, width=_terminal_width(), chars=args.chars), end="")
    2286            if result.status == "failed" and not args.continue_on_error:
    2287                print("stopped after failed step; pass --continue-on-error to keep going")
    2288                return
    2289            if index < args.steps and args.poll_seconds > 0:
    2290                time.sleep(args.poll_seconds)
    2291    finally:
    2292        db.close()
    2293
    2294
    2295def _pause_job_for_recoverable_provider_preflight(
    2296    db: AgentDB,
    2297    config: Any,
    2298    job_id: str,
    2299    *,
    2300    fake: bool,
    2301    failures: list[str] | None = None,
    2302) -> bool:
    2303    if fake:
    2304        return False
    2305    failures = failures if failures is not None else _recoverable_remote_model_preflight_failures(config)
    2306    if not failures:
    2307        return False
    2308    now = utc_now()
    2309    detail = "; ".join(failures)
    2310    job = db.get_job(job_id)
    2311    already_provider_blocked = job_provider_blocked(job)
    2312    if already_provider_blocked and str(job.get("status") or "") == "paused":
    2313        db.update_job_metadata(
    2314            job_id,
    2315            {
    2316                "provider_last_probe_at": now,
    2317                "provider_last_probe_detail": detail[:1000],
    2318                "last_note": "Model provider still unavailable; daemon will check again later.",
    2319            },
    2320        )
    2321        return True
    2322    note = "Model provider is unavailable; daemon will monitor and resume this job when calls succeed."
    2323    db.update_job_status(
    2324        job_id,
    2325        "paused",
    2326        metadata_patch={
    2327            "last_note": note,
    2328            "provider_blocked_at": str(job.get("metadata", {}).get("provider_blocked_at") or now)
    2329            if already_provider_blocked
    2330            else now,
    2331            "provider_last_probe_at": now,
    2332            "provider_last_probe_detail": detail[:1000],
    2333        },
    2334    )
    2335    if not already_provider_blocked:
    2336        db.append_agent_update(
    2337            job_id,
    2338            note,
    2339            category="error",
    2340            metadata={"reason": "llm_provider_blocked", "detail": detail[:1000]},
    2341        )
    2342    return True
    2343
    2344
    2345def cmd_run(args: argparse.Namespace) -> None:
    2346    config = load_config()
    2347    preflight_failures = [] if args.fake or _model_setup_verified(config) else _remote_model_preflight_failures(config)
    2348    preflight_recoverable = _provider_preflight_is_recoverable(preflight_failures)
    2349    can_prepare_job = not preflight_failures or preflight_recoverable
    2350    requested = _job_ref_text(args.job_id)
    2351    if requested:
    2352        db, _ = _db()
    2353        try:
    2354            job = _find_job(db, requested)
    2355            if not job:
    2356                print(f"No job matched: {requested}")
    2357                return
    2358            args.job_id = job["id"]
    2359            _write_shell_state({"focus_job_id": job["id"]})
    2360            if can_prepare_job:
    2361                already_provider_blocked = preflight_recoverable and job_provider_blocked(job)
    2362                if not already_provider_blocked:
    2363                    _ensure_job_runnable(db, job["id"])
    2364                if preflight_recoverable:
    2365                    _pause_job_for_recoverable_provider_preflight(
    2366                        db,
    2367                        config,
    2368                        job["id"],
    2369                        fake=bool(args.fake),
    2370                        failures=preflight_failures,
    2371                    )
    2372            job = db.get_job(job["id"])
    2373            daemon = daemon_lock_status(load_config().runtime.home / "agentd.lock")
    2374            print(f"focus set: {job['title']} | job {_job_display_state(job, bool(daemon['running']))}")
    2375        finally:
    2376            db.close()
    2377    else:
    2378        db, _ = _db()
    2379        try:
    2380            job_id = _default_job_id(db)
    2381            if job_id:
    2382                if can_prepare_job:
    2383                    job = db.get_job(job_id)
    2384                    already_provider_blocked = preflight_recoverable and job_provider_blocked(job)
    2385                    if not already_provider_blocked:
    2386                        _ensure_job_runnable(db, job_id)
    2387                    if preflight_recoverable:
    2388                        _pause_job_for_recoverable_provider_preflight(
    2389                            db,
    2390                            config,
    2391                            job_id,
    2392                            fake=bool(args.fake),
    2393                            failures=preflight_failures,
    2394                        )
    2395            else:
    2396                print("No jobs found. Create one with /new OBJECTIVE.")
    2397                return
    2398        finally:
    2399            db.close()
    2400    _start_daemon_if_needed(
    2401        poll_seconds=args.poll_seconds,
    2402        fake=args.fake,
    2403        quiet=args.quiet,
    2404        log_file=args.log_file,
    2405    )
    2406    if args.no_follow:
    2407        return
    2408    cmd_activity(
    2409        argparse.Namespace(
    2410            job_id=args.job_id,
    2411            limit=args.limit,
    2412            chars=args.chars,
    2413            follow=True,
    2414            interval=args.interval,
    2415            verbose=args.verbose,
    2416            paths=args.paths,
    2417        )
    2418    )
    2419
    2420
    2421def cmd_digest(args: argparse.Namespace) -> None:
    2422    db, config = _db()
    2423    try:
    2424        job_id = _resolve_job_id(db, args.job_id)
    2425        if not job_id:
    2426            print(f"No job matched: {_job_ref_text(args.job_id)}")
    2427            return
    2428        print(
    2429            render_job_digest(
    2430                db,
    2431                job_id,
    2432                model=config.model.model,
    2433                base_url=config.model.base_url,
    2434                context_length=config.model.context_length,
    2435                input_cost_per_million=config.model.input_cost_per_million,
    2436                output_cost_per_million=config.model.output_cost_per_million,
    2437            ),
    2438            end="",
    2439        )
    2440    finally:
    2441        db.close()
    2442
    2443
    2444def cmd_daily_digest(args: argparse.Namespace) -> None:
    2445    db, config = _db()
    2446    try:
    2447        result = write_daily_digest(config, db, day=args.day)
    2448        print(json.dumps(result, ensure_ascii=False, indent=2))
    2449    finally:
    2450        db.close()
    2451
    2452
    2453def cmd_daemon(args: argparse.Namespace) -> None:
    2454    config = load_config()
    2455    if not _ensure_remote_model_ready_for_worker(config, fake=args.fake):
    2456        raise SystemExit(2)
    2457    daemon = Daemon.open(config=config)
    2458    try:
    2459        if args.once:
    2460            result = daemon.run_once(fake=args.fake, verbose=args.verbose)
    2461            print(json.dumps(result.__dict__ if result else None, ensure_ascii=False, indent=2))
    2462            return
    2463        daemon.run_forever(fake=args.fake, poll_seconds=args.poll_seconds, quiet=args.quiet, verbose=args.verbose)
    2464    except DaemonAlreadyRunning as exc:
    2465        raise SystemExit(str(exc)) from exc
    2466    finally:
    2467        daemon.close()
    2468
    2469
    2470def cmd_doctor(args: argparse.Namespace) -> None:
    2471    config = load_config()
    2472    checks = run_doctor(config=config, check_model=args.check_model)
    2473    for check in checks:
    2474        status = "ok" if check.ok else "fail"
    2475        print(f"{status}\t{check.name}\t{check.detail}")
    2476    ok = all(check.ok for check in checks)
    2477    if args.check_model:
    2478        if ok:
    2479            _mark_model_setup_verified(config)
    2480            print("ok\tmodel_setup\tverified for workspace and chat")
    2481        else:
    2482            _clear_model_setup_verified()
    2483    if not ok:
    2484        raise SystemExit(1)
    2485
    2486
    2487def _verify_model_setup_from_first_run() -> list[str]:
    2488    stream = StringIO()
    2489    with redirect_stdout(stream):
    2490        try:
    2491            cmd_doctor(argparse.Namespace(check_model=True))
    2492        except SystemExit as exc:
    2493            if exc.code not in (None, 0):
    2494                print("Model setup is not ready. Fix the failed check above before creating a job.")
    2495                print("Use /base-url URL, /api-key KEY, or /model MODEL here, then run Doctor again.")
    2496                print("For a local endpoint, start the local server or change the endpoint.")
    2497    lines = [" ".join(item.split()) for item in stream.getvalue().splitlines() if item.strip()]
    2498    return lines[-12:] or ["done"]
    2499
    2500
    2501def _chat_handle_line(job_id: str, line: str, *, reply_fn=None) -> bool:
    2502    line = line.strip()
    2503    if not line:
    2504        return True
    2505    if line.startswith("chat "):
    2506        db, _ = _db()
    2507        try:
    2508            job = db.get_job(job_id)
    2509            print(f"already chatting with {job['title']}; type your message, /run, or /exit")
    2510            return True
    2511        finally:
    2512            db.close()
    2513    if line in {"/exit", "/quit", "exit", "quit"}:
    2514        return False
    2515    if line in {"/help", "help"}:
    2516        print("Core workflow:")
    2517        print("  /new OBJECTIVE       create a job and start work")
    2518        print("  /run                 resume/start the focused job")
    2519        print("  /jobs                switch or inspect jobs")
    2520        print("  /status              current job state")
    2521        print("  /outcomes            durable progress")
    2522        print("  /artifacts           saved files")
    2523        print("  /activity            tool calls")
    2524        print("  /pause /resume       control the focused job")
    2525        print()
    2526        print("All commands:")
    2527        print("  /jobs /focus JOB_TITLE /switch JOB_TITLE /new OBJECTIVE /delete [JOB_TITLE]")
    2528        print("  /history /events /activity /outputs /updates /outcomes [all] /status /usage /config /settings /health")
    2529        print("  /artifacts /artifact QUERY /findings /tasks /roadmap /experiments /sources /memory /metrics /lessons")
    2530        print("  /model MODEL /base-url URL /api-key KEY /api-key-env ENV /context TOKENS")
    2531        print("  /input-cost DOLLARS_PER_1M_INPUT_TOKENS /output-cost DOLLARS_PER_1M_OUTPUT_TOKENS")
    2532        print("  /browser true|false /web true|false /cli-access true|false /file-access true|false")
    2533        print("  /timeout SECONDS /home PATH /step-limit SECONDS /output-chars CHARS /daily-digest BOOL /digest-time HH:MM /doctor")
    2534        print("  /run /start /restart /work N /work-verbose N /stop /pause [note] /resume /cancel [note]")
    2535        print("  /learn LESSON /note MESSAGE /follow MESSAGE /digest /clear /exit")
    2536        print("Plain text gets a model reply and is saved as model-visible steering.")
    2537        return True
    2538    if line in {"clear", "/clear"}:
    2539        print("\033[2J\033[H", end="")
    2540        return True
    2541    if line == "jobs" or line == "ls" or line.startswith("jobs "):
    2542        cmd_jobs(argparse.Namespace())
    2543        return True
    2544    if line.startswith(("focus ", "switch ")):
    2545        parts = shlex.split(line)
    2546        cmd_focus(argparse.Namespace(query=parts[1:]))
    2547        return True
    2548    if line.startswith("/"):
    2549        parts = shlex.split(line[1:])
    2550        if not parts:
    2551            return True
    2552        return _handle_chat_slash_command(job_id, parts[0], parts[1:], deps=_chat_command_deps())
    2553    if reply_fn is None:
    2554        reply_fn = _reply_to_chat
    2555    _handle_chat_message(job_id, line, reply_fn=reply_fn)
    2556    return True
    2557
    2558
    2559def _handle_chat_message(job_id: str, line: str, *, reply_fn=None, quiet: bool = False) -> tuple[bool, str]:
    2560    if not _model_setup_verified(load_config()):
    2561        message = (
    2562            "Model setup is not verified. Complete setup or run /doctor after configuring a working provider."
    2563        )
    2564        if not quiet:
    2565            print(message)
    2566        return True, message
    2567    if job_id == WORKSPACE_CHAT_ID:
    2568        return _handle_workspace_chat_message(line, quiet=quiet)
    2569    return _controller_handle_chat_message(
    2570        job_id,
    2571        line,
    2572        deps=_chat_controller_deps(),
    2573        reply_fn=reply_fn,
    2574        quiet=quiet,
    2575    )
    2576
    2577
    2578def _chat_reply_text_and_metadata(reply: Any) -> tuple[str, dict[str, Any]]:
    2579    return _controller_reply_text_and_metadata(reply)
    2580
    2581
    2582def _workspace_chat_events() -> list[dict[str, Any]]:
    2583    events = _read_shell_state().get("workspace_chat_events")
    2584    if not isinstance(events, list):
    2585        return []
    2586    return [event for event in events if isinstance(event, dict)][-120:]
    2587
    2588
    2589def _append_workspace_chat_event(event_type: str, title: str, body: str, metadata: dict[str, Any] | None = None) -> None:
    2590    events = _workspace_chat_events()
    2591    events.append(
    2592        {
    2593            "id": f"workspace_{len(events) + 1}_{int(time.time() * 1000)}",
    2594            "job_id": WORKSPACE_CHAT_ID,
    2595            "event_type": event_type,
    2596            "created_at": utc_now(),
    2597            "title": title,
    2598            "body": body,
    2599            "metadata": metadata or {},
    2600        }
    2601    )
    2602    _write_shell_state({"workspace_chat_events": events[-120:]})
    2603
    2604
    2605def _handle_workspace_chat_message(line: str, *, quiet: bool = False) -> tuple[bool, str]:
    2606    _append_workspace_chat_event("operator_message", "chat", line, {"source": "workspace"})
    2607    objective = _extract_job_objective_from_message(line)
    2608    if objective:
    2609        message = _create_workspace_job_from_chat(line, objective)
    2610        _append_workspace_chat_event("agent_message", "chat", message, {"source": "workspace"})
    2611        if not quiet:
    2612            print(message)
    2613        return True, message
    2614    control_command = chat_control_command(line)
    2615    if control_command:
    2616        keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, control_command)
    2617        compact = _compact_command_output(output)
    2618        message = " | ".join(compact[-4:]) if compact else f"{control_command.lstrip('/')} done"
    2619        _append_workspace_chat_event(
    2620            "agent_message",
    2621            "chat",
    2622            message,
    2623            {"source": "workspace", "command": control_command},
    2624        )
    2625        if not quiet:
    2626            print(message)
    2627        return keep_running, message
    2628    try:
    2629        reply = _reply_to_workspace_chat(line)
    2630    except Exception as exc:
    2631        message = _friendly_error_text(f"{type(exc).__name__}: {exc}")
    2632        _append_workspace_chat_event("agent_message", "chat", message, {"source": "workspace", "error": True})
    2633        if not quiet:
    2634            print(message)
    2635        return True, message
    2636    reply_text, reply_metadata = _chat_reply_text_and_metadata(reply)
    2637    text = reply_text.strip() or "I did not get a usable model reply."
    2638    _append_workspace_chat_event("agent_message", "chat", text, {"source": "workspace", **reply_metadata})
    2639    if not quiet:
    2640        print(text)
    2641    return True, text
    2642
    2643
    2644def _create_workspace_job_from_chat(message: str, objective: str) -> str:
    2645    refined = _refine_job_objective_for_worker(message=message, objective=objective)
    2646    job_id, title = _create_job(objective=refined, title=None, kind="generic", cadence=None)
    2647    _write_shell_state({"focus_job_id": job_id})
    2648    db, _config = _db()
    2649    try:
    2650        db.append_operator_message(job_id, message, source="workspace_chat", mode="steer")
    2651        db.append_agent_update(
    2652            job_id,
    2653            "Created from Nipux workspace chat with an expanded long-running objective.",
    2654            category="chat",
    2655        )
    2656    finally:
    2657        db.close()
    2658    run_now = not message_requests_queued_job(message) or message_requests_immediate_run(message)
    2659    text = f"Created worker job: {title}."
    2660    if run_now:
    2661        if _start_worker_from_chat_context():
    2662            text += " Started worker."
    2663        else:
    2664            text += " Worker is waiting for a working model."
    2665    else:
    2666        text += " It is queued; tell me to run it when ready."
    2667    return text
    2668
    2669
    2670def _refine_job_objective_for_worker(*, message: str, objective: str) -> str:
    2671    fallback = _durable_job_objective(objective)
    2672    try:
    2673        from nipux_cli.llm import OpenAIChatLLM
    2674
    2675        _db_handle, config = _db()
    2676        _db_handle.close()
    2677        prompt = [
    2678            {
    2679                "role": "system",
    2680                "content": (
    2681                    "You rewrite operator requests into strong, generic Nipux worker objectives. "
    2682                    "Nipux workers are long-running autonomous jobs with browser, web, CLI, file, artifact, "
    2683                    "memory, roadmap, task, source, finding, and experiment tools. "
    2684                    "Return only the objective text for the worker. Start with one concise title line, then add "
    2685                    "clear success criteria, output expectations, constraints, evidence requirements, progress "
    2686                    "reporting expectations, and instructions to keep improving until no useful progress remains. "
    2687                    "Do not invent hosts, credentials, accounts, domains, models, or private details that the operator did not provide."
    2688                ),
    2689            },
    2690            {
    2691                "role": "user",
    2692                "content": f"Operator message:\n{message}\n\nExtracted objective:\n{objective}",
    2693            },
    2694        ]
    2695        refined = OpenAIChatLLM(config.model).complete(messages=prompt).strip()
    2696    except Exception:
    2697        return fallback
    2698    if len(refined) < 20:
    2699        return fallback
    2700    return refined[:8000]
    2701
    2702
    2703def _durable_job_objective(objective: str) -> str:
    2704    cleaned = " ".join(str(objective or "").split()).strip() or "Long-running Nipux job"
    2705    title = _one_line(cleaned, 96)
    2706    return (
    2707        f"{title}\n\n"
    2708        "Run this as a durable long-running Nipux worker job.\n"
    2709        "- Clarify and preserve the operator's actual goal, constraints, and success criteria.\n"
    2710        "- Build a roadmap before deep work, then keep the task queue current as evidence changes.\n"
    2711        "- Produce concrete outputs as artifacts or files when the work creates something useful.\n"
    2712        "- Record findings, sources, lessons, experiments, and measurable results when they apply.\n"
    2713        "- Separate activity from progress: report what changed, what was learned, what failed, and what branch is next.\n"
    2714        "- Keep improving autonomously until no useful progress remains, the operator pauses the job, or a real blocker needs operator input."
    2715    )
    2716
    2717
    2718def _workspace_chat_job_dossier(db: AgentDB, jobs: list[dict[str, Any]], *, limit: int = 8) -> str:
    2719    """Compact job context for the left-side workspace chat model."""
    2720
    2721    if not jobs:
    2722        return "No worker jobs yet."
    2723    sections: list[str] = []
    2724    for index, job in enumerate(jobs[:limit], start=1):
    2725        job_id = str(job.get("id") or "")
    2726        metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    2727        counts = _safe_job_counts(db, job_id)
    2728        findings = _metadata_records(job, "finding_ledger")
    2729        sources = _metadata_records(job, "source_ledger")
    2730        tasks = _metadata_records(job, "task_queue")
    2731        experiments = _metadata_records(job, "experiment_ledger")
    2732        lessons = _metadata_records(job, "lessons")
    2733        roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
    2734        milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
    2735        open_tasks = sum(1 for task in tasks if str(task.get("status") or "open") in {"open", "active", "blocked"})
    2736        artifacts = _safe_list_artifacts(db, job_id, limit=3)
    2737        events = _safe_list_events(db, job_id, limit=40)
    2738        steps = _safe_list_steps(db, job_id, limit=1)
    2739        current_task = _workspace_current_task(tasks)
    2740        recent_outcomes = _workspace_recent_outcomes(events, limit=5)
    2741        latest_outputs = [
    2742            _one_line(str(artifact.get("title") or artifact.get("id") or "saved output"), 160)
    2743            for artifact in artifacts[:3]
    2744        ]
    2745        progress = (
    2746            f"actions={counts.get('steps', 0)} outputs={counts.get('artifacts', 0)} "
    2747            f"findings={len(findings)} sources={len(sources)} tasks={len(tasks)}/{open_tasks} open "
    2748            f"experiments={len(experiments)} lessons={len(lessons)} memory={counts.get('memory', 0)} "
    2749            f"roadmap={len(milestones)}"
    2750        )
    2751        latest_step = _one_line(_step_line(steps[-1]), 180) if steps else "no worker steps yet"
    2752        lines = [
    2753            f"{index}. {job.get('title') or job_id} | state={job.get('status') or 'unknown'} kind={job.get('kind') or 'generic'}",
    2754            f"   objective: {_one_line(job.get('objective') or '', 220)}",
    2755            f"   progress: {progress}",
    2756            f"   latest step: {latest_step}",
    2757        ]
    2758        if current_task:
    2759            lines.append(f"   active task: {current_task}")
    2760        if latest_outputs:
    2761            lines.append(f"   latest outputs: {'; '.join(latest_outputs)}")
    2762        if recent_outcomes:
    2763            lines.append(f"   recent outcomes: {'; '.join(recent_outcomes)}")
    2764        sections.append("\n".join(lines))
    2765    if len(jobs) > limit:
    2766        sections.append(f"... {len(jobs) - limit} more job(s) available.")
    2767    return "\n\n".join(sections)
    2768
    2769
    2770def _safe_job_counts(db: AgentDB, job_id: str) -> dict[str, int]:
    2771    if not job_id:
    2772        return {"steps": 0, "artifacts": 0, "memory": 0, "events": 0}
    2773    try:
    2774        return db.job_record_counts(job_id)
    2775    except Exception:
    2776        return {"steps": 0, "artifacts": 0, "memory": 0, "events": 0}
    2777
    2778
    2779def _safe_list_artifacts(db: AgentDB, job_id: str, *, limit: int) -> list[dict[str, Any]]:
    2780    try:
    2781        return db.list_artifacts(job_id, limit=limit)
    2782    except Exception:
    2783        return []
    2784
    2785
    2786def _safe_list_events(db: AgentDB, job_id: str, *, limit: int) -> list[dict[str, Any]]:
    2787    try:
    2788        return db.list_timeline_events(job_id, limit=limit)
    2789    except Exception:
    2790        return []
    2791
    2792
    2793def _safe_list_steps(db: AgentDB, job_id: str, *, limit: int) -> list[dict[str, Any]]:
    2794    try:
    2795        return db.list_steps(job_id=job_id, limit=limit)
    2796    except Exception:
    2797        return []
    2798
    2799
    2800def _workspace_current_task(tasks: list[dict[str, Any]]) -> str:
    2801    visible = [
    2802        task
    2803        for task in tasks
    2804        if str(task.get("status") or "open") in {"active", "open", "blocked"}
    2805    ]
    2806    if not visible:
    2807        return ""
    2808    visible.sort(
    2809        key=lambda task: (
    2810            {"active": 0, "open": 1, "blocked": 2}.get(str(task.get("status") or "open"), 9),
    2811            -int(task.get("priority") or 0),
    2812        )
    2813    )
    2814    task = visible[0]
    2815    status = str(task.get("status") or "open")
    2816    contract = str(task.get("output_contract") or "")
    2817    suffix = f" [{contract}]" if contract else ""
    2818    return _one_line(f"{status} {task.get('title') or 'task'}{suffix}", 180)
    2819
    2820
    2821def _workspace_recent_outcomes(events: list[dict[str, Any]], *, limit: int) -> list[str]:
    2822    outcomes: list[str] = []
    2823    seen: set[str] = set()
    2824    for event in reversed(events):
    2825        parsed = _model_update_event_parts(event, width=240, compact=True)
    2826        if not parsed:
    2827            continue
    2828        label, text, _clock = parsed
    2829        if label == "DONE":
    2830            continue
    2831        piece = _one_line(f"{label.lower()} {text}", 180)
    2832        if piece in seen:
    2833            continue
    2834        seen.add(piece)
    2835        outcomes.append(piece)
    2836        if len(outcomes) >= limit:
    2837            break
    2838    return outcomes
    2839
    2840
    2841def _reply_to_workspace_chat(message: str) -> Any:
    2842    from nipux_cli.llm import OpenAIChatLLM
    2843
    2844    db, config = _db()
    2845    try:
    2846        jobs = db.list_jobs()[:12]
    2847        job_dossier = _workspace_chat_job_dossier(db, jobs)
    2848        workspace_events = _workspace_chat_events()[-12:]
    2849        history_lines = [
    2850            f"- {event.get('event_type')} {event.get('title')}: {_one_line(event.get('body') or '', 220)}"
    2851            for event in workspace_events
    2852        ]
    2853        messages = [
    2854            {
    2855                "role": "system",
    2856                "content": (
    2857                    "You are Nipux, the workspace chat model for a generic long-running agent CLI. "
    2858                    "Your job is to help the operator create, start, inspect, pause, resume, and steer worker jobs. "
    2859                    "You know the CLI concepts: jobs are long-running workers; artifacts are saved outputs; outcomes summarize durable progress; "
    2860                    "the updates page shows durable worker outcomes; the jobs page shows state, outputs, tasks, memory, findings, sources, experiments, and cost. "
    2861                    "Answer job-status questions from the job dossier. Mention concrete outputs, tasks, measurements, sources, blockers, and next branches when present. "
    2862                    "When the operator asks you to do new work, explain that Nipux will spin up a worker job; the harness will create the job from plain language. "
    2863                    "Keep replies concise, concrete, and operator-facing. Do not expose hidden chain-of-thought."
    2864                ),
    2865            },
    2866            {
    2867                "role": "user",
    2868                "content": (
    2869                    f"Model: {config.model.model}\n"
    2870                    f"Endpoint: {config.model.base_url}\n"
    2871                    f"Tools: browser={config.tools.browser}, web={config.tools.web}, CLI={config.tools.shell}, files={config.tools.files}\n\n"
    2872                    f"Job dossier:\n{job_dossier}\n\n"
    2873                    f"Recent workspace chat:\n{chr(10).join(history_lines) or 'None yet.'}\n\n"
    2874                    f"Operator message:\n{message}"
    2875                ),
    2876            },
    2877        ]
    2878    finally:
    2879        db.close()
    2880    return OpenAIChatLLM(config.model).complete_response(messages=messages)
    2881
    2882
    2883def _handle_chat_control_intent(job_id: str, line: str, *, quiet: bool = False) -> tuple[bool, str] | None:
    2884    return _controller_handle_chat_control_intent(job_id, line, deps=_chat_controller_deps(), quiet=quiet)
    2885
    2886
    2887def _maybe_spawn_job_from_chat(job_id: str, message: str, *, quiet: bool = False) -> str:
    2888    return _controller_maybe_spawn_job_from_chat(job_id, message, deps=_chat_controller_deps(), quiet=quiet)
    2889
    2890
    2891def _queue_chat_note(job_id: str, message: str, *, mode: str = "steer", quiet: bool = False) -> None:
    2892    _controller_queue_chat_note(job_id, message, deps=_chat_controller_deps(), mode=mode, quiet=quiet)
    2893
    2894
    2895def _chat_controller_deps() -> ChatControllerDeps:
    2896    return ChatControllerDeps(
    2897        db_factory=_db,
    2898        reply_fn=_reply_to_chat,
    2899        create_job=_create_job,
    2900        write_shell_state=_write_shell_state,
    2901        start_daemon=_start_worker_from_chat_context,
    2902        capture_command=_capture_chat_command,
    2903        compact_command_output=_compact_command_output,
    2904        friendly_error_text=_friendly_error_text,
    2905    )
    2906
    2907
    2908def _chat_command_deps() -> ChatCommandDeps:
    2909    return ChatCommandDeps(
    2910        db_factory=_db,
    2911        jobs=cmd_jobs,
    2912        history=cmd_history,
    2913        events=cmd_events,
    2914        logs=cmd_logs,
    2915        updates=cmd_updates,
    2916        artifacts=cmd_artifacts,
    2917        artifact=cmd_artifact,
    2918        lessons=cmd_lessons,
    2919        findings=cmd_findings,
    2920        tasks=cmd_tasks,
    2921        roadmap=cmd_roadmap,
    2922        experiments=cmd_experiments,
    2923        sources=cmd_sources,
    2924        memory=cmd_memory,
    2925        metrics=cmd_metrics,
    2926        activity=cmd_activity,
    2927        digest=cmd_digest,
    2928        status=cmd_status,
    2929        usage=cmd_usage,
    2930        handle_setting=_handle_chat_setting_command,
    2931        doctor=cmd_doctor,
    2932        init=cmd_init,
    2933        health=cmd_health,
    2934        start=_start_worker_from_chat_namespace,
    2935        ensure_job_runnable=_ensure_job_runnable,
    2936        run=cmd_run,
    2937        restart=cmd_restart,
    2938        work=cmd_work,
    2939        pause=cmd_pause,
    2940        resume=cmd_resume,
    2941        cancel=cmd_cancel,
    2942        queue_note=_queue_chat_note,
    2943        create_job=_create_job,
    2944        focus=cmd_focus,
    2945        delete=cmd_delete,
    2946    )
    2947
    2948
    2949def _reply_to_chat(job_id: str, message: str) -> Any:
    2950    from nipux_cli.llm import OpenAIChatLLM
    2951
    2952    db, config = _db()
    2953    try:
    2954        job = db.get_job(job_id)
    2955        messages = _build_chat_messages(db, job, message)
    2956    finally:
    2957        db.close()
    2958    return OpenAIChatLLM(config.model).complete_response(messages=messages)
    2959
    2960
    2961def cmd_shell(args: argparse.Namespace) -> None:
    2962    _install_readline_history()
    2963    _print_shell_header()
    2964    print()
    2965    if args.status:
    2966        _print_shell_status(limit=args.limit, chars=args.chars)
    2967    while True:
    2968        try:
    2969            line = input(_shell_prompt())
    2970        except EOFError:
    2971            print()
    2972            return
    2973        except KeyboardInterrupt:
    2974            print()
    2975            continue
    2976        if not _run_shell_line(line):
    2977            return
    2978
    2979
    2980def _print_shell_header() -> None:
    2981    print(NIPUX_BANNER)
    2982    print(_rule("="))
    2983    print(_shell_summary())
    2984    print("Type 'chat' to talk, 'history' or 'artifacts' to inspect output, or plain text to steer.")
    2985    print("Trace output is observable state and tool I/O, not hidden chain-of-thought.")
    2986    print(_rule("="))
    2987
    2988
    2989def _shell_summary() -> str:
    2990    db, config = _db()
    2991    try:
    2992        daemon = daemon_lock_status(config.runtime.home / "agentd.lock")
    2993        job_id = _default_job_id(db)
    2994        if not job_id:
    2995            focus = "no jobs"
    2996        else:
    2997            job = db.get_job(job_id)
    2998            state = _job_display_state(job, bool(daemon["running"]))
    2999            focus = f"{job['title']} [job {state} | worker {_worker_label(job, bool(daemon['running']))}]"
    3000        daemon_text = "running" if daemon["running"] else "stopped"
    3001        return f"daemon: {daemon_text} | model: {config.model.model} | focus: {focus}"
    3002    finally:
    3003        db.close()
    3004
    3005
    3006def _shell_prompt() -> str:
    3007    db, _ = _db()
    3008    try:
    3009        job_id = _default_job_id(db)
    3010        if not job_id:
    3011            return "nipux> "
    3012        job = db.get_job(job_id)
    3013        title = str(job.get("title") or job_id).strip()[:22]
    3014        daemon = daemon_lock_status(load_config().runtime.home / "agentd.lock")
    3015        worker = _worker_label(job, bool(daemon["running"]))
    3016        return f"nipux[{title}:{worker}]> "
    3017    except Exception:
    3018        return "nipux> "
    3019    finally:
    3020        db.close()
    3021
    3022
    3023def _install_readline_history() -> None:
    3024    try:
    3025        import atexit
    3026        import readline
    3027    except ImportError:
    3028        return
    3029    config = load_config()
    3030    config.ensure_dirs()
    3031    history_path = config.runtime.home / "shell_history"
    3032    try:
    3033        readline.read_history_file(history_path)
    3034    except OSError:
    3035        pass
    3036    atexit.register(readline.write_history_file, history_path)
    3037
    3038
    3039def _print_shell_status(*, limit: int, chars: int) -> None:
    3040    db, config = _db()
    3041    try:
    3042        state = collect_dashboard_state(db, config, limit=limit)
    3043        print(render_dashboard(state, width=_terminal_width(), chars=chars), end="")
    3044        print()
    3045    finally:
    3046        db.close()
    3047
    3048
    3049def _print_shell_help() -> None:
    3050    _render_shell_help(rule=_rule)
    3051
    3052
    3053def _run_shell_line(line: str) -> bool:
    3054    line = line.strip()
    3055    if not line:
    3056        return True
    3057    if line in {"exit", "quit", ":q"}:
    3058        return False
    3059    if line in {"help", "?", "commands"}:
    3060        _print_shell_help()
    3061        return True
    3062    if line == "clear":
    3063        print("\033[2J\033[H", end="")
    3064        return True
    3065    try:
    3066        tokens = shlex.split(line)
    3067    except ValueError as exc:
    3068        print(f"parse error: {exc}")
    3069        return True
    3070    if tokens and tokens[0] == "nipux":
    3071        tokens = tokens[1:]
    3072    if not tokens:
    3073        return True
    3074    natural = natural_command_for(" ".join(tokens))
    3075    if natural:
    3076        tokens = [natural]
    3077    if tokens[0] == "ls":
    3078        tokens[0] = "jobs"
    3079    if tokens[0] == "new":
    3080        tokens[0] = "create"
    3081    if tokens[0] == "focus" and len(tokens) > 1 and tokens[1].lower() in {"on", "more", "only"}:
    3082        _steer_default_job(line)
    3083        return True
    3084    if tokens[0] not in SHELL_COMMAND_NAMES and tokens[0] not in SHELL_BUILTINS:
    3085        _steer_default_job(line)
    3086        return True
    3087    try:
    3088        parser = build_parser()
    3089        parsed = parser.parse_args(tokens)
    3090        if parsed.func is cmd_shell:
    3091            print("already in nipux shell")
    3092            return True
    3093        parsed.func(parsed)
    3094    except SystemExit as exc:
    3095        code = exc.code if isinstance(exc.code, int) else 1
    3096        if code:
    3097            print(f"command exited with status {code}")
    3098    return True
    3099
    3100
    3101def _steer_default_job(message: str) -> None:
    3102    db, _ = _db()
    3103    try:
    3104        job_id = _default_job_id(db)
    3105        if not job_id:
    3106            print('No focused job. Create one first, or run: create "objective"')
    3107            return
    3108        job = db.get_job(job_id)
    3109        entry = db.append_operator_message(job_id, message, source="shell")
    3110        print(f"waiting for {job['title']}: {entry['message']}")
    3111        print("Waiting for the next worker step.")
    3112    finally:
    3113        db.close()
    3114
    3115
    3116def build_parser() -> argparse.ArgumentParser:
    3117    return build_arg_parser(
    3118        handlers={
    3119            "init": cmd_init,
    3120            "update": cmd_update,
    3121            "uninstall": cmd_uninstall,
    3122            "create": cmd_create,
    3123            "jobs": cmd_jobs,
    3124            "focus": cmd_focus,
    3125            "rename": cmd_rename,
    3126            "delete": cmd_delete,
    3127            "chat": cmd_chat,
    3128            "shell": cmd_shell,
    3129            "steer": cmd_steer,
    3130            "pause": cmd_pause,
    3131            "resume": cmd_resume,
    3132            "cancel": cmd_cancel,
    3133            "status": cmd_status,
    3134            "health": cmd_health,
    3135            "history": cmd_history,
    3136            "events": cmd_events,
    3137            "dashboard": cmd_dashboard,
    3138            "start": cmd_start,
    3139            "stop": cmd_stop,
    3140            "restart": cmd_restart,
    3141            "browser_dashboard": cmd_browser_dashboard,
    3142            "autostart": cmd_autostart,
    3143            "service": cmd_service,
    3144            "artifacts": cmd_artifacts,
    3145            "artifact": cmd_artifact,
    3146            "lessons": cmd_lessons,
    3147            "learn": cmd_learn,
    3148            "findings": cmd_findings,
    3149            "tasks": cmd_tasks,
    3150            "roadmap": cmd_roadmap,
    3151            "experiments": cmd_experiments,
    3152            "sources": cmd_sources,
    3153            "memory": cmd_memory,
    3154            "metrics": cmd_metrics,
    3155            "usage": cmd_usage,
    3156            "logs": cmd_logs,
    3157            "activity": cmd_activity,
    3158            "updates": cmd_updates,
    3159            "watch": cmd_watch,
    3160            "run_one": cmd_run_one,
    3161            "work": cmd_work,
    3162            "run": cmd_run,
    3163            "digest": cmd_digest,
    3164            "daily_digest": cmd_daily_digest,
    3165            "daemon": cmd_daemon,
    3166            "doctor": cmd_doctor,
    3167        },
    3168        version=__version__,
    3169        default_context_length=DEFAULT_CONTEXT_LENGTH,
    3170    )
    3171
    3172
    3173def main(argv: list[str] | None = None) -> None:
    3174    argv = sys.argv[1:] if argv is None else argv
    3175    try:
    3176        if not argv:
    3177            cmd_home(argparse.Namespace(history_limit=12))
    3178            return
    3179        parser = build_parser()
    3180        args = parser.parse_args(argv)
    3181        args.func(args)
    3182    except KeyboardInterrupt:
    3183        print()
    3184        return
    3185
    3186
    3187if __name__ == "__main__":
    3188    main()

nipux_cli/cli_help.py 92 lines

       1"""Help text and static branding for the Nipux command console."""
       2
       3from __future__ import annotations
       4
       5from typing import Callable
       6
       7
       8NIPUX_BANNER = r"""
       9 _   _ _                  ____ _     ___
      10| \ | (_)_ __  _   ___  _/ ___| |   |_ _|
      11|  \| | | '_ \| | | \ \/ / |   | |    | |
      12| |\  | | |_) | |_| |>  <| |___| |___ | |
      13|_| \_|_| .__/ \__,_/_/\_\\____|_____|___|
      14        |_|
      15""".strip("\n")
      16
      17
      18def print_shell_help(*, rule: Callable[[str], str]) -> None:
      19    print(NIPUX_BANNER)
      20    print(rule("="))
      21    _print_group(
      22        "Jobs",
      23        (
      24            'create "objective" --title TITLE',
      25            "ls",
      26            "focus [JOB_TITLE]",
      27            "rename JOB_TITLE --title NEW_TITLE",
      28            "delete JOB_TITLE",
      29            "chat [JOB_TITLE]",
      30            "steer [--job JOB_TITLE] MESSAGE",
      31            "pause [JOB_TITLE] [note...]",
      32            "resume [JOB_TITLE]",
      33            "cancel [JOB_TITLE] [note...]",
      34        ),
      35    )
      36    _print_group(
      37        "Inspect",
      38        (
      39            "status [JOB_TITLE]",
      40            "health",
      41            "history [JOB_TITLE]",
      42            "events [JOB_TITLE] [--follow] [--json]",
      43            "activity [JOB_TITLE] [--follow]",
      44            "updates [JOB_TITLE]",
      45            "outputs [JOB_TITLE] --verbose",
      46            "findings [JOB_TITLE]",
      47            "tasks [JOB_TITLE]",
      48            "roadmap [JOB_TITLE]",
      49            "experiments [JOB_TITLE]",
      50            "sources [JOB_TITLE]",
      51            "memory [JOB_TITLE]",
      52            "metrics [JOB_TITLE]",
      53            "usage [JOB_TITLE]",
      54            "artifacts [JOB_TITLE]",
      55            "artifact QUERY_OR_TITLE",
      56            "lessons [JOB_TITLE]",
      57        ),
      58    )
      59    _print_group(
      60        "Worker",
      61        (
      62            "work [JOB_TITLE] --steps N [--verbose]",
      63            "run [JOB_TITLE] --poll-seconds N",
      64            "start --poll-seconds N",
      65            "restart --poll-seconds N",
      66            "stop  # daemon",
      67            "stop [JOB_TITLE]  # pause job",
      68        ),
      69    )
      70    _print_group(
      71        "System",
      72        (
      73            "learn [--job JOB_TITLE] LESSON",
      74            "digest JOB_TITLE",
      75            "daily-digest",
      76            "update",
      77            "service install|status|uninstall",
      78            "autostart install|status|uninstall",
      79            "dashboard [JOB_TITLE] --no-follow",
      80            "doctor --check-model",
      81            "browser-dashboard --port 4848",
      82            "help",
      83            "exit",
      84        ),
      85    )
      86
      87
      88def _print_group(title: str, commands: tuple[str, ...]) -> None:
      89    print(title)
      90    for command in commands:
      91        print(f"  {command}")
      92    print()

nipux_cli/cli_render.py 280 lines

       1"""Reusable text renderers for non-frame CLI commands."""
       2
       3from __future__ import annotations
       4
       5import json
       6import os
       7import shutil
       8import textwrap
       9from pathlib import Path
      10from typing import Any
      11
      12from nipux_cli.event_render import event_display_parts
      13from nipux_cli.tui_event_format import clean_step_summary
      14from nipux_cli.tui_status import job_display_state, worker_label
      15from nipux_cli.tui_style import _accent, _event_badge, _fancy_ui, _muted, _one_line, _status_badge
      16
      17
      18def clip_json(value: Any, limit: int) -> str:
      19    text = json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True)
      20    if len(text) <= limit:
      21        return text
      22    return text[:limit] + f"\n... truncated {len(text) - limit} chars"
      23
      24
      25def print_step(step: dict[str, Any], *, verbose: bool = False, chars: int = 4000) -> None:
      26    tool = step.get("tool_name") or "-"
      27    summary = _one_line(clean_step_summary(step.get("summary") or ""), chars)
      28    error = _one_line(step["error"], chars) if step.get("error") else ""
      29    print(f"step #{step['step_no']} {step['started_at']} {step['status']} {step['kind']} {tool}")
      30    if summary:
      31        print(f"  summary: {summary}")
      32    if error:
      33        print(f"  error: {error}")
      34    output_data = step.get("output") or {}
      35    if not verbose and isinstance(output_data, dict):
      36        artifact_id = output_data.get("artifact_id")
      37        if artifact_id:
      38            print(f"  artifact: {artifact_id} (view with: artifact {artifact_id})")
      39        lesson = output_data.get("lesson") if isinstance(output_data.get("lesson"), dict) else None
      40        if lesson:
      41            print(f"  lesson: {_one_line(lesson.get('lesson') or '', chars)}")
      42        update = output_data.get("update") if isinstance(output_data.get("update"), dict) else None
      43        if update:
      44            print(f"  update: {_one_line(update.get('message') or '', chars)}")
      45        source = output_data.get("source") if isinstance(output_data.get("source"), dict) else None
      46        if source:
      47            print(f"  source: {_one_line(source.get('source') or '', chars)} score={source.get('usefulness_score')}")
      48        if isinstance(output_data.get("findings"), list):
      49            print(f"  findings: {output_data.get('added', 0)} new, {output_data.get('updated', 0)} updated")
      50        checkpoint = output_data.get("auto_checkpoint") if isinstance(output_data.get("auto_checkpoint"), dict) else None
      51        if checkpoint:
      52            print(f"  auto checkpoint: {checkpoint.get('artifact_id')}")
      53    if verbose:
      54        input_data = step.get("input") or {}
      55        if input_data:
      56            print("  input:")
      57            print(clip_json(input_data, chars))
      58        if output_data:
      59            print("  output:")
      60            print(clip_json(output_data, chars))
      61
      62
      63def print_artifact(artifact: dict[str, Any]) -> None:
      64    title = artifact.get("title") or artifact["id"]
      65    print(f"artifact {artifact['created_at']} {artifact['type']} {title}")
      66    print(f"  {artifact['path']}")
      67
      68
      69def print_run(run: dict[str, Any]) -> None:
      70    print(f"run {run['started_at']} {run['status']} {run['id']} {run.get('model') or ''}")
      71    if run.get("error"):
      72        print(f"  error: {run['error']}")
      73
      74
      75def print_wrapped(prefix: str, text: Any, *, width: int, subsequent_indent: str = "") -> None:
      76    content = " ".join(str(text).split())
      77    if not content:
      78        print(prefix.rstrip())
      79        return
      80    available = max(20, min(width, 96) - len(prefix))
      81    wrapped = textwrap.wrap(content, width=available) or [content]
      82    print(prefix + wrapped[0])
      83    for line in wrapped[1:]:
      84        print(subsequent_indent + line)
      85
      86
      87def section_title(title: str, subtitle: str = "") -> str:
      88    text = title.upper()
      89    if subtitle:
      90        text = f"{text} - {_one_line(subtitle, 52)}"
      91    width = min(terminal_width(), 96)
      92    if len(text) >= width - 2:
      93        return text[:width]
      94    if _fancy_ui():
      95        return _accent(f"╭─ {text} " + "─" * max(0, width - len(text) - 4))
      96    return f"{text} " + "-" * max(0, width - len(text) - 1)
      97
      98
      99def print_metric_grid(items: list[tuple[str, Any]]) -> None:
     100    width = min(terminal_width(), 96)
     101    cell_width = 24 if width >= 80 else 18
     102    cells = [f"{label:<12} {value}"[:cell_width].ljust(cell_width) for label, value in items]
     103    columns = max(1, width // cell_width)
     104    for start in range(0, len(cells), columns):
     105        print("  " + "  ".join(cells[start : start + columns]).rstrip())
     106
     107
     108def short_path(path: Path | str, *, max_width: int = 80) -> str:
     109    text = str(path)
     110    home = str(Path.home())
     111    if text.startswith(home + os.sep):
     112        text = "~" + text[len(home) :]
     113    if len(text) <= max_width:
     114        return text
     115    keep = max(12, max_width - 4)
     116    return "..." + text[-keep:]
     117
     118
     119def print_jobs_panel(jobs: list[dict[str, Any]], *, focused_job_id: str, daemon_running: bool) -> None:
     120    print(section_title("Jobs"))
     121    if not jobs:
     122        print("  No jobs yet. Type an objective or use /new OBJECTIVE.")
     123        return
     124    print("  #  job                         state       worker      kind")
     125    for index, item in enumerate(jobs[:8], start=1):
     126        marker = "*" if str(item.get("id")) == focused_job_id else " "
     127        state = job_display_state(item, daemon_running)
     128        worker = worker_label(item, daemon_running)
     129        title = _one_line(item.get("title") or item.get("id") or "job", 27)
     130        print(f"  {marker}{index:<2} {title:<27} {_status_badge(state):<11} {_status_badge(worker):<11} {item.get('kind') or ''}")
     131    if len(jobs) > 8:
     132        print(f"  ... {len(jobs) - 8} more. Use /jobs for the full list.")
     133    print("  switch: /focus JOB_TITLE")
     134
     135
     136def next_operator_action(job: dict[str, Any], daemon_running: bool) -> str:
     137    status = str(job.get("status") or "")
     138    if status == "planning":
     139        return "review the plan, or run when ready"
     140    if status == "cancelled":
     141        return "resume to reopen this job, or delete it"
     142    if status == "paused":
     143        return "resume, then run to continue"
     144    if status in {"queued", "running"} and not daemon_running:
     145        return "run to start background work"
     146    if status in {"queued", "running"} and daemon_running:
     147        return "daemon is active; live steps will stream here"
     148    if status == "completed":
     149        return "inspect history or artifacts"
     150    if status == "failed":
     151        return "resume, then run one worker step to test recovery"
     152    return ""
     153
     154
     155def important_startup_events(events: list[dict[str, Any]], *, limit: int) -> list[dict[str, Any]]:
     156    if len(events) <= limit:
     157        return events
     158    important_types = {
     159        "operator_message",
     160        "agent_message",
     161        "artifact",
     162        "finding",
     163        "task",
     164        "experiment",
     165        "lesson",
     166        "reflection",
     167        "error",
     168        "compaction",
     169    }
     170    selected: list[dict[str, Any]] = []
     171    for event in reversed(events):
     172        if event.get("event_type") in important_types:
     173            selected.append(event)
     174        if len(selected) >= limit:
     175            break
     176    if len(selected) < limit:
     177        for event in reversed(events):
     178            if event not in selected:
     179                selected.append(event)
     180            if len(selected) >= limit:
     181                break
     182    selected.sort(key=lambda event: (str(event.get("created_at") or ""), str(event.get("id") or "")))
     183    return selected
     184
     185
     186def print_event_card(event: dict[str, Any], *, chars: int, artifact_indexes: dict[str, int] | None = None) -> None:
     187    when, label, detail, access = event_display_parts(event, chars=chars, full=False)
     188    artifact_indexes = artifact_indexes or {}
     189    artifact_index = artifact_indexes.get(str(event.get("ref_id") or ""))
     190    if artifact_index and event.get("event_type") == "artifact":
     191        access = f"open: /artifact {artifact_index}"
     192    print(f"  {_event_badge(label):<8} {_muted(when):<16} {_one_line(detail, chars)}")
     193    if access:
     194        print(f"  {'':<8} {'':<16} {access}")
     195
     196
     197def public_event(event: dict[str, Any]) -> dict[str, Any]:
     198    public = dict(event)
     199    public.pop("metadata_json", None)
     200    return public
     201
     202
     203def print_event_details(event: dict[str, Any], *, chars: int) -> None:
     204    metadata = event.get("metadata") if isinstance(event.get("metadata"), dict) else {}
     205    if not metadata:
     206        return
     207    compact = {
     208        key: value
     209        for key, value in metadata.items()
     210        if key not in {"input", "output"} and value not in (None, "", [], {})
     211    }
     212    if compact:
     213        print(f"     meta: {_one_line(json.dumps(compact, ensure_ascii=False, sort_keys=True, default=str), chars)}")
     214    if isinstance(metadata.get("input"), dict):
     215        print(f"     input: {_one_line(json.dumps(metadata['input'], ensure_ascii=False, sort_keys=True, default=str), chars)}")
     216    if isinstance(metadata.get("output"), dict):
     217        print(f"     output: {_one_line(json.dumps(metadata['output'], ensure_ascii=False, sort_keys=True, default=str), chars)}")
     218
     219
     220def step_line(step: dict[str, Any], *, chars: int = 180) -> str:
     221    tool = step.get("tool_name") or step.get("kind") or "-"
     222    summary = clean_step_summary(step.get("summary") or step.get("error") or "-")
     223    error = " ERROR" if step.get("error") else ""
     224    return f"#{step['step_no']:<4} {step['status']:<9} {tool:<18} {_one_line(summary, chars)}{error}"
     225
     226
     227def terminal_width() -> int:
     228    return shutil.get_terminal_size((120, 40)).columns
     229
     230
     231def rule(char: str = "-", width: int | None = None) -> str:
     232    return char * min(width or terminal_width(), 96)
     233
     234
     235def json_default(value: Any) -> str:
     236    return str(value)
     237
     238
     239def daemon_state_line(lock: dict[str, Any]) -> str:
     240    metadata = lock.get("metadata") if isinstance(lock.get("metadata"), dict) else {}
     241    if lock.get("running"):
     242        pid = metadata.get("pid") or "unknown"
     243        stale = " stale-runtime" if lock.get("stale") else ""
     244        return f"running pid={pid}{stale}"
     245    return "ready when work starts"
     246
     247
     248def daemon_event_line(event: dict[str, Any], *, chars: int, job_titles: dict[str, str] | None = None) -> str:
     249    at = str(event.get("at") or "?")
     250    name = str(event.get("event") or "?")
     251    pieces = []
     252    job_titles = job_titles or {}
     253    for key in ("status", "tool", "job_id", "step_id", "error_type", "detail", "error"):
     254        value = event.get(key)
     255        if value not in (None, ""):
     256            label = key
     257            if key == "job_id":
     258                value = job_titles.get(str(value), value)
     259            pieces.append(f"{label}={value}")
     260    suffix = " ".join(pieces)
     261    return _one_line(f"{at} {name} {suffix}".strip(), chars)
     262
     263
     264def job_ref_text(value: Any) -> str | None:
     265    if value is None:
     266        return None
     267    if isinstance(value, list):
     268        text = " ".join(str(item) for item in value)
     269    else:
     270        text = str(value)
     271    text = " ".join(text.split())
     272    return text or None
     273
     274
     275def note_text(value: Any) -> str:
     276    if value is None:
     277        return ""
     278    if isinstance(value, list):
     279        return " ".join(str(item) for item in value).strip()
     280    return str(value).strip()

nipux_cli/cli_state.py 126 lines

       1"""Persistent CLI focus state and job lookup helpers."""
       2
       3from __future__ import annotations
       4
       5import hashlib
       6import json
       7from datetime import datetime, timezone
       8from pathlib import Path
       9from typing import Any
      10
      11from nipux_cli.config import AppConfig, load_config
      12from nipux_cli.db import AgentDB
      13
      14
      15def default_job_id(db: AgentDB) -> str | None:
      16    configured = configured_focus_job_id(db)
      17    if configured:
      18        return configured
      19    jobs = db.list_jobs()
      20    for status in ("running", "queued", "planning", "paused", "failed", "completed"):
      21        for job in jobs:
      22            if job.get("status") == status:
      23                return str(job["id"])
      24    return str(jobs[0]["id"]) if jobs else None
      25
      26
      27def configured_focus_job_id(db: AgentDB) -> str | None:
      28    job_id = read_shell_state().get("focus_job_id")
      29    if not isinstance(job_id, str) or not job_id:
      30        return None
      31    try:
      32        db.get_job(job_id)
      33    except KeyError:
      34        return None
      35    return job_id
      36
      37
      38def find_job(db: AgentDB, query: str) -> dict[str, Any] | None:
      39    needle = " ".join(query.split()).lower()
      40    if not needle:
      41        return None
      42    jobs = db.list_jobs()
      43    for job in jobs:
      44        if str(job["id"]).lower() == needle:
      45            return job
      46    for job in jobs:
      47        if str(job.get("title") or "").lower() == needle:
      48            return job
      49    for job in jobs:
      50        if needle in str(job.get("title") or "").lower():
      51            return job
      52    return None
      53
      54
      55def shell_state_path() -> Path:
      56    config = load_config()
      57    config.ensure_dirs()
      58    return config.runtime.home / "shell_state.json"
      59
      60
      61def read_shell_state() -> dict[str, Any]:
      62    path = shell_state_path()
      63    if not path.exists():
      64        return {}
      65    try:
      66        parsed = json.loads(path.read_text(encoding="utf-8"))
      67    except (OSError, json.JSONDecodeError):
      68        return {}
      69    return parsed if isinstance(parsed, dict) else {}
      70
      71
      72def write_shell_state(patch: dict[str, Any]) -> None:
      73    state = read_shell_state()
      74    state.update(patch)
      75    shell_state_path().write_text(
      76        json.dumps(state, ensure_ascii=False, indent=2, sort_keys=True) + "\n", encoding="utf-8"
      77    )
      78
      79
      80def setup_completed() -> bool:
      81    return bool(read_shell_state().get("setup_completed"))
      82
      83
      84def mark_setup_completed() -> None:
      85    write_shell_state({"setup_completed": True})
      86
      87
      88def model_setup_fingerprint(config: AppConfig | None = None) -> str:
      89    config = config or load_config()
      90    key_hash = hashlib.sha256(config.model.api_key.encode("utf-8")).hexdigest() if config.model.api_key else ""
      91    payload = {
      92        "model": config.model.model,
      93        "base_url": config.model.base_url,
      94        "api_key_env": config.model.api_key_env,
      95        "api_key_hash": key_hash,
      96    }
      97    return hashlib.sha256(json.dumps(payload, sort_keys=True).encode("utf-8")).hexdigest()
      98
      99
     100def model_setup_verified(config: AppConfig | None = None) -> bool:
     101    state = read_shell_state()
     102    marker = state.get("model_setup_verified")
     103    if not isinstance(marker, dict) or not marker.get("ok"):
     104        return False
     105    return marker.get("fingerprint") == model_setup_fingerprint(config)
     106
     107
     108def mark_model_setup_verified(config: AppConfig | None = None) -> None:
     109    config = config or load_config()
     110    write_shell_state(
     111        {
     112            "setup_completed": True,
     113            "model_setup_verified": {
     114                "ok": True,
     115                "fingerprint": model_setup_fingerprint(config),
     116                "checked_at": datetime.now(timezone.utc).isoformat(),
     117                "model": config.model.model,
     118                "base_url": config.model.base_url,
     119                "api_key_env": config.model.api_key_env,
     120            },
     121        }
     122    )
     123
     124
     125def clear_model_setup_verified() -> None:
     126    write_shell_state({"model_setup_verified": {}})

nipux_cli/compression.py 246 lines

       1"""Deterministic rolling memory summaries for long-running jobs."""
       2
       3from __future__ import annotations
       4
       5from nipux_cli.db import AgentDB
       6from nipux_cli.memory_graph import rank_memory_nodes
       7from nipux_cli.operator_context import active_prompt_operator_entries
       8
       9
      10def _clip_text(value: object, limit: int) -> str:
      11    text = " ".join(str(value or "").split())
      12    if len(text) <= limit:
      13        return text
      14    return text[: max(0, limit - 3)].rstrip() + "..."
      15
      16
      17def refresh_memory_index(db: AgentDB, job_id: str, *, max_steps: int = 8, max_artifacts: int = 8) -> str:
      18    """Write a compact, artifact-referenced job memory entry.
      19
      20    This is deliberately deterministic. A local model can later improve the
      21    prose, but the daemon should always have a cheap compaction path that runs
      22    after every step and survives model failures.
      23    """
      24
      25    job = db.get_job(job_id)
      26    steps = db.list_steps(job_id=job_id)[-max_steps:]
      27    artifacts = db.list_artifacts(job_id, limit=max_artifacts)
      28    artifact_refs = [artifact["id"] for artifact in artifacts]
      29    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
      30    operator_messages = metadata.get("operator_messages") if isinstance(metadata.get("operator_messages"), list) else []
      31    active_operator = [
      32        entry
      33        for entry in active_prompt_operator_entries(operator_messages)
      34        if str(entry.get("mode") or "steer") in {"steer", "follow_up"}
      35    ][-5:]
      36    operator_notes = [
      37        entry for entry in operator_messages
      38        if isinstance(entry, dict)
      39        and str(entry.get("mode") or "steer") == "note"
      40    ][-3:]
      41
      42    lines = [
      43        f"Job lifecycle status: {job['status']}",
      44        f"Objective: {job['objective']}",
      45        "",
      46        "Active operator context:",
      47    ]
      48    if not active_operator and not operator_notes:
      49        lines.append("- none")
      50    for entry in active_operator:
      51        lines.append(
      52            f"- {entry.get('mode') or 'steer'} {entry.get('event_id') or ''}: "
      53            f"{_clip_text(entry.get('message') or '', 300)}"
      54        )
      55    for entry in operator_notes:
      56        lines.append(f"- note {entry.get('event_id') or ''}: {_clip_text(entry.get('message') or '', 300)}")
      57
      58    lines.extend([
      59        "",
      60        "Recent steps:",
      61    ])
      62    if not steps:
      63        lines.append("- none")
      64    for step in steps:
      65        tool = f" tool={step['tool_name']}" if step.get("tool_name") else ""
      66        summary = step.get("summary") or step.get("error") or ""
      67        lines.append(f"- #{step['step_no']} {step['kind']} {step['status']}{tool}: {_clip_text(summary, 280)}")
      68
      69    lines.extend(["", "Recent artifacts:"])
      70    if not artifacts:
      71        lines.append("- none")
      72    for artifact in artifacts:
      73        title = artifact.get("title") or artifact["id"]
      74        summary = artifact.get("summary") or ""
      75        lines.append(f"- {artifact['id']} {_clip_text(title, 120)} ({artifact['type']}): {_clip_text(summary, 240)}")
      76
      77    tasks = _metadata_list(metadata, "task_queue")
      78    findings = _metadata_list(metadata, "finding_ledger")
      79    sources = _metadata_list(metadata, "source_ledger")
      80    experiments = _metadata_list(metadata, "experiment_ledger")
      81    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
      82    memory_graph = metadata.get("memory_graph") if isinstance(metadata.get("memory_graph"), dict) else {}
      83    memory_nodes = _metadata_list(memory_graph, "nodes")
      84    memory_edges = _metadata_list(memory_graph, "edges")
      85    pending_measurement = (
      86        metadata.get("pending_measurement_obligation")
      87        if isinstance(metadata.get("pending_measurement_obligation"), dict)
      88        and metadata.get("pending_measurement_obligation")
      89        and not metadata.get("pending_measurement_obligation", {}).get("resolved_at")
      90        else {}
      91    )
      92
      93    lines.extend(["", "Durable progress ledgers:"])
      94    lines.append(
      95        "- "
      96        + ", ".join(
      97            [
      98                f"tasks={len(tasks)}",
      99                f"findings={len(findings)}",
     100                f"sources={len(sources)}",
     101                f"experiments={len(experiments)}",
     102                f"memory_nodes={len(memory_nodes)}",
     103                f"roadmap={'yes' if roadmap else 'no'}",
     104            ]
     105        )
     106    )
     107    for node in rank_memory_nodes(memory_nodes, limit=4):
     108        lines.append(
     109            "- memory "
     110            f"{node.get('status') or 'active'} "
     111            f"{node.get('kind') or 'fact'} "
     112            f"{_clip_text(node.get('title') or node.get('key') or '', 120)}"
     113        )
     114    if memory_edges:
     115        lines.append(f"- memory_links={len(memory_edges)}")
     116    for task in _rank_tasks(tasks)[:4]:
     117        lines.append(
     118            "- task "
     119            f"{task.get('status') or 'open'} "
     120            f"{_clip_text(task.get('title') or '', 120)} "
     121            f"contract={task.get('output_contract') or '?'}"
     122        )
     123    for experiment in experiments[-3:]:
     124        metric = ""
     125        if experiment.get("metric_value") not in (None, ""):
     126            metric = (
     127                f" metric={experiment.get('metric_name') or 'value'}="
     128                f"{experiment.get('metric_value')}{experiment.get('metric_unit') or ''}"
     129            )
     130        lines.append(
     131            "- experiment "
     132            f"{experiment.get('status') or 'planned'} "
     133            f"{_clip_text(experiment.get('title') or '', 120)}{metric}"
     134        )
     135    if pending_measurement:
     136        candidates = pending_measurement.get("metric_candidates")
     137        candidate_text = "; ".join(str(item) for item in candidates[:3]) if isinstance(candidates, list) else ""
     138        lines.append(
     139            "- pending_measurement "
     140            f"step=#{pending_measurement.get('source_step_no') or '?'} "
     141            f"tool={pending_measurement.get('tool') or '?'} "
     142            f"{_clip_text(candidate_text or pending_measurement.get('summary') or '', 220)}"
     143        )
     144    for finding in findings[-3:]:
     145        lines.append(f"- finding {_clip_text(finding.get('name') or finding.get('title') or '', 140)}")
     146    for source in sources[-3:]:
     147        score = source.get("usefulness_score")
     148        lines.append(f"- source {_clip_text(source.get('source') or '', 140)} score={score if score is not None else '?'}")
     149    if roadmap:
     150        lines.append(
     151            "- roadmap "
     152            f"{roadmap.get('status') or 'planned'} "
     153            f"{_clip_text(roadmap.get('title') or 'Roadmap', 140)} "
     154            f"current={_clip_text(roadmap.get('current_milestone') or '', 120)}"
     155        )
     156
     157    usage = db.job_token_usage(job_id)
     158    if int(usage.get("calls") or 0) > 0:
     159        lines.extend(["", "Model usage:"])
     160        latest_prompt = _compact_count(usage.get("latest_prompt_tokens"))
     161        latest_total = _compact_count(usage.get("latest_total_tokens"))
     162        context_length = _first_positive_int(usage.get("latest_context_length"), usage.get("context_length"))
     163        context_fraction = _context_fraction(usage, context_length=context_length)
     164        lines.append(
     165            "- "
     166            + ", ".join(
     167                [
     168                    f"calls={usage.get('calls') or 0}",
     169                    f"total_tokens={_compact_count(usage.get('total_tokens'))}",
     170                    f"output_tokens={_compact_count(usage.get('completion_tokens'))}",
     171                    f"latest_context={latest_prompt}",
     172                    f"latest_total={latest_total}",
     173                    f"estimated_calls={usage.get('estimated_calls') or 0}",
     174                ]
     175            )
     176        )
     177        if context_fraction >= 0.65:
     178            lines.append(
     179                "- context_pressure "
     180                f"latest_context={latest_prompt}"
     181                + (f"/{_compact_count(context_length)}" if context_length else "")
     182                + f" ({context_fraction:.0%}); prefer compact ledgers, artifacts, and decisions over raw history."
     183            )
     184
     185    return db.upsert_memory(
     186        job_id=job_id,
     187        key="rolling_state",
     188        summary="\n".join(lines).strip(),
     189        artifact_refs=artifact_refs,
     190    )
     191
     192
     193def _metadata_list(metadata: dict, key: str) -> list[dict]:
     194    values = metadata.get(key)
     195    if not isinstance(values, list):
     196        return []
     197    return [value for value in values if isinstance(value, dict)]
     198
     199
     200def _rank_tasks(tasks: list[dict]) -> list[dict]:
     201    status_rank = {"active": 0, "open": 1, "blocked": 2, "validating": 3, "done": 4, "skipped": 5}
     202    return sorted(
     203        tasks,
     204        key=lambda task: (
     205            status_rank.get(str(task.get("status") or "open"), 9),
     206            -int(task.get("priority") or 0),
     207            str(task.get("title") or ""),
     208        ),
     209    )
     210
     211
     212def _compact_count(value: object) -> str:
     213    try:
     214        number = int(float(value or 0))
     215    except (TypeError, ValueError):
     216        number = 0
     217    if number >= 1_000_000:
     218        return f"{number / 1_000_000:.1f}M"
     219    if number >= 1_000:
     220        return f"{number / 1_000:.1f}K"
     221    return str(number)
     222
     223
     224def _context_fraction(usage: dict, *, context_length: int) -> float:
     225    raw_fraction = usage.get("latest_context_fraction") or usage.get("context_fraction")
     226    try:
     227        fraction = float(raw_fraction)
     228    except (TypeError, ValueError):
     229        fraction = 0.0
     230    if fraction > 0:
     231        return fraction
     232    latest_prompt = _first_positive_int(usage.get("latest_prompt_tokens"), usage.get("prompt_tokens"))
     233    if context_length <= 0 or latest_prompt <= 0:
     234        return 0.0
     235    return latest_prompt / context_length
     236
     237
     238def _first_positive_int(*values: object) -> int:
     239    for value in values:
     240        try:
     241            number = int(float(value or 0))
     242        except (TypeError, ValueError):
     243            continue
     244        if number > 0:
     245            return number
     246    return 0

nipux_cli/config.py 271 lines

       1"""Configuration for the Nipux long-running agent runtime."""
       2
       3from __future__ import annotations
       4
       5import os
       6from dataclasses import dataclass, field
       7from pathlib import Path
       8from typing import Any
       9
      10import yaml
      11
      12
      13DEFAULT_OPENROUTER_MODEL = "openrouter/auto"
      14DEFAULT_OPENROUTER_API_KEY_ENV = <redacted>
      15DEFAULT_MODEL = "local-model"
      16DEFAULT_BASE_URL = "http://localhost:8000/v1"
      17DEFAULT_API_KEY_ENV = <redacted>
      18DEFAULT_CONTEXT_LENGTH = 262_144
      19DEFAULT_REQUEST_TIMEOUT_SECONDS = 300.0
      20
      21
      22def get_agent_home() -> Path:
      23    """Return the Nipux agent home directory."""
      24
      25    value = os.environ.get("NIPUX_HOME", "").strip()
      26    return Path(value).expanduser() if value else Path.home() / ".nipux"
      27
      28
      29def load_env_file(path: str | Path) -> None:
      30    """Load KEY=value pairs from a local env file without overriding the shell."""
      31
      32    env_path = Path(path).expanduser()
      33    if not env_path.exists():
      34        return
      35    ensure_private_file_permissions(env_path)
      36    for raw_line in env_path.read_text(encoding="utf-8").splitlines():
      37        line = raw_line.strip()
      38        if not line or line.startswith("#") or "=" not in line:
      39            continue
      40        key, value = line.split("=", 1)
      41        key = key.strip()
      42        value = value.strip().strip("\"'")
      43        if key and key not in os.environ:
      44            os.environ[key] = value
      45
      46
      47def ensure_private_file_permissions(path: str | Path) -> None:
      48    """Best-effort POSIX privacy for local config/secret files."""
      49
      50    if os.name == "nt":
      51        return
      52    try:
      53        Path(path).chmod(0o600)
      54    except OSError:
      55        pass
      56
      57
      58def ensure_private_dir_permissions(path: str | Path) -> None:
      59    """Best-effort POSIX privacy for the local Nipux state directory."""
      60
      61    if os.name == "nt":
      62        return
      63    try:
      64        Path(path).chmod(0o700)
      65    except OSError:
      66        pass
      67
      68
      69def write_private_text(path: str | Path, text: str) -> None:
      70    """Write text with private file permissions from creation time."""
      71
      72    target = Path(path).expanduser()
      73    target.parent.mkdir(parents=True, exist_ok=True)
      74    flags = os.O_WRONLY | os.O_CREAT | os.O_TRUNC
      75    fd = os.open(target, flags, 0o600)
      76    try:
      77        with os.fdopen(fd, "w", encoding="utf-8") as handle:
      78            fd = -1
      79            handle.write(text)
      80    finally:
      81        if fd >= 0:
      82            os.close(fd)
      83    ensure_private_file_permissions(target)
      84
      85
      86@dataclass(frozen=True)
      87class ModelConfig:
      88    model: str = DEFAULT_MODEL
      89    base_url: str = DEFAULT_BASE_URL
      90    api_key_env: str = DEFAULT_API_KEY_ENV
      91    context_length: int = DEFAULT_CONTEXT_LENGTH
      92    request_timeout_seconds: float = DEFAULT_REQUEST_TIMEOUT_SECONDS
      93    input_cost_per_million: float | None = None
      94    output_cost_per_million: float | None = None
      95
      96    @property
      97    def api_key(self) -> str:
      98        return os.environ.get(self.api_key_env, "")
      99
     100
     101@dataclass(frozen=True)
     102class RuntimeConfig:
     103    home: Path = field(default_factory=get_agent_home)
     104    max_step_seconds: int = 600
     105    max_steps_per_run: int = 1
     106    artifact_inline_char_limit: int = 12_000
     107    daily_digest_enabled: bool = True
     108    daily_digest_time: str = "08:00"
     109    max_job_cost_usd: float | None = None
     110
     111    @property
     112    def state_db_path(self) -> Path:
     113        return self.home / "state.db"
     114
     115    @property
     116    def jobs_dir(self) -> Path:
     117        return self.home / "jobs"
     118
     119    @property
     120    def logs_dir(self) -> Path:
     121        return self.home / "logs"
     122
     123    @property
     124    def digests_dir(self) -> Path:
     125        return self.home / "digests"
     126
     127
     128@dataclass(frozen=True)
     129class ToolAccessConfig:
     130    browser: bool = True
     131    web: bool = True
     132    shell: bool = True
     133    files: bool = True
     134
     135
     136@dataclass(frozen=True)
     137class EmailConfig:
     138    enabled: bool = False
     139    smtp_host: str = ""
     140    smtp_port: int = 587
     141    username: str = ""
     142    password_env: str = "NIPUX_EMAIL_PASSWORD"
     143    from_addr: str = ""
     144    to_addr: str = ""
     145    use_tls: bool = True
     146
     147    @property
     148    def password(self) -> str:
     149        return os.environ.get(self.password_env, "")
     150
     151
     152@dataclass(frozen=True)
     153class AppConfig:
     154    runtime: RuntimeConfig = field(default_factory=RuntimeConfig)
     155    model: ModelConfig = field(default_factory=ModelConfig)
     156    tools: ToolAccessConfig = field(default_factory=ToolAccessConfig)
     157    email: EmailConfig = field(default_factory=EmailConfig)
     158
     159    def ensure_dirs(self) -> None:
     160        for directory in (
     161            self.runtime.home,
     162            self.runtime.jobs_dir,
     163            self.runtime.logs_dir,
     164            self.runtime.digests_dir,
     165        ):
     166            directory.mkdir(parents=True, exist_ok=True)
     167            ensure_private_dir_permissions(directory)
     168
     169
     170def _as_dict(value: Any) -> dict[str, Any]:
     171    return value if isinstance(value, dict) else {}
     172
     173
     174def _optional_float(value: Any) -> float | None:
     175    if value in (None, ""):
     176        return None
     177    return float(value)
     178
     179
     180def load_config(path: str | Path | None = None) -> AppConfig:
     181    """Load config.yaml, falling back to a local OpenAI-compatible endpoint."""
     182
     183    home = get_agent_home()
     184    load_env_file(home / ".env")
     185    cfg_path = Path(path).expanduser() if path else home / "config.yaml"
     186    raw: dict[str, Any] = {}
     187    if cfg_path.exists():
     188        loaded = yaml.safe_load(cfg_path.read_text(encoding="utf-8")) or {}
     189        raw = _as_dict(loaded)
     190
     191    runtime_raw = _as_dict(raw.get("runtime"))
     192    model_raw = _as_dict(raw.get("model"))
     193    tools_raw = _as_dict(raw.get("tools"))
     194    email_raw = _as_dict(raw.get("email"))
     195
     196    runtime_home = Path(runtime_raw.get("home") or home).expanduser()
     197    runtime = RuntimeConfig(
     198        home=runtime_home,
     199        max_step_seconds=int(runtime_raw.get("max_step_seconds", 600)),
     200        max_steps_per_run=int(runtime_raw.get("max_steps_per_run", 1)),
     201        artifact_inline_char_limit=int(runtime_raw.get("artifact_inline_char_limit", 12_000)),
     202        daily_digest_enabled=bool(runtime_raw.get("daily_digest_enabled", True)),
     203        daily_digest_time=str(runtime_raw.get("daily_digest_time") or "08:00"),
     204        max_job_cost_usd=_optional_float(runtime_raw.get("max_job_cost_usd")),
     205    )
     206    model = ModelConfig(
     207        model=str(model_raw.get("name") or model_raw.get("model") or DEFAULT_MODEL),
     208        base_url=str(model_raw.get("base_url") or DEFAULT_BASE_URL).rstrip("/"),
     209        api_key_env=str(model_raw.get("api_key_env") or DEFAULT_API_KEY_ENV),
     210        context_length=int(model_raw.get("context_length", DEFAULT_CONTEXT_LENGTH)),
     211        request_timeout_seconds=float(model_raw.get("request_timeout_seconds", DEFAULT_REQUEST_TIMEOUT_SECONDS)),
     212        input_cost_per_million=_optional_float(model_raw.get("input_cost_per_million")),
     213        output_cost_per_million=_optional_float(model_raw.get("output_cost_per_million")),
     214    )
     215    tools = ToolAccessConfig(
     216        browser=bool(tools_raw.get("browser", True)),
     217        web=bool(tools_raw.get("web", True)),
     218        shell=bool(tools_raw.get("shell", True)),
     219        files=bool(tools_raw.get("files", True)),
     220    )
     221    email = EmailConfig(
     222        enabled=bool(email_raw.get("enabled", False)),
     223        smtp_host=str(email_raw.get("smtp_host") or ""),
     224        smtp_port=int(email_raw.get("smtp_port", 587)),
     225        username=str(email_raw.get("username") or ""),
     226        password_env=str(email_raw.get("password_env") or "NIPUX_EMAIL_PASSWORD"),
     227        from_addr=str(email_raw.get("from_addr") or ""),
     228        to_addr=str(email_raw.get("to_addr") or ""),
     229        use_tls=bool(email_raw.get("use_tls", True)),
     230    )
     231    return AppConfig(runtime=runtime, model=model, tools=tools, email=email)
     232
     233
     234def default_config_yaml(
     235    *,
     236    model: str = DEFAULT_MODEL,
     237    base_url: str = DEFAULT_BASE_URL,
     238    api_key_env: str = DEFAULT_API_KEY_ENV,
     239    context_length: int = DEFAULT_CONTEXT_LENGTH,
     240) -> str:
     241    """Return a starter config file for an OpenAI-compatible model server."""
     242
     243    return (
     244        "model:\n"
     245        f"  name: {model}\n"
     246        f"  base_url: {base_url.rstrip('/')}\n"
     247        f"  api_key_env: {api_key_env}\n"
     248        f"  context_length: {context_length}\n"
     249        "  input_cost_per_million: null\n"
     250        "  output_cost_per_million: null\n"
     251        "runtime:\n"
     252        "  max_step_seconds: 600\n"
     253        "  max_steps_per_run: 1\n"
     254        "  artifact_inline_char_limit: 12000\n"
     255        "  daily_digest_enabled: true\n"
     256        "  daily_digest_time: \"08:00\"\n"
     257        "  max_job_cost_usd: null\n"
     258        "tools:\n"
     259        "  browser: true\n"
     260        "  web: true\n"
     261        "  shell: true\n"
     262        "  files: true\n"
     263        "email:\n"
     264        "  enabled: false\n"
     265        "  smtp_host: \"\"\n"
     266        "  smtp_port: 587\n"
     267        "  username: \"\"\n"
     268        "  password_env: NIPUX_EMAIL_PASSWORD\n"
     269        "  from_addr: \"\"\n"
     270        "  to_addr: \"\"\n"
     271    )

nipux_cli/context_pressure.py 254 lines

       1"""Context-pressure signals for long-running worker prompts."""
       2
       3from __future__ import annotations
       4
       5from datetime import datetime, timezone
       6from typing import Any
       7
       8from nipux_cli.db import AgentDB
       9
      10
      11CONTEXT_PRESSURE_BANDS = (
      12    (0.95, "critical"),
      13    (0.85, "high"),
      14    (0.65, "watch"),
      15)
      16USAGE_TOKEN_BANDS = <redacted>
      17    (20_000_000, "critical"),
      18    (5_000_000, "high"),
      19    (1_000_000, "watch"),
      20)
      21USAGE_CALL_BANDS = (
      22    (2_000, "critical"),
      23    (1_000, "high"),
      24    (200, "watch"),
      25)
      26USAGE_COST_BANDS = (
      27    (10.0, "critical"),
      28    (5.0, "high"),
      29    (1.0, "watch"),
      30)
      31USAGE_BAND_RANK = {"": 0, "watch": 1, "high": 2, "critical": 3}
      32
      33
      34def context_pressure_for_prompt(job: dict[str, Any]) -> str:
      35    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
      36    pressure = metadata.get("context_pressure") if isinstance(metadata.get("context_pressure"), dict) else {}
      37    band = str(pressure.get("band") or "")
      38    if band not in {"watch", "high", "critical"}:
      39        return "None."
      40    prompt_tokens = compact_token_count(pressure.get("prompt_tokens"))
      41    context_length = compact_token_count(pressure.get("context_length"))
      42    context_text = prompt_tokens
      43    if context_length != "0":
      44        context_text = f"{context_text}/{context_length}"
      45    fraction = _as_float(pressure.get("fraction"))
      46    fraction_text = f" ({fraction:.0%})" if fraction else ""
      47    return (
      48        f"Context pressure is {band}: latest prompt used {context_text}{fraction_text}. "
      49        "Keep the next turn compact; prefer durable memory, ledgers, artifact references, and explicit decisions "
      50        "over copying raw history."
      51    )
      52
      53
      54def usage_pressure_for_prompt(job: dict[str, Any], usage: dict[str, Any] | None) -> str:
      55    usage = usage if isinstance(usage, dict) else {}
      56    band = _usage_pressure_band(usage)
      57    if not band:
      58        return "None."
      59    calls = _as_int(usage.get("calls"))
      60    prompt_tokens = _as_int(usage.get("prompt_tokens"))
      61    completion_tokens = _as_int(usage.get("completion_tokens"))
      62    total_tokens = _as_int(usage.get("total_tokens")) or prompt_tokens + completion_tokens
      63    latest_prompt_tokens = _as_int(usage.get("latest_prompt_tokens"))
      64    latest_context_length = _as_int(usage.get("latest_context_length"))
      65    durable_records = _durable_usage_signal_count(job)
      66    tokens_per_record = total_tokens / max(1, durable_records)
      67    latest_context = compact_token_count(latest_prompt_tokens)
      68    if latest_context_length:
      69        latest_context = f"{latest_context}/{compact_token_count(latest_context_length)}"
      70    bits = [
      71        f"calls={calls}",
      72        f"tokens={compact_token_count(total_tokens)}",
      73        f"prompt={compact_token_count(prompt_tokens)}",
      74        f"output={compact_token_count(completion_tokens)}",
      75    ]
      76    if bool(usage.get("has_cost")):
      77        bits.append(f"cost=${_as_float(usage.get('cost')):.4f}")
      78    if latest_prompt_tokens:
      79        bits.append(f"latest_context={latest_context}")
      80    lines = [
      81        f"Cumulative model usage pressure is {band}: " + " ".join(bits) + ".",
      82        (
      83            f"Durable progress records={durable_records}; "
      84            f"approximately {compact_token_count(int(tokens_per_record))} tokens per durable record."
      85        ),
      86        (
      87            "Next action should be high leverage: execute, measure, validate, consolidate, defer, or mark a branch "
      88            "blocked/skipped from concrete evidence. Avoid low-yield retries, broad rereads, or new research unless it "
      89            "directly resolves an active contract or unlocks the next experiment."
      90        ),
      91    ]
      92    return "\n".join(lines)
      93
      94
      95def emit_usage_pressure_update(db: AgentDB, job_id: str, usage: dict[str, Any]) -> None:
      96    band = _usage_pressure_band(usage)
      97    if not band:
      98        return
      99    job = db.get_job(job_id)
     100    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     101    previous = metadata.get("usage_pressure") if isinstance(metadata.get("usage_pressure"), dict) else {}
     102    previous_band = str(previous.get("band") or "")
     103    total_tokens = _as_int(usage.get("total_tokens"))
     104    previous_high_tokens = _as_int(previous.get("high_water_tokens"))
     105    should_emit = (
     106        previous_band != band
     107        or (previous_high_tokens > 0 and total_tokens >= int(previous_high_tokens * 1.5))
     108        or (previous_high_tokens <= 0)
     109    )
     110    pressure = {
     111        "band": band,
     112        "calls": _as_int(usage.get("calls")),
     113        "total_tokens": total_tokens,
     114        "prompt_tokens": _as_int(usage.get("prompt_tokens")),
     115        "completion_tokens": _as_int(usage.get("completion_tokens")),
     116        "cost": _as_float(usage.get("cost")) if bool(usage.get("has_cost")) else None,
     117        "has_cost": bool(usage.get("has_cost")),
     118        "high_water_tokens": max(total_tokens, previous_high_tokens),
     119        "updated_at": datetime.now(timezone.utc).isoformat(),
     120    }
     121    db.update_job_metadata(job_id, {"usage_pressure": pressure})
     122    if not should_emit:
     123        return
     124    cost_text = ""
     125    if pressure["has_cost"]:
     126        cost_text = f" cost=${pressure['cost']:.4f}"
     127    db.append_agent_update(
     128        job_id,
     129        (
     130            f"Usage pressure {band}: {compact_token_count(total_tokens)} tokens across "
     131            f"{pressure['calls']} model calls.{cost_text} Prefer high-leverage actions, measurement, "
     132            "consolidation, or explicit blocked/deferred branches over low-yield churn."
     133        ),
     134        category="update",
     135        metadata={"kind": "usage_pressure", "usage_pressure": pressure},
     136    )
     137
     138
     139def emit_context_pressure_update(db: AgentDB, job_id: str, usage: dict[str, Any]) -> None:
     140    fraction = _as_float(usage.get("context_fraction"))
     141    band = _context_pressure_band(fraction)
     142    if not band:
     143        return
     144    job = db.get_job(job_id)
     145    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     146    previous = metadata.get("context_pressure") if isinstance(metadata.get("context_pressure"), dict) else {}
     147    previous_band = str(previous.get("band") or "")
     148    previous_high = _as_float(previous.get("high_water_fraction"))
     149    should_emit = previous_band != band or fraction >= previous_high + 0.10
     150    prompt_tokens = _as_int(usage.get("prompt_tokens"))
     151    context_length = _as_int(usage.get("context_length"))
     152    pressure = {
     153        "band": band,
     154        "fraction": round(fraction, 6),
     155        "high_water_fraction": round(max(fraction, previous_high), 6),
     156        "prompt_tokens": prompt_tokens,
     157        "context_length": context_length,
     158        "updated_at": datetime.now(timezone.utc).isoformat(),
     159    }
     160    db.update_job_metadata(job_id, {"context_pressure": pressure})
     161    if not should_emit:
     162        return
     163    denominator = f"/{compact_token_count(context_length)}" if context_length else ""
     164    estimated = ", estimated" if usage.get("estimated") else ""
     165    db.append_agent_update(
     166        job_id,
     167        (
     168            f"Context pressure {band}: latest prompt "
     169            f"{compact_token_count(prompt_tokens)}{denominator} ({fraction:.0%}{estimated}). "
     170            "Prefer compact memory, ledgers, artifact references, and explicit decisions over raw history."
     171        ),
     172        category="update",
     173        metadata={"kind": "context_pressure", "context_pressure": pressure},
     174    )
     175
     176
     177def compact_token_count(value: object) -> str:
     178    number = _as_int(value)
     179    if number >= 1_000_000:
     180        return f"{number / 1_000_000:.1f}M"
     181    if number >= 1_000:
     182        return f"{number / 1_000:.1f}K"
     183    return str(number)
     184
     185
     186def _usage_pressure_band(usage: dict[str, Any]) -> str:
     187    total_tokens = _as_int(usage.get("total_tokens"))
     188    if total_tokens <= 0:
     189        total_tokens = _as_int(usage.get("prompt_tokens")) + _as_int(usage.get("completion_tokens"))
     190    calls = _as_int(usage.get("calls"))
     191    cost = _as_float(usage.get("cost")) if bool(usage.get("has_cost")) else 0.0
     192    band = ""
     193    for threshold, candidate in USAGE_TOKEN_BANDS:
     194        if total_tokens >= threshold and USAGE_BAND_RANK[candidate] > USAGE_BAND_RANK[band]:
     195            band = candidate
     196            break
     197    for threshold, candidate in USAGE_CALL_BANDS:
     198        if calls >= threshold and USAGE_BAND_RANK[candidate] > USAGE_BAND_RANK[band]:
     199            band = candidate
     200            break
     201    if bool(usage.get("has_cost")):
     202        for threshold, candidate in USAGE_COST_BANDS:
     203            if cost >= threshold and USAGE_BAND_RANK[candidate] > USAGE_BAND_RANK[band]:
     204                band = candidate
     205                break
     206    return band
     207
     208
     209def _context_pressure_band(fraction: float) -> str:
     210    for threshold, band in CONTEXT_PRESSURE_BANDS:
     211        if fraction >= threshold:
     212            return band
     213    return ""
     214
     215
     216def _durable_usage_signal_count(job: dict[str, Any]) -> int:
     217    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     218    count = 0
     219    for key in ("finding_ledger", "source_ledger", "experiment_ledger", "lessons"):
     220        records = metadata.get(key)
     221        if isinstance(records, list):
     222            count += sum(1 for record in records if isinstance(record, dict))
     223    tasks = metadata.get("task_queue")
     224    if isinstance(tasks, list):
     225        count += sum(
     226            1
     227            for task in tasks
     228            if isinstance(task, dict)
     229            and str(task.get("status") or "open").lower() in {"done", "blocked", "skipped"}
     230            and (task.get("result") or task.get("evidence_needed") or task.get("acceptance_criteria"))
     231        )
     232    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
     233    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
     234    count += sum(
     235        1
     236        for milestone in milestones
     237        if isinstance(milestone, dict)
     238        and str(milestone.get("status") or "planned").lower() in {"active", "validating", "done", "blocked", "skipped"}
     239    )
     240    return count
     241
     242
     243def _as_float(value: Any, default: float = 0.0) -> float:
     244    try:
     245        return float(value)
     246    except (TypeError, ValueError):
     247        return default
     248
     249
     250def _as_int(value: Any, default: int = 0) -> int:
     251    try:
     252        return int(float(value))
     253    except (TypeError, ValueError):
     254        return default

nipux_cli/daemon.py 695 lines

       1"""Daemon runner for restartable background jobs."""
       2
       3from __future__ import annotations
       4
       5import contextlib
       6import fcntl
       7import hashlib
       8import json
       9import os
      10import signal
      11import threading
      12import time
      13from dataclasses import dataclass
      14from datetime import datetime, timezone
      15from email.utils import parsedate_to_datetime
      16from functools import lru_cache
      17from pathlib import Path
      18from typing import Any, Callable
      19
      20from nipux_cli.config import AppConfig, load_config
      21from nipux_cli.db import AgentDB
      22from nipux_cli.digest import write_daily_digest
      23from nipux_cli.doctor import run_doctor
      24from nipux_cli.provider_errors import provider_rate_limited
      25from nipux_cli.scheduling import job_deferred_until, job_is_deferred, job_provider_blocked
      26from nipux_cli.shell_tools import cleanup_registered_shell_processes
      27
      28
      29class DaemonAlreadyRunning(RuntimeError):
      30    pass
      31
      32
      33RUNTIME_CODE_GLOB = "*.py"
      34
      35
      36def runtime_code_file_names() -> tuple[str, ...]:
      37    package_dir = Path(__file__).resolve().parent
      38    return tuple(path.name for path in _runtime_code_paths(package_dir))
      39
      40
      41@lru_cache(maxsize=1)
      42def current_runtime_fingerprint() -> dict[str, Any]:
      43    """Return a stable fingerprint for code that affects daemon behavior."""
      44
      45    from nipux_cli import __version__
      46    from nipux_cli.tools import DEFAULT_REGISTRY
      47    from nipux_cli.worker_policy import SYSTEM_PROMPT, WORKER_PROTOCOL_VERSION
      48
      49    tool_schema = DEFAULT_REGISTRY.openai_tools()
      50    tool_schema_hash = hashlib.sha256(json.dumps(tool_schema, sort_keys=True, default=str).encode("utf-8")).hexdigest()
      51    prompt_hash = hashlib.sha256(SYSTEM_PROMPT.encode("utf-8")).hexdigest()
      52    code_fingerprint = _runtime_code_fingerprint()
      53    payload = {
      54        "nipux_version": __version__,
      55        "worker_protocol": WORKER_PROTOCOL_VERSION,
      56        "tool_schema_hash": tool_schema_hash[:16],
      57        "prompt_hash": prompt_hash[:16],
      58        "code_hash": code_fingerprint["code_hash"],
      59        "code_mtime": code_fingerprint["code_mtime"],
      60        "tool_count": len(DEFAULT_REGISTRY.names()),
      61    }
      62    hash_payload = {key: value for key, value in payload.items() if key != "code_mtime"}
      63    payload["runtime_hash"] = hashlib.sha256(json.dumps(hash_payload, sort_keys=True).encode("utf-8")).hexdigest()[:16]
      64    return payload
      65
      66
      67@lru_cache(maxsize=1)
      68def _runtime_code_fingerprint() -> dict[str, Any]:
      69    package_dir = Path(__file__).resolve().parent
      70    digest = hashlib.sha256()
      71    mtimes: list[float] = []
      72    for path in _runtime_code_paths(package_dir):
      73        name = path.name
      74        digest.update(name.encode("utf-8"))
      75        data = path.read_bytes()
      76        digest.update(hashlib.sha256(data).digest())
      77        mtimes.append(path.stat().st_mtime)
      78    return {
      79        "code_hash": digest.hexdigest()[:16],
      80        "code_mtime": max(mtimes) if mtimes else 0,
      81    }
      82
      83
      84def _runtime_code_paths(package_dir: Path) -> list[Path]:
      85    return sorted(path for path in package_dir.glob(RUNTIME_CODE_GLOB) if path.is_file())
      86
      87
      88RUNTIME_CODE_FILES = runtime_code_file_names()
      89PROVIDER_RECOVERY_PROBE_SECONDS = 300.0
      90WORK_HEARTBEAT_INTERVAL_SECONDS = 15.0
      91
      92
      93def runtime_stale(metadata: dict[str, Any] | None) -> bool:
      94    if not isinstance(metadata, dict):
      95        return False
      96    recorded = metadata.get("runtime")
      97    if not isinstance(recorded, dict):
      98        return True
      99    return recorded.get("runtime_hash") != current_runtime_fingerprint().get("runtime_hash")
     100
     101
     102def _parse_lock_metadata(raw: str) -> dict[str, Any]:
     103    raw = raw.strip()
     104    if not raw:
     105        return {}
     106    try:
     107        parsed = json.loads(raw)
     108        return parsed if isinstance(parsed, dict) else {"raw": raw}
     109    except json.JSONDecodeError:
     110        return {"raw": raw}
     111
     112
     113def daemon_lock_status(path: str | Path) -> dict[str, Any]:
     114    """Return whether another process currently holds the daemon lock."""
     115
     116    path = Path(path)
     117    path.parent.mkdir(parents=True, exist_ok=True)
     118    with path.open("a+", encoding="utf-8") as handle:
     119        handle.seek(0)
     120        metadata = _parse_lock_metadata(handle.read())
     121        try:
     122            fcntl.flock(handle.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
     123        except BlockingIOError:
     124            stale = runtime_stale(metadata)
     125            return {
     126                "running": True,
     127                "lock_path": str(path),
     128                "metadata": metadata,
     129                "stale": stale,
     130                "current_runtime": current_runtime_fingerprint(),
     131                "detail": "daemon lock is held",
     132            }
     133        with contextlib.suppress(OSError):
     134            fcntl.flock(handle.fileno(), fcntl.LOCK_UN)
     135    return {
     136        "running": False,
     137        "lock_path": str(path),
     138        "metadata": metadata,
     139        "stale": False,
     140        "current_runtime": current_runtime_fingerprint(),
     141        "detail": "daemon lock is free",
     142    }
     143
     144
     145@contextlib.contextmanager
     146def single_instance_lock(path: str | Path):
     147    """Hold an exclusive non-blocking daemon lock for this state directory."""
     148
     149    path = Path(path)
     150    path.parent.mkdir(parents=True, exist_ok=True)
     151    with path.open("w+", encoding="utf-8") as handle:
     152        try:
     153            fcntl.flock(handle.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
     154        except BlockingIOError as exc:
     155            raise DaemonAlreadyRunning(f"Another nipux daemon holds {path}") from exc
     156        payload = {
     157            "pid": os.getpid(),
     158            "started_at": datetime.now(timezone.utc).isoformat(),
     159            "runtime": current_runtime_fingerprint(),
     160        }
     161        handle.seek(0)
     162        handle.truncate()
     163        handle.write(json.dumps(payload, sort_keys=True))
     164        handle.flush()
     165        try:
     166            yield handle
     167        finally:
     168            fcntl.flock(handle.fileno(), fcntl.LOCK_UN)
     169
     170
     171def update_lock_metadata(handle, **patch: Any) -> None:
     172    handle.seek(0)
     173    metadata = _parse_lock_metadata(handle.read())
     174    metadata.setdefault("pid", os.getpid())
     175    metadata.setdefault("started_at", datetime.now(timezone.utc).isoformat())
     176    metadata.update(patch)
     177    handle.seek(0)
     178    handle.truncate()
     179    handle.write(json.dumps(metadata, sort_keys=True))
     180    handle.flush()
     181
     182
     183@contextlib.contextmanager
     184def _work_heartbeat(
     185    update_metadata: Callable[..., None],
     186    *,
     187    interval_seconds: float | None = None,
     188    state: str = "working",
     189    **metadata: Any,
     190):
     191    """Keep daemon lock metadata fresh while a worker turn is in progress."""
     192
     193    raw_interval = WORK_HEARTBEAT_INTERVAL_SECONDS if interval_seconds is None else interval_seconds
     194    interval = max(0.01, float(raw_interval))
     195    stop = threading.Event()
     196
     197    def beat() -> None:
     198        while not stop.wait(interval):
     199            update_metadata(
     200                last_heartbeat=datetime.now(timezone.utc).isoformat(),
     201                last_state=state,
     202                runtime=current_runtime_fingerprint(),
     203                **metadata,
     204            )
     205
     206    update_metadata(
     207        last_heartbeat=datetime.now(timezone.utc).isoformat(),
     208        last_state=state,
     209        runtime=current_runtime_fingerprint(),
     210        **metadata,
     211    )
     212    thread = threading.Thread(target=beat, name="nipux-daemon-heartbeat", daemon=True)
     213    thread.start()
     214    try:
     215        yield
     216    finally:
     217        stop.set()
     218        thread.join(timeout=1.0)
     219
     220
     221def append_daemon_event(config: AppConfig, event: str, **fields: Any) -> Path:
     222    """Append a small daemon event that the CLI can tail without parsing stdout."""
     223
     224    config.ensure_dirs()
     225    path = config.runtime.logs_dir / "daemon-events.jsonl"
     226    payload = {
     227        "at": datetime.now(timezone.utc).isoformat(),
     228        "event": event,
     229        **fields,
     230    }
     231    with path.open("a", encoding="utf-8") as handle:
     232        handle.write(json.dumps(payload, ensure_ascii=False, sort_keys=True, default=str) + "\n")
     233    return path
     234
     235
     236def read_daemon_events(config: AppConfig, *, limit: int = 20) -> list[dict[str, Any]]:
     237    path = config.runtime.logs_dir / "daemon-events.jsonl"
     238    if not path.exists():
     239        return []
     240    lines = path.read_text(encoding="utf-8", errors="replace").splitlines()[-limit:]
     241    events: list[dict[str, Any]] = []
     242    for line in lines:
     243        try:
     244            parsed = json.loads(line)
     245        except json.JSONDecodeError:
     246            events.append({"event": "unparseable", "raw": line})
     247            continue
     248        if isinstance(parsed, dict):
     249            events.append(parsed)
     250    return events
     251
     252
     253def fake_step_llm():
     254    from nipux_cli.llm import LLMResponse, ScriptedLLM, ToolCall
     255
     256    nonce = datetime.now(timezone.utc).isoformat()
     257    return ScriptedLLM([
     258        LLMResponse(tool_calls=[
     259            ToolCall(
     260                name="write_artifact",
     261                arguments={
     262                    "title": "daemon-fake-step",
     263                    "type": "text",
     264                    "summary": "Fake daemon step",
     265                    "content": f"This is a fake daemon worker step.\n\nnonce: {nonce}",
     266                },
     267            )
     268        ])
     269    ])
     270
     271
     272@dataclass
     273class Daemon:
     274    config: AppConfig
     275    db: AgentDB
     276
     277    @classmethod
     278    def open(cls, config: AppConfig | None = None) -> "Daemon":
     279        config = config or load_config()
     280        config.ensure_dirs()
     281        return cls(config=config, db=AgentDB(config.runtime.state_db_path))
     282
     283    @property
     284    def lock_path(self) -> Path:
     285        return self.config.runtime.home / "agentd.lock"
     286
     287    def close(self) -> None:
     288        self.db.close()
     289
     290    def next_runnable_job(self) -> dict | None:
     291        """Return the next runnable job by priority/age.
     292
     293        UI focus is intentionally not used here. Focus is for the operator's
     294        chat view; the daemon should keep all runnable jobs advancing.
     295        """
     296
     297        now = datetime.now(timezone.utc)
     298        self._maybe_recover_provider_blocked_jobs(now=now)
     299        runnable_jobs = self.db.list_jobs(statuses=["queued", "running"])
     300        for job in runnable_jobs:
     301            if job_provider_blocked(job):
     302                self.db.update_job_status(
     303                    job["id"],
     304                    "paused",
     305                    metadata_patch={
     306                        "last_note": "Model provider still requires operator action; paused before retrying failed calls.",
     307                    },
     308                )
     309                self.db.append_agent_update(
     310                    job["id"],
     311                    "Model provider still requires operator action; paused before retrying failed calls.",
     312                    category="error",
     313                    metadata={"reason": "llm_provider_blocked"},
     314                )
     315                continue
     316        for job in runnable_jobs:
     317            if job_provider_blocked(job):
     318                continue
     319            if job_is_deferred(job, now=now):
     320                continue
     321            return job
     322        return None
     323
     324    def _maybe_recover_provider_blocked_jobs(self, *, now: datetime) -> None:
     325        paused = [job for job in self.db.list_jobs(statuses=["paused"]) if job_provider_blocked(job)]
     326        due = [job for job in paused if _provider_probe_due(job, now=now)]
     327        if not due:
     328            return
     329        ok, detail = _model_generation_ready(self.config)
     330        timestamp = now.isoformat()
     331        if not ok:
     332            for job in due:
     333                self.db.update_job_status(
     334                    job["id"],
     335                    "paused",
     336                    metadata_patch={
     337                        "provider_last_probe_at": timestamp,
     338                        "provider_last_probe_detail": detail[:1000],
     339                        "last_note": "Model provider still unavailable; daemon will check again later.",
     340                    },
     341                )
     342            append_daemon_event(
     343                self.config,
     344                "provider_recovery_wait",
     345                checked_jobs=len(due),
     346                detail=detail[:500],
     347                next_probe_seconds=PROVIDER_RECOVERY_PROBE_SECONDS,
     348            )
     349            return
     350        for job in paused:
     351            self.db.update_job_status(
     352                job["id"],
     353                "queued",
     354                metadata_patch={
     355                    "provider_last_probe_at": timestamp,
     356                    "provider_unblocked_at": timestamp,
     357                    "last_note": "Model provider recovered; daemon resumed this job.",
     358                },
     359            )
     360            self.db.append_agent_update(
     361                job["id"],
     362                "Model provider recovered; continuing queued work.",
     363                category="progress",
     364                metadata={"reason": "llm_provider_recovered"},
     365            )
     366        append_daemon_event(self.config, "provider_recovered", resumed_jobs=len(paused), detail=detail[:500])
     367
     368    def idle_sleep_seconds(self, *, poll_seconds: float, now: datetime | None = None) -> float:
     369        """Return the next idle sleep, capped by the nearest deferred job wake."""
     370
     371        fallback = max(5.0, poll_seconds)
     372        now = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
     373        due_times: list[datetime] = []
     374        for job in self.db.list_jobs(statuses=["queued", "running"]):
     375            due = job_deferred_until(job, now=now)
     376            if due is not None:
     377                due_times.append(due)
     378        if not due_times:
     379            return fallback
     380        wait_seconds = min((due - now).total_seconds() for due in due_times)
     381        return max(0.5, min(fallback, wait_seconds))
     382
     383    def run_once(self, *, fake: bool = False, verbose: bool = False):
     384        from nipux_cli.worker import run_one_step
     385
     386        job = self.next_runnable_job()
     387        if job is None:
     388            return None
     389        if verbose:
     390            print(f"thinking job={job['id']} title={job['title']} kind={job['kind']}", flush=True)
     391            print(f"objective: {job['objective']}", flush=True)
     392        llm = fake_step_llm() if fake else None
     393        return run_one_step(job["id"], config=self.config, db=self.db, llm=llm)
     394
     395    def send_due_daily_digest(self, *, now: datetime | None = None) -> dict | None:
     396        if not self.config.runtime.daily_digest_enabled:
     397            return None
     398        now = now or datetime.now()
     399        if not _is_digest_due(now, self.config.runtime.daily_digest_time):
     400            return None
     401        day = now.date().isoformat()
     402        target = self.config.email.to_addr or "dry-run"
     403        if self.db.digest_exists(day=day, target=target):
     404            return None
     405        return write_daily_digest(self.config, self.db, day=day)
     406
     407    def run_forever(
     408        self,
     409        *,
     410        fake: bool = False,
     411        poll_seconds: float = 30.0,
     412        quiet: bool = False,
     413        verbose: bool = False,
     414        max_iterations: int | None = None,
     415    ) -> None:
     416        consecutive_failures = 0
     417        iterations = 0
     418        with single_instance_lock(self.lock_path) as lock_handle:
     419            metadata_lock = threading.Lock()
     420
     421            def locked_update_metadata(**patch: Any) -> None:
     422                with metadata_lock:
     423                    update_lock_metadata(lock_handle, **patch)
     424
     425            previous_sigterm = signal.getsignal(signal.SIGTERM)
     426            signal.signal(signal.SIGTERM, _raise_keyboard_interrupt)
     427            cleaned_shell_processes = cleanup_registered_shell_processes(self.config.runtime.home)
     428            recovered = self.db.mark_interrupted_running(reason="daemon recovered abandoned running work from a previous process")
     429            append_daemon_event(
     430                self.config,
     431                "daemon_started",
     432                pid=os.getpid(),
     433                fake=fake,
     434                poll_seconds=poll_seconds,
     435                recovered_steps=recovered["steps"],
     436                recovered_runs=recovered["runs"],
     437                cleaned_shell_processes=len(cleaned_shell_processes),
     438                runtime=current_runtime_fingerprint(),
     439            )
     440            if cleaned_shell_processes:
     441                append_daemon_event(
     442                    self.config,
     443                    "shell_processes_cleaned",
     444                    count=len(cleaned_shell_processes),
     445                    processes=cleaned_shell_processes[:12],
     446                )
     447            if recovered["steps"] or recovered["runs"]:
     448                append_daemon_event(self.config, "stale_work_recovered", **recovered)
     449            if not quiet:
     450                print(f"nipux daemon started; db={self.config.runtime.state_db_path}", flush=True)
     451            try:
     452                while True:
     453                    iterations += 1
     454                    locked_update_metadata(
     455                        last_heartbeat=datetime.now(timezone.utc).isoformat(),
     456                        last_state="checking",
     457                        consecutive_failures=consecutive_failures,
     458                        runtime=current_runtime_fingerprint(),
     459                    )
     460
     461                    try:
     462                        digest = self.send_due_daily_digest()
     463                        if digest:
     464                            append_daemon_event(self.config, "daily_digest", **digest)
     465                            if not quiet:
     466                                print(f"daily_digest {json.dumps(digest, ensure_ascii=False)}", flush=True)
     467                        with _work_heartbeat(
     468                            locked_update_metadata,
     469                            consecutive_failures=consecutive_failures,
     470                        ):
     471                            result = self.run_once(fake=fake, verbose=verbose and not quiet)
     472                    except Exception as exc:
     473                        consecutive_failures += 1
     474                        payload = _exception_payload(exc)
     475                        locked_update_metadata(
     476                            last_heartbeat=datetime.now(timezone.utc).isoformat(),
     477                            last_state="error",
     478                            last_error=payload["error"],
     479                            last_error_type=payload["error_type"],
     480                            consecutive_failures=consecutive_failures,
     481                            runtime=current_runtime_fingerprint(),
     482                        )
     483                        append_daemon_event(self.config, "daemon_error", **payload, consecutive_failures=consecutive_failures)
     484                        if not quiet:
     485                            print(
     486                                f"daemon_error type={payload['error_type']} error={payload['error'][:240]}",
     487                                flush=True,
     488                            )
     489                        _sleep_or_stop(_exception_backoff(exc, poll_seconds, consecutive_failures), max_iterations, iterations)
     490                        if max_iterations is not None and iterations >= max_iterations:
     491                            return
     492                        continue
     493
     494                    if result is None:
     495                        locked_update_metadata(
     496                            last_heartbeat=datetime.now(timezone.utc).isoformat(),
     497                            last_state="idle",
     498                            runtime=current_runtime_fingerprint(),
     499                        )
     500                        idle_sleep = self.idle_sleep_seconds(poll_seconds=poll_seconds)
     501                        if not quiet:
     502                            print(f"idle; sleeping {idle_sleep:g}s", flush=True)
     503                        _sleep_or_stop(idle_sleep, max_iterations, iterations)
     504                    else:
     505                        consecutive_failures = consecutive_failures + 1 if result.status == "failed" else 0
     506                        locked_update_metadata(
     507                            last_heartbeat=datetime.now(timezone.utc).isoformat(),
     508                            last_state="step",
     509                            last_job_id=result.job_id,
     510                            last_run_id=result.run_id,
     511                            last_step_id=result.step_id,
     512                            last_status=result.status,
     513                            last_tool=result.tool_name,
     514                            last_error="" if result.status != "failed" else str(result.result.get("error") or ""),
     515                            last_error_type="" if result.status != "failed" else str(result.result.get("error_type") or ""),
     516                            consecutive_failures=consecutive_failures,
     517                            runtime=current_runtime_fingerprint(),
     518                        )
     519                        detail = result.result.get("error") or result.result.get("artifact_id") or result.result.get("content", "")
     520                        append_daemon_event(
     521                            self.config,
     522                            "step",
     523                            job_id=result.job_id,
     524                            run_id=result.run_id,
     525                            step_id=result.step_id,
     526                            status=result.status,
     527                            tool=result.tool_name,
     528                            detail=str(detail)[:500],
     529                            consecutive_failures=consecutive_failures,
     530                        )
     531                        if not quiet:
     532                            print(
     533                                f"step job={result.job_id} run={result.run_id} step={result.step_id} "
     534                                f"status={result.status} tool={result.tool_name or '-'} detail={str(detail)[:240]}",
     535                                flush=True,
     536                            )
     537                            if verbose:
     538                                print(json.dumps(result.result, ensure_ascii=False, indent=2)[:8000], flush=True)
     539                        sleep_seconds = (
     540                            _step_failure_backoff(result, poll_seconds, consecutive_failures)
     541                            if result.status == "failed"
     542                            else max(0.0, poll_seconds)
     543                        )
     544                        _sleep_or_stop(sleep_seconds, max_iterations, iterations)
     545                    if max_iterations is not None and iterations >= max_iterations:
     546                        return
     547            except KeyboardInterrupt:
     548                interrupted = self.db.mark_interrupted_running(reason="daemon stopped during active work")
     549                locked_update_metadata(
     550                    last_heartbeat=datetime.now(timezone.utc).isoformat(),
     551                    last_state="stopped",
     552                    consecutive_failures=consecutive_failures,
     553                    runtime=current_runtime_fingerprint(),
     554                )
     555                append_daemon_event(self.config, "daemon_stopped", pid=os.getpid(), interrupted_steps=interrupted["steps"], interrupted_runs=interrupted["runs"])
     556                if not quiet:
     557                    print("nipux daemon stopped", flush=True)
     558            finally:
     559                signal.signal(signal.SIGTERM, previous_sigterm)
     560
     561
     562def _is_digest_due(now: datetime, configured_time: str) -> bool:
     563    try:
     564        hour_text, minute_text = configured_time.split(":", 1)
     565        hour = int(hour_text)
     566        minute = int(minute_text)
     567    except ValueError:
     568        hour, minute = 8, 0
     569    return (now.hour, now.minute) >= (hour, minute)
     570
     571
     572def _raise_keyboard_interrupt(signum, frame) -> None:
     573    raise KeyboardInterrupt
     574
     575
     576def _exception_payload(exc: Exception) -> dict[str, str]:
     577    return {
     578        "error": str(exc),
     579        "error_type": type(exc).__name__,
     580    }
     581
     582
     583def _failure_backoff(poll_seconds: float, consecutive_failures: int) -> float:
     584    base = max(1.0, poll_seconds)
     585    return min(60.0, base * min(8, max(1, consecutive_failures)))
     586
     587
     588def _step_failure_backoff(result: Any, poll_seconds: float, consecutive_failures: int) -> float:
     589    """Return a retry delay for failed worker steps.
     590
     591    Worker failures are recorded as failed steps rather than escaping as daemon
     592    exceptions, so they use the same generic throttling path here.
     593    """
     594
     595    fallback = _failure_backoff(poll_seconds, consecutive_failures)
     596    return fallback
     597
     598
     599def _exception_backoff(exc: Exception, poll_seconds: float, consecutive_failures: int) -> float:
     600    fallback = _failure_backoff(poll_seconds, consecutive_failures)
     601    if not _is_rate_limit_error(exc):
     602        return fallback
     603    retry_after = _retry_after_seconds(exc)
     604    if retry_after is None:
     605        return max(fallback, 10.0)
     606    return max(fallback, min(300.0, retry_after))
     607
     608
     609def _is_rate_limit_error(exc: Exception) -> bool:
     610    status_code = getattr(exc, "status_code", None)
     611    if status_code == 429:
     612        return True
     613    return _is_rate_limit_text(f"{type(exc).__name__} {exc}")
     614
     615
     616def _is_rate_limit_text(text: str) -> bool:
     617    return provider_rate_limited(text)
     618
     619
     620def _retry_after_seconds(exc: Exception) -> float | None:
     621    headers = _exception_headers(exc)
     622    for key, value in headers.items():
     623        normalized = key.lower()
     624        if normalized in {"retry-after", "x-ratelimit-reset", "x-rate-limit-reset"}:
     625            parsed = _parse_retry_after(value)
     626            if parsed is not None:
     627                return parsed
     628    return None
     629
     630
     631def _exception_headers(exc: Exception) -> dict[str, str]:
     632    response = getattr(exc, "response", None)
     633    headers = getattr(response, "headers", None)
     634    if headers:
     635        return {str(key): str(value) for key, value in dict(headers).items()}
     636    return {}
     637
     638
     639def _parse_retry_after(value: str) -> float | None:
     640    text = str(value).strip()
     641    if not text:
     642        return None
     643    with contextlib.suppress(ValueError):
     644        number = float(text)
     645        if number > 10_000_000_000:
     646            number = number / 1000
     647        if number > 1_000_000_000:
     648            return max(0.0, number - time.time())
     649        return max(0.0, number)
     650    with contextlib.suppress(ValueError, TypeError, OSError):
     651        parsed = parsedate_to_datetime(text)
     652        if parsed.tzinfo is None:
     653            parsed = parsed.replace(tzinfo=timezone.utc)
     654        return max(0.0, parsed.timestamp() - time.time())
     655    return None
     656
     657
     658def _provider_probe_due(job: dict[str, Any], *, now: datetime) -> bool:
     659    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     660    raw = str(metadata.get("provider_last_probe_at") or "").strip()
     661    if not raw:
     662        return True
     663    with contextlib.suppress(ValueError):
     664        previous = datetime.fromisoformat(raw.replace("Z", "+00:00"))
     665        if previous.tzinfo is None:
     666            previous = previous.replace(tzinfo=timezone.utc)
     667        return (now.astimezone(timezone.utc) - previous.astimezone(timezone.utc)).total_seconds() >= PROVIDER_RECOVERY_PROBE_SECONDS
     668    return True
     669
     670
     671def _model_generation_ready(config: AppConfig) -> tuple[bool, str]:
     672    checks = run_doctor(config=config, check_model=True)
     673    failures = [check for check in checks if not check.ok and check.name in {"model_config", "model_auth", "model_endpoint", "model_generation"}]
     674    if not failures:
     675        return True, "model_generation accepted"
     676    detail = "; ".join(f"{check.name}: {check.detail}" for check in failures)
     677    return False, detail
     678
     679
     680def _sleep_or_stop(seconds: float, max_iterations: int | None, iterations: int) -> None:
     681    if max_iterations is not None and iterations >= max_iterations:
     682        return
     683    time.sleep(seconds)
     684
     685
     686def _focused_job_id(config: AppConfig) -> str | None:
     687    path = config.runtime.home / "shell_state.json"
     688    if not path.exists():
     689        return None
     690    try:
     691        parsed = json.loads(path.read_text(encoding="utf-8"))
     692    except (OSError, json.JSONDecodeError):
     693        return None
     694    job_id = parsed.get("focus_job_id") if isinstance(parsed, dict) else None
     695    return job_id if isinstance(job_id, str) and job_id else None

nipux_cli/daemon_control.py 243 lines

       1"""Daemon process control helpers used by CLI commands."""
       2
       3from __future__ import annotations
       4
       5import argparse
       6import os
       7import signal
       8import subprocess
       9import sys
      10import time
      11from pathlib import Path
      12from typing import Any, Callable
      13
      14from nipux_cli.config import AppConfig, load_config
      15from nipux_cli.cli_state import clear_model_setup_verified, mark_model_setup_verified
      16from nipux_cli.daemon import daemon_lock_status
      17from nipux_cli.doctor import run_doctor
      18from nipux_cli.provider_errors import provider_action_required, provider_rate_limited
      19
      20
      21ReadyFn = Callable[[Any], bool]
      22StartFn = Callable[[argparse.Namespace], None]
      23StopFn = Callable[[Any], bool]
      24PidAliveFn = Callable[[int], bool]
      25
      26
      27def remote_model_preflight_failures(config: Any, *, doctor_fn: Callable[..., list[Any]] = run_doctor) -> list[str]:
      28    blocking = {"model_config", "model_auth", "model_endpoint", "model_generation"}
      29    checks = doctor_fn(config=config, check_model=True)
      30    return [f"{check.name}: {check.detail}" for check in checks if not check.ok and check.name in blocking]
      31
      32
      33def _recoverable_provider_preflight(failures: list[str]) -> bool:
      34    if not failures:
      35        return False
      36    for failure in failures:
      37        name = failure.split(":", 1)[0].strip()
      38        if name != "model_generation":
      39            return False
      40        if not (provider_action_required(failure) or provider_rate_limited(failure)):
      41            return False
      42    return True
      43
      44
      45def recoverable_remote_model_preflight_failures(
      46    config: Any,
      47    *,
      48    doctor_fn: Callable[..., list[Any]] = run_doctor,
      49) -> list[str]:
      50    failures = remote_model_preflight_failures(config, doctor_fn=doctor_fn)
      51    return failures if _recoverable_provider_preflight(failures) else []
      52
      53
      54def provider_preflight_is_recoverable(failures: list[str]) -> bool:
      55    return _recoverable_provider_preflight(failures)
      56
      57
      58def ensure_remote_model_ready_for_worker(
      59    config: Any,
      60    *,
      61    fake: bool,
      62    doctor_fn: Callable[..., list[Any]] = run_doctor,
      63) -> bool:
      64    if fake:
      65        return True
      66    failures = remote_model_preflight_failures(config, doctor_fn=doctor_fn)
      67    if not failures:
      68        mark_model_setup_verified(config)
      69        return True
      70    if _recoverable_provider_preflight(failures):
      71        clear_model_setup_verified()
      72        print("model provider is not ready; starting daemon in recovery monitor mode")
      73        for failure in failures:
      74            print(f"  wait {failure}")
      75        print("The daemon will periodically re-check the configured model and resume provider-blocked jobs when it works.")
      76        return True
      77    clear_model_setup_verified()
      78    print("model is not ready; daemon not started")
      79    for failure in failures:
      80        print(f"  fail {failure}")
      81    print("Run `nipux doctor --check-model` after fixing the model configuration.")
      82    return False
      83
      84
      85def cmd_start_impl(
      86    args: argparse.Namespace,
      87    *,
      88    ready_fn: Callable[[Any, bool], bool],
      89    stop_fn: Callable[[AppConfig, float, bool], bool],
      90) -> None:
      91    config = load_config()
      92    config.ensure_dirs()
      93    status = daemon_lock_status(config.runtime.home / "agentd.lock")
      94    if status["running"]:
      95        metadata = status.get("metadata") or {}
      96        if status.get("stale"):
      97            print(f"nipux daemon stale pid={metadata.get('pid', 'unknown')}; restarting")
      98            stop_fn(config, 5.0, True)
      99            time.sleep(0.5)
     100        else:
     101            print(f"nipux daemon already running pid={metadata.get('pid', 'unknown')}")
     102            return
     103    if not ready_fn(config, bool(args.fake)):
     104        return
     105    log_path = Path(args.log_file).expanduser() if args.log_file else config.runtime.logs_dir / "daemon.log"
     106    log_path.parent.mkdir(parents=True, exist_ok=True)
     107    command = [
     108        sys.executable,
     109        "-m",
     110        "nipux_cli.cli",
     111        "daemon",
     112        "--poll-seconds",
     113        str(args.poll_seconds),
     114    ]
     115    if args.fake:
     116        command.append("--fake")
     117    command.append("--quiet" if args.quiet else "--verbose")
     118    with log_path.open("a", encoding="utf-8") as log_file:
     119        process = subprocess.Popen(
     120            command,
     121            cwd=str(Path.cwd()),
     122            stdout=log_file,
     123            stderr=subprocess.STDOUT,
     124            start_new_session=True,
     125        )
     126    time.sleep(0.5)
     127    status = daemon_lock_status(config.runtime.home / "agentd.lock")
     128    if status["running"]:
     129        metadata = status.get("metadata") or {}
     130        print(f"nipux daemon started pid={metadata.get('pid') or process.pid}")
     131        print(f"log: {log_path}")
     132        return
     133    if process.poll() is None:
     134        print(f"nipux daemon process started pid={process.pid}, waiting for lock")
     135        print(f"log: {log_path}")
     136        return
     137    raise SystemExit(f"nipux daemon exited immediately with code {process.returncode}; see {log_path}")
     138
     139
     140def start_daemon_if_needed_impl(
     141    *,
     142    poll_seconds: float,
     143    fake: bool,
     144    quiet: bool,
     145    log_file: str | None,
     146    start_fn: StartFn,
     147    stop_fn: Callable[[AppConfig, float, bool], bool],
     148) -> None:
     149    config = load_config()
     150    config.ensure_dirs()
     151    status = daemon_lock_status(config.runtime.home / "agentd.lock")
     152    if status["running"]:
     153        metadata = status.get("metadata") or {}
     154        if status.get("stale"):
     155            print(f"daemon stale pid={metadata.get('pid', 'unknown')}; restarting")
     156            stop_fn(config, 5.0, True)
     157            time.sleep(0.5)
     158            start_fn(argparse.Namespace(poll_seconds=poll_seconds, fake=fake, quiet=quiet, log_file=log_file))
     159            return
     160        print(f"daemon already running pid={metadata.get('pid', 'unknown')}")
     161        return
     162    start_fn(argparse.Namespace(poll_seconds=poll_seconds, fake=fake, quiet=quiet, log_file=log_file))
     163
     164
     165def cmd_restart_impl(
     166    args: argparse.Namespace,
     167    *,
     168    start_fn: StartFn,
     169    stop_fn: Callable[[AppConfig, float, bool], bool],
     170) -> None:
     171    config = load_config()
     172    config.ensure_dirs()
     173    stopped = stop_fn(config, float(args.wait), False)
     174    if stopped:
     175        time.sleep(0.5)
     176    start_fn(argparse.Namespace(poll_seconds=args.poll_seconds, fake=args.fake, quiet=args.quiet, log_file=args.log_file))
     177
     178
     179def stop_daemon_process_impl(
     180    config: AppConfig,
     181    *,
     182    wait: float,
     183    quiet: bool,
     184    pid_alive: PidAliveFn,
     185) -> bool:
     186    status = daemon_lock_status(config.runtime.home / "agentd.lock")
     187    if not status["running"]:
     188        if not quiet:
     189            print("nipux daemon is not running")
     190        return False
     191    metadata = status.get("metadata") or {}
     192    pid = metadata.get("pid")
     193    if not isinstance(pid, int):
     194        recovered = _find_single_daemon_process()
     195        if recovered is None:
     196            raise SystemExit("daemon is running but lock file has no pid; stop it from the terminal that owns it")
     197        pid = recovered
     198        if not quiet:
     199            print(f"daemon lock had no pid; recovered daemon pid={pid}")
     200    os.kill(pid, signal.SIGTERM)
     201    deadline = time.time() + wait
     202    while time.time() < deadline:
     203        if not pid_alive(pid):
     204            if not quiet:
     205                print(f"nipux daemon stopped pid={pid}")
     206            return True
     207        time.sleep(0.2)
     208    if not quiet:
     209        print(f"sent SIGTERM to nipux daemon pid={pid}; it may still be shutting down")
     210    return False
     211
     212
     213def _find_single_daemon_process() -> int | None:
     214    """Best-effort recovery for older locks that lost pid metadata."""
     215
     216    try:
     217        result = subprocess.run(["ps", "-eo", "pid=,args="], capture_output=True, text=True, timeout=5)
     218    except (OSError, subprocess.SubprocessError):
     219        return None
     220    if result.returncode != 0:
     221        return None
     222    candidates: list[int] = []
     223    current_pid = os.getpid()
     224    for raw_line in result.stdout.splitlines():
     225        line = raw_line.strip()
     226        if not line:
     227            continue
     228        pid_text, _, command = line.partition(" ")
     229        try:
     230            pid = int(pid_text)
     231        except ValueError:
     232            continue
     233        if pid == current_pid:
     234            continue
     235        normalized = " ".join(command.split())
     236        if "-m nipux_cli.cli daemon" in normalized or " nipux_cli.cli daemon" in normalized:
     237            candidates.append(pid)
     238            continue
     239        parts = normalized.split()
     240        if parts and Path(parts[0]).name == "nipux" and "daemon" in parts[1:]:
     241            candidates.append(pid)
     242    unique = sorted(set(candidates))
     243    return unique[0] if len(unique) == 1 else None

nipux_cli/dashboard.py 493 lines

       1"""Operator-facing dashboard state and rendering."""
       2
       3from __future__ import annotations
       4
       5from collections import Counter
       6from datetime import datetime, timezone
       7from pathlib import Path
       8from textwrap import shorten
       9from typing import Any
      10
      11from nipux_cli.config import AppConfig
      12from nipux_cli.daemon import daemon_lock_status
      13from nipux_cli.db import AgentDB
      14from nipux_cli.operator_context import active_prompt_operator_entries
      15from nipux_cli.scheduling import job_deferred_until
      16from nipux_cli.tools import DEFAULT_REGISTRY
      17
      18
      19def collect_dashboard_state(
      20    db: AgentDB,
      21    config: AppConfig,
      22    *,
      23    job_id: str | None = None,
      24    limit: int = 12,
      25) -> dict[str, Any]:
      26    """Build a serializable snapshot for status and dashboard commands."""
      27
      28    jobs = db.list_jobs()
      29    selected = _select_focus_job(db, jobs, job_id)
      30    job_cards = [_job_card(db, job) for job in jobs]
      31    focus = _focus_state(db, selected, limit=limit) if selected else None
      32    return {
      33        "generated_at": datetime.now(timezone.utc).isoformat(),
      34        "daemon": daemon_lock_status(config.runtime.home / "agentd.lock"),
      35        "runtime": {
      36            "home": str(config.runtime.home),
      37            "state_db": str(config.runtime.state_db_path),
      38            "logs_dir": str(config.runtime.logs_dir),
      39            "model": config.model.model,
      40            "base_url": config.model.base_url,
      41            "tool_count": len(DEFAULT_REGISTRY.names()),
      42        },
      43        "jobs": job_cards,
      44        "focus": focus,
      45    }
      46
      47
      48def render_dashboard(state: dict[str, Any], *, width: int = 120, chars: int = 260) -> str:
      49    """Render a compact terminal dashboard."""
      50
      51    width = max(72, min(width, 160))
      52    line = "-" * width
      53    runtime = state["runtime"]
      54    daemon = state["daemon"]
      55    focus = state.get("focus")
      56    generated_at = _compact_time(state["generated_at"])
      57    daemon_text = _daemon_text(daemon)
      58    lines = [
      59        "Nipux CLI Dashboard".ljust(width - len(generated_at)) + generated_at,
      60        line,
      61        f"daemon: {daemon_text}",
      62        f"model: {runtime['model']} | endpoint: {runtime['base_url']} | tools: {runtime['tool_count']}",
      63        f"home: {runtime['home']}",
      64        "trace: model-visible state, tool calls, outputs, artifacts, and errors. Hidden chain-of-thought is not exposed.",
      65        line,
      66        "Jobs",
      67    ]
      68    jobs = state.get("jobs") or []
      69    if not jobs:
      70        lines.append("  no jobs yet")
      71    else:
      72        lines.append("  title                         state      kind            steps  artifacts  last action")
      73        for job in jobs[:12]:
      74            latest = job.get("latest_step") or {}
      75            last_action = _one_line(latest.get("summary") or latest.get("error") or "-", 42)
      76            display_state = _job_state_text(job, bool(daemon.get("running")))
      77            lines.append(
      78                f"  {_one_line(job['title'], 29):<29} {display_state:<10} {job['kind']:<15} "
      79                f"{job['step_count']:>5} {job['artifact_count']:>10}  {last_action}"
      80            )
      81    if focus:
      82        lines.extend(_render_focus(focus, width=width, chars=chars, daemon_running=bool(daemon.get("running"))))
      83    return "\n".join(lines).rstrip() + "\n"
      84
      85
      86def render_overview(state: dict[str, Any], *, width: int = 100) -> str:
      87    """Render a human-sized status view for the interactive shell."""
      88
      89    width = max(72, min(width, 120))
      90    runtime = state["runtime"]
      91    daemon = state["daemon"]
      92    focus = state.get("focus")
      93    jobs = state.get("jobs") or []
      94    latest_step = ((focus or {}).get("recent_steps") or [{}])[-1] if focus else {}
      95    lines = [
      96        "Nipux Status",
      97        "=" * min(width, 96),
      98        f"daemon: {_daemon_health_text(daemon, latest_step=latest_step)}",
      99        f"model: {runtime['model']}",
     100        f"jobs: {len(jobs)} total | tools: {runtime['tool_count']} | home: {runtime['home']}",
     101    ]
     102    if not focus:
     103        lines.append("focus: no job yet")
     104        lines.append("")
     105        lines.append("next: create \"your objective\" --title \"name\"")
     106        return "\n".join(lines).rstrip() + "\n"
     107
     108    job = focus["job"]
     109    counts = focus["counts"]
     110    artifacts = focus.get("artifacts") or []
     111    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     112    operator = (metadata.get("last_operator_message") if isinstance(metadata, dict) else None) or {}
     113    agent_update = (metadata.get("last_agent_update") if isinstance(metadata, dict) else None) or {}
     114    lesson = (metadata.get("last_lesson") if isinstance(metadata, dict) else None) or {}
     115    findings = metadata.get("finding_ledger") if isinstance(metadata.get("finding_ledger"), list) else []
     116    sources = metadata.get("source_ledger") if isinstance(metadata.get("source_ledger"), list) else []
     117    tasks = metadata.get("task_queue") if isinstance(metadata.get("task_queue"), list) else []
     118    experiments = metadata.get("experiment_ledger") if isinstance(metadata.get("experiment_ledger"), list) else []
     119    active_operator = _active_operator_messages(metadata)
     120    pending_measurement = metadata.get("pending_measurement_obligation") if isinstance(metadata.get("pending_measurement_obligation"), dict) else {}
     121    lines.extend([
     122        "",
     123        f"focus: {job['title']}",
     124        (
     125            f"state: {_job_state_text(job, bool(daemon.get('running')))} | "
     126            f"worker: {_worker_text(job, bool(daemon.get('running')))} | kind: {job['kind']} | "
     127            f"steps: {counts['steps']} | artifacts: {counts['artifacts']} | failures: {counts['failed_steps']}"
     128        ),
     129        f"learning: findings={len(findings)} | sources={len(sources)} | tasks={len(tasks)} | experiments={len(experiments)} | lessons={counts.get('lessons', 0)} | reflections={counts.get('reflections', 0)}",
     130        f"objective: {_one_line(job['objective'], width - 11)}",
     131    ])
     132    if active_operator:
     133        lines.append(f"operator context: {len(active_operator)} active | {_one_line(active_operator[-1].get('message') or '', width - 28)}")
     134    if pending_measurement:
     135        lines.append(f"measurement: pending from step #{pending_measurement.get('source_step_no') or '?'}")
     136    if latest_step:
     137        tool = latest_step.get("tool_name") or latest_step.get("kind") or "-"
     138        status = latest_step.get("status") or "-"
     139        summary = latest_step.get("summary") or latest_step.get("error") or "-"
     140        lines.append(f"latest: #{latest_step.get('step_no')} {status} {tool}: {_one_line(summary, width - 22)}")
     141    if artifacts:
     142        artifact = artifacts[0]
     143        lines.append(f"latest artifact: {artifact.get('title') or artifact['id']}")
     144    if operator:
     145        lines.append(f"last steering: {_one_line(operator.get('message') or '', width - 15)}")
     146    if agent_update:
     147        lines.append(f"agent note: {_one_line(agent_update.get('message') or '', width - 12)}")
     148    if lesson:
     149        lines.append(f"latest lesson: {_one_line(lesson.get('lesson') or '', width - 16)}")
     150    lines.extend([
     151        "",
     152        "commands: activity | updates | findings | tasks | sources | memory | metrics | work --steps 3 | start | stop",
     153    ])
     154    return "\n".join(lines).rstrip() + "\n"
     155
     156
     157def _select_focus_job(db: AgentDB, jobs: list[dict[str, Any]], job_id: str | None) -> dict[str, Any] | None:
     158    if job_id:
     159        return db.get_job(job_id)
     160    for status in ("running", "queued", "paused", "failed", "completed"):
     161        for job in jobs:
     162            if job.get("status") == status:
     163                return job
     164    return jobs[0] if jobs else None
     165
     166
     167def _job_card(db: AgentDB, job: dict[str, Any]) -> dict[str, Any]:
     168    steps = db.list_steps(job_id=job["id"])
     169    artifacts = db.list_artifacts(job["id"], limit=500)
     170    runs = db.list_runs(job["id"], limit=500)
     171    return {
     172        "id": job["id"],
     173        "status": job["status"],
     174        "kind": job["kind"],
     175        "title": job["title"],
     176        "updated_at": job["updated_at"],
     177        "step_count": _step_count(steps),
     178        "run_count": len(runs),
     179        "failed_steps": sum(1 for step in steps if step.get("status") == "failed"),
     180        "artifact_count": len(artifacts),
     181        "latest_step": _public_step(steps[-1]) if steps else None,
     182    }
     183
     184
     185def _focus_state(db: AgentDB, job: dict[str, Any], *, limit: int) -> dict[str, Any]:
     186    steps = db.list_steps(job_id=job["id"])
     187    runs = db.list_runs(job["id"], limit=limit)
     188    artifacts = db.list_artifacts(job["id"], limit=limit)
     189    memory = db.list_memory(job["id"])
     190    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     191    lessons = metadata.get("lessons") if isinstance(metadata.get("lessons"), list) else []
     192    findings = metadata.get("finding_ledger") if isinstance(metadata.get("finding_ledger"), list) else []
     193    sources = metadata.get("source_ledger") if isinstance(metadata.get("source_ledger"), list) else []
     194    tasks = metadata.get("task_queue") if isinstance(metadata.get("task_queue"), list) else []
     195    experiments = metadata.get("experiment_ledger") if isinstance(metadata.get("experiment_ledger"), list) else []
     196    reflections = metadata.get("reflections") if isinstance(metadata.get("reflections"), list) else []
     197    active_operator = _active_operator_messages(metadata)
     198    tool_counts = Counter(step.get("tool_name") or step.get("kind") or "unknown" for step in steps)
     199    blocked = [step for step in steps if str(step.get("error") or "").endswith("blocked") or "blocked" in str(step.get("summary") or "")]
     200    return {
     201        "job": {
     202            "id": job["id"],
     203            "title": job["title"],
     204            "kind": job["kind"],
     205            "status": job["status"],
     206            "objective": job["objective"],
     207            "updated_at": job["updated_at"],
     208            "metadata": job.get("metadata") or {},
     209        },
     210        "counts": {
     211            "steps": _step_count(steps),
     212            "runs": len(db.list_runs(job["id"], limit=1000)),
     213            "artifacts": len(db.list_artifacts(job["id"], limit=1000)),
     214            "failed_steps": sum(1 for step in steps if step.get("status") == "failed"),
     215            "blocked_steps": len(blocked),
     216            "findings": len(findings),
     217            "sources": len(sources),
     218            "tasks": len(tasks),
     219            "experiments": len(experiments),
     220            "active_operator_messages": len(active_operator),
     221            "lessons": len(lessons),
     222            "reflections": len(reflections),
     223        },
     224        "tool_counts": dict(tool_counts.most_common(8)),
     225        "recent_runs": [_public_run(run) for run in runs],
     226        "recent_steps": [_public_step(step) for step in steps[-limit:]],
     227        "artifacts": [_public_artifact(artifact) for artifact in artifacts],
     228        "memory": [
     229            {
     230                "key": entry.get("key"),
     231                "summary": entry.get("summary"),
     232                "artifact_refs": entry.get("artifact_refs") or [],
     233                "updated_at": entry.get("updated_at"),
     234            }
     235            for entry in memory[:4]
     236        ],
     237        "lessons": [
     238            {
     239                "at": entry.get("at"),
     240                "category": entry.get("category") or "memory",
     241                "lesson": entry.get("lesson") or "",
     242                "confidence": entry.get("confidence"),
     243            }
     244            for entry in lessons[-5:]
     245            if isinstance(entry, dict)
     246        ],
     247        "findings": findings[-8:],
     248        "tasks": tasks[-12:],
     249        "experiments": experiments[-12:],
     250        "active_operator_messages": active_operator[-12:],
     251        "sources": sources[-8:],
     252        "reflections": reflections[-4:],
     253    }
     254
     255
     256def _render_focus(focus: dict[str, Any], *, width: int, chars: int, daemon_running: bool) -> list[str]:
     257    job = focus["job"]
     258    counts = focus["counts"]
     259    lines = [
     260        "-" * width,
     261        f"Focus Job: {job['title']} | state {_job_state_text(job, daemon_running)} | {job['kind']}",
     262        f"objective: {_one_line(job['objective'], width - 11)}",
     263        (
     264            f"counts: steps={counts['steps']} runs={counts['runs']} artifacts={counts['artifacts']} "
     265            f"failed_steps={counts['failed_steps']} blocked_steps={counts['blocked_steps']}"
     266        ),
     267        f"learning: findings={counts.get('findings', 0)} sources={counts.get('sources', 0)} tasks={counts.get('tasks', 0)} experiments={counts.get('experiments', 0)} lessons={counts.get('lessons', 0)} reflections={counts.get('reflections', 0)}",
     268        f"tool mix: {_tool_mix(focus.get('tool_counts') or {})}",
     269    ]
     270    active_operator = focus.get("active_operator_messages") or []
     271    if active_operator:
     272        lines.append(f"operator context: {len(active_operator)} active | {_one_line(active_operator[-1].get('message') or '', chars)}")
     273    pending_measurement = (job.get("metadata") or {}).get("pending_measurement_obligation") if isinstance(job.get("metadata"), dict) else {}
     274    if isinstance(pending_measurement, dict) and pending_measurement:
     275        lines.append(f"measurement obligation: pending from step #{pending_measurement.get('source_step_no') or '?'}")
     276    lines.extend(["", "Recent Steps"])
     277    recent_steps = focus.get("recent_steps") or []
     278    if not recent_steps:
     279        lines.append("  no steps recorded")
     280    for step in recent_steps:
     281        error = f" | error={_one_line(step['error'], 70)}" if step.get("error") else ""
     282        lines.append(
     283            f"  #{step['step_no']:<4} {step['status']:<9} {step.get('tool_name') or step['kind']:<18} "
     284            f"{_one_line(step.get('summary') or '-', chars)}{error}"
     285        )
     286        args = step.get("arguments") or {}
     287        if args:
     288            lines.append(f"       args: {_one_line(_compact_value(args), chars)}")
     289    lines.append("")
     290    lines.append("Artifacts")
     291    artifacts = focus.get("artifacts") or []
     292    if not artifacts:
     293        lines.append("  no artifacts yet")
     294    for artifact in artifacts[:8]:
     295        title = artifact.get("title") or artifact["id"]
     296        lines.append(f"  {artifact['created_at']} {artifact['type']} {title}")
     297        if artifact.get("summary"):
     298            lines.append(f"       {_one_line(artifact['summary'], chars)}")
     299    lessons = focus.get("lessons") or []
     300    if lessons:
     301        lines.append("")
     302        lines.append("Lessons")
     303        for lesson in lessons:
     304            lines.append(f"  {lesson.get('category') or 'memory'}: {_one_line(lesson.get('lesson') or '', chars)}")
     305    findings = focus.get("findings") or []
     306    if findings:
     307        lines.append("")
     308        lines.append("Recent Findings")
     309        for finding in findings[-5:]:
     310            lines.append(f"  {_one_line(finding.get('name') or 'unknown', 48)} score={finding.get('score')} {finding.get('category') or ''}")
     311    tasks = focus.get("tasks") or []
     312    if tasks:
     313        lines.append("")
     314        lines.append("Task Queue")
     315        for task in tasks[-6:]:
     316            lines.append(f"  {task.get('status') or 'open':<7} p={task.get('priority') or 0:<3} {_one_line(task.get('title') or 'untitled', 56)}")
     317    sources = focus.get("sources") or []
     318    if sources:
     319        lines.append("")
     320        lines.append("Recent Sources")
     321        for source in sources[-5:]:
     322            lines.append(f"  {_one_line(source.get('source') or 'unknown', 48)} score={source.get('usefulness_score')} findings={source.get('yield_count') or 0}")
     323    memory = focus.get("memory") or []
     324    if memory:
     325        lines.append("")
     326        lines.append("Compact Memory")
     327        for entry in memory:
     328            refs = ", ".join(entry.get("artifact_refs") or [])
     329            suffix = f" refs={refs}" if refs else ""
     330            lines.append(f"  {entry['key']}: {_one_line(entry.get('summary') or '', chars)}{suffix}")
     331    return lines
     332
     333
     334def _public_run(run: dict[str, Any]) -> dict[str, Any]:
     335    return {
     336        "id": run["id"],
     337        "status": run["status"],
     338        "started_at": run["started_at"],
     339        "ended_at": run.get("ended_at"),
     340        "model": run.get("model"),
     341        "error": run.get("error"),
     342    }
     343
     344
     345def _public_step(step: dict[str, Any]) -> dict[str, Any]:
     346    input_data = step.get("input") if isinstance(step.get("input"), dict) else {}
     347    args = input_data.get("arguments") if isinstance(input_data.get("arguments"), dict) else {}
     348    return {
     349        "id": step["id"],
     350        "step_no": step["step_no"],
     351        "kind": step["kind"],
     352        "status": step["status"],
     353        "tool_name": step.get("tool_name"),
     354        "started_at": step["started_at"],
     355        "ended_at": step.get("ended_at"),
     356        "summary": _clean_step_summary(step.get("summary")),
     357        "error": step.get("error"),
     358        "arguments": args,
     359    }
     360
     361
     362def _public_artifact(artifact: dict[str, Any]) -> dict[str, Any]:
     363    return {
     364        "id": artifact["id"],
     365        "created_at": artifact["created_at"],
     366        "type": artifact["type"],
     367        "title": artifact.get("title"),
     368        "summary": artifact.get("summary"),
     369        "path": artifact["path"],
     370    }
     371
     372
     373def _step_count(steps: list[dict[str, Any]]) -> int:
     374    numbers = [int(step.get("step_no") or 0) for step in steps]
     375    return max(numbers, default=0)
     376
     377
     378def _active_operator_messages(metadata: dict[str, Any]) -> list[dict[str, Any]]:
     379    messages = metadata.get("operator_messages") if isinstance(metadata.get("operator_messages"), list) else []
     380    prompt_entries = active_prompt_operator_entries(messages)
     381    return [
     382        entry for entry in messages
     383        if isinstance(entry, dict)
     384        and entry in prompt_entries
     385        and str(entry.get("mode") or "steer") in {"steer", "follow_up"}
     386    ]
     387
     388
     389def _daemon_text(daemon: dict[str, Any]) -> str:
     390    metadata = daemon.get("metadata") or {}
     391    if daemon.get("running"):
     392        pid = metadata.get("pid") or "unknown"
     393        started = metadata.get("started_at") or "unknown start"
     394        stale = " stale-runtime" if daemon.get("stale") else ""
     395        return f"running pid={pid}{stale} started={started}"
     396    return "ready when work starts"
     397
     398
     399def _daemon_health_text(daemon: dict[str, Any], *, latest_step: dict[str, Any] | None = None) -> str:
     400    if not daemon.get("running"):
     401        return "ready when work starts"
     402    metadata = daemon.get("metadata") or {}
     403    heartbeat = metadata.get("last_heartbeat")
     404    status = "running"
     405    if daemon.get("stale"):
     406        status = "running stale-runtime"
     407    if heartbeat:
     408        age = _age_seconds(heartbeat)
     409        if age is not None:
     410            status += f" | heartbeat {int(age)}s ago"
     411            running_step = latest_step or {}
     412            if age > 120 and running_step.get("status") == "running":
     413                tool = running_step.get("tool_name") or running_step.get("kind") or "step"
     414                step_age = _age_seconds(running_step.get("started_at") or "")
     415                if step_age is not None:
     416                    status += f" | busy #{running_step.get('step_no')} {tool} for {int(step_age)}s"
     417                else:
     418                    status += f" | busy #{running_step.get('step_no')} {tool}"
     419            elif age > 120:
     420                status += " (stale)"
     421    failures = metadata.get("consecutive_failures")
     422    if failures:
     423        status += f" | consecutive failures: {failures}"
     424    tool = metadata.get("last_tool")
     425    step_status = metadata.get("last_status")
     426    if tool or step_status:
     427        status += f" | last: {step_status or '?'} {tool or '-'}"
     428    if metadata.get("last_error"):
     429        status += f" | error: {_one_line(metadata.get('last_error'), 48)}"
     430    return status
     431
     432
     433def _worker_text(job: dict[str, Any], daemon_running: bool) -> str:
     434    status = str(job.get("status") or "")
     435    if status in {"paused", "completed", "cancelled", "failed"}:
     436        return status
     437    if job_deferred_until(job):
     438        return "waiting"
     439    return "active" if daemon_running and status in {"running", "queued"} else "idle"
     440
     441
     442def _job_state_text(job: dict[str, Any], daemon_running: bool) -> str:
     443    status = str(job.get("status") or "")
     444    if status in {"running", "queued"}:
     445        if job_deferred_until(job):
     446            return "waiting"
     447        return "advancing" if daemon_running else "open"
     448    return status or "unknown"
     449
     450
     451def _age_seconds(value: str) -> float | None:
     452    try:
     453        parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
     454    except ValueError:
     455        return None
     456    return max(0.0, (datetime.now(timezone.utc) - parsed.astimezone(timezone.utc)).total_seconds())
     457
     458
     459def _compact_time(value: str) -> str:
     460    try:
     461        parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
     462    except ValueError:
     463        return value
     464    return parsed.astimezone().strftime("%Y-%m-%d %H:%M:%S %Z")
     465
     466
     467def _one_line(value: Any, width: int) -> str:
     468    text = " ".join(str(value).split())
     469    return shorten(text, width=max(8, width), placeholder="...")
     470
     471
     472def _clean_step_summary(summary: Any) -> str:
     473    text = " ".join(str(summary or "").split())
     474    if text.startswith("write_artifact saved ") and " at /" in text:
     475        return text.split(" at /", 1)[0]
     476    return text
     477
     478
     479def _compact_value(value: Any) -> str:
     480    if isinstance(value, dict):
     481        parts = [f"{key}={value[key]!r}" for key in sorted(value)]
     482        return ", ".join(parts)
     483    return str(value)
     484
     485
     486def _tool_mix(tool_counts: dict[str, int]) -> str:
     487    if not tool_counts:
     488        return "none"
     489    return ", ".join(f"{name}:{count}" for name, count in tool_counts.items())
     490
     491
     492def resolve_artifact_path(path: str | Path) -> str:
     493    return str(Path(path).expanduser())

nipux_cli/db.py 2752 lines

       1"""SQLite state store for the Nipux agent."""
       2
       3from __future__ import annotations
       4
       5import json
       6import random
       7import re
       8import sqlite3
       9import threading
      10import time
      11import uuid
      12from datetime import datetime, timezone
      13from pathlib import Path
      14from typing import Any, Callable, Iterable, TypeVar
      15
      16from nipux_cli.metric_format import format_metric_value
      17from nipux_cli.memory_graph import DEFAULT_NODE_KIND, DEFAULT_NODE_STATUS, NODE_KINDS, NODE_STATUSES
      18
      19T = TypeVar("T")
      20
      21SCHEMA_VERSION = 1
      22
      23SCHEMA_SQL = """
      24CREATE TABLE IF NOT EXISTS schema_version (
      25    version INTEGER NOT NULL
      26);
      27
      28CREATE TABLE IF NOT EXISTS jobs (
      29    id TEXT PRIMARY KEY,
      30    title TEXT NOT NULL,
      31    objective TEXT NOT NULL,
      32    kind TEXT NOT NULL DEFAULT 'generic',
      33    status TEXT NOT NULL DEFAULT 'queued',
      34    priority INTEGER NOT NULL DEFAULT 0,
      35    cadence TEXT,
      36    created_at TEXT NOT NULL,
      37    updated_at TEXT NOT NULL,
      38    metadata_json TEXT NOT NULL DEFAULT '{}'
      39);
      40
      41CREATE TABLE IF NOT EXISTS job_runs (
      42    id TEXT PRIMARY KEY,
      43    job_id TEXT NOT NULL REFERENCES jobs(id),
      44    status TEXT NOT NULL,
      45    started_at TEXT NOT NULL,
      46    ended_at TEXT,
      47    model TEXT,
      48    config_hash TEXT,
      49    score REAL,
      50    error TEXT
      51);
      52
      53CREATE TABLE IF NOT EXISTS steps (
      54    id TEXT PRIMARY KEY,
      55    job_id TEXT NOT NULL REFERENCES jobs(id),
      56    run_id TEXT NOT NULL REFERENCES job_runs(id),
      57    step_no INTEGER NOT NULL,
      58    kind TEXT NOT NULL,
      59    status TEXT NOT NULL,
      60    tool_name TEXT,
      61    started_at TEXT NOT NULL,
      62    ended_at TEXT,
      63    summary TEXT,
      64    input_json TEXT NOT NULL DEFAULT '{}',
      65    output_json TEXT NOT NULL DEFAULT '{}',
      66    error TEXT
      67);
      68
      69CREATE TABLE IF NOT EXISTS artifacts (
      70    id TEXT PRIMARY KEY,
      71    job_id TEXT NOT NULL REFERENCES jobs(id),
      72    run_id TEXT,
      73    step_id TEXT,
      74    type TEXT NOT NULL,
      75    path TEXT NOT NULL,
      76    sha256 TEXT NOT NULL,
      77    title TEXT,
      78    summary TEXT,
      79    metadata_json TEXT NOT NULL DEFAULT '{}',
      80    created_at TEXT NOT NULL
      81);
      82
      83CREATE TABLE IF NOT EXISTS evidence (
      84    id TEXT PRIMARY KEY,
      85    job_id TEXT NOT NULL REFERENCES jobs(id),
      86    url_or_source TEXT NOT NULL,
      87    artifact_id TEXT REFERENCES artifacts(id),
      88    extracted_text_path TEXT,
      89    summary TEXT,
      90    score_json TEXT NOT NULL DEFAULT '{}',
      91    created_at TEXT NOT NULL
      92);
      93
      94CREATE TABLE IF NOT EXISTS memory_index (
      95    id TEXT PRIMARY KEY,
      96    job_id TEXT NOT NULL REFERENCES jobs(id),
      97    key TEXT NOT NULL,
      98    summary TEXT NOT NULL,
      99    artifact_refs_json TEXT NOT NULL DEFAULT '[]',
     100    updated_at TEXT NOT NULL,
     101    UNIQUE(job_id, key)
     102);
     103
     104CREATE TABLE IF NOT EXISTS digests (
     105    id TEXT PRIMARY KEY,
     106    day TEXT NOT NULL,
     107    target TEXT,
     108    subject TEXT,
     109    body_path TEXT,
     110    sent_at TEXT,
     111    status TEXT NOT NULL,
     112    error TEXT
     113);
     114
     115CREATE TABLE IF NOT EXISTS events (
     116    id TEXT PRIMARY KEY,
     117    job_id TEXT REFERENCES jobs(id),
     118    event_type TEXT NOT NULL,
     119    created_at TEXT NOT NULL,
     120    title TEXT,
     121    body TEXT,
     122    ref_table TEXT,
     123    ref_id TEXT,
     124    metadata_json TEXT NOT NULL DEFAULT '{}'
     125);
     126
     127CREATE INDEX IF NOT EXISTS idx_jobs_status_priority ON jobs(status, priority DESC, updated_at);
     128CREATE INDEX IF NOT EXISTS idx_runs_job ON job_runs(job_id, started_at DESC);
     129CREATE INDEX IF NOT EXISTS idx_steps_run ON steps(run_id, step_no);
     130CREATE INDEX IF NOT EXISTS idx_artifacts_job ON artifacts(job_id, created_at DESC);
     131CREATE INDEX IF NOT EXISTS idx_events_job_time ON events(job_id, created_at DESC);
     132CREATE INDEX IF NOT EXISTS idx_events_ref ON events(ref_table, ref_id);
     133"""
     134
     135
     136def utc_now() -> str:
     137    return datetime.now(timezone.utc).isoformat()
     138
     139
     140def new_id(prefix: str) -> str:
     141    return f"{prefix}_{uuid.uuid4().hex[:16]}"
     142
     143
     144def _slugify(value: str) -> str:
     145    slug = re.sub(r"[^a-z0-9]+", "-", value.lower()).strip("-")
     146    return slug[:72].strip("-") or new_id("job")
     147
     148
     149def _unique_job_id(conn: sqlite3.Connection, seed: str) -> str:
     150    base = _slugify(seed)
     151    candidate = base
     152    suffix = 2
     153    while conn.execute("SELECT 1 FROM jobs WHERE id = ? LIMIT 1", (candidate,)).fetchone():
     154        candidate = f"{base[:68]}-{suffix}"
     155        suffix += 1
     156    return candidate
     157
     158
     159def _json_dumps(value: Any) -> str:
     160    return json.dumps(value if value is not None else {}, ensure_ascii=False, sort_keys=True)
     161
     162
     163def _json_loads(value: str | None) -> dict[str, Any]:
     164    try:
     165        loaded = json.loads(value or "{}")
     166    except json.JSONDecodeError:
     167        return {}
     168    return loaded if isinstance(loaded, dict) else {}
     169
     170
     171def _bounded_float(value: Any, low: float, high: float) -> float:
     172    try:
     173        number = float(value)
     174    except (TypeError, ValueError):
     175        return low
     176    return min(high, max(low, number))
     177
     178
     179def _merge_string_lists(existing: Any, incoming: Any, *, limit: int) -> list[str]:
     180    values: list[str] = []
     181    for source in (existing, incoming):
     182        if isinstance(source, list):
     183            items = source
     184        elif isinstance(source, str) and source.strip():
     185            items = [source]
     186        else:
     187            items = []
     188        for item in items:
     189            text = " ".join(str(item).split())
     190            if text and text not in values:
     191                values.append(text)
     192    return values[-limit:]
     193
     194
     195def _memory_edge_key(edge: dict[str, Any]) -> str:
     196    from_key = str(edge.get("from_key") or "")
     197    relation = str(edge.get("relation") or "")
     198    to_key = str(edge.get("to_key") or "")
     199    return f"{from_key}|{relation}|{to_key}" if from_key and relation and to_key else ""
     200
     201
     202def _as_int(value: Any) -> int:
     203    try:
     204        return int(float(value))
     205    except (TypeError, ValueError):
     206        return 0
     207
     208
     209def _as_float(value: Any) -> float | None:
     210    try:
     211        return float(value)
     212    except (TypeError, ValueError):
     213        return None
     214
     215
     216def _nested_value(value: dict[str, Any], *keys: str) -> Any:
     217    current: Any = value
     218    for key in keys:
     219        if not isinstance(current, dict):
     220            return None
     221        current = current.get(key)
     222    return current
     223
     224
     225def _metadata_list(metadata: dict[str, Any], key: str) -> list[dict[str, Any]]:
     226    values = metadata.get(key)
     227    if not isinstance(values, list):
     228        return []
     229    return [value for value in values if isinstance(value, dict)]
     230
     231
     232def _change_fingerprint(entry: dict[str, Any], fields: Iterable[str]) -> str:
     233    return _json_dumps({field: entry.get(field) for field in fields})
     234
     235
     236def _norm_key(value: str) -> str:
     237    return re.sub(r"[^a-z0-9]+", "-", value.lower()).strip("-")[:120]
     238
     239
     240def _clean_status(value: str, allowed: set[str], default: str) -> str:
     241    status = (value.strip().lower() or default).replace(" ", "_")
     242    return status if status in allowed else default
     243
     244
     245def _experiment_metric_value(entry: dict[str, Any]) -> float | None:
     246    try:
     247        value = entry.get("metric_value")
     248        if value is None:
     249            return None
     250        return float(value)
     251    except (TypeError, ValueError):
     252        return None
     253
     254
     255def _same_metric_group(
     256    entry: dict[str, Any],
     257    *,
     258    metric_name: str,
     259    metric_unit: str,
     260    higher_is_better: bool,
     261) -> bool:
     262    return (
     263        str(entry.get("metric_name") or "").strip().lower() == metric_name.strip().lower()
     264        and str(entry.get("metric_unit") or "").strip().lower() == metric_unit.strip().lower()
     265        and bool(entry.get("higher_is_better", True)) == bool(higher_is_better)
     266        and _experiment_metric_value(entry) is not None
     267    )
     268
     269
     270def _best_experiment_for_metric(
     271    experiments: list[dict[str, Any]],
     272    *,
     273    metric_name: str,
     274    metric_unit: str,
     275    higher_is_better: bool,
     276    exclude_key: str = "",
     277) -> dict[str, Any] | None:
     278    candidates = [
     279        experiment
     280        for experiment in experiments
     281        if experiment.get("key") != exclude_key
     282        and _same_metric_group(
     283            experiment,
     284            metric_name=metric_name,
     285            metric_unit=metric_unit,
     286            higher_is_better=higher_is_better,
     287        )
     288    ]
     289    if not candidates:
     290        return None
     291    return max(candidates, key=lambda item: _experiment_metric_value(item) or 0.0) if higher_is_better else min(candidates, key=lambda item: _experiment_metric_value(item) or 0.0)
     292
     293
     294def _metric_delta(
     295    *,
     296    metric_value: Any,
     297    previous_best: dict[str, Any] | None,
     298    higher_is_better: bool,
     299) -> float | None:
     300    try:
     301        current = float(metric_value)
     302    except (TypeError, ValueError):
     303        return None
     304    if previous_best is None:
     305        return None
     306    previous = _experiment_metric_value(previous_best)
     307    if previous is None:
     308        return None
     309    delta = current - previous if higher_is_better else previous - current
     310    return round(delta, 6)
     311
     312
     313def _mark_best_experiments(experiments: list[dict[str, Any]]) -> dict[str, Any] | None:
     314    groups: dict[tuple[str, str, bool], list[dict[str, Any]]] = {}
     315    for experiment in experiments:
     316        metric_name = str(experiment.get("metric_name") or "").strip().lower()
     317        if _experiment_metric_value(experiment) is None or not metric_name:
     318            experiment["best_observed"] = False
     319            continue
     320        key = (
     321            metric_name,
     322            str(experiment.get("metric_unit") or "").strip().lower(),
     323            bool(experiment.get("higher_is_better", True)),
     324        )
     325        groups.setdefault(key, []).append(experiment)
     326    winners: list[dict[str, Any]] = []
     327    for (_metric_name, _metric_unit, higher_is_better), entries in groups.items():
     328        winner = max(entries, key=lambda item: _experiment_metric_value(item) or 0.0) if higher_is_better else min(entries, key=lambda item: _experiment_metric_value(item) or 0.0)
     329        for entry in entries:
     330            entry["best_observed"] = entry is winner
     331        winners.append(winner)
     332    if not winners:
     333        return None
     334    return max(winners, key=lambda item: str(item.get("updated_at") or item.get("created_at") or ""))
     335
     336
     337def _row_to_dict(row: sqlite3.Row | None) -> dict[str, Any] | None:
     338    if row is None:
     339        return None
     340    result = dict(row)
     341    for key in ("metadata_json", "input_json", "output_json", "score_json", "artifact_refs_json"):
     342        if key in result:
     343            try:
     344                result[key.removesuffix("_json")] = json.loads(result[key] or "{}")
     345            except json.JSONDecodeError:
     346                result[key.removesuffix("_json")] = {}
     347    return result
     348
     349
     350def _insert_event(
     351    conn: sqlite3.Connection,
     352    *,
     353    job_id: str | None,
     354    event_type: str,
     355    title: str = "",
     356    body: str = "",
     357    ref_table: str = "",
     358    ref_id: str = "",
     359    metadata: dict[str, Any] | None = None,
     360    created_at: str | None = None,
     361) -> dict[str, Any]:
     362    event_id = new_id("evt")
     363    when = created_at or utc_now()
     364    conn.execute(
     365        """
     366        INSERT INTO events(id, job_id, event_type, created_at, title, body, ref_table, ref_id, metadata_json)
     367        VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
     368        """,
     369        (
     370            event_id,
     371            job_id,
     372            event_type.strip().lower() or "event",
     373            when,
     374            title.strip(),
     375            body.strip(),
     376            ref_table.strip(),
     377            ref_id.strip(),
     378            _json_dumps(metadata or {}),
     379        ),
     380    )
     381    return {
     382        "id": event_id,
     383        "job_id": job_id,
     384        "event_type": event_type.strip().lower() or "event",
     385        "created_at": when,
     386        "title": title.strip(),
     387        "body": body.strip(),
     388        "ref_table": ref_table.strip(),
     389        "ref_id": ref_id.strip(),
     390        "metadata": metadata or {},
     391    }
     392
     393
     394def _projected_event(
     395    *,
     396    event_id: str,
     397    job_id: str,
     398    event_type: str,
     399    created_at: str,
     400    title: str = "",
     401    body: str = "",
     402    ref_table: str = "",
     403    ref_id: str = "",
     404    metadata: dict[str, Any] | None = None,
     405) -> dict[str, Any]:
     406    return {
     407        "id": event_id,
     408        "job_id": job_id,
     409        "event_type": event_type,
     410        "created_at": created_at,
     411        "title": title,
     412        "body": body,
     413        "ref_table": ref_table,
     414        "ref_id": ref_id,
     415        "metadata": metadata or {},
     416        "projected": True,
     417    }
     418
     419
     420class AgentDB:
     421    """Small SQLite wrapper with WAL and jittered write retries."""
     422
     423    _WRITE_RETRIES = 12
     424
     425    def __init__(self, path: str | Path):
     426        self.path = Path(path)
     427        self.path.parent.mkdir(parents=True, exist_ok=True)
     428        self._lock = threading.RLock()
     429        self._conn = sqlite3.connect(
     430            str(self.path),
     431            check_same_thread=False,
     432            timeout=1.0,
     433            isolation_level=None,
     434        )
     435        self._conn.row_factory = sqlite3.Row
     436        self._conn.execute("PRAGMA journal_mode=WAL")
     437        self._conn.execute("PRAGMA foreign_keys=ON")
     438        self._init_schema()
     439
     440    def close(self) -> None:
     441        with self._lock:
     442            if self._conn is not None:
     443                try:
     444                    self._conn.execute("PRAGMA wal_checkpoint(PASSIVE)")
     445                finally:
     446                    self._conn.close()
     447                    self._conn = None
     448
     449    def _init_schema(self) -> None:
     450        with self._lock:
     451            self._conn.executescript(SCHEMA_SQL)
     452            row = self._conn.execute("SELECT version FROM schema_version LIMIT 1").fetchone()
     453            if row is None:
     454                self._conn.execute("INSERT INTO schema_version(version) VALUES (?)", (SCHEMA_VERSION,))
     455            elif int(row["version"]) != SCHEMA_VERSION:
     456                raise RuntimeError(f"Unsupported nipux schema version: {row['version']}")
     457
     458    def _write(self, fn: Callable[[sqlite3.Connection], T]) -> T:
     459        last_error: Exception | None = None
     460        for attempt in range(self._WRITE_RETRIES):
     461            try:
     462                with self._lock:
     463                    self._conn.execute("BEGIN IMMEDIATE")
     464                    try:
     465                        result = fn(self._conn)
     466                        self._conn.commit()
     467                        return result
     468                    except BaseException:
     469                        self._conn.rollback()
     470                        raise
     471            except sqlite3.OperationalError as exc:
     472                if "locked" not in str(exc).lower() and "busy" not in str(exc).lower():
     473                    raise
     474                last_error = exc
     475                if attempt < self._WRITE_RETRIES - 1:
     476                    time.sleep(random.uniform(0.02, 0.15))
     477        raise last_error or sqlite3.OperationalError("database is locked")
     478
     479    def append_event(
     480        self,
     481        job_id: str | None = None,
     482        *,
     483        event_type: str,
     484        title: str = "",
     485        body: str = "",
     486        ref_table: str = "",
     487        ref_id: str = "",
     488        metadata: dict[str, Any] | None = None,
     489        created_at: str | None = None,
     490    ) -> dict[str, Any]:
     491        def op(conn: sqlite3.Connection) -> dict[str, Any]:
     492            return _insert_event(
     493                conn,
     494                job_id=job_id,
     495                event_type=event_type,
     496                title=title,
     497                body=body,
     498                ref_table=ref_table,
     499                ref_id=ref_id,
     500                metadata=metadata,
     501                created_at=created_at,
     502            )
     503
     504        return self._write(op)
     505
     506    def list_events(
     507        self,
     508        *,
     509        job_id: str | None = None,
     510        limit: int = 100,
     511        event_types: Iterable[str] | None = None,
     512    ) -> list[dict[str, Any]]:
     513        filters = []
     514        params: list[Any] = []
     515        if job_id is not None:
     516            filters.append("job_id = ?")
     517            params.append(job_id)
     518        if event_types:
     519            values = [str(value).strip().lower() for value in event_types if str(value).strip()]
     520            if values:
     521                filters.append(f"event_type IN ({','.join('?' for _ in values)})")
     522                params.extend(values)
     523        where = f"WHERE {' AND '.join(filters)}" if filters else ""
     524        rows = self._conn.execute(
     525            f"""
     526            SELECT * FROM (
     527                SELECT * FROM events
     528                {where}
     529                ORDER BY created_at DESC, id DESC
     530                LIMIT ?
     531            )
     532            ORDER BY created_at ASC, id ASC
     533            """,
     534            [*params, int(limit)],
     535        ).fetchall()
     536        return [_row_to_dict(row) for row in rows]
     537
     538    def list_timeline_events(self, job_id: str, *, limit: int = 100) -> list[dict[str, Any]]:
     539        """Return visible job history, combining durable events with old projected state."""
     540
     541        actual = self.list_events(job_id=job_id, limit=max(limit * 4, 250))
     542        actual_ids = {str(event.get("id")) for event in actual}
     543        actual_refs = {
     544            (str(event.get("ref_table") or ""), str(event.get("ref_id") or ""))
     545            for event in actual
     546            if event.get("ref_table") and event.get("ref_id")
     547        }
     548        timeline: list[dict[str, Any]] = list(actual)
     549        job = self.get_job(job_id)
     550        metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     551
     552        for index, entry in enumerate(_metadata_list(metadata, "operator_messages")):
     553            if entry.get("event_id") in actual_ids:
     554                continue
     555            timeline.append(_projected_event(
     556                event_id=f"projected_operator_{index}",
     557                job_id=job_id,
     558                event_type="operator_message",
     559                created_at=str(entry.get("at") or job.get("updated_at") or job.get("created_at")),
     560                title=str(entry.get("source") or "operator"),
     561                body=str(entry.get("message") or ""),
     562                metadata={
     563                    "source": entry.get("source") or "operator",
     564                    "mode": entry.get("mode") or "steer",
     565                    "claimed_at": entry.get("claimed_at"),
     566                    "acknowledged_at": entry.get("acknowledged_at"),
     567                    "superseded_at": entry.get("superseded_at"),
     568                },
     569            ))
     570
     571        for index, entry in enumerate(_metadata_list(metadata, "agent_updates")):
     572            if entry.get("event_id") in actual_ids:
     573                continue
     574            timeline.append(_projected_event(
     575                event_id=f"projected_agent_{index}",
     576                job_id=job_id,
     577                event_type="agent_message",
     578                created_at=str(entry.get("at") or job.get("updated_at") or job.get("created_at")),
     579                title=str(entry.get("category") or "progress"),
     580                body=str(entry.get("message") or ""),
     581                metadata=entry.get("metadata") if isinstance(entry.get("metadata"), dict) else {},
     582            ))
     583
     584        for index, lesson in enumerate(_metadata_list(metadata, "lessons")):
     585            if lesson.get("event_id") in actual_ids:
     586                continue
     587            timeline.append(_projected_event(
     588                event_id=f"projected_lesson_{index}",
     589                job_id=job_id,
     590                event_type="lesson",
     591                created_at=str(lesson.get("at") or lesson.get("last_seen") or job.get("updated_at") or job.get("created_at")),
     592                title=str(lesson.get("category") or "memory"),
     593                body=str(lesson.get("lesson") or ""),
     594                metadata={"confidence": lesson.get("confidence"), **(lesson.get("metadata") if isinstance(lesson.get("metadata"), dict) else {})},
     595            ))
     596
     597        for index, source in enumerate(_metadata_list(metadata, "source_ledger")):
     598            if source.get("event_id") in actual_ids:
     599                continue
     600            timeline.append(_projected_event(
     601                event_id=f"projected_source_{index}",
     602                job_id=job_id,
     603                event_type="source",
     604                created_at=str(source.get("last_seen") or source.get("first_seen") or job.get("updated_at") or job.get("created_at")),
     605                title=str(source.get("source") or "source"),
     606                body=str(source.get("last_outcome") or ""),
     607                metadata=source,
     608            ))
     609
     610        for index, finding in enumerate(_metadata_list(metadata, "finding_ledger")):
     611            if finding.get("event_id") in actual_ids:
     612                continue
     613            timeline.append(_projected_event(
     614                event_id=f"projected_finding_{index}",
     615                job_id=job_id,
     616                event_type="finding",
     617                created_at=str(finding.get("updated_at") or finding.get("created_at") or job.get("updated_at") or job.get("created_at")),
     618                title=str(finding.get("name") or "finding"),
     619                body=str(finding.get("reason") or finding.get("category") or ""),
     620                metadata=finding,
     621            ))
     622
     623        for index, task in enumerate(_metadata_list(metadata, "task_queue")):
     624            if task.get("event_id") in actual_ids:
     625                continue
     626            timeline.append(_projected_event(
     627                event_id=f"projected_task_{index}",
     628                job_id=job_id,
     629                event_type="task",
     630                created_at=str(task.get("updated_at") or task.get("created_at") or job.get("updated_at") or job.get("created_at")),
     631                title=str(task.get("title") or "task"),
     632                body=str(task.get("result") or task.get("goal") or ""),
     633                metadata=task,
     634            ))
     635
     636        for index, experiment in enumerate(_metadata_list(metadata, "experiment_ledger")):
     637            if experiment.get("event_id") in actual_ids:
     638                continue
     639            metric = ""
     640            if experiment.get("metric_value") is not None:
     641                metric = format_metric_value(
     642                    experiment.get("metric_name") or "metric",
     643                    experiment.get("metric_value"),
     644                    experiment.get("metric_unit") or "",
     645                )
     646            timeline.append(_projected_event(
     647                event_id=f"projected_experiment_{index}",
     648                job_id=job_id,
     649                event_type="experiment",
     650                created_at=str(experiment.get("updated_at") or experiment.get("created_at") or job.get("updated_at") or job.get("created_at")),
     651                title=str(experiment.get("title") or "experiment"),
     652                body=str(experiment.get("result") or metric or experiment.get("hypothesis") or ""),
     653                metadata=experiment,
     654            ))
     655
     656        for index, reflection in enumerate(_metadata_list(metadata, "reflections")):
     657            if reflection.get("event_id") in actual_ids:
     658                continue
     659            timeline.append(_projected_event(
     660                event_id=f"projected_reflection_{index}",
     661                job_id=job_id,
     662                event_type="reflection",
     663                created_at=str(reflection.get("at") or job.get("updated_at") or job.get("created_at")),
     664                title="reflection",
     665                body=str(reflection.get("summary") or reflection.get("strategy") or ""),
     666                metadata=reflection.get("metadata") if isinstance(reflection.get("metadata"), dict) else {},
     667            ))
     668
     669        for step in self.list_steps(job_id=job_id):
     670            ref = ("steps", str(step["id"]))
     671            if ref in actual_refs:
     672                continue
     673            event_type = "error" if step.get("status") == "failed" or step.get("error") else "tool_result"
     674            title = str(step.get("tool_name") or step.get("kind") or "step")
     675            body = str(step.get("summary") or step.get("error") or "")
     676            timeline.append(_projected_event(
     677                event_id=f"projected_step_{step['id']}",
     678                job_id=job_id,
     679                event_type=event_type,
     680                created_at=str(step.get("ended_at") or step.get("started_at")),
     681                title=title,
     682                body=body,
     683                ref_table="steps",
     684                ref_id=str(step["id"]),
     685                metadata={"step_no": step.get("step_no"), "status": step.get("status"), "kind": step.get("kind")},
     686            ))
     687
     688        for artifact in self.list_artifacts(job_id, limit=10000):
     689            ref = ("artifacts", str(artifact["id"]))
     690            if ref in actual_refs:
     691                continue
     692            timeline.append(_projected_event(
     693                event_id=f"projected_artifact_{artifact['id']}",
     694                job_id=job_id,
     695                event_type="artifact",
     696                created_at=str(artifact.get("created_at")),
     697                title=str(artifact.get("title") or artifact["id"]),
     698                body=str(artifact.get("summary") or artifact.get("path") or ""),
     699                ref_table="artifacts",
     700                ref_id=str(artifact["id"]),
     701                metadata={"type": artifact.get("type"), "path": artifact.get("path")},
     702            ))
     703
     704        for memory in self.list_memory(job_id):
     705            ref = ("memory_index", str(memory["id"]))
     706            if ref in actual_refs:
     707                continue
     708            timeline.append(_projected_event(
     709                event_id=f"projected_memory_{memory['id']}",
     710                job_id=job_id,
     711                event_type="compaction",
     712                created_at=str(memory.get("updated_at")),
     713                title=str(memory.get("key") or "compact memory"),
     714                body=str(memory.get("summary") or ""),
     715                ref_table="memory_index",
     716                ref_id=str(memory["id"]),
     717                metadata={"artifact_refs": memory.get("artifact_refs") or []},
     718            ))
     719
     720        timeline = [event for event in timeline if event.get("created_at")]
     721        timeline.sort(key=lambda event: (str(event.get("created_at") or ""), str(event.get("id") or "")))
     722        return timeline[-int(limit):]
     723
     724    def create_job(
     725        self,
     726        objective: str,
     727        *,
     728        title: str | None = None,
     729        kind: str = "generic",
     730        priority: int = 0,
     731        cadence: str | None = None,
     732        metadata: dict[str, Any] | None = None,
     733    ) -> str:
     734        now = utc_now()
     735        title = title or objective.strip().splitlines()[0][:80] or "Untitled job"
     736
     737        def op(conn: sqlite3.Connection) -> str:
     738            job_id = _unique_job_id(conn, title)
     739            conn.execute(
     740                """
     741                INSERT INTO jobs(id, title, objective, kind, status, priority, cadence, created_at, updated_at, metadata_json)
     742                VALUES (?, ?, ?, ?, 'queued', ?, ?, ?, ?, ?)
     743                """,
     744                (job_id, title, objective, kind, priority, cadence, now, now, _json_dumps(metadata)),
     745            )
     746            _insert_event(
     747                conn,
     748                job_id=job_id,
     749                event_type="daemon",
     750                title="job created",
     751                body=objective,
     752                metadata={"title": title, "kind": kind, "cadence": cadence},
     753                created_at=now,
     754            )
     755            return job_id
     756
     757        return self._write(op)
     758
     759    def get_job(self, job_id: str) -> dict[str, Any]:
     760        row = self._conn.execute("SELECT * FROM jobs WHERE id = ?", (job_id,)).fetchone()
     761        job = _row_to_dict(row)
     762        if job is None:
     763            raise KeyError(f"Job not found: {job_id}")
     764        return job
     765
     766    def list_jobs(self, *, statuses: Iterable[str] | None = None) -> list[dict[str, Any]]:
     767        if statuses:
     768            values = list(statuses)
     769            placeholders = ",".join("?" for _ in values)
     770            rows = self._conn.execute(
     771                f"SELECT * FROM jobs WHERE status IN ({placeholders}) ORDER BY priority DESC, updated_at",
     772                values,
     773            ).fetchall()
     774        else:
     775            rows = self._conn.execute("SELECT * FROM jobs ORDER BY updated_at DESC").fetchall()
     776        return [_row_to_dict(row) for row in rows]
     777
     778    def update_job_status(self, job_id: str, status: str, *, metadata_patch: dict[str, Any] | None = None) -> None:
     779        now = utc_now()
     780
     781        def op(conn: sqlite3.Connection) -> None:
     782            metadata_json = None
     783            if metadata_patch:
     784                row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
     785                if row is None:
     786                    raise KeyError(f"Job not found: {job_id}")
     787                current = json.loads(row["metadata_json"] or "{}")
     788                current.update(metadata_patch)
     789                metadata_json = _json_dumps(current)
     790            if metadata_json is None:
     791                conn.execute("UPDATE jobs SET status = ?, updated_at = ? WHERE id = ?", (status, now, job_id))
     792            else:
     793                conn.execute(
     794                    "UPDATE jobs SET status = ?, updated_at = ?, metadata_json = ? WHERE id = ?",
     795                    (status, now, metadata_json, job_id),
     796                )
     797            _insert_event(
     798                conn,
     799                job_id=job_id,
     800                event_type="daemon",
     801                title=f"job {status}",
     802                body=str((metadata_patch or {}).get("last_note") or ""),
     803                metadata={"status": status, "metadata_patch": metadata_patch or {}},
     804                created_at=now,
     805            )
     806
     807        self._write(op)
     808
     809    def update_job_metadata(self, job_id: str, metadata_patch: dict[str, Any]) -> None:
     810        now = utc_now()
     811
     812        def op(conn: sqlite3.Connection) -> None:
     813            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
     814            if row is None:
     815                raise KeyError(f"Job not found: {job_id}")
     816            current = json.loads(row["metadata_json"] or "{}")
     817            current.update(metadata_patch)
     818            conn.execute(
     819                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
     820                (now, _json_dumps(current), job_id),
     821            )
     822
     823        self._write(op)
     824
     825    def claim_operator_messages(
     826        self,
     827        job_id: str,
     828        *,
     829        modes: Iterable[str] = ("steer",),
     830        limit: int = 1,
     831    ) -> list[dict[str, Any]]:
     832        now = utc_now()
     833        allowed = {mode.strip().lower().replace("-", "_") for mode in modes}
     834
     835        def op(conn: sqlite3.Connection) -> list[dict[str, Any]]:
     836            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
     837            if row is None:
     838                raise KeyError(f"Job not found: {job_id}")
     839            metadata = json.loads(row["metadata_json"] or "{}")
     840            messages = metadata.get("operator_messages")
     841            if not isinstance(messages, list):
     842                return []
     843            claimed: list[dict[str, Any]] = []
     844            for entry in messages:
     845                if len(claimed) >= limit:
     846                    break
     847                if not isinstance(entry, dict):
     848                    continue
     849                mode = str(entry.get("mode") or "steer").strip().lower().replace("-", "_")
     850                if mode not in allowed or entry.get("claimed_at"):
     851                    continue
     852                if entry.get("acknowledged_at") or entry.get("superseded_at"):
     853                    continue
     854                entry["claimed_at"] = now
     855                entry["delivered_at"] = now
     856                claimed.append(dict(entry))
     857            if not claimed:
     858                return []
     859            metadata["operator_messages"] = messages[-200:]
     860            metadata["last_claimed_operator_messages"] = claimed
     861            conn.execute(
     862                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
     863                (now, _json_dumps(metadata), job_id),
     864            )
     865            for entry in claimed:
     866                _insert_event(
     867                    conn,
     868                    job_id=job_id,
     869                    event_type="loop",
     870                    title="steering claimed",
     871                    body=str(entry.get("message") or ""),
     872                    metadata={
     873                        "source": entry.get("source"),
     874                        "mode": entry.get("mode"),
     875                        "operator_event_id": entry.get("event_id"),
     876                    },
     877                    created_at=now,
     878                )
     879            return claimed
     880
     881        return self._write(op)
     882
     883    def acknowledge_operator_messages(
     884        self,
     885        job_id: str,
     886        *,
     887        message_ids: Iterable[str] | None = None,
     888        summary: str = "",
     889        status: str = "acknowledged",
     890    ) -> dict[str, Any]:
     891        now = utc_now()
     892        wanted = {str(message_id).strip() for message_id in (message_ids or []) if str(message_id).strip()}
     893        status = status.strip().lower().replace("-", "_") or "acknowledged"
     894        if status not in {"acknowledged", "superseded"}:
     895            status = "acknowledged"
     896
     897        def op(conn: sqlite3.Connection) -> dict[str, Any]:
     898            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
     899            if row is None:
     900                raise KeyError(f"Job not found: {job_id}")
     901            metadata = json.loads(row["metadata_json"] or "{}")
     902            messages = metadata.get("operator_messages")
     903            if not isinstance(messages, list):
     904                messages = []
     905            acknowledged: list[dict[str, Any]] = []
     906            for entry in messages:
     907                if not isinstance(entry, dict):
     908                    continue
     909                mode = str(entry.get("mode") or "steer").strip().lower().replace("-", "_")
     910                if mode not in {"steer", "follow_up"}:
     911                    continue
     912                event_id = str(entry.get("event_id") or "")
     913                if wanted and event_id not in wanted:
     914                    continue
     915                if not wanted and not entry.get("claimed_at"):
     916                    continue
     917                if entry.get("acknowledged_at") or entry.get("superseded_at"):
     918                    continue
     919                if status == "superseded":
     920                    entry["superseded_at"] = now
     921                else:
     922                    entry["acknowledged_at"] = now
     923                if summary:
     924                    entry["acknowledgement_summary"] = summary.strip()
     925                acknowledged.append(dict(entry))
     926            metadata["operator_messages"] = messages[-200:]
     927            metadata["last_operator_context_ack"] = {
     928                "at": now,
     929                "status": status,
     930                "summary": summary.strip(),
     931                "message_ids": [entry.get("event_id") for entry in acknowledged if entry.get("event_id")],
     932                "count": len(acknowledged),
     933            }
     934            conn.execute(
     935                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
     936                (now, _json_dumps(metadata), job_id),
     937            )
     938            event = _insert_event(
     939                conn,
     940                job_id=job_id,
     941                event_type="operator_context",
     942                title=f"operator {status}",
     943                body=summary.strip() or f"{len(acknowledged)} operator message(s) {status}",
     944                metadata={
     945                    "status": status,
     946                    "message_ids": [entry.get("event_id") for entry in acknowledged if entry.get("event_id")],
     947                    "count": len(acknowledged),
     948                },
     949                created_at=now,
     950            )
     951            return {"event": event, "messages": acknowledged, "count": len(acknowledged), "status": status}
     952
     953        return self._write(op)
     954
     955    def rename_job(self, job_id: str, title: str) -> dict[str, Any]:
     956        now = utc_now()
     957        new_title = title.strip()
     958        if not new_title:
     959            raise ValueError("title is required")
     960
     961        def op(conn: sqlite3.Connection) -> dict[str, Any]:
     962            row = conn.execute("SELECT * FROM jobs WHERE id = ?", (job_id,)).fetchone()
     963            if row is None:
     964                raise KeyError(f"Job not found: {job_id}")
     965            conn.execute("UPDATE jobs SET title = ?, updated_at = ? WHERE id = ?", (new_title, now, job_id))
     966            _insert_event(
     967                conn,
     968                job_id=job_id,
     969                event_type="daemon",
     970                title="job renamed",
     971                body=f"{row['title']} -> {new_title}",
     972                metadata={"old_title": row["title"], "new_title": new_title},
     973                created_at=now,
     974            )
     975            updated = dict(row)
     976            updated["title"] = new_title
     977            updated["updated_at"] = now
     978            return _row_to_dict(updated)
     979
     980        return self._write(op)
     981
     982    def delete_job(self, job_id: str) -> dict[str, Any]:
     983        def op(conn: sqlite3.Connection) -> dict[str, Any]:
     984            row = conn.execute("SELECT * FROM jobs WHERE id = ?", (job_id,)).fetchone()
     985            if row is None:
     986                raise KeyError(f"Job not found: {job_id}")
     987            artifact_rows = conn.execute("SELECT path FROM artifacts WHERE job_id = ?", (job_id,)).fetchall()
     988            artifact_paths = [str(artifact["path"]) for artifact in artifact_rows if artifact["path"]]
     989            counts = {
     990                "evidence": conn.execute("SELECT COUNT(*) AS n FROM evidence WHERE job_id = ?", (job_id,)).fetchone()["n"],
     991                "artifacts": conn.execute("SELECT COUNT(*) AS n FROM artifacts WHERE job_id = ?", (job_id,)).fetchone()["n"],
     992                "memory": conn.execute("SELECT COUNT(*) AS n FROM memory_index WHERE job_id = ?", (job_id,)).fetchone()["n"],
     993                "steps": conn.execute("SELECT COUNT(*) AS n FROM steps WHERE job_id = ?", (job_id,)).fetchone()["n"],
     994                "runs": conn.execute("SELECT COUNT(*) AS n FROM job_runs WHERE job_id = ?", (job_id,)).fetchone()["n"],
     995                "events": conn.execute("SELECT COUNT(*) AS n FROM events WHERE job_id = ?", (job_id,)).fetchone()["n"],
     996            }
     997            conn.execute("DELETE FROM evidence WHERE job_id = ?", (job_id,))
     998            conn.execute("DELETE FROM artifacts WHERE job_id = ?", (job_id,))
     999            conn.execute("DELETE FROM memory_index WHERE job_id = ?", (job_id,))
    1000            conn.execute("DELETE FROM steps WHERE job_id = ?", (job_id,))
    1001            conn.execute("DELETE FROM job_runs WHERE job_id = ?", (job_id,))
    1002            conn.execute("DELETE FROM events WHERE job_id = ?", (job_id,))
    1003            conn.execute("DELETE FROM jobs WHERE id = ?", (job_id,))
    1004            return {
    1005                "job": _row_to_dict(row),
    1006                "artifact_paths": artifact_paths,
    1007                "counts": counts,
    1008            }
    1009
    1010        return self._write(op)
    1011
    1012    def append_operator_message(
    1013        self,
    1014        job_id: str,
    1015        message: str,
    1016        *,
    1017        source: str = "operator",
    1018        mode: str = "steer",
    1019    ) -> dict[str, Any]:
    1020        now = utc_now()
    1021        text = message.strip()
    1022        if not text:
    1023            raise ValueError("message is required")
    1024        mode = mode.strip().lower().replace("-", "_") or "steer"
    1025        if mode not in {"steer", "follow_up", "note"}:
    1026            mode = "steer"
    1027        entry = {"at": now, "source": source, "mode": mode, "message": text}
    1028
    1029        def op(conn: sqlite3.Connection) -> dict[str, Any]:
    1030            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
    1031            if row is None:
    1032                raise KeyError(f"Job not found: {job_id}")
    1033            event = _insert_event(
    1034                conn,
    1035                job_id=job_id,
    1036                event_type="operator_message",
    1037                title=source,
    1038                body=text,
    1039                metadata={"source": source, "mode": mode},
    1040                created_at=now,
    1041            )
    1042            entry["event_id"] = event["id"]
    1043            metadata = json.loads(row["metadata_json"] or "{}")
    1044            messages = metadata.get("operator_messages")
    1045            if not isinstance(messages, list):
    1046                messages = []
    1047            messages.append(entry)
    1048            metadata["operator_messages"] = messages[-200:]
    1049            metadata["last_operator_message"] = entry
    1050            conn.execute(
    1051                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
    1052                (now, _json_dumps(metadata), job_id),
    1053            )
    1054            return entry
    1055
    1056        return self._write(op)
    1057
    1058    def append_agent_update(
    1059        self,
    1060        job_id: str,
    1061        message: str,
    1062        *,
    1063        category: str = "progress",
    1064        metadata: dict[str, Any] | None = None,
    1065    ) -> dict[str, Any]:
    1066        now = utc_now()
    1067        text = message.strip()
    1068        if not text:
    1069            raise ValueError("message is required")
    1070        entry = {
    1071            "at": now,
    1072            "category": category.strip() or "progress",
    1073            "message": text,
    1074            "metadata": metadata or {},
    1075        }
    1076
    1077        def op(conn: sqlite3.Connection) -> dict[str, Any]:
    1078            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
    1079            if row is None:
    1080                raise KeyError(f"Job not found: {job_id}")
    1081            event = _insert_event(
    1082                conn,
    1083                job_id=job_id,
    1084                event_type="agent_message",
    1085                title=entry["category"],
    1086                body=text,
    1087                metadata=entry["metadata"],
    1088                created_at=now,
    1089            )
    1090            entry["event_id"] = event["id"]
    1091            job_metadata = json.loads(row["metadata_json"] or "{}")
    1092            updates = job_metadata.get("agent_updates")
    1093            if not isinstance(updates, list):
    1094                updates = []
    1095            updates.append(entry)
    1096            job_metadata["agent_updates"] = updates[-100:]
    1097            job_metadata["last_agent_update"] = entry
    1098            conn.execute(
    1099                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
    1100                (now, _json_dumps(job_metadata), job_id),
    1101            )
    1102            return entry
    1103
    1104        return self._write(op)
    1105
    1106    def append_lesson(
    1107        self,
    1108        job_id: str,
    1109        lesson: str,
    1110        *,
    1111        category: str = "memory",
    1112        confidence: float | None = None,
    1113        metadata: dict[str, Any] | None = None,
    1114    ) -> dict[str, Any]:
    1115        now = utc_now()
    1116        text = lesson.strip()
    1117        if not text:
    1118            raise ValueError("lesson is required")
    1119        entry = {
    1120            "at": now,
    1121            "category": category.strip().lower() or "memory",
    1122            "key": _norm_key(f"{category}:{text}"),
    1123            "lesson": text,
    1124            "confidence": confidence,
    1125            "metadata": metadata or {},
    1126        }
    1127
    1128        def op(conn: sqlite3.Connection) -> dict[str, Any]:
    1129            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
    1130            if row is None:
    1131                raise KeyError(f"Job not found: {job_id}")
    1132            job_metadata = json.loads(row["metadata_json"] or "{}")
    1133            lessons = job_metadata.get("lessons")
    1134            if not isinstance(lessons, list):
    1135                lessons = []
    1136            existing = next(
    1137                (
    1138                    item
    1139                    for item in lessons
    1140                    if isinstance(item, dict)
    1141                    and (item.get("key") or _norm_key(f"{item.get('category', 'memory')}:{item.get('lesson', '')}"))
    1142                    == entry["key"]
    1143                ),
    1144                None,
    1145            )
    1146            if existing is None:
    1147                lessons.append(entry)
    1148                current = entry
    1149                current["created"] = True
    1150                current["substantive_update"] = True
    1151                event = _insert_event(
    1152                    conn,
    1153                    job_id=job_id,
    1154                    event_type="lesson",
    1155                    title=current.get("category") or "memory",
    1156                    body=current.get("lesson") or text,
    1157                    metadata={
    1158                        "confidence": current.get("confidence"),
    1159                        "seen_count": current.get("seen_count"),
    1160                        **(current.get("metadata") if isinstance(current.get("metadata"), dict) else {}),
    1161                    },
    1162                    created_at=now,
    1163                )
    1164                current["event_id"] = event["id"]
    1165            else:
    1166                existing["last_seen"] = now
    1167                existing["seen_count"] = int(existing.get("seen_count") or 1) + 1
    1168                if confidence is not None:
    1169                    existing["confidence"] = confidence
    1170                if metadata:
    1171                    merged = existing.get("metadata") if isinstance(existing.get("metadata"), dict) else {}
    1172                    merged.update(metadata)
    1173                    existing["metadata"] = merged
    1174                existing["key"] = entry["key"]
    1175                current = existing
    1176                current["created"] = False
    1177                current["substantive_update"] = False
    1178            job_metadata["lessons"] = lessons[-200:]
    1179            job_metadata["last_lesson"] = current
    1180            conn.execute(
    1181                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
    1182                (now, _json_dumps(job_metadata), job_id),
    1183            )
    1184            return current
    1185
    1186        return self._write(op)
    1187
    1188    def append_memory_graph_records(
    1189        self,
    1190        job_id: str,
    1191        *,
    1192        nodes: list[dict[str, Any]] | None = None,
    1193        edges: list[dict[str, Any]] | None = None,
    1194    ) -> dict[str, Any]:
    1195        now = utc_now()
    1196        node_items = [node for node in (nodes or []) if isinstance(node, dict)]
    1197        edge_items = [edge for edge in (edges or []) if isinstance(edge, dict)]
    1198        if not node_items and not edge_items:
    1199            raise ValueError("nodes or edges are required")
    1200
    1201        def op(conn: sqlite3.Connection) -> dict[str, Any]:
    1202            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
    1203            if row is None:
    1204                raise KeyError(f"Job not found: {job_id}")
    1205            job_metadata = json.loads(row["metadata_json"] or "{}")
    1206            graph = job_metadata.get("memory_graph") if isinstance(job_metadata.get("memory_graph"), dict) else {}
    1207            stored_nodes = _metadata_list(graph, "nodes")
    1208            stored_edges = _metadata_list(graph, "edges")
    1209            node_by_key = {str(node.get("key") or ""): node for node in stored_nodes if node.get("key")}
    1210            added_nodes = 0
    1211            updated_nodes = 0
    1212            touched_nodes: list[dict[str, Any]] = []
    1213
    1214            for node in node_items[:50]:
    1215                title = str(node.get("title") or node.get("name") or "").strip()
    1216                summary = str(node.get("summary") or node.get("body") or "").strip()
    1217                if not title and not summary:
    1218                    continue
    1219                key = _norm_key(str(node.get("key") or title or summary[:80]))
    1220                current = node_by_key.get(key)
    1221                created = current is None
    1222                if current is None:
    1223                    current = {
    1224                        "key": key,
    1225                        "title": title or key,
    1226                        "kind": DEFAULT_NODE_KIND,
    1227                        "status": DEFAULT_NODE_STATUS,
    1228                        "summary": "",
    1229                        "tags": [],
    1230                        "evidence_refs": [],
    1231                        "links": [],
    1232                        "metadata": {},
    1233                        "created_at": now,
    1234                    }
    1235                    stored_nodes.append(current)
    1236                    node_by_key[key] = current
    1237                    added_nodes += 1
    1238                else:
    1239                    updated_nodes += 1
    1240                if title:
    1241                    current["title"] = title
    1242                if summary:
    1243                    current["summary"] = summary
    1244                kind = str(node.get("kind") or current.get("kind") or DEFAULT_NODE_KIND).strip().lower()
    1245                current["kind"] = kind if kind in NODE_KINDS else DEFAULT_NODE_KIND
    1246                status = str(node.get("status") or current.get("status") or DEFAULT_NODE_STATUS).strip().lower()
    1247                current["status"] = status if status in NODE_STATUSES else DEFAULT_NODE_STATUS
    1248                if "salience" in node:
    1249                    current["salience"] = _bounded_float(node.get("salience"), 0.0, 1.0)
    1250                elif "salience" not in current:
    1251                    current["salience"] = 0.5
    1252                if "confidence" in node:
    1253                    current["confidence"] = _bounded_float(node.get("confidence"), 0.0, 1.0)
    1254                elif "confidence" not in current:
    1255                    current["confidence"] = 0.5
    1256                parent_key = str(node.get("parent_key") or node.get("parent") or "").strip()
    1257                if parent_key:
    1258                    current["parent_key"] = _norm_key(parent_key)
    1259                current["tags"] = _merge_string_lists(current.get("tags"), node.get("tags"), limit=24)
    1260                current["evidence_refs"] = _merge_string_lists(current.get("evidence_refs"), node.get("evidence_refs") or node.get("evidence"), limit=24)
    1261                current["links"] = _merge_string_lists(current.get("links"), node.get("links"), limit=50)
    1262                if isinstance(node.get("metadata"), dict):
    1263                    merged = current.get("metadata") if isinstance(current.get("metadata"), dict) else {}
    1264                    merged.update(node["metadata"])
    1265                    current["metadata"] = merged
    1266                current["created"] = created
    1267                current["updated_at"] = now
    1268                current["use_count"] = int(current.get("use_count") or 0)
    1269                touched_nodes.append(current)
    1270
    1271            existing_edge_keys = {
    1272                _memory_edge_key(edge)
    1273                for edge in stored_edges
    1274                if _memory_edge_key(edge)
    1275            }
    1276            added_edges = 0
    1277            touched_edges: list[dict[str, Any]] = []
    1278            for edge in edge_items[:100]:
    1279                from_key = _norm_key(str(edge.get("from_key") or edge.get("from") or "").strip())
    1280                to_key = _norm_key(str(edge.get("to_key") or edge.get("to") or "").strip())
    1281                if not from_key or not to_key:
    1282                    continue
    1283                relation = str(edge.get("relation") or "related_to").strip().lower().replace(" ", "_")
    1284                relation = re.sub(r"[^a-z0-9_-]+", "_", relation).strip("_") or "related_to"
    1285                edge_key = f"{from_key}|{relation}|{to_key}"
    1286                if edge_key in existing_edge_keys:
    1287                    continue
    1288                stored = {
    1289                    "key": edge_key,
    1290                    "from_key": from_key,
    1291                    "to_key": to_key,
    1292                    "relation": relation,
    1293                    "evidence_refs": _merge_string_lists([], edge.get("evidence_refs") or edge.get("evidence"), limit=24),
    1294                    "metadata": edge.get("metadata") if isinstance(edge.get("metadata"), dict) else {},
    1295                    "created_at": now,
    1296                    "updated_at": now,
    1297                }
    1298                stored_edges.append(stored)
    1299                existing_edge_keys.add(edge_key)
    1300                touched_edges.append(stored)
    1301                added_edges += 1
    1302
    1303            graph = {
    1304                "nodes": stored_nodes[-1000:],
    1305                "edges": stored_edges[-2000:],
    1306                "updated_at": now,
    1307            }
    1308            event = _insert_event(
    1309                conn,
    1310                job_id=job_id,
    1311                event_type="memory_node",
    1312                title="memory graph",
    1313                body=f"nodes +{added_nodes}/~{updated_nodes}; edges +{added_edges}",
    1314                metadata={
    1315                    "added_nodes": added_nodes,
    1316                    "updated_nodes": updated_nodes,
    1317                    "added_edges": added_edges,
    1318                    "node_keys": [node.get("key") for node in touched_nodes[-20:]],
    1319                    "edge_keys": [edge.get("key") for edge in touched_edges[-20:]],
    1320                },
    1321                created_at=now,
    1322            )
    1323            graph["event_id"] = event["id"]
    1324            job_metadata["memory_graph"] = graph
    1325            job_metadata["last_memory_graph_record"] = {
    1326                "at": now,
    1327                "event_id": event["id"],
    1328                "added_nodes": added_nodes,
    1329                "updated_nodes": updated_nodes,
    1330                "added_edges": added_edges,
    1331                "nodes": touched_nodes[-20:],
    1332                "edges": touched_edges[-20:],
    1333            }
    1334            conn.execute(
    1335                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
    1336                (now, _json_dumps(job_metadata), job_id),
    1337            )
    1338            return {
    1339                "added_nodes": added_nodes,
    1340                "updated_nodes": updated_nodes,
    1341                "added_edges": added_edges,
    1342                "nodes": touched_nodes,
    1343                "edges": touched_edges,
    1344                "event_id": event["id"],
    1345            }
    1346
    1347        return self._write(op)
    1348
    1349    def append_source_record(
    1350        self,
    1351        job_id: str,
    1352        source: str,
    1353        *,
    1354        source_type: str = "",
    1355        usefulness_score: float | None = None,
    1356        yield_count: int = 0,
    1357        fail_count_delta: int = 0,
    1358        warnings: list[str] | None = None,
    1359        outcome: str = "",
    1360        metadata: dict[str, Any] | None = None,
    1361    ) -> dict[str, Any]:
    1362        now = utc_now()
    1363        text = source.strip()
    1364        if not text:
    1365            raise ValueError("source is required")
    1366        key = _norm_key(text)
    1367
    1368        def op(conn: sqlite3.Connection) -> dict[str, Any]:
    1369            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
    1370            if row is None:
    1371                raise KeyError(f"Job not found: {job_id}")
    1372            job_metadata = json.loads(row["metadata_json"] or "{}")
    1373            sources = _metadata_list(job_metadata, "source_ledger")
    1374            current = next((entry for entry in sources if entry.get("key") == key), None)
    1375            created = current is None
    1376            change_fields = (
    1377                "source",
    1378                "source_type",
    1379                "usefulness_score",
    1380                "fail_count",
    1381                "yield_count",
    1382                "warnings",
    1383                "last_outcome",
    1384                "metadata",
    1385            )
    1386            before = "" if created else _change_fingerprint(current, change_fields)
    1387            if current is None:
    1388                current = {
    1389                    "key": key,
    1390                    "source": text,
    1391                    "source_type": source_type.strip() or "unknown",
    1392                    "usefulness_score": 0.0,
    1393                    "fail_count": 0,
    1394                    "yield_count": 0,
    1395                    "warnings": [],
    1396                    "last_outcome": "",
    1397                    "metadata": {},
    1398                    "first_seen": now,
    1399                }
    1400                sources.append(current)
    1401            if source_type:
    1402                current["source_type"] = source_type.strip()
    1403            if usefulness_score is not None:
    1404                current["usefulness_score"] = float(usefulness_score)
    1405            if yield_count:
    1406                current["yield_count"] = int(current.get("yield_count") or 0) + int(yield_count)
    1407            if fail_count_delta:
    1408                current["fail_count"] = int(current.get("fail_count") or 0) + int(fail_count_delta)
    1409            if warnings:
    1410                merged = list(dict.fromkeys([*current.get("warnings", []), *[str(warning) for warning in warnings]]))
    1411                current["warnings"] = merged[-20:]
    1412            if outcome:
    1413                current["last_outcome"] = outcome.strip()
    1414            if metadata:
    1415                merged_metadata = current.get("metadata") if isinstance(current.get("metadata"), dict) else {}
    1416                merged_metadata.update(metadata)
    1417                current["metadata"] = merged_metadata
    1418            current["created"] = created
    1419            substantive_update = created or before != _change_fingerprint(current, change_fields)
    1420            current["substantive_update"] = substantive_update
    1421            if substantive_update:
    1422                current["updated_at"] = now
    1423            current["last_seen"] = now
    1424            if substantive_update:
    1425                event = _insert_event(
    1426                    conn,
    1427                    job_id=job_id,
    1428                    event_type="source",
    1429                    title=current.get("source") or text,
    1430                    body=current.get("last_outcome") or outcome,
    1431                    metadata={
    1432                        "created": created,
    1433                        "substantive_update": substantive_update,
    1434                        "source_type": current.get("source_type"),
    1435                        "usefulness_score": current.get("usefulness_score"),
    1436                        "yield_count": current.get("yield_count"),
    1437                        "fail_count": current.get("fail_count"),
    1438                        "warnings": current.get("warnings") or [],
    1439                        **(current.get("metadata") if isinstance(current.get("metadata"), dict) else {}),
    1440                    },
    1441                    created_at=now,
    1442                )
    1443                current["event_id"] = event["id"]
    1444            job_metadata["source_ledger"] = sources[-250:]
    1445            job_metadata["last_source_record"] = current
    1446            conn.execute(
    1447                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
    1448                (now, _json_dumps(job_metadata), job_id),
    1449            )
    1450            return current
    1451
    1452        return self._write(op)
    1453
    1454    def append_finding_record(
    1455        self,
    1456        job_id: str,
    1457        *,
    1458        name: str,
    1459        url: str = "",
    1460        source_url: str = "",
    1461        category: str = "",
    1462        location: str = "",
    1463        contact: str = "",
    1464        reason: str = "",
    1465        status: str = "new",
    1466        score: float | None = None,
    1467        evidence_artifact: str = "",
    1468        metadata: dict[str, Any] | None = None,
    1469    ) -> dict[str, Any]:
    1470        now = utc_now()
    1471        name = name.strip()
    1472        if not name:
    1473            raise ValueError("name is required")
    1474        url = url.strip()
    1475        source_url = source_url.strip()
    1476        key = _norm_key(f"{name}|{url or source_url}")
    1477
    1478        def op(conn: sqlite3.Connection) -> dict[str, Any]:
    1479            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
    1480            if row is None:
    1481                raise KeyError(f"Job not found: {job_id}")
    1482            job_metadata = json.loads(row["metadata_json"] or "{}")
    1483            findings = _metadata_list(job_metadata, "finding_ledger")
    1484            current = next((entry for entry in findings if entry.get("key") == key), None)
    1485            created = current is None
    1486            change_fields = (
    1487                "url",
    1488                "source_url",
    1489                "category",
    1490                "location",
    1491                "contact",
    1492                "reason",
    1493                "status",
    1494                "score",
    1495                "evidence_artifact",
    1496                "metadata",
    1497            )
    1498            before = "" if created else _change_fingerprint(current, change_fields)
    1499            if current is None:
    1500                current = {
    1501                    "key": key,
    1502                    "name": name,
    1503                    "url": url,
    1504                    "source_url": source_url,
    1505                    "category": category.strip(),
    1506                    "location": location.strip(),
    1507                    "contact": contact.strip(),
    1508                    "reason": reason.strip(),
    1509                    "status": status.strip() or "new",
    1510                    "score": score,
    1511                    "evidence_artifact": evidence_artifact.strip(),
    1512                    "metadata": metadata or {},
    1513                    "created_at": now,
    1514                }
    1515                findings.append(current)
    1516            else:
    1517                for field, value in {
    1518                    "url": url,
    1519                    "source_url": source_url,
    1520                    "category": category.strip(),
    1521                    "location": location.strip(),
    1522                    "contact": contact.strip(),
    1523                    "reason": reason.strip(),
    1524                    "status": status.strip(),
    1525                    "evidence_artifact": evidence_artifact.strip(),
    1526                }.items():
    1527                    if value:
    1528                        current[field] = value
    1529                if score is not None:
    1530                    current["score"] = score
    1531                if metadata:
    1532                    merged_metadata = current.get("metadata") if isinstance(current.get("metadata"), dict) else {}
    1533                    merged_metadata.update(metadata)
    1534                    current["metadata"] = merged_metadata
    1535            current["created"] = created
    1536            substantive_update = created or before != _change_fingerprint(current, change_fields)
    1537            current["substantive_update"] = substantive_update
    1538            if substantive_update:
    1539                current["updated_at"] = now
    1540            if substantive_update:
    1541                event = _insert_event(
    1542                    conn,
    1543                    job_id=job_id,
    1544                    event_type="finding",
    1545                    title=current.get("name") or name,
    1546                    body=current.get("reason") or current.get("category") or "",
    1547                    metadata={
    1548                        "created": created,
    1549                        "substantive_update": substantive_update,
    1550                        "score": current.get("score"),
    1551                        "status": current.get("status"),
    1552                        "source_url": current.get("source_url"),
    1553                        "evidence_artifact": current.get("evidence_artifact"),
    1554                        **(current.get("metadata") if isinstance(current.get("metadata"), dict) else {}),
    1555                    },
    1556                    created_at=now,
    1557                )
    1558                current["event_id"] = event["id"]
    1559            job_metadata["finding_ledger"] = findings[-1000:]
    1560            job_metadata["last_finding_record"] = current
    1561            conn.execute(
    1562                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
    1563                (now, _json_dumps(job_metadata), job_id),
    1564            )
    1565            return current
    1566
    1567        return self._write(op)
    1568
    1569    def append_roadmap_record(
    1570        self,
    1571        job_id: str,
    1572        *,
    1573        title: str,
    1574        status: str = "planned",
    1575        objective: str = "",
    1576        scope: str = "",
    1577        current_milestone: str = "",
    1578        validation_contract: str = "",
    1579        milestones: list[dict[str, Any]] | None = None,
    1580        metadata: dict[str, Any] | None = None,
    1581    ) -> dict[str, Any]:
    1582        now = utc_now()
    1583        title = title.strip()
    1584        if not title:
    1585            raise ValueError("title is required")
    1586        status = _clean_status(status, {"planned", "active", "validating", "done", "blocked", "paused"}, "planned")
    1587        milestone_items = milestones if isinstance(milestones, list) else []
    1588
    1589        def merge_feature(existing_features: list[dict[str, Any]], feature: dict[str, Any]) -> tuple[dict[str, Any] | None, bool, bool]:
    1590            feature_title = str(feature.get("title") or feature.get("name") or "").strip()
    1591            if not feature_title:
    1592                return None, False, False
    1593            feature_key = _norm_key(str(feature.get("key") or feature_title))
    1594            feature_title_key = _norm_key(feature_title)
    1595            current = next(
    1596                (
    1597                    entry for entry in existing_features
    1598                    if entry.get("key") == feature_key
    1599                    or _norm_key(str(entry.get("title") or "")) == feature_title_key
    1600                ),
    1601                None,
    1602            )
    1603            created = current is None
    1604            change_fields = (
    1605                "title",
    1606                "status",
    1607                "goal",
    1608                "output_contract",
    1609                "acceptance_criteria",
    1610                "evidence_needed",
    1611                "result",
    1612                "metadata",
    1613            )
    1614            before = "" if created else _change_fingerprint(current, change_fields)
    1615            if current is None:
    1616                current = {
    1617                    "key": feature_key,
    1618                    "title": feature_title,
    1619                    "status": _clean_status(str(feature.get("status") or "planned"), {"planned", "active", "done", "blocked", "skipped"}, "planned"),
    1620                    "goal": str(feature.get("goal") or feature.get("description") or "").strip(),
    1621                    "output_contract": str(feature.get("output_contract") or feature.get("contract") or "").strip().lower().replace(" ", "_"),
    1622                    "acceptance_criteria": str(feature.get("acceptance_criteria") or "").strip(),
    1623                    "evidence_needed": str(feature.get("evidence_needed") or "").strip(),
    1624                    "result": str(feature.get("result") or feature.get("outcome") or "").strip(),
    1625                    "metadata": feature.get("metadata") if isinstance(feature.get("metadata"), dict) else {},
    1626                    "created_at": now,
    1627                }
    1628                existing_features.append(current)
    1629            else:
    1630                current["status"] = _clean_status(str(feature.get("status") or current.get("status") or "planned"), {"planned", "active", "done", "blocked", "skipped"}, "planned")
    1631                for field, value in {
    1632                    "title": feature_title,
    1633                    "goal": str(feature.get("goal") or feature.get("description") or "").strip(),
    1634                    "output_contract": str(feature.get("output_contract") or feature.get("contract") or "").strip().lower().replace(" ", "_"),
    1635                    "acceptance_criteria": str(feature.get("acceptance_criteria") or "").strip(),
    1636                    "evidence_needed": str(feature.get("evidence_needed") or "").strip(),
    1637                    "result": str(feature.get("result") or feature.get("outcome") or "").strip(),
    1638                }.items():
    1639                    if value:
    1640                        current[field] = value
    1641                if isinstance(feature.get("metadata"), dict):
    1642                    merged = current.get("metadata") if isinstance(current.get("metadata"), dict) else {}
    1643                    merged.update(feature["metadata"])
    1644                    current["metadata"] = merged
    1645            if current.get("output_contract") not in {"research", "artifact", "experiment", "action", "monitor", "decision", "report", "validation"}:
    1646                current["output_contract"] = ""
    1647            current["created"] = created
    1648            changed = created or before != _change_fingerprint(current, change_fields)
    1649            current["substantive_update"] = changed
    1650            if changed:
    1651                current["updated_at"] = now
    1652            return current, created, changed
    1653
    1654        def op(conn: sqlite3.Connection) -> dict[str, Any]:
    1655            row = conn.execute("SELECT objective, metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
    1656            if row is None:
    1657                raise KeyError(f"Job not found: {job_id}")
    1658            job_metadata = json.loads(row["metadata_json"] or "{}")
    1659            roadmap = job_metadata.get("roadmap")
    1660            created = not isinstance(roadmap, dict)
    1661            roadmap_change_fields = (
    1662                "title",
    1663                "status",
    1664                "objective",
    1665                "scope",
    1666                "validation_contract",
    1667                "current_milestone",
    1668                "metadata",
    1669            )
    1670            roadmap_before = "" if created else _change_fingerprint(roadmap, roadmap_change_fields)
    1671            if created:
    1672                roadmap = {
    1673                    "key": _norm_key(title),
    1674                    "title": title,
    1675                    "status": status,
    1676                    "objective": objective.strip() or str(row["objective"] or "").strip(),
    1677                    "scope": scope.strip(),
    1678                    "validation_contract": validation_contract.strip(),
    1679                    "current_milestone": current_milestone.strip(),
    1680                    "milestones": [],
    1681                    "metadata": metadata or {},
    1682                    "created_at": now,
    1683                }
    1684            else:
    1685                roadmap["title"] = title or roadmap.get("title") or "Roadmap"
    1686                roadmap["status"] = status
    1687                for field, value in {
    1688                    "objective": objective.strip(),
    1689                    "scope": scope.strip(),
    1690                    "validation_contract": validation_contract.strip(),
    1691                    "current_milestone": current_milestone.strip(),
    1692                }.items():
    1693                    if value:
    1694                        roadmap[field] = value
    1695                if metadata:
    1696                    merged_metadata = roadmap.get("metadata") if isinstance(roadmap.get("metadata"), dict) else {}
    1697                    merged_metadata.update(metadata)
    1698                    roadmap["metadata"] = merged_metadata
    1699
    1700            stored_milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
    1701            added_milestones = 0
    1702            updated_milestones = 0
    1703            added_features = 0
    1704            updated_features = 0
    1705            touched: list[dict[str, Any]] = []
    1706            for milestone in milestone_items[:100]:
    1707                if not isinstance(milestone, dict):
    1708                    continue
    1709                milestone_title = str(milestone.get("title") or milestone.get("name") or "").strip()
    1710                if not milestone_title:
    1711                    continue
    1712                milestone_key = _norm_key(str(milestone.get("key") or milestone_title))
    1713                milestone_title_key = _norm_key(milestone_title)
    1714                current = next(
    1715                    (
    1716                        entry for entry in stored_milestones
    1717                        if entry.get("key") == milestone_key
    1718                        or _norm_key(str(entry.get("title") or "")) == milestone_title_key
    1719                    ),
    1720                    None,
    1721                )
    1722                milestone_created = current is None
    1723                milestone_change_fields = (
    1724                    "title",
    1725                    "status",
    1726                    "priority",
    1727                    "goal",
    1728                    "acceptance_criteria",
    1729                    "evidence_needed",
    1730                    "validation_status",
    1731                    "validation_result",
    1732                    "next_action",
    1733                    "metadata",
    1734                )
    1735                milestone_before = "" if milestone_created else _change_fingerprint(current, milestone_change_fields)
    1736                if current is None:
    1737                    current = {
    1738                        "key": milestone_key,
    1739                        "title": milestone_title,
    1740                        "status": _clean_status(str(milestone.get("status") or "planned"), {"planned", "active", "validating", "done", "blocked", "skipped"}, "planned"),
    1741                        "priority": int(milestone.get("priority") or 0),
    1742                        "goal": str(milestone.get("goal") or milestone.get("description") or "").strip(),
    1743                        "acceptance_criteria": str(milestone.get("acceptance_criteria") or "").strip(),
    1744                        "evidence_needed": str(milestone.get("evidence_needed") or "").strip(),
    1745                        "validation_status": _clean_status(str(milestone.get("validation_status") or "not_started"), {"not_started", "pending", "passed", "failed", "blocked"}, "not_started"),
    1746                        "validation_result": str(milestone.get("validation_result") or "").strip(),
    1747                        "next_action": str(milestone.get("next_action") or "").strip(),
    1748                        "features": [],
    1749                        "metadata": milestone.get("metadata") if isinstance(milestone.get("metadata"), dict) else {},
    1750                        "created_at": now,
    1751                    }
    1752                    stored_milestones.append(current)
    1753                    added_milestones += 1
    1754                else:
    1755                    current["status"] = _clean_status(str(milestone.get("status") or current.get("status") or "planned"), {"planned", "active", "validating", "done", "blocked", "skipped"}, "planned")
    1756                    if "priority" in milestone:
    1757                        current["priority"] = int(milestone.get("priority") or 0)
    1758                    for field, value in {
    1759                        "title": milestone_title,
    1760                        "goal": str(milestone.get("goal") or milestone.get("description") or "").strip(),
    1761                        "acceptance_criteria": str(milestone.get("acceptance_criteria") or "").strip(),
    1762                        "evidence_needed": str(milestone.get("evidence_needed") or "").strip(),
    1763                        "validation_status": _clean_status(str(milestone.get("validation_status") or ""), {"not_started", "pending", "passed", "failed", "blocked"}, ""),
    1764                        "validation_result": str(milestone.get("validation_result") or "").strip(),
    1765                        "next_action": str(milestone.get("next_action") or "").strip(),
    1766                    }.items():
    1767                        if value:
    1768                            current[field] = value
    1769                    if isinstance(milestone.get("metadata"), dict):
    1770                        merged_metadata = current.get("metadata") if isinstance(current.get("metadata"), dict) else {}
    1771                        merged_metadata.update(milestone["metadata"])
    1772                        current["metadata"] = merged_metadata
    1773                feature_items = milestone.get("features") if isinstance(milestone.get("features"), list) else []
    1774                features = current.get("features") if isinstance(current.get("features"), list) else []
    1775                feature_changed = False
    1776                for feature in feature_items[:100]:
    1777                    if not isinstance(feature, dict):
    1778                        continue
    1779                    stored_feature, feature_created, feature_updated = merge_feature(features, feature)
    1780                    if stored_feature is None:
    1781                        continue
    1782                    if feature_created:
    1783                        added_features += 1
    1784                    elif feature_updated:
    1785                        updated_features += 1
    1786                    feature_changed = feature_changed or feature_updated
    1787                current["features"] = features[-500:]
    1788                current["created"] = milestone_created
    1789                milestone_changed = milestone_created or milestone_before != _change_fingerprint(current, milestone_change_fields)
    1790                current["substantive_update"] = milestone_changed or feature_changed
    1791                if current["substantive_update"]:
    1792                    current["updated_at"] = now
    1793                if not milestone_created and milestone_changed:
    1794                    updated_milestones += 1
    1795                touched.append(current)
    1796
    1797            roadmap["milestones"] = stored_milestones[-500:]
    1798            roadmap["created"] = created
    1799            roadmap_substantive_update = created or roadmap_before != _change_fingerprint(roadmap, roadmap_change_fields)
    1800            roadmap["substantive_update"] = roadmap_substantive_update
    1801            if roadmap_substantive_update or added_milestones or updated_milestones or added_features or updated_features:
    1802                roadmap["updated_at"] = now
    1803            roadmap["added_milestones"] = added_milestones
    1804            roadmap["updated_milestones"] = updated_milestones
    1805            roadmap["added_features"] = added_features
    1806            roadmap["updated_features"] = updated_features
    1807            roadmap_has_change = bool(
    1808                roadmap_substantive_update
    1809                or added_milestones
    1810                or updated_milestones
    1811                or added_features
    1812                or updated_features
    1813            )
    1814            if roadmap_has_change:
    1815                event = _insert_event(
    1816                    conn,
    1817                    job_id=job_id,
    1818                    event_type="roadmap",
    1819                    title=roadmap.get("title") or title,
    1820                    body=f"{roadmap.get('status')} | milestones +{added_milestones}/~{updated_milestones} | features +{added_features}/~{updated_features}",
    1821                    metadata={
    1822                        "created": created,
    1823                        "substantive_update": roadmap_substantive_update,
    1824                        "status": roadmap.get("status"),
    1825                        "current_milestone": roadmap.get("current_milestone"),
    1826                        "milestone_count": len(roadmap.get("milestones") or []),
    1827                        "added_milestones": added_milestones,
    1828                        "updated_milestones": updated_milestones,
    1829                        "added_features": added_features,
    1830                        "updated_features": updated_features,
    1831                        "roadmap_updated": roadmap_substantive_update and not created,
    1832                    },
    1833                    created_at=now,
    1834                )
    1835                roadmap["event_id"] = event["id"]
    1836            job_metadata["roadmap"] = roadmap
    1837            job_metadata["last_roadmap_record"] = {
    1838                "at": now,
    1839                "updated_at": roadmap.get("updated_at") or now,
    1840                "event_id": roadmap.get("event_id"),
    1841                "created": created,
    1842                "substantive_update": roadmap_substantive_update,
    1843                "title": roadmap.get("title"),
    1844                "status": roadmap.get("status"),
    1845                "added_milestones": added_milestones,
    1846                "updated_milestones": updated_milestones,
    1847                "added_features": added_features,
    1848                "updated_features": updated_features,
    1849                "roadmap_updated": roadmap_substantive_update and not created,
    1850                "milestones": touched[-10:],
    1851            }
    1852            conn.execute(
    1853                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
    1854                (now, _json_dumps(job_metadata), job_id),
    1855            )
    1856            return roadmap
    1857
    1858        return self._write(op)
    1859
    1860    def append_milestone_validation_record(
    1861        self,
    1862        job_id: str,
    1863        *,
    1864        milestone: str,
    1865        validation_status: str = "pending",
    1866        result: str = "",
    1867        evidence: str = "",
    1868        issues: list[str] | None = None,
    1869        next_action: str = "",
    1870        metadata: dict[str, Any] | None = None,
    1871    ) -> dict[str, Any]:
    1872        now = utc_now()
    1873        milestone = milestone.strip()
    1874        if not milestone:
    1875            raise ValueError("milestone is required")
    1876        validation_status = _clean_status(validation_status, {"pending", "passed", "failed", "blocked"}, "pending")
    1877        issue_values = [str(issue).strip() for issue in (issues or []) if str(issue).strip()]
    1878        milestone_key = _norm_key(milestone)
    1879
    1880        def op(conn: sqlite3.Connection) -> dict[str, Any]:
    1881            row = conn.execute("SELECT objective, metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
    1882            if row is None:
    1883                raise KeyError(f"Job not found: {job_id}")
    1884            job_metadata = json.loads(row["metadata_json"] or "{}")
    1885            roadmap = job_metadata.get("roadmap")
    1886            if not isinstance(roadmap, dict):
    1887                roadmap = {
    1888                    "key": _norm_key(str(row["objective"] or "roadmap")),
    1889                    "title": "Roadmap",
    1890                    "status": "active",
    1891                    "objective": str(row["objective"] or ""),
    1892                    "scope": "",
    1893                    "validation_contract": "",
    1894                    "current_milestone": milestone,
    1895                    "milestones": [],
    1896                    "metadata": {},
    1897                    "created_at": now,
    1898                }
    1899            milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
    1900            current = next(
    1901                (
    1902                    entry for entry in milestones
    1903                    if entry.get("key") == milestone_key
    1904                    or _norm_key(str(entry.get("title") or "")) == milestone_key
    1905                ),
    1906                None,
    1907            )
    1908            created = current is None
    1909            if current is None:
    1910                current = {
    1911                    "key": milestone_key,
    1912                    "title": milestone,
    1913                    "status": "validating" if validation_status == "pending" else ("done" if validation_status == "passed" else "blocked"),
    1914                    "priority": 0,
    1915                    "goal": "",
    1916                    "acceptance_criteria": "",
    1917                    "evidence_needed": "",
    1918                    "features": [],
    1919                    "metadata": {},
    1920                    "created_at": now,
    1921                }
    1922                milestones.append(current)
    1923            current["validation_status"] = validation_status
    1924            current["validation_result"] = result.strip()
    1925            current["validation_evidence"] = evidence.strip()
    1926            current["validation_issues"] = issue_values
    1927            current["next_action"] = next_action.strip()
    1928            if validation_status == "passed":
    1929                current["status"] = "done"
    1930            elif validation_status == "pending":
    1931                current["status"] = "validating"
    1932            elif validation_status in {"failed", "blocked"}:
    1933                current["status"] = "blocked"
    1934            if metadata:
    1935                merged_metadata = current.get("metadata") if isinstance(current.get("metadata"), dict) else {}
    1936                merged_metadata.update(metadata)
    1937                current["metadata"] = merged_metadata
    1938            current["updated_at"] = now
    1939            current["created"] = created
    1940            roadmap["milestones"] = milestones[-500:]
    1941            roadmap["status"] = "active" if validation_status in {"failed", "blocked"} else ("validating" if validation_status == "pending" else roadmap.get("status") or "active")
    1942            roadmap["current_milestone"] = current.get("title") or milestone
    1943            roadmap["updated_at"] = now
    1944            event = _insert_event(
    1945                conn,
    1946                job_id=job_id,
    1947                event_type="milestone_validation",
    1948                title=current.get("title") or milestone,
    1949                body=result.strip() or validation_status,
    1950                metadata={
    1951                    "created": created,
    1952                    "validation_status": validation_status,
    1953                    "evidence": evidence.strip(),
    1954                    "issues": issue_values,
    1955                    "next_action": next_action.strip(),
    1956                    **(metadata or {}),
    1957                },
    1958                created_at=now,
    1959            )
    1960            current["validation_event_id"] = event["id"]
    1961            job_metadata["roadmap"] = roadmap
    1962            job_metadata["last_milestone_validation"] = {
    1963                "at": now,
    1964                "validated_at": now,
    1965                "event_id": event["id"],
    1966                "milestone": current.get("title"),
    1967                "validation_status": validation_status,
    1968                "result": result.strip(),
    1969                "issues": issue_values,
    1970                "next_action": next_action.strip(),
    1971            }
    1972            conn.execute(
    1973                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
    1974                (now, _json_dumps(job_metadata), job_id),
    1975            )
    1976            return current
    1977
    1978        return self._write(op)
    1979
    1980    def append_task_record(
    1981        self,
    1982        job_id: str,
    1983        *,
    1984        title: str,
    1985        status: str = "open",
    1986        priority: int = 0,
    1987        goal: str = "",
    1988        source_hint: str = "",
    1989        result: str = "",
    1990        parent: str = "",
    1991        output_contract: str = "",
    1992        acceptance_criteria: str = "",
    1993        evidence_needed: str = "",
    1994        stall_behavior: str = "",
    1995        metadata: dict[str, Any] | None = None,
    1996    ) -> dict[str, Any]:
    1997        now = utc_now()
    1998        title = title.strip()
    1999        if not title:
    2000            raise ValueError("title is required")
    2001        status = (status.strip().lower() or "open").replace(" ", "_")
    2002        if status not in {"open", "active", "done", "blocked", "skipped"}:
    2003            status = "open"
    2004        output_contract = output_contract.strip().lower().replace(" ", "_")
    2005        if output_contract not in {"research", "artifact", "experiment", "action", "monitor", "decision", "report"}:
    2006            output_contract = ""
    2007        key = _norm_key(f"{parent}|{title}")
    2008
    2009        def op(conn: sqlite3.Connection) -> dict[str, Any]:
    2010            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
    2011            if row is None:
    2012                raise KeyError(f"Job not found: {job_id}")
    2013            job_metadata = json.loads(row["metadata_json"] or "{}")
    2014            tasks = _metadata_list(job_metadata, "task_queue")
    2015            current = next(
    2016                (
    2017                    entry
    2018                    for entry in tasks
    2019                    if entry.get("key") == key
    2020                    or (
    2021                        not entry.get("key")
    2022                        and _norm_key(f"{entry.get('parent') or ''}|{entry.get('title') or ''}") == key
    2023                    )
    2024                ),
    2025                None,
    2026            )
    2027            created = current is None
    2028            change_fields = (
    2029                "status",
    2030                "priority",
    2031                "goal",
    2032                "source_hint",
    2033                "result",
    2034                "parent",
    2035                "output_contract",
    2036                "acceptance_criteria",
    2037                "evidence_needed",
    2038                "stall_behavior",
    2039                "metadata",
    2040            )
    2041            before = "" if created else _change_fingerprint(current, change_fields)
    2042            if current is None:
    2043                current = {
    2044                    "key": key,
    2045                    "title": title,
    2046                    "status": status,
    2047                    "priority": int(priority),
    2048                    "goal": goal.strip(),
    2049                    "source_hint": source_hint.strip(),
    2050                    "result": result.strip(),
    2051                    "parent": parent.strip(),
    2052                    "output_contract": output_contract,
    2053                    "acceptance_criteria": acceptance_criteria.strip(),
    2054                    "evidence_needed": evidence_needed.strip(),
    2055                    "stall_behavior": stall_behavior.strip(),
    2056                    "metadata": metadata or {},
    2057                    "created_at": now,
    2058                }
    2059                tasks.append(current)
    2060            else:
    2061                current["status"] = status
    2062                current["priority"] = int(priority)
    2063                for field, value in {
    2064                    "goal": goal.strip(),
    2065                    "source_hint": source_hint.strip(),
    2066                    "result": result.strip(),
    2067                    "parent": parent.strip(),
    2068                    "output_contract": output_contract,
    2069                    "acceptance_criteria": acceptance_criteria.strip(),
    2070                    "evidence_needed": evidence_needed.strip(),
    2071                    "stall_behavior": stall_behavior.strip(),
    2072                }.items():
    2073                    if value:
    2074                        current[field] = value
    2075                if metadata:
    2076                    merged_metadata = current.get("metadata") if isinstance(current.get("metadata"), dict) else {}
    2077                    merged_metadata.update(metadata)
    2078                    current["metadata"] = merged_metadata
    2079            current["created"] = created
    2080            substantive_update = created or before != _change_fingerprint(current, change_fields)
    2081            current["substantive_update"] = substantive_update
    2082            if substantive_update:
    2083                current["updated_at"] = now
    2084            if substantive_update:
    2085                event = _insert_event(
    2086                    conn,
    2087                    job_id=job_id,
    2088                    event_type="task",
    2089                    title=current.get("title") or title,
    2090                    body=current.get("result") or current.get("goal") or "",
    2091                    metadata={
    2092                        "created": created,
    2093                        "substantive_update": substantive_update,
    2094                        "status": current.get("status"),
    2095                        "priority": current.get("priority"),
    2096                        "parent": current.get("parent"),
    2097                        "source_hint": current.get("source_hint"),
    2098                        "output_contract": current.get("output_contract"),
    2099                        "acceptance_criteria": current.get("acceptance_criteria"),
    2100                        "evidence_needed": current.get("evidence_needed"),
    2101                        "stall_behavior": current.get("stall_behavior"),
    2102                        **(current.get("metadata") if isinstance(current.get("metadata"), dict) else {}),
    2103                    },
    2104                    created_at=now,
    2105                )
    2106                current["event_id"] = event["id"]
    2107            job_metadata["task_queue"] = tasks[-500:]
    2108            job_metadata["last_task_record"] = current
    2109            conn.execute(
    2110                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
    2111                (now, _json_dumps(job_metadata), job_id),
    2112            )
    2113            return current
    2114
    2115        return self._write(op)
    2116
    2117    def append_experiment_record(
    2118        self,
    2119        job_id: str,
    2120        *,
    2121        title: str,
    2122        hypothesis: str = "",
    2123        status: str = "planned",
    2124        metric_name: str = "",
    2125        metric_value: float | None = None,
    2126        metric_unit: str = "",
    2127        higher_is_better: bool = True,
    2128        baseline_value: float | None = None,
    2129        config: dict[str, Any] | None = None,
    2130        result: str = "",
    2131        evidence_artifact: str = "",
    2132        next_action: str = "",
    2133        metadata: dict[str, Any] | None = None,
    2134    ) -> dict[str, Any]:
    2135        now = utc_now()
    2136        title = title.strip()
    2137        if not title:
    2138            raise ValueError("title is required")
    2139        status = (status.strip().lower() or "planned").replace(" ", "_")
    2140        if status not in {"planned", "running", "measured", "failed", "blocked", "skipped"}:
    2141            status = "planned"
    2142        config_value = config if isinstance(config, dict) else {}
    2143        key = _norm_key(f"{title}|{_json_dumps(config_value)}")
    2144
    2145        def op(conn: sqlite3.Connection) -> dict[str, Any]:
    2146            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
    2147            if row is None:
    2148                raise KeyError(f"Job not found: {job_id}")
    2149            job_metadata = json.loads(row["metadata_json"] or "{}")
    2150            experiments = _metadata_list(job_metadata, "experiment_ledger")
    2151            current = next((entry for entry in experiments if entry.get("key") == key), None)
    2152            created = current is None
    2153            change_fields = (
    2154                "hypothesis",
    2155                "status",
    2156                "metric_name",
    2157                "metric_value",
    2158                "metric_unit",
    2159                "higher_is_better",
    2160                "baseline_value",
    2161                "config",
    2162                "result",
    2163                "evidence_artifact",
    2164                "next_action",
    2165                "metadata",
    2166                "delta_from_previous_best",
    2167                "best_observed",
    2168            )
    2169            before = "" if created else _change_fingerprint(current, change_fields)
    2170            previous_best = _best_experiment_for_metric(
    2171                experiments,
    2172                metric_name=metric_name,
    2173                metric_unit=metric_unit,
    2174                higher_is_better=higher_is_better,
    2175                exclude_key=key,
    2176            )
    2177            if current is None:
    2178                current = {
    2179                    "key": key,
    2180                    "title": title,
    2181                    "hypothesis": hypothesis.strip(),
    2182                    "status": status,
    2183                    "metric_name": metric_name.strip(),
    2184                    "metric_value": metric_value,
    2185                    "metric_unit": metric_unit.strip(),
    2186                    "higher_is_better": bool(higher_is_better),
    2187                    "baseline_value": baseline_value,
    2188                    "config": config_value,
    2189                    "result": result.strip(),
    2190                    "evidence_artifact": evidence_artifact.strip(),
    2191                    "next_action": next_action.strip(),
    2192                    "metadata": metadata or {},
    2193                    "created_at": now,
    2194                }
    2195                experiments.append(current)
    2196            else:
    2197                current["status"] = status
    2198                for field, value in {
    2199                    "hypothesis": hypothesis.strip(),
    2200                    "metric_name": metric_name.strip(),
    2201                    "metric_unit": metric_unit.strip(),
    2202                    "result": result.strip(),
    2203                    "evidence_artifact": evidence_artifact.strip(),
    2204                    "next_action": next_action.strip(),
    2205                }.items():
    2206                    if value:
    2207                        current[field] = value
    2208                current["higher_is_better"] = bool(higher_is_better)
    2209                if metric_value is not None:
    2210                    current["metric_value"] = metric_value
    2211                if baseline_value is not None:
    2212                    current["baseline_value"] = baseline_value
    2213                if config_value:
    2214                    current["config"] = config_value
    2215                if metadata:
    2216                    merged_metadata = current.get("metadata") if isinstance(current.get("metadata"), dict) else {}
    2217                    merged_metadata.update(metadata)
    2218                    current["metadata"] = merged_metadata
    2219            current["created"] = created
    2220            current["delta_from_previous_best"] = _metric_delta(
    2221                metric_value=current.get("metric_value"),
    2222                previous_best=previous_best,
    2223                higher_is_better=bool(current.get("higher_is_better", True)),
    2224            )
    2225            best = _mark_best_experiments(experiments)
    2226            substantive_update = created or before != _change_fingerprint(current, change_fields)
    2227            current["substantive_update"] = substantive_update
    2228            if substantive_update:
    2229                current["updated_at"] = now
    2230            event_body = current.get("result") or ""
    2231            if current.get("metric_value") is not None:
    2232                event_body = format_metric_value(
    2233                    current.get("metric_name") or "metric",
    2234                    current.get("metric_value"),
    2235                    current.get("metric_unit") or "",
    2236                )
    2237                if current.get("delta_from_previous_best") is not None:
    2238                    event_body += f" delta={current.get('delta_from_previous_best')}"
    2239                if current.get("result"):
    2240                    event_body += f" | {current.get('result')}"
    2241            if substantive_update:
    2242                event = _insert_event(
    2243                    conn,
    2244                    job_id=job_id,
    2245                    event_type="experiment",
    2246                    title=current.get("title") or title,
    2247                    body=event_body,
    2248                    metadata={
    2249                        "created": created,
    2250                        "substantive_update": substantive_update,
    2251                        "status": current.get("status"),
    2252                        "metric_name": current.get("metric_name"),
    2253                        "metric_value": current.get("metric_value"),
    2254                        "metric_unit": current.get("metric_unit"),
    2255                        "higher_is_better": current.get("higher_is_better"),
    2256                        "best_observed": current.get("best_observed"),
    2257                        "delta_from_previous_best": current.get("delta_from_previous_best"),
    2258                        "evidence_artifact": current.get("evidence_artifact"),
    2259                        **(current.get("metadata") if isinstance(current.get("metadata"), dict) else {}),
    2260                    },
    2261                    created_at=now,
    2262                )
    2263                current["event_id"] = event["id"]
    2264            job_metadata["experiment_ledger"] = experiments[-1000:]
    2265            job_metadata["last_experiment_record"] = current
    2266            if best:
    2267                job_metadata["best_experiment_record"] = best
    2268            conn.execute(
    2269                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
    2270                (now, _json_dumps(job_metadata), job_id),
    2271            )
    2272            return current
    2273
    2274        return self._write(op)
    2275
    2276    def append_reflection(
    2277        self,
    2278        job_id: str,
    2279        summary: str,
    2280        *,
    2281        strategy: str = "",
    2282        metadata: dict[str, Any] | None = None,
    2283    ) -> dict[str, Any]:
    2284        now = utc_now()
    2285        text = summary.strip()
    2286        if not text:
    2287            raise ValueError("summary is required")
    2288        entry = {
    2289            "at": now,
    2290            "summary": text,
    2291            "strategy": strategy.strip(),
    2292            "metadata": metadata or {},
    2293        }
    2294
    2295        def op(conn: sqlite3.Connection) -> dict[str, Any]:
    2296            row = conn.execute("SELECT metadata_json FROM jobs WHERE id = ?", (job_id,)).fetchone()
    2297            if row is None:
    2298                raise KeyError(f"Job not found: {job_id}")
    2299            event = _insert_event(
    2300                conn,
    2301                job_id=job_id,
    2302                event_type="reflection",
    2303                title="reflection",
    2304                body=text,
    2305                metadata={"strategy": strategy.strip(), **(metadata or {})},
    2306                created_at=now,
    2307            )
    2308            entry["event_id"] = event["id"]
    2309            job_metadata = json.loads(row["metadata_json"] or "{}")
    2310            reflections = _metadata_list(job_metadata, "reflections")
    2311            reflections.append(entry)
    2312            job_metadata["reflections"] = reflections[-100:]
    2313            job_metadata["last_reflection"] = entry
    2314            conn.execute(
    2315                "UPDATE jobs SET updated_at = ?, metadata_json = ? WHERE id = ?",
    2316                (now, _json_dumps(job_metadata), job_id),
    2317            )
    2318            return entry
    2319
    2320        return self._write(op)
    2321
    2322    def start_run(self, job_id: str, *, model: str = "", config_hash: str = "") -> str:
    2323        run_id = new_id("run")
    2324        now = utc_now()
    2325
    2326        def op(conn: sqlite3.Connection) -> str:
    2327            conn.execute(
    2328                """
    2329                INSERT INTO job_runs(id, job_id, status, started_at, model, config_hash)
    2330                VALUES (?, ?, 'running', ?, ?, ?)
    2331                """,
    2332                (run_id, job_id, now, model, config_hash),
    2333            )
    2334            conn.execute("UPDATE jobs SET status = 'running', updated_at = ? WHERE id = ?", (now, job_id))
    2335            _insert_event(
    2336                conn,
    2337                job_id=job_id,
    2338                event_type="daemon",
    2339                title="run started",
    2340                body=f"model={model}" if model else "",
    2341                ref_table="job_runs",
    2342                ref_id=run_id,
    2343                metadata={"model": model, "config_hash": config_hash},
    2344                created_at=now,
    2345            )
    2346            return run_id
    2347
    2348        return self._write(op)
    2349
    2350    def finish_run(self, run_id: str, status: str, *, score: float | None = None, error: str | None = None) -> None:
    2351        now = utc_now()
    2352
    2353        def op(conn: sqlite3.Connection) -> None:
    2354            conn.execute(
    2355                "UPDATE job_runs SET status = ?, ended_at = ?, score = ?, error = ? WHERE id = ?",
    2356                (status, now, score, error, run_id),
    2357            )
    2358
    2359        self._write(op)
    2360
    2361    def mark_interrupted_running(self, *, reason: str = "daemon interrupted active work") -> dict[str, int]:
    2362        now = utc_now()
    2363        output = {"success": False, "error": reason, "error_type": "Interrupted"}
    2364
    2365        def op(conn: sqlite3.Connection) -> dict[str, int]:
    2366            step_result = conn.execute(
    2367                """
    2368                UPDATE steps
    2369                SET status = 'failed',
    2370                    ended_at = ?,
    2371                    summary = COALESCE(summary, ?),
    2372                    output_json = ?,
    2373                    error = ?
    2374                WHERE status = 'running'
    2375                """,
    2376                (now, reason, _json_dumps(output), reason),
    2377            )
    2378            run_result = conn.execute(
    2379                """
    2380                UPDATE job_runs
    2381                SET status = 'failed',
    2382                    ended_at = ?,
    2383                    error = ?
    2384                WHERE status = 'running'
    2385                """,
    2386                (now, reason),
    2387            )
    2388            return {"steps": int(step_result.rowcount or 0), "runs": int(run_result.rowcount or 0)}
    2389
    2390        return self._write(op)
    2391
    2392    def next_step_no(self, job_id: str) -> int:
    2393        row = self._conn.execute("SELECT COALESCE(MAX(step_no), 0) + 1 AS next_step FROM steps WHERE job_id = ?", (job_id,)).fetchone()
    2394        return int(row["next_step"])
    2395
    2396    def add_step(
    2397        self,
    2398        *,
    2399        job_id: str,
    2400        run_id: str,
    2401        kind: str,
    2402        status: str = "running",
    2403        tool_name: str | None = None,
    2404        summary: str | None = None,
    2405        input_data: dict[str, Any] | None = None,
    2406    ) -> str:
    2407        step_id = new_id("step")
    2408        step_no = self.next_step_no(job_id)
    2409        now = utc_now()
    2410
    2411        def op(conn: sqlite3.Connection) -> str:
    2412            conn.execute(
    2413                """
    2414                INSERT INTO steps(id, job_id, run_id, step_no, kind, status, tool_name, started_at, summary, input_json)
    2415                VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
    2416                """,
    2417                (step_id, job_id, run_id, step_no, kind, status, tool_name, now, summary, _json_dumps(input_data)),
    2418            )
    2419            _insert_event(
    2420                conn,
    2421                job_id=job_id,
    2422                event_type="tool_call" if tool_name else kind,
    2423                title=tool_name or kind,
    2424                body=summary or "",
    2425                ref_table="steps",
    2426                ref_id=step_id,
    2427                metadata={"run_id": run_id, "step_no": step_no, "kind": kind, "status": status, "input": input_data or {}},
    2428                created_at=now,
    2429            )
    2430            return step_id
    2431
    2432        return self._write(op)
    2433
    2434    def finish_step(
    2435        self,
    2436        step_id: str,
    2437        *,
    2438        status: str,
    2439        summary: str | None = None,
    2440        output_data: dict[str, Any] | None = None,
    2441        error: str | None = None,
    2442    ) -> None:
    2443        now = utc_now()
    2444
    2445        def op(conn: sqlite3.Connection) -> None:
    2446            row = conn.execute("SELECT job_id, run_id, step_no, kind, tool_name FROM steps WHERE id = ?", (step_id,)).fetchone()
    2447            conn.execute(
    2448                """
    2449                UPDATE steps
    2450                SET status = ?, ended_at = ?, summary = COALESCE(?, summary), output_json = ?, error = ?
    2451                WHERE id = ?
    2452                """,
    2453                (status, now, summary, _json_dumps(output_data), error, step_id),
    2454            )
    2455            if row is not None:
    2456                event_type = "error" if status == "failed" or error else "tool_result"
    2457                if row["kind"] == "reflection" and not error:
    2458                    event_type = "reflection"
    2459                _insert_event(
    2460                    conn,
    2461                    job_id=row["job_id"],
    2462                    event_type=event_type,
    2463                    title=row["tool_name"] or row["kind"],
    2464                    body=summary or error or "",
    2465                    ref_table="steps",
    2466                    ref_id=step_id,
    2467                    metadata={
    2468                        "run_id": row["run_id"],
    2469                        "step_no": row["step_no"],
    2470                        "kind": row["kind"],
    2471                        "status": status,
    2472                        "output": output_data or {},
    2473                        "error": error,
    2474                    },
    2475                    created_at=now,
    2476                )
    2477
    2478        self._write(op)
    2479
    2480    def list_steps(self, *, job_id: str | None = None, run_id: str | None = None, limit: int | None = None) -> list[dict[str, Any]]:
    2481        if run_id:
    2482            if limit:
    2483                rows = self._conn.execute(
    2484                    """
    2485                    SELECT * FROM (
    2486                        SELECT * FROM steps WHERE run_id = ? ORDER BY step_no DESC LIMIT ?
    2487                    ) ORDER BY step_no
    2488                    """,
    2489                    (run_id, int(limit)),
    2490                ).fetchall()
    2491            else:
    2492                rows = self._conn.execute("SELECT * FROM steps WHERE run_id = ? ORDER BY step_no", (run_id,)).fetchall()
    2493        elif job_id:
    2494            if limit:
    2495                rows = self._conn.execute(
    2496                    """
    2497                    SELECT * FROM (
    2498                        SELECT * FROM steps WHERE job_id = ? ORDER BY step_no DESC LIMIT ?
    2499                    ) ORDER BY step_no
    2500                    """,
    2501                    (job_id, int(limit)),
    2502                ).fetchall()
    2503            else:
    2504                rows = self._conn.execute("SELECT * FROM steps WHERE job_id = ? ORDER BY started_at", (job_id,)).fetchall()
    2505        else:
    2506            if limit:
    2507                rows = self._conn.execute(
    2508                    """
    2509                    SELECT * FROM (
    2510                        SELECT * FROM steps ORDER BY started_at DESC LIMIT ?
    2511                    ) ORDER BY started_at
    2512                    """,
    2513                    (int(limit),),
    2514                ).fetchall()
    2515            else:
    2516                rows = self._conn.execute("SELECT * FROM steps ORDER BY started_at").fetchall()
    2517        return [_row_to_dict(row) for row in rows]
    2518
    2519    def job_record_counts(self, job_id: str) -> dict[str, int]:
    2520        row = self._conn.execute(
    2521            """
    2522            SELECT
    2523                (SELECT COUNT(*) FROM steps WHERE job_id = ?) AS steps,
    2524                (SELECT COUNT(*) FROM artifacts WHERE job_id = ?) AS artifacts,
    2525                (SELECT COUNT(*) FROM memory_index WHERE job_id = ?) AS memory,
    2526                (SELECT COUNT(*) FROM events WHERE job_id = ?) AS events
    2527            """,
    2528            (job_id, job_id, job_id, job_id),
    2529        ).fetchone()
    2530        return {
    2531            "steps": int(row["steps"] or 0),
    2532            "artifacts": int(row["artifacts"] or 0),
    2533            "memory": int(row["memory"] or 0),
    2534            "events": int(row["events"] or 0),
    2535        }
    2536
    2537    def job_token_usage(self, job_id: str) -> dict[str, Any]:
    2538        rows = self._conn.execute(
    2539            """
    2540            SELECT created_at, metadata_json
    2541            FROM events
    2542            WHERE job_id = ? AND event_type = 'loop' AND title = 'message_end'
    2543            ORDER BY created_at ASC, id ASC
    2544            """,
    2545            (job_id,),
    2546        ).fetchall()
    2547        totals: dict[str, Any] = {
    2548            "prompt_tokens": 0,
    2549            "completion_tokens": 0,
    2550            "total_tokens": 0,
    2551            "reasoning_tokens": 0,
    2552            "cached_tokens": 0,
    2553            "cost": 0.0,
    2554            "calls": 0,
    2555            "estimated_calls": 0,
    2556            "latest_prompt_tokens": 0,
    2557            "latest_completion_tokens": 0,
    2558            "latest_total_tokens": 0,
    2559            "latest_context_length": 0,
    2560            "latest_context_fraction": 0.0,
    2561            "latest_at": "",
    2562            "has_cost": False,
    2563        }
    2564        for row in rows:
    2565            metadata = _json_loads(row["metadata_json"])
    2566            usage = metadata.get("usage")
    2567            if not isinstance(usage, dict):
    2568                continue
    2569            prompt = _as_int(usage.get("prompt_tokens"))
    2570            completion = _as_int(usage.get("completion_tokens"))
    2571            total = _as_int(usage.get("total_tokens")) or prompt + completion
    2572            totals["prompt_tokens"] += prompt
    2573            totals["completion_tokens"] += completion
    2574            totals["total_tokens"] += total
    2575            totals["reasoning_tokens"] += _as_int(_nested_value(usage, "completion_tokens_details", "reasoning_tokens"))
    2576            totals["cached_tokens"] += _as_int(_nested_value(usage, "prompt_tokens_details", "cached_tokens"))
    2577            cost = _as_float(usage.get("cost"))
    2578            if cost is not None:
    2579                totals["cost"] += cost
    2580                totals["has_cost"] = True
    2581            totals["calls"] += 1
    2582            if bool(usage.get("estimated")):
    2583                totals["estimated_calls"] += 1
    2584            totals["latest_prompt_tokens"] = prompt
    2585            totals["latest_completion_tokens"] = completion
    2586            totals["latest_total_tokens"] = total
    2587            totals["latest_context_length"] = _as_int(usage.get("context_length"))
    2588            totals["latest_context_fraction"] = _as_float(usage.get("context_fraction")) or 0.0
    2589            totals["latest_at"] = str(row["created_at"] or "")
    2590        return totals
    2591
    2592    def list_runs(self, job_id: str, *, limit: int = 50) -> list[dict[str, Any]]:
    2593        rows = self._conn.execute(
    2594            "SELECT * FROM job_runs WHERE job_id = ? ORDER BY started_at DESC LIMIT ?",
    2595            (job_id, limit),
    2596        ).fetchall()
    2597        return [_row_to_dict(row) for row in rows]
    2598
    2599    def add_artifact(
    2600        self,
    2601        *,
    2602        job_id: str,
    2603        path: str | Path,
    2604        sha256: str,
    2605        artifact_type: str,
    2606        run_id: str | None = None,
    2607        step_id: str | None = None,
    2608        title: str | None = None,
    2609        summary: str | None = None,
    2610        metadata: dict[str, Any] | None = None,
    2611    ) -> str:
    2612        artifact_id = new_id("art")
    2613        now = utc_now()
    2614
    2615        def op(conn: sqlite3.Connection) -> str:
    2616            conn.execute(
    2617                """
    2618                INSERT INTO artifacts(id, job_id, run_id, step_id, type, path, sha256, title, summary, metadata_json, created_at)
    2619                VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
    2620                """,
    2621                (
    2622                    artifact_id,
    2623                    job_id,
    2624                    run_id,
    2625                    step_id,
    2626                    artifact_type,
    2627                    str(path),
    2628                    sha256,
    2629                    title,
    2630                    summary,
    2631                    _json_dumps(metadata),
    2632                    now,
    2633                ),
    2634            )
    2635            _insert_event(
    2636                conn,
    2637                job_id=job_id,
    2638                event_type="artifact",
    2639                title=title or artifact_id,
    2640                body=summary or str(path),
    2641                ref_table="artifacts",
    2642                ref_id=artifact_id,
    2643                metadata={"type": artifact_type, "path": str(path), "sha256": sha256, **(metadata or {})},
    2644                created_at=now,
    2645            )
    2646            return artifact_id
    2647
    2648        return self._write(op)
    2649
    2650    def get_artifact(self, artifact_id: str) -> dict[str, Any]:
    2651        row = self._conn.execute("SELECT * FROM artifacts WHERE id = ?", (artifact_id,)).fetchone()
    2652        artifact = _row_to_dict(row)
    2653        if artifact is None:
    2654            raise KeyError(f"Artifact not found: {artifact_id}")
    2655        return artifact
    2656
    2657    def list_artifacts(self, job_id: str, *, limit: int = 100) -> list[dict[str, Any]]:
    2658        rows = self._conn.execute(
    2659            "SELECT * FROM artifacts WHERE job_id = ? ORDER BY created_at DESC LIMIT ?",
    2660            (job_id, limit),
    2661        ).fetchall()
    2662        return [_row_to_dict(row) for row in rows]
    2663
    2664    def upsert_memory(
    2665        self,
    2666        *,
    2667        job_id: str,
    2668        key: str,
    2669        summary: str,
    2670        artifact_refs: list[str] | None = None,
    2671    ) -> str:
    2672        memory_id = new_id("mem")
    2673        now = utc_now()
    2674
    2675        def op(conn: sqlite3.Connection) -> str:
    2676            conn.execute(
    2677                """
    2678                INSERT INTO memory_index(id, job_id, key, summary, artifact_refs_json, updated_at)
    2679                VALUES (?, ?, ?, ?, ?, ?)
    2680                ON CONFLICT(job_id, key) DO UPDATE SET
    2681                    summary = excluded.summary,
    2682                    artifact_refs_json = excluded.artifact_refs_json,
    2683                    updated_at = excluded.updated_at
    2684                """,
    2685                (memory_id, job_id, key, summary, _json_dumps(artifact_refs or []), now),
    2686            )
    2687            row = conn.execute("SELECT id FROM memory_index WHERE job_id = ? AND key = ?", (job_id, key)).fetchone()
    2688            current_id = str(row["id"])
    2689            _insert_event(
    2690                conn,
    2691                job_id=job_id,
    2692                event_type="compaction",
    2693                title=key,
    2694                body=summary,
    2695                ref_table="memory_index",
    2696                ref_id=current_id,
    2697                metadata={"artifact_refs": artifact_refs or []},
    2698                created_at=now,
    2699            )
    2700            return current_id
    2701
    2702        return self._write(op)
    2703
    2704    def list_memory(self, job_id: str) -> list[dict[str, Any]]:
    2705        rows = self._conn.execute(
    2706            "SELECT * FROM memory_index WHERE job_id = ? ORDER BY updated_at DESC",
    2707            (job_id,),
    2708        ).fetchall()
    2709        return [_row_to_dict(row) for row in rows]
    2710
    2711    def digest_exists(self, *, day: str, target: str) -> bool:
    2712        row = self._conn.execute(
    2713            "SELECT 1 FROM digests WHERE day = ? AND target = ? AND status IN ('sent', 'dry_run') LIMIT 1",
    2714            (day, target),
    2715        ).fetchone()
    2716        return row is not None
    2717
    2718    def record_digest(
    2719        self,
    2720        *,
    2721        day: str,
    2722        target: str,
    2723        subject: str,
    2724        body_path: str | Path,
    2725        status: str,
    2726        error: str | None = None,
    2727    ) -> str:
    2728        digest_id = new_id("dig")
    2729        sent_at = utc_now() if status in {"sent", "dry_run"} else None
    2730
    2731        def op(conn: sqlite3.Connection) -> str:
    2732            conn.execute(
    2733                """
    2734                INSERT INTO digests(id, day, target, subject, body_path, sent_at, status, error)
    2735                VALUES (?, ?, ?, ?, ?, ?, ?, ?)
    2736                """,
    2737                (digest_id, day, target, subject, str(body_path), sent_at, status, error),
    2738            )
    2739            _insert_event(
    2740                conn,
    2741                job_id=None,
    2742                event_type="digest",
    2743                title=subject,
    2744                body=str(body_path),
    2745                ref_table="digests",
    2746                ref_id=digest_id,
    2747                metadata={"day": day, "target": target, "status": status, "error": error},
    2748                created_at=sent_at or utc_now(),
    2749            )
    2750            return digest_id
    2751
    2752        return self._write(op)

nipux_cli/digest.py 380 lines

       1"""Digest rendering and optional email delivery."""
       2
       3from __future__ import annotations
       4
       5import smtplib
       6from datetime import date
       7from email.message import EmailMessage
       8from pathlib import Path
       9from typing import Any
      10
      11from nipux_cli.config import AppConfig, EmailConfig
      12from nipux_cli.db import AgentDB
      13from nipux_cli.operator_context import active_prompt_operator_entries
      14from nipux_cli.tui_layout import _format_compact_count, _format_usage_cost
      15
      16
      17def _metadata_list(job: dict, key: str) -> list[dict]:
      18    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
      19    values = metadata.get(key)
      20    return [value for value in values if isinstance(value, dict)] if isinstance(values, list) else []
      21
      22
      23def _active_operator_messages(messages: list[dict]) -> list[dict]:
      24    prompt_entries = active_prompt_operator_entries(messages)
      25    return [
      26        entry for entry in messages
      27        if str(entry.get("mode") or "steer") in {"steer", "follow_up"}
      28        and entry in prompt_entries
      29    ]
      30
      31
      32def _safe_int(value: Any) -> int:
      33    try:
      34        return int(float(value))
      35    except (TypeError, ValueError):
      36        return 0
      37
      38
      39def _latest_run_model(db: AgentDB, job_id: str) -> str:
      40    runs = db.list_runs(job_id, limit=1)
      41    if runs:
      42        return str(runs[0].get("model") or "unknown")
      43    return "unknown"
      44
      45
      46def _usage_lines(
      47    db: AgentDB,
      48    job_id: str,
      49    *,
      50    model: str | None = None,
      51    base_url: str = "",
      52    context_length: int = 0,
      53    input_cost_per_million: float | None = None,
      54    output_cost_per_million: float | None = None,
      55) -> list[str]:
      56    usage = db.job_token_usage(job_id)
      57    usage["input_cost_per_million"] = input_cost_per_million
      58    usage["output_cost_per_million"] = output_cost_per_million
      59    calls = _safe_int(usage.get("calls"))
      60    if calls <= 0:
      61        return ["- No model usage recorded yet."]
      62    model_name = model or _latest_run_model(db, job_id)
      63    prompt = _safe_int(usage.get("prompt_tokens"))
      64    completion = _safe_int(usage.get("completion_tokens"))
      65    total = _safe_int(usage.get("total_tokens")) or prompt + completion
      66    latest_prompt = _safe_int(usage.get("latest_prompt_tokens"))
      67    latest_completion = _safe_int(usage.get("latest_completion_tokens"))
      68    context_text = _format_compact_count(latest_prompt)
      69    if context_length > 0:
      70        context_text = f"{context_text}/{_format_compact_count(context_length)}"
      71    cost_text = _format_usage_cost(usage, model=model_name, base_url=base_url)
      72    lines = [
      73        (
      74            f"- {model_name}: {calls} calls, {_format_compact_count(total)} tokens "
      75            f"({_format_compact_count(prompt)} prompt, {_format_compact_count(completion)} output), "
      76            f"latest ctx={context_text}, latest output={_format_compact_count(latest_completion)}, cost={cost_text}"
      77        )
      78    ]
      79    if _safe_int(usage.get("estimated_calls")):
      80        lines.append("- Some token/cost values are estimated because the provider did not return complete usage metadata.")
      81    elif not bool(usage.get("has_cost")) and cost_text == "pending":
      82        lines.append("- Cost is pending until the provider returns cost metadata or the model is configured as local/free.")
      83    return lines
      84
      85
      86def render_job_digest(
      87    db: AgentDB,
      88    job_id: str,
      89    *,
      90    model: str | None = None,
      91    base_url: str = "",
      92    context_length: int = 0,
      93    input_cost_per_million: float | None = None,
      94    output_cost_per_million: float | None = None,
      95) -> str:
      96    job = db.get_job(job_id)
      97    artifacts = db.list_artifacts(job_id, limit=50)
      98    steps = db.list_steps(job_id=job_id)
      99    findings = _metadata_list(job, "finding_ledger")
     100    sources = _metadata_list(job, "source_ledger")
     101    tasks = _metadata_list(job, "task_queue")
     102    experiments = _metadata_list(job, "experiment_ledger")
     103    lessons = _metadata_list(job, "lessons")
     104    reflections = _metadata_list(job, "reflections")
     105    operator_messages = _metadata_list(job, "operator_messages")
     106    active_operator = _active_operator_messages(operator_messages)
     107    lines = [
     108        f"# {job['title']}",
     109        "",
     110        f"Status: {job['status']}",
     111        f"Findings: {len(findings)}",
     112        f"Sources: {len(sources)}",
     113        f"Tasks: {len(tasks)}",
     114        f"Experiments: {len(experiments)}",
     115        f"Lessons: {len(lessons)}",
     116        "",
     117        "## Model Usage",
     118        "",
     119        *_usage_lines(
     120            db,
     121            job_id,
     122            model=model,
     123            base_url=base_url,
     124            context_length=context_length,
     125            input_cost_per_million=input_cost_per_million,
     126            output_cost_per_million=output_cost_per_million,
     127        ),
     128        "",
     129        "## Objective",
     130        "",
     131        job["objective"],
     132        "",
     133        "## Active Operator Context",
     134        "",
     135    ]
     136    if not active_operator:
     137        lines.append("- none")
     138    for entry in active_operator[-8:]:
     139        lines.append(f"- {entry.get('mode') or 'steer'}: {entry.get('message') or ''}")
     140    lines.extend([
     141        "",
     142        "## Recent Steps",
     143        "",
     144    ])
     145    if not steps:
     146        lines.append("- No steps have run yet.")
     147    for step in steps[-20:]:
     148        tool = f" `{step['tool_name']}`" if step.get("tool_name") else ""
     149        lines.append(f"- #{step['step_no']} {step['kind']}{tool}: {step['status']} - {step.get('summary') or ''}")
     150    lines.extend(["", "## Best Findings", ""])
     151    if not findings:
     152        lines.append("- No findings recorded yet.")
     153    for finding in sorted(findings, key=lambda item: float(item.get("score") or 0), reverse=True)[:15]:
     154        details = " | ".join(str(finding.get(key) or "") for key in ("category", "location", "contact") if finding.get(key))
     155        suffix = f" - {details}" if details else ""
     156        lines.append(f"- {finding.get('name') or 'unknown'} (score={finding.get('score')}){suffix}")
     157        if finding.get("reason"):
     158            lines.append(f"  - {finding['reason']}")
     159    lines.extend(["", "## Source Learning", ""])
     160    if not sources:
     161        lines.append("- No sources scored yet.")
     162    for source in sorted(sources, key=lambda item: float(item.get("usefulness_score") or 0), reverse=True)[:12]:
     163        lines.append(
     164            f"- {source.get('source')} score={source.get('usefulness_score')} "
     165            f"findings={source.get('yield_count') or 0} fails={source.get('fail_count') or 0}: {source.get('last_outcome') or ''}"
     166        )
     167    lines.extend(["", "## Task Queue", ""])
     168    if not tasks:
     169        lines.append("- No tasks recorded yet.")
     170    status_order = {"active": 0, "open": 1, "blocked": 2, "done": 3, "skipped": 4}
     171    for task in sorted(tasks, key=lambda item: (status_order.get(str(item.get("status") or "open"), 9), -int(item.get("priority") or 0)))[:15]:
     172        contract = f" [{task.get('output_contract')}]" if task.get("output_contract") else ""
     173        lines.append(f"- {task.get('status') or 'open'} p={task.get('priority') or 0}{contract}: {task.get('title') or 'untitled'}")
     174        for key, label in (("acceptance_criteria", "accept"), ("evidence_needed", "evidence"), ("stall_behavior", "stall")):
     175            if task.get(key):
     176                lines.append(f"  - {label}: {task[key]}")
     177        if task.get("result"):
     178            lines.append(f"  - {task['result']}")
     179    lines.extend(["", "## Experiments", ""])
     180    if not experiments:
     181        lines.append("- No experiments recorded yet.")
     182    measured = [experiment for experiment in experiments if experiment.get("metric_value") is not None]
     183    for experiment in sorted(measured or experiments, key=lambda item: (not bool(item.get("best_observed")), str(item.get("updated_at") or item.get("created_at") or "")))[:15]:
     184        metric = ""
     185        if experiment.get("metric_value") is not None:
     186            metric = f" {experiment.get('metric_name') or 'metric'}={experiment.get('metric_value')}{experiment.get('metric_unit') or ''}"
     187        best = " best" if experiment.get("best_observed") else ""
     188        lines.append(f"- {experiment.get('status') or 'planned'}: {experiment.get('title') or 'experiment'}{metric}{best}")
     189        if experiment.get("result"):
     190            lines.append(f"  - {experiment['result']}")
     191    lines.extend(["", "## Lessons", ""])
     192    if not lessons:
     193        lines.append("- No lessons recorded yet.")
     194    for lesson in lessons[-12:]:
     195        lines.append(f"- {lesson.get('category') or 'memory'}: {lesson.get('lesson') or ''}")
     196    if reflections:
     197        lines.extend(["", "## Current Strategy", ""])
     198        reflection = reflections[-1]
     199        lines.append(reflection.get("summary") or "")
     200        if reflection.get("strategy"):
     201            lines.append("")
     202            lines.append(reflection["strategy"])
     203    lines.extend(["", "## Artifacts", ""])
     204    if not artifacts:
     205        lines.append("- No artifacts yet.")
     206    for artifact in artifacts[:20]:
     207        title = artifact.get("title") or artifact["id"]
     208        lines.append(f"- {title} ({artifact['type']}): {artifact['path']}")
     209    return "\n".join(lines).rstrip() + "\n"
     210
     211
     212def send_digest_email(config: EmailConfig, *, subject: str, body: str, to_addr: str | None = None) -> dict:
     213    if not config.enabled:
     214        return {"sent": False, "dry_run": True, "reason": "email.disabled", "subject": subject, "body": body}
     215    target = to_addr or config.to_addr
     216    if not all([config.smtp_host, config.from_addr, target]):
     217        raise ValueError("Email is enabled but smtp_host/from_addr/to_addr is incomplete")
     218    message = EmailMessage()
     219    message["Subject"] = subject
     220    message["From"] = config.from_addr
     221    message["To"] = target
     222    message.set_content(body)
     223    with smtplib.SMTP(config.smtp_host, config.smtp_port, timeout=30) as smtp:
     224        if config.use_tls:
     225            smtp.starttls()
     226        if config.username:
     227            smtp.login(config.username, config.password)
     228        smtp.send_message(message)
     229    return {"sent": True, "target": target, "subject": subject}
     230
     231
     232def render_daily_digest(
     233    db: AgentDB,
     234    *,
     235    model: str | None = None,
     236    base_url: str = "",
     237    context_length: int = 0,
     238    input_cost_per_million: float | None = None,
     239    output_cost_per_million: float | None = None,
     240) -> str:
     241    jobs = [job for job in db.list_jobs() if job["status"] not in {"cancelled"}]
     242    lines = ["# Nipux CLI Daily Digest", ""]
     243    if not jobs:
     244        lines.append("No jobs are currently tracked.")
     245        return "\n".join(lines).rstrip() + "\n"
     246
     247    for job in jobs:
     248        artifacts = db.list_artifacts(job["id"], limit=10)
     249        steps = db.list_steps(job_id=job["id"])[-5:]
     250        findings = _metadata_list(job, "finding_ledger")
     251        sources = _metadata_list(job, "source_ledger")
     252        tasks = _metadata_list(job, "task_queue")
     253        experiments = _metadata_list(job, "experiment_ledger")
     254        lessons = _metadata_list(job, "lessons")
     255        reflections = _metadata_list(job, "reflections")
     256        operator_messages = _metadata_list(job, "operator_messages")
     257        active_operator = _active_operator_messages(operator_messages)
     258        finding_batches = [artifact for artifact in artifacts if "finding" in str(artifact.get("title") or artifact.get("summary") or "").lower()]
     259        lines.extend([
     260            f"## {job['title']}",
     261            "",
     262            f"Status: {job['status']}",
     263            f"Kind: {job['kind']}",
     264            f"Counts: {len(findings)} findings, {len(sources)} sources, {len(tasks)} tasks, {len(experiments)} experiments, {len(lessons)} lessons, {len(finding_batches)} recent finding artifacts",
     265            "",
     266            "Model usage:",
     267        ])
     268        lines.extend(
     269            _usage_lines(
     270                db,
     271                job["id"],
     272                model=model,
     273                base_url=base_url,
     274                context_length=context_length,
     275                input_cost_per_million=input_cost_per_million,
     276                output_cost_per_million=output_cost_per_million,
     277            )
     278        )
     279        lines.extend([
     280            "",
     281            "Recent steps:",
     282        ])
     283        if not steps:
     284            lines.append("- none")
     285        for step in steps:
     286            tool = f" `{step['tool_name']}`" if step.get("tool_name") else ""
     287            lines.append(f"- #{step['step_no']} {step['kind']}{tool}: {step['status']} - {step.get('summary') or ''}")
     288        lines.extend(["", "Active operator context:"])
     289        if not active_operator:
     290            lines.append("- none")
     291        for entry in active_operator[-5:]:
     292            lines.append(f"- {entry.get('mode') or 'steer'}: {entry.get('message') or ''}")
     293        lines.extend(["", "Best findings:"])
     294        if not findings:
     295            lines.append("- none")
     296        for finding in sorted(findings, key=lambda item: float(item.get("score") or 0), reverse=True)[:8]:
     297            lines.append(f"- {finding.get('name') or 'unknown'} (score={finding.get('score')}) - {finding.get('reason') or finding.get('category') or ''}")
     298        lines.extend(["", "Task queue:"])
     299        if not tasks:
     300            lines.append("- none")
     301        status_order = {"active": 0, "open": 1, "blocked": 2, "done": 3, "skipped": 4}
     302        for task in sorted(tasks, key=lambda item: (status_order.get(str(item.get("status") or "open"), 9), -int(item.get("priority") or 0)))[:8]:
     303            contract = f" [{task.get('output_contract')}]" if task.get("output_contract") else ""
     304            lines.append(f"- {task.get('status') or 'open'} p={task.get('priority') or 0}{contract}: {task.get('title') or 'untitled'}")
     305        lines.extend(["", "Experiments:"])
     306        if not experiments:
     307            lines.append("- none")
     308        measured = [experiment for experiment in experiments if experiment.get("metric_value") is not None]
     309        for experiment in (measured or experiments)[-8:]:
     310            metric = ""
     311            if experiment.get("metric_value") is not None:
     312                metric = f" {experiment.get('metric_name') or 'metric'}={experiment.get('metric_value')}{experiment.get('metric_unit') or ''}"
     313            best = " best" if experiment.get("best_observed") else ""
     314            lines.append(f"- {experiment.get('status') or 'planned'}: {experiment.get('title') or 'experiment'}{metric}{best}")
     315        lines.extend(["", "Lessons learned:"])
     316        if not lessons:
     317            lines.append("- none")
     318        for lesson in lessons[-8:]:
     319            lines.append(f"- {lesson.get('category') or 'memory'}: {lesson.get('lesson') or ''}")
     320        lines.extend(["", "Source quality:"])
     321        if not sources:
     322            lines.append("- none")
     323        for source in sorted(sources, key=lambda item: float(item.get("usefulness_score") or 0), reverse=True)[:8]:
     324            lines.append(f"- {source.get('source')} score={source.get('usefulness_score')} findings={source.get('yield_count') or 0}: {source.get('last_outcome') or ''}")
     325        if reflections:
     326            reflection = reflections[-1]
     327            lines.extend(["", "Current strategy:", f"- {reflection.get('strategy') or reflection.get('summary') or ''}"])
     328        lines.extend(["", "Next branches:"])
     329        lines.append("- Continue with high-yield source types, avoid low-yield paths, and save durable findings as artifacts.")
     330        lines.extend(["", "Recent artifacts:"])
     331        if not artifacts:
     332            lines.append("- none")
     333        for artifact in artifacts:
     334            title = artifact.get("title") or artifact["id"]
     335            lines.append(f"- {title} ({artifact['type']}): {artifact['path']}")
     336        lines.append("")
     337    return "\n".join(lines).rstrip() + "\n"
     338
     339
     340def write_daily_digest(config: AppConfig, db: AgentDB, *, day: str | None = None) -> dict:
     341    day = day or date.today().isoformat()
     342    target = config.email.to_addr or "dry-run"
     343    subject = f"Nipux CLI daily digest - {day}"
     344    if db.digest_exists(day=day, target=target):
     345        return {"sent": False, "skipped": True, "reason": "already_recorded", "day": day, "target": target}
     346
     347    body = render_daily_digest(
     348        db,
     349        model=config.model.model,
     350        base_url=config.model.base_url,
     351        context_length=config.model.context_length,
     352        input_cost_per_million=config.model.input_cost_per_million,
     353        output_cost_per_million=config.model.output_cost_per_million,
     354    )
     355    config.runtime.digests_dir.mkdir(parents=True, exist_ok=True)
     356    body_path = Path(config.runtime.digests_dir) / f"{day}-daily.md"
     357    body_path.write_text(body, encoding="utf-8")
     358
     359    try:
     360        email_result = send_digest_email(config.email, subject=subject, body=body)
     361        status = "sent" if email_result.get("sent") else "dry_run"
     362        digest_id = db.record_digest(day=day, target=target, subject=subject, body_path=body_path, status=status)
     363        return {
     364            "digest_id": digest_id,
     365            "status": status,
     366            "day": day,
     367            "target": target,
     368            "path": str(body_path),
     369            "email": email_result,
     370        }
     371    except Exception as exc:
     372        digest_id = db.record_digest(
     373            day=day,
     374            target=target,
     375            subject=subject,
     376            body_path=body_path,
     377            status="failed",
     378            error=str(exc),
     379        )
     380        return {"digest_id": digest_id, "status": "failed", "day": day, "target": target, "path": str(body_path), "error": str(exc)}

nipux_cli/doctor.py 260 lines

       1"""Runtime checks for the Nipux agent."""
       2
       3from __future__ import annotations
       4
       5import json
       6import shutil
       7import urllib.error
       8import urllib.request
       9from dataclasses import dataclass
      10from pathlib import Path
      11from typing import Any
      12from urllib.parse import urlparse
      13
      14from nipux_cli.config import AppConfig, load_config
      15from nipux_cli.db import AgentDB
      16from nipux_cli.tools import DEFAULT_REGISTRY
      17
      18
      19@dataclass(frozen=True)
      20class Check:
      21    name: str
      22    ok: bool
      23    detail: str
      24
      25    def as_dict(self) -> dict[str, Any]:
      26        return {"name": self.name, "ok": self.ok, "detail": self.detail}
      27
      28
      29def _check_writable_dir(path: Path) -> Check:
      30    try:
      31        path.mkdir(parents=True, exist_ok=True)
      32        probe = path / ".write-test"
      33        probe.write_text("ok", encoding="utf-8")
      34        probe.unlink()
      35        return Check("state_dir_writable", True, str(path))
      36    except OSError as exc:
      37        return Check("state_dir_writable", False, f"{path}: {exc}")
      38
      39
      40def _check_db(config: AppConfig) -> Check:
      41    try:
      42        db = AgentDB(config.runtime.state_db_path)
      43        db.close()
      44        return Check("sqlite", True, str(config.runtime.state_db_path))
      45    except Exception as exc:
      46        return Check("sqlite", False, str(exc))
      47
      48
      49def _check_tool_surface() -> Check:
      50    names = DEFAULT_REGISTRY.names()
      51    forbidden = sorted({"terminal", "delegate_task", "skill_manage", "image_generate"} & set(names))
      52    if forbidden:
      53        return Check("tool_surface", False, f"forbidden tools exposed: {', '.join(forbidden)}")
      54    return Check("tool_surface", True, f"{len(names)} tools: {', '.join(names)}")
      55
      56
      57def _check_model_config(config: AppConfig) -> Check:
      58    base_url = config.model.base_url
      59    host = (urlparse(base_url).hostname or "").lower()
      60    local_hosts = {"", "localhost", "127.0.0.1", "::1", "0.0.0.0"}
      61    if host in local_hosts or host.endswith(".local"):
      62        return Check("model_config", True, f"{config.model.model} at {base_url}")
      63    if config.model.api_key:
      64        return Check("model_config", True, f"{config.model.model} at {base_url}; key read from {config.model.api_key_env}")
      65    return Check(
      66        "model_config",
      67        False,
      68        f"{config.model.api_key_env} is not set for remote endpoint {base_url}; put it in the shell or ~/.nipux/.env",
      69    )
      70
      71
      72def _check_browser_runtime() -> Check:
      73    direct = shutil.which("agent-browser")
      74    if direct:
      75        return Check("browser_runtime", True, f"agent-browser: {direct}")
      76    npx = shutil.which("npx")
      77    if npx:
      78        return Check("browser_runtime", True, f"agent-browser available through npx fallback: {npx}")
      79    return Check(
      80        "browser_runtime",
      81        False,
      82        "agent-browser not found and npx is unavailable; install with: npm install -g agent-browser && agent-browser install",
      83    )
      84
      85
      86def _check_model_endpoint(config: AppConfig) -> Check:
      87    if "openrouter.ai" in config.model.base_url and not config.model.api_key:
      88        return Check("model_endpoint", False, "API key is not set")
      89    auth = _check_openrouter_auth(config)
      90    if auth is not None and not auth.ok:
      91        return auth
      92    url = config.model.base_url.rstrip("/") + "/models"
      93    request = urllib.request.Request(url, headers={"Authorization": f"Bearer {config.model.api_key or 'local-no-key'}"})
      94    try:
      95        with urllib.request.urlopen(request, timeout=5) as response:
      96            payload = response.read(512_000).decode("utf-8", errors="replace")
      97        try:
      98            data = json.loads(payload)
      99            count = len(data.get("data", [])) if isinstance(data, dict) else "unknown"
     100            available = _model_available(data, config.model.model)
     101            if available is False:
     102                return Check("model_endpoint", False, f"{config.model.model} not found at {url}; models={count}")
     103            generation = _check_model_generation(config)
     104            if not generation.ok:
     105                return generation
     106            return Check("model_endpoint", True, f"{url} returned models={count}; {config.model.model} available; generation accepted")
     107        except json.JSONDecodeError:
     108            generation = _check_model_generation(config)
     109            if not generation.ok:
     110                return generation
     111            return Check("model_endpoint", True, f"{url} responded; generation accepted")
     112    except (urllib.error.URLError, TimeoutError, OSError) as exc:
     113        return Check("model_endpoint", False, f"{url}: {exc}")
     114
     115
     116def _check_model_generation(config: AppConfig) -> Check:
     117    url = config.model.base_url.rstrip("/") + "/chat/completions"
     118    payload = {
     119        "model": config.model.model,
     120        "messages": [{"role": "user", "content": "Reply with exactly: ok"}],
     121        "max_tokens": 8,
     122        "temperature": 0,
     123        "tools": [
     124            {
     125                "type": "function",
     126                "function": {
     127                    "name": "noop",
     128                    "description": "No-op model readiness probe.",
     129                    "parameters": {
     130                        "type": "object",
     131                        "properties": {"reason": {"type": "string"}},
     132                        "required": ["reason"],
     133                    },
     134                },
     135            }
     136        ],
     137    }
     138    request = urllib.request.Request(
     139        url,
     140        data=json.dumps(payload).encode("utf-8"),
     141        headers={
     142            "Authorization": f"Bearer {config.model.api_key or 'local-no-key'}",
     143            "Content-Type": "application/json",
     144        },
     145        method="POST",
     146    )
     147    try:
     148        with urllib.request.urlopen(request, timeout=15) as response:
     149            body = response.read(64_000).decode("utf-8", errors="replace")
     150        data = json.loads(body)
     151        choices = data.get("choices") if isinstance(data, dict) else None
     152        if isinstance(choices, list) and choices:
     153            return Check("model_generation", True, f"{url} accepted chat/tool request")
     154        return Check("model_generation", False, f"{url} returned no choices")
     155    except urllib.error.HTTPError as exc:
     156        body = exc.read(2048).decode("utf-8", errors="replace")
     157        detail = _extract_error_message(body) or str(exc)
     158        return Check("model_generation", False, f"{url}: {detail}")
     159    except (json.JSONDecodeError, urllib.error.URLError, TimeoutError, OSError) as exc:
     160        return Check("model_generation", False, f"{url}: {exc}")
     161
     162
     163def _check_openrouter_auth(config: AppConfig) -> Check | None:
     164    if "openrouter.ai" not in config.model.base_url:
     165        return None
     166    if not config.model.api_key:
     167        return Check("model_auth", False, "OpenRouter API key is not set")
     168    url = "https://openrouter.ai/api/v1/key"
     169    request = urllib.request.Request(url, headers={"Authorization": f"Bearer {config.model.api_key}"})
     170    try:
     171        with urllib.request.urlopen(request, timeout=5) as response:
     172            response.read(2048)
     173        return Check("model_auth", True, "OpenRouter API key accepted")
     174    except urllib.error.HTTPError as exc:
     175        body = exc.read(512).decode("utf-8", errors="replace")
     176        detail = _extract_error_message(body) or str(exc)
     177        return Check("model_auth", False, f"OpenRouter rejected API key: {detail}")
     178    except (urllib.error.URLError, TimeoutError, OSError) as exc:
     179        return Check("model_auth", False, f"{url}: {exc}")
     180
     181
     182def _extract_error_message(body: str) -> str:
     183    try:
     184        data = json.loads(body)
     185    except json.JSONDecodeError:
     186        return body.strip()
     187    error = data.get("error") if isinstance(data, dict) else None
     188    if isinstance(error, dict):
     189        message = str(error.get("message") or "").strip()
     190        code = str(error.get("code") or "").strip()
     191        metadata = error.get("metadata")
     192        raw = _extract_error_raw(metadata)
     193        provider = _metadata_value(metadata, "provider_name")
     194        byok = _metadata_value(metadata, "is_byok")
     195
     196        primary = message or code
     197        if raw and raw != primary:
     198            primary = f"{primary}: {raw}" if primary else raw
     199        details = []
     200        if code and code not in primary:
     201            details.append(f"code={code}")
     202        if provider:
     203            details.append(f"provider={provider}")
     204        if byok not in {"", None}:
     205            details.append(f"byok={byok}")
     206        if details:
     207            primary = f"{primary} ({'; '.join(details)})" if primary else "; ".join(details)
     208        return primary.strip()
     209    return ""
     210
     211
     212def _metadata_value(metadata: Any, key: str) -> str:
     213    if not isinstance(metadata, dict):
     214        return ""
     215    value = metadata.get(key)
     216    if value is None:
     217        return ""
     218    return str(value).strip()
     219
     220
     221def _extract_error_raw(metadata: Any) -> str:
     222    if not isinstance(metadata, dict):
     223        return ""
     224    raw = metadata.get("raw")
     225    if raw is None:
     226        return ""
     227    if isinstance(raw, dict):
     228        return _extract_error_message(json.dumps(raw))
     229    raw_text = str(raw).strip()
     230    if not raw_text:
     231        return ""
     232    try:
     233        raw_json = json.loads(raw_text)
     234    except json.JSONDecodeError:
     235        return raw_text
     236    if isinstance(raw_json, dict):
     237        nested = _extract_error_message(json.dumps(raw_json))
     238        return nested or raw_text
     239    return raw_text
     240
     241
     242def _model_available(data: Any, model: str) -> bool | None:
     243    if not isinstance(data, dict) or not isinstance(data.get("data"), list):
     244        return None
     245    ids = {str(item.get("id") or "") for item in data["data"] if isinstance(item, dict)}
     246    return model in ids
     247
     248
     249def run_doctor(*, config: AppConfig | None = None, check_model: bool = False) -> list[Check]:
     250    config = config or load_config()
     251    checks = [
     252        _check_writable_dir(config.runtime.home),
     253        _check_db(config),
     254        _check_model_config(config),
     255        _check_tool_surface(),
     256        _check_browser_runtime(),
     257    ]
     258    if check_model:
     259        checks.append(_check_model_endpoint(config))
     260    return checks

nipux_cli/event_render.py 118 lines

       1"""Readable event rendering shared by CLI history and chat context."""
       2
       3from __future__ import annotations
       4
       5import shlex
       6from typing import Any
       7
       8from nipux_cli.metric_format import format_metric_value
       9from nipux_cli.tui_event_format import clean_step_summary, generic_display_text
      10from nipux_cli.tui_style import _one_line
      11
      12
      13def event_line(event: dict[str, Any], *, chars: int, full: bool = False) -> str:
      14    when, label, detail, access = event_display_parts(event, chars=chars, full=full)
      15    suffix = f" | {access}" if access and full else ""
      16    return f"{when:<16} {label:<8} {_one_line(detail + suffix, chars)}"
      17
      18
      19def event_display_parts(event: dict[str, Any], *, chars: int, full: bool = False) -> tuple[str, str, str, str]:
      20    when = compact_time(str(event.get("created_at") or "?"))
      21    kind = str(event.get("event_type") or "event")
      22    title = str(event.get("title") or "").strip()
      23    body = generic_display_text(event.get("body") or "")
      24    ref_table = str(event.get("ref_table") or "")
      25    ref_id = str(event.get("ref_id") or "")
      26    metadata = event.get("metadata") if isinstance(event.get("metadata"), dict) else {}
      27    label = event_label(kind, metadata)
      28    access = ""
      29    if kind == "tool_result" and metadata.get("status"):
      30        label = event_label(f"{kind}:{metadata.get('status')}", metadata)
      31    if kind == "error":
      32        label = "ERROR"
      33    if kind.startswith("tool_result") or kind == "error":
      34        body = clean_step_summary(body)
      35    if kind == "artifact":
      36        title = title or ref_id
      37        if body.startswith("/") or "/.nipux/jobs/" in body or "/jobs/job_" in body:
      38            body = generic_display_text(metadata.get("summary") or "saved output")
      39        if title:
      40            access = f"open: /artifact {shlex.quote(title)}"
      41    if kind == "operator_message" and metadata.get("mode"):
      42        title = f"{title or 'operator'} {metadata.get('mode')}"
      43    if kind == "operator_context":
      44        body = body or f"{metadata.get('count') or 0} message(s)"
      45    if kind in {"tool_call", "tool_result", "error"} and metadata.get("step_no"):
      46        title = f"#{metadata.get('step_no')} {title}".strip()
      47    if not body and kind == "artifact" and metadata.get("path"):
      48        body = str(metadata.get("type") or "saved artifact")
      49    if not body and kind == "finding" and metadata.get("category"):
      50        body = str(metadata.get("category") or "")
      51    if not body and kind == "task" and metadata.get("status"):
      52        body = str(metadata.get("status") or "")
      53    if not body and kind == "roadmap" and metadata.get("status"):
      54        body = str(metadata.get("status") or "")
      55    if not body and kind == "milestone_validation" and metadata.get("validation_status"):
      56        body = str(metadata.get("validation_status") or "")
      57    if not body and kind == "experiment":
      58        metric_value = metadata.get("metric_value")
      59        if metric_value is not None:
      60            body = format_metric_value(
      61                metadata.get("metric_name") or "metric",
      62                metric_value,
      63                metadata.get("metric_unit") or "",
      64            )
      65    if kind == "compaction":
      66        body = _one_line(body, min(chars, 140))
      67    if kind == "daemon" and title == "run started":
      68        body = body or str(metadata.get("model") or "")
      69    detail = title if title else kind
      70    if body:
      71        detail = f"{detail} - {body}"
      72    if ref_table and ref_id and full:
      73        detail = f"{detail} [{ref_table}:{ref_id}]"
      74    return when, label, detail, access
      75
      76
      77def event_label(kind: str, metadata: dict[str, Any]) -> str:
      78    if kind == "operator_message":
      79        mode = str(metadata.get("mode") or "")
      80        return "FOLLOW" if mode == "follow_up" else "USER"
      81    if kind == "operator_context":
      82        return "ACK"
      83    if kind == "agent_message":
      84        return "AGENT"
      85    if kind == "roadmap":
      86        return "ROAD"
      87    if kind == "milestone_validation":
      88        return "VALID"
      89    if kind == "tool_call":
      90        return "TOOL"
      91    if kind.startswith("tool_result"):
      92        status = str(metadata.get("status") or "")
      93        if status == "blocked":
      94            return "BLOCK"
      95        if status == "failed" or kind.endswith(":failed"):
      96            return "ERROR"
      97        return "DONE"
      98    labels = {
      99        "artifact": "OUTPUT",
     100        "compaction": "MEMORY",
     101        "daemon": "SYSTEM",
     102        "digest": "DIGEST",
     103        "error": "ERROR",
     104        "experiment": "TEST",
     105        "finding": "FIND",
     106        "lesson": "LEARN",
     107        "reflection": "PLAN",
     108        "source": "SOURCE",
     109        "task": "TASK",
     110    }
     111    return labels.get(kind, kind.upper()[:8])
     112
     113
     114def compact_time(value: str) -> str:
     115    text = value.replace("T", " ")
     116    if len(text) >= 16 and text[4:5] == "-" and text[13:14] == ":":
     117        return text[:16]
     118    return _one_line(text, 16)

nipux_cli/first_run_controller.py 170 lines

       1"""First-run command decisions for the Nipux TUI."""
       2
       3from __future__ import annotations
       4
       5import shlex
       6from contextlib import redirect_stdout
       7from dataclasses import dataclass
       8from io import StringIO
       9from typing import Callable
      10
      11from nipux_cli.settings import config_field_value
      12from nipux_cli.tui_commands import CHAT_SETTING_COMMANDS
      13from nipux_cli.frame_snapshot import WORKSPACE_CHAT_ID
      14
      15
      16TOGGLE_SETTING_COMMANDS = {
      17    "tools.browser": "browser",
      18    "tools.web": "web",
      19    "tools.shell": "cli-access",
      20    "tools.files": "file-access",
      21}
      22
      23
      24@dataclass(frozen=True)
      25class FirstRunFrameDeps:
      26    capture_command: Callable[[str], list[str]]
      27    capture_setting_command: Callable[[str], list[str]]
      28    create_job: Callable[..., tuple[str, str]]
      29    current_default_job_id: Callable[[], str | None]
      30    extract_objective: Callable[[str], str]
      31    model_setup_verified: Callable[[], bool]
      32    verify_model_setup: Callable[[], list[str]]
      33    shell_command_names: set[str]
      34
      35
      36def handle_first_run_action(action: str, *, deps: FirstRunFrameDeps) -> tuple[str, str | list[str] | None]:
      37    if action == "open_workspace" and not deps.model_setup_verified():
      38        return "notice", "Run Doctor first. The workspace opens only after the configured model accepts a chat request."
      39    if action == "open_workspace":
      40        return "open", WORKSPACE_CHAT_ID
      41    if action.startswith("view:"):
      42        return "view", action.split(":", 1)[1]
      43    if action == "preset:local":
      44        notices = [
      45            *deps.capture_setting_command("model local-model"),
      46            *deps.capture_setting_command("base-url http://localhost:8000/v1"),
      47            *deps.capture_setting_command("api-key-env OPENAI_API_KEY"),
      48            "Local connector selected. Start your OpenAI-compatible server, then run Doctor.",
      49        ]
      50        return "notice", notices
      51    if action.startswith("toggle:"):
      52        field = action.split(":", 1)[1]
      53        command = TOGGLE_SETTING_COMMANDS.get(field)
      54        if not command:
      55            return "notice", f"Unknown setup toggle: {field}"
      56        next_value = "false" if bool(config_field_value(field)) else "true"
      57        return "notice", deps.capture_setting_command(f"{command} {next_value}")
      58    if action.startswith("edit:"):
      59        return "edit", action.split(":", 1)[1]
      60    if action.startswith("secret:"):
      61        return "edit", action
      62    if action == "new":
      63        return "notice", "Finish setup first. Then describe worker jobs in the chat workspace."
      64    if action == "back":
      65        return "view", "endpoint"
      66    if action == "jobs":
      67        return "notice", "Finish setup first. Jobs are available after Doctor verifies the configured model."
      68    if action == "doctor":
      69        notices = deps.verify_model_setup()
      70        if deps.model_setup_verified():
      71            return "open", WORKSPACE_CHAT_ID
      72        return "notice", notices
      73    if action == "init":
      74        return "notice", deps.capture_command("init")
      75    if action == "exit":
      76        return "exit", None
      77    return "notice", f"Unknown action: {action}"
      78
      79
      80def handle_first_run_frame_line(line: str, *, deps: FirstRunFrameDeps) -> tuple[str, str | list[str] | None]:
      81    original = line.strip()
      82    if original.startswith("/"):
      83        original = original[1:].strip()
      84    lowered = original.lower()
      85    if lowered in {"exit", "quit", ":q", "5"}:
      86        return "exit", None
      87    if lowered in {"clear"}:
      88        return "clear", None
      89    if lowered in {"help", "?", "commands"}:
      90        return "notice", [
      91            "Finish setup before chat or jobs are available.",
      92            "Enter endpoint, API key, and model id when prompted.",
      93            "Doctor must verify the configured model before the workspace opens.",
      94        ]
      95    if lowered in {"1", "new"}:
      96        return "notice", "Finish setup first. Then tell Nipux what job to create from the chat workspace."
      97    if lowered.startswith("new "):
      98        return "notice", "Finish setup first. Then describe worker jobs in the chat workspace."
      99    if lowered in {"2", "jobs", "ls"}:
     100        return "notice", "Finish setup first. Jobs are available after Doctor verifies the configured model."
     101    if lowered == "settings":
     102        return "notice", "Config is changed with slash commands: /model, /api-key, /base-url, /context."
     103    if lowered in {"back"}:
     104        return "notice", "Setup is linear during first run. Continue forward, then edit settings later if needed."
     105    if lowered in {"3", "doctor"}:
     106        notices = deps.verify_model_setup()
     107        if deps.model_setup_verified():
     108            return "open", WORKSPACE_CHAT_ID
     109        return "notice", notices
     110    if lowered in {"4", "init"}:
     111        return "notice", deps.capture_command("init")
     112    if lowered == "shell":
     113        return "notice", "The old console is only available as `nipux shell` from your terminal."
     114    first = first_token(original)
     115    if first == "shell":
     116        return "notice", "The old console is only available as `nipux shell` from your terminal."
     117    if first in {"create", "new"}:
     118        return "notice", "Finish setup first. Then describe worker jobs in the chat workspace."
     119    if first in CHAT_SETTING_COMMANDS or first in {"api-key", "key"}:
     120        return "notice", deps.capture_setting_command(original)
     121    if first in deps.shell_command_names:
     122        before_job_id = deps.current_default_job_id()
     123        output = deps.capture_command(original)
     124        after_job_id = deps.current_default_job_id()
     125        if first == "create" and after_job_id and after_job_id != before_job_id:
     126            return "open", after_job_id
     127        return "notice", output
     128    objective = deps.extract_objective(original)
     129    if objective:
     130        return "notice", "Finish setup first. Then describe worker jobs in the chat workspace."
     131    return "notice", first_run_chat_reply(original)
     132
     133
     134def first_run_chat_reply(message: str) -> str:
     135    del message
     136    return "Setup must be completed before chat is available."
     137
     138
     139def create_first_run_job(objective: str, *, deps: FirstRunFrameDeps) -> str | list[str]:
     140    objective = objective.strip()
     141    if not objective:
     142        return ["No job created. Type an objective first."]
     143    if not deps.model_setup_verified():
     144        return [
     145            "No job created.",
     146            "Finish model setup first: choose a connector, set the endpoint/key if needed, then run Doctor.",
     147            "Doctor must confirm that the configured model accepts a chat request.",
     148        ]
     149    job_id, _title = deps.create_job(objective=objective, title=None, kind="generic", cadence=None)
     150    return job_id
     151
     152
     153def capture_first_run_command(line: str, run_shell_line: Callable[[str], bool]) -> list[str]:
     154    stream = StringIO()
     155    with redirect_stdout(stream):
     156        try:
     157            run_shell_line(line)
     158        except SystemExit as exc:
     159            if exc.code not in (None, 0):
     160                print(f"command exited with status {exc.code}")
     161    lines = [" ".join(item.split()) for item in stream.getvalue().splitlines() if item.strip()]
     162    return lines[-8:] or ["done"]
     163
     164
     165def first_token(line: str) -> str:
     166    try:
     167        parts = shlex.split(line)
     168    except ValueError:
     169        parts = line.split()
     170    return parts[0].lower() if parts else ""

nipux_cli/first_run_frame_runtime.py 436 lines

       1"""Terminal runtime for the first-run Nipux workspace."""
       2
       3from __future__ import annotations
       4
       5import select
       6import shutil
       7import sys
       8import termios
       9import time
      10import tty
      11from dataclasses import dataclass
      12from typing import Callable
      13from urllib.parse import urlparse
      14
      15from nipux_cli.config import load_config
      16from nipux_cli.settings import inline_setting_notice
      17from nipux_cli.tui_commands import FIRST_RUN_SLASH_COMMANDS, autocomplete_slash, cycle_slash, slash_completion_for_submit
      18from nipux_cli.tui_input import (
      19    decode_terminal_escape,
      20    drain_pending_input,
      21    read_escape_sequence,
      22    read_terminal_char,
      23)
      24from nipux_cli.tui_style import _frame_enter_sequence, _frame_exit_sequence
      25
      26
      27@dataclass(frozen=True)
      28class FirstRunRuntimeDeps:
      29    render_frame: Callable[[str, list[str], int, str, str | None, str], str]
      30    actions: Callable[[str], list[tuple[str, str, str]]]
      31    handle_action: Callable[[str], tuple[str, str | list[str] | None]]
      32    handle_line: Callable[[str], tuple[str, str | list[str] | None]]
      33    click_action: Callable[[int, int, str], int | str | None]
      34
      35
      36def run_first_run_frame(*, deps: FirstRunRuntimeDeps) -> str | None:
      37    buffer = ""
      38    notices: list[str] = []
      39    next_job_id: str | None = None
      40    view = "endpoint"
      41    selected = 0
      42    editing_field: str | None = required_first_run_edit_field(view)
      43    old_attrs = termios.tcgetattr(sys.stdin)
      44    print(_frame_enter_sequence(), end="", flush=True)
      45    try:
      46        stdin_fd = sys.stdin.fileno()
      47        tty.setcbreak(stdin_fd)
      48        needs_render = True
      49        last_render = 0.0
      50        last_frame = ""
      51        while next_job_id is None:
      52            now = time.monotonic()
      53            if needs_render or now - last_render >= 1.0:
      54                selected = clamp_selection(selected, deps.actions(view))
      55                last_frame = _safe_render_frame(
      56                    deps,
      57                    buffer=buffer,
      58                    notices=notices,
      59                    selected=selected,
      60                    view=view,
      61                    editing_field=editing_field,
      62                    previous_frame=last_frame,
      63                )
      64                needs_render = False
      65                last_render = now
      66            try:
      67                readable, _, _ = select.select([stdin_fd], [], [], 0.05)
      68            except OSError as exc:
      69                _append_notice(notices, f"terminal read failed: {type(exc).__name__}: {_one_line(exc, 90)}")
      70                needs_render = True
      71                continue
      72            if not readable:
      73                continue
      74            try:
      75                char = read_terminal_char(stdin_fd)
      76            except OSError as exc:
      77                _append_notice(notices, f"terminal input failed: {type(exc).__name__}: {_one_line(exc, 90)}")
      78                needs_render = True
      79                continue
      80            if editing_field is not None:
      81                previous_edit = editing_field
      82                try:
      83                    buffer, editing_field, should_exit = _handle_edit_input(
      84                        char,
      85                        buffer=buffer,
      86                        editing_field=editing_field,
      87                        notices=notices,
      88                        stdin_fd=stdin_fd,
      89                    )
      90                except Exception as exc:
      91                    buffer = ""
      92                    editing_field = None
      93                    _append_notice(notices, f"edit failed: {type(exc).__name__}: {_one_line(exc, 90)}")
      94                    needs_render = True
      95                    continue
      96                if should_exit:
      97                    return None
      98                if previous_edit and editing_field is None:
      99                    next_view = next_first_run_view_after_edit(view)
     100                    if next_view:
     101                        view = next_view
     102                        selected = 0
     103                        editing_field = required_first_run_edit_field(view)
     104                needs_render = True
     105                continue
     106            if char in {"\r", "\n"}:
     107                buffer, should_submit = slash_completion_for_submit(buffer, FIRST_RUN_SLASH_COMMANDS)
     108                if not should_submit:
     109                    needs_render = True
     110                    continue
     111                try:
     112                    action, payload = _submit_first_run_line(buffer, selected=selected, view=view, deps=deps)
     113                except Exception as exc:
     114                    action, payload = "notice", f"input failed: {type(exc).__name__}: {_one_line(exc, 100)}"
     115                buffer = ""
     116                try:
     117                    state = _apply_first_run_action(action, payload, view=view, selected=selected, notices=notices)
     118                except Exception as exc:
     119                    _append_notice(notices, f"action failed: {type(exc).__name__}: {_one_line(exc, 100)}")
     120                    state = (view, selected, None, None, False)
     121                view, selected, editing_field, next_job_id, should_exit = state
     122                editing_field = editing_field or required_first_run_edit_field(view)
     123                if should_exit:
     124                    return None
     125                needs_render = True
     126                continue
     127            if char in {"\x04"}:
     128                return None
     129            if char == "\x03":
     130                buffer = ""
     131                _append_notice(notices, "cancelled input")
     132                needs_render = True
     133                continue
     134            if char == "\x15":
     135                buffer = ""
     136                needs_render = True
     137                continue
     138            if char in {"\x7f", "\b"}:
     139                buffer = buffer[:-1]
     140                needs_render = True
     141                continue
     142            if char == "\t":
     143                try:
     144                    buffer = autocomplete_slash(buffer, FIRST_RUN_SLASH_COMMANDS)
     145                except Exception as exc:
     146                    _append_notice(notices, f"autocomplete failed: {type(exc).__name__}: {_one_line(exc, 90)}")
     147                needs_render = True
     148                continue
     149            if char == "\x1b":
     150                try:
     151                    view, selected, editing_field, next_job_id, should_exit, buffer = _handle_first_run_escape(
     152                        stdin_fd,
     153                        view=view,
     154                        selected=selected,
     155                        buffer=buffer,
     156                        notices=notices,
     157                        deps=deps,
     158                    )
     159                except Exception as exc:
     160                    _append_notice(notices, f"navigation failed: {type(exc).__name__}: {_one_line(exc, 90)}")
     161                    should_exit = False
     162                if should_exit:
     163                    return None
     164                editing_field = editing_field or required_first_run_edit_field(view)
     165                needs_render = True
     166                continue
     167            if char.isprintable():
     168                buffer += char
     169                needs_render = True
     170    except KeyboardInterrupt:
     171        return None
     172    finally:
     173        termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_attrs)
     174        print(_frame_exit_sequence(), flush=True)
     175    return next_job_id
     176
     177
     178def clamp_selection(selected: int, actions: list[tuple[str, str, str]]) -> int:
     179    if not actions:
     180        return 0
     181    return max(0, min(selected, len(actions) - 1))
     182
     183
     184def _safe_render_frame(
     185    deps: FirstRunRuntimeDeps,
     186    *,
     187    buffer: str,
     188    notices: list[str],
     189    selected: int,
     190    view: str,
     191    editing_field: str | None,
     192    previous_frame: str,
     193) -> str:
     194    try:
     195        return deps.render_frame(buffer, notices, selected, view, editing_field, previous_frame)
     196    except Exception as exc:
     197        _append_notice(notices, f"render failed: {type(exc).__name__}: {_one_line(exc, 100)}")
     198        frame = _fallback_first_run_frame(buffer=buffer, notices=notices, view=view)
     199        print("\033[H" + frame, end="", flush=True)
     200        return frame
     201
     202
     203def _fallback_first_run_frame(*, buffer: str, notices: list[str], view: str) -> str:
     204    width, height = shutil.get_terminal_size((100, 30))
     205    width = max(60, width)
     206    lines = [
     207        _fit_plain("NIPUX - setup safe mode", width),
     208        _fit_plain("=" * width, width),
     209        _fit_plain(f"Screen: {view}", width),
     210        _fit_plain("A UI render error was caught. You can keep typing; /exit leaves.", width),
     211        "",
     212        "Recent notices:",
     213    ]
     214    lines.extend(f"- {_one_line(notice, width - 3)}" for notice in notices[-8:])
     215    lines.extend(["", f"> {_one_line(buffer, width - 3)}"])
     216    return "\n".join(_fit_plain(line, width) for line in lines[:height])
     217
     218
     219def _submit_first_run_line(
     220    buffer: str,
     221    *,
     222    selected: int,
     223    view: str,
     224    deps: FirstRunRuntimeDeps,
     225) -> tuple[str, str | list[str] | None]:
     226    line = buffer.strip()
     227    if not line:
     228        actions = deps.actions(view)
     229        if not actions:
     230            return "notice", "This setup step requires an explicit value."
     231        return deps.handle_action(actions[clamp_selection(selected, actions)][0])
     232    if not line.startswith("/"):
     233        return "notice", "Complete the active setup field before continuing."
     234    return deps.handle_line(line)
     235
     236
     237def _handle_first_run_escape(
     238    stdin_fd: int,
     239    *,
     240    view: str,
     241    selected: int,
     242    buffer: str,
     243    notices: list[str],
     244    deps: FirstRunRuntimeDeps,
     245) -> tuple[str, int, str | None, str | None, bool, str]:
     246    key, payload = decode_terminal_escape(read_escape_sequence("\x1b", fd=stdin_fd))
     247    if key in {"up", "down"} and buffer.startswith("/"):
     248        buffer = cycle_slash(buffer, FIRST_RUN_SLASH_COMMANDS, direction=-1 if key == "up" else 1)
     249        return view, selected, None, None, False, buffer
     250    if key == "up":
     251        actions = deps.actions(view)
     252        if not actions:
     253            return view, selected, None, None, False, buffer
     254        return view, (selected - 1) % len(actions), None, None, False, buffer
     255    if key == "down":
     256        actions = deps.actions(view)
     257        if not actions:
     258            return view, selected, None, None, False, buffer
     259        return view, (selected + 1) % len(actions), None, None, False, buffer
     260    if key in {"left", "right"}:
     261        actions = deps.actions(view)
     262        if not actions:
     263            return view, selected, None, None, False, buffer
     264        delta = 1 if key == "right" else -1
     265        return view, (selected + delta) % len(actions), None, None, False, buffer
     266    if key == "click" and isinstance(payload, tuple):
     267        clicked = deps.click_action(payload[0], payload[1], view)
     268        if clicked is not None:
     269            if isinstance(clicked, str):
     270                action, payload = deps.handle_action(clicked)
     271                next_view, next_selected, editing_field, next_job_id, should_exit = _apply_first_run_action(
     272                    action,
     273                    payload,
     274                    view=view,
     275                    selected=selected,
     276                    notices=notices,
     277                )
     278                return next_view, next_selected, editing_field, next_job_id, should_exit, buffer
     279            actions = deps.actions(view)
     280            if not actions:
     281                return view, selected, None, None, False, buffer
     282            action, payload = deps.handle_action(actions[clamp_selection(clicked, actions)][0])
     283            next_view, next_selected, editing_field, next_job_id, should_exit = _apply_first_run_action(
     284                action,
     285                payload,
     286                view=view,
     287                selected=clicked,
     288                notices=notices,
     289            )
     290            return next_view, next_selected, editing_field, next_job_id, should_exit, buffer
     291    drain_pending_input(stdin_fd)
     292    return view, selected, None, None, False, buffer
     293
     294
     295def directional_first_run_action(actions: list[tuple[str, str, str]], *, direction: int) -> str | None:
     296    """Return the setup-screen action for left/right navigation."""
     297
     298    if direction >= 0:
     299        for key, label, _detail in actions:
     300            if key.startswith("view:") and label.lower() in {"begin setup", "continue"}:
     301                return key
     302        return None
     303    for key, label, _detail in reversed(actions):
     304        if key.startswith("view:") and label.lower() == "back":
     305            return key
     306    return None
     307
     308
     309def _apply_first_run_action(
     310    action: str,
     311    payload: str | list[str] | None,
     312    *,
     313    view: str,
     314    selected: int,
     315    notices: list[str],
     316) -> tuple[str, int, str | None, str | None, bool]:
     317    if action == "view":
     318        notices.clear()
     319        return str(payload or "start"), 0, None, None, False
     320    if action == "exit":
     321        return view, selected, None, None, True
     322    if action == "clear":
     323        notices.clear()
     324        return view, selected, None, None, False
     325    if action == "open":
     326        return view, selected, None, str(payload), False
     327    if action == "edit":
     328        editing_field = str(payload)
     329        _append_notice(notices, f"editing {editing_field}; enter saves, escape cancels", limit=10)
     330        return view, selected, editing_field, None, False
     331    if isinstance(payload, list):
     332        for item in payload:
     333            if str(item).strip():
     334                _append_notice(notices, str(item), limit=10)
     335    elif payload:
     336        _append_notice(notices, str(payload), limit=10)
     337    return view, selected, None, None, False
     338
     339
     340def _handle_edit_input(
     341    char: str,
     342    *,
     343    buffer: str,
     344    editing_field: str,
     345    notices: list[str],
     346    stdin_fd: int,
     347) -> tuple[str, str | None, bool]:
     348    if char in {"\r", "\n"}:
     349        saved, notice = _save_first_run_edit(editing_field, buffer)
     350        _append_notice(notices, notice, limit=10)
     351        return ("", None, False) if saved else (buffer, editing_field, False)
     352    if char in {"\x04"}:
     353        return buffer, editing_field, True
     354    if char == "\x03":
     355        _append_notice(notices, "cancelled edit", limit=10)
     356        return "", editing_field, False
     357    if char == "\x15":
     358        return "", editing_field, False
     359    if char in {"\x7f", "\b"}:
     360        return buffer[:-1], editing_field, False
     361    if char == "\x1b":
     362        key, _payload = decode_terminal_escape(read_escape_sequence(char, fd=stdin_fd))
     363        if key == "unknown":
     364            _append_notice(notices, "cancelled edit", limit=10)
     365            return "", editing_field, False
     366        return buffer, editing_field, False
     367    if char.isprintable():
     368        return buffer + char, editing_field, False
     369    return buffer, editing_field, False
     370
     371
     372def required_first_run_edit_field(view: str) -> str | None:
     373    return {
     374        "endpoint": "model.base_url",
     375        "api": "secret:model.api_key",
     376        "model": "model.name",
     377    }.get(view)
     378
     379
     380def next_first_run_view_after_edit(view: str) -> str | None:
     381    return {
     382        "endpoint": "api",
     383        "api": "model",
     384        "model": "access",
     385    }.get(view)
     386
     387
     388def _save_first_run_edit(field: str, raw_value: str) -> tuple[bool, str]:
     389    value = raw_value.strip()
     390    if field == "model.base_url":
     391        if not value:
     392            return False, "Endpoint URL is required."
     393        parsed = urlparse(value)
     394        if parsed.scheme not in {"http", "https"} or not parsed.netloc:
     395            return False, "Endpoint must be a full http:// or https:// URL."
     396        if not parsed.path.rstrip("/").endswith("/v1"):
     397            return False, "Endpoint must point at an OpenAI-compatible /v1 path."
     398        return True, inline_setting_notice(field, value)
     399    if field == "model.name":
     400        if not value:
     401            return False, "Model id is required."
     402        return True, inline_setting_notice(field, value)
     403    if field == "secret:model.api_key":
     404        if not value:
     405            return False, "API key is required, or type skip for a local endpoint."
     406        if value.lower() in {"skip", "none", "local"}:
     407            config = load_config()
     408            if not _is_local_endpoint(config.model.base_url):
     409                return False, "Only local endpoints can skip the API key."
     410            return True, "skipped API key for local endpoint"
     411        return True, inline_setting_notice(field, value)
     412    return True, inline_setting_notice(field, value)
     413
     414
     415def _is_local_endpoint(value: str) -> bool:
     416    host = (urlparse(value).hostname or "").lower()
     417    return host in {"localhost", "127.0.0.1", "::1", "0.0.0.0"} or host.endswith(".local")
     418
     419
     420def _append_notice(notices: list[str], message: str, *, limit: int = 10) -> None:
     421    notices.append(message)
     422    notices[:] = notices[-limit:]
     423
     424
     425def _one_line(value: object, width: int) -> str:
     426    text = " ".join(str(value).split())
     427    if len(text) <= width:
     428        return text
     429    return text[: max(0, width - 3)] + "..."
     430
     431
     432def _fit_plain(text: object, width: int) -> str:
     433    content = str(text)
     434    if len(content) > width:
     435        content = _one_line(content, width)
     436    return content + " " * max(0, width - len(content))

nipux_cli/first_run_tui.py 571 lines

       1"""First-run terminal UI rendering for Nipux."""
       2
       3from __future__ import annotations
       4
       5from typing import Any
       6
       7from nipux_cli.config import AppConfig
       8from nipux_cli.settings import (
       9    config_field_value,
      10    edit_target_hint,
      11    edit_target_label,
      12    edit_target_masks_input,
      13)
      14from nipux_cli.tui_layout import _compose_bar
      15from nipux_cli.tui_style import (
      16    _accent,
      17    _bold,
      18    _center_ansi,
      19    _fit_ansi,
      20    _muted,
      21    _one_line,
      22    _style,
      23    _strip_ansi,
      24    _themed_lines,
      25)
      26
      27
      28INSTALL_FLOW = [
      29    ("endpoint", "Endpoint", "OpenAI-compatible /v1"),
      30    ("api", "API key", "secret stored in .env"),
      31    ("model", "Model", "choose the model id"),
      32    ("access", "Tools", "browser, web, CLI, files"),
      33    ("doctor", "Doctor", "check setup"),
      34]
      35
      36
      37FIRST_RUN_ACTIONS_BY_VIEW: dict[str, list[tuple[str, str, str]]] = {
      38    "endpoint": [],
      39    "api": [],
      40    "model": [],
      41    "access": [
      42        ("toggle:tools.browser", "Browser", "automation"),
      43        ("toggle:tools.web", "Web", "search/extract"),
      44        ("toggle:tools.shell", "CLI", "terminal commands"),
      45        ("toggle:tools.files", "Files", "write files"),
      46        ("view:doctor", "Continue", "run checks"),
      47    ],
      48    "doctor": [
      49        ("doctor", "Run doctor", "verify setup"),
      50        ("open_workspace", "Open chat", "talk to Nipux"),
      51    ],
      52}
      53
      54
      55def build_first_run_frame(
      56    input_buffer: str,
      57    notices: list[str],
      58    *,
      59    width: int,
      60    height: int,
      61    selected: int = 0,
      62    view: str = "start",
      63    editing_field: str | None = None,
      64    config: AppConfig,
      65    daemon_text: str,
      66    jobs: list[dict[str, Any]],
      67    home: str,
      68    config_path: str,
      69) -> str:
      70    del daemon_text
      71    width = max(92, width)
      72    height = max(22, height)
      73    view = _normalize_first_run_view(view)
      74    selected = _clamp_first_run_selection(selected, view)
      75    header: list[str] = []
      76    if editing_field:
      77        hint = _first_run_edit_hint(editing_field, config)
      78        prompt_label = _first_run_prompt_label(editing_field)
      79    else:
      80        hint = _first_run_hint(view)
      81        prompt_label = "Setup"
      82    suggestions: list[str] = []
      83    compose_lines = _compose_bar(
      84        input_buffer,
      85        width=width,
      86        hint=hint,
      87        suggestions=suggestions,
      88        prompt_label=prompt_label,
      89        title="setup",
      90        mask_input=edit_target_masks_input(editing_field),
      91    )
      92    footer_rows = len(compose_lines)
      93    body_rows = max(10, height - len(header) - 1 - footer_rows)
      94    body_lines = _wizard_body_lines(
      95        notices=notices,
      96        jobs=jobs,
      97        config=config,
      98        home=home,
      99        config_path=config_path,
     100        selected=selected,
     101        view=view,
     102        width=width,
     103        rows=body_rows,
     104    )
     105    lines = [*header, *body_lines, *compose_lines]
     106    return "\n".join(first_run_themed_lines(lines[:height], width=width))
     107
     108
     109def first_run_columns(width: int) -> tuple[int, int]:
     110    right_width = min(max(40, int(width * 0.34)), 54)
     111    left_width = max(48, width - right_width - 3)
     112    if left_width < 48:
     113        left_width = 48
     114        right_width = max(36, width - left_width - 3)
     115    return left_width, right_width
     116
     117
     118def first_run_actions(view: str) -> list[tuple[str, str, str]]:
     119    return FIRST_RUN_ACTIONS_BY_VIEW[_normalize_first_run_view(view)]
     120
     121
     122def first_run_themed_lines(lines: list[str], *, width: int) -> list[str]:
     123    return _themed_lines(lines, width=width)
     124
     125
     126def _wizard_body_lines(
     127    *,
     128    notices: list[str],
     129    jobs: list[dict[str, Any]],
     130    config: AppConfig,
     131    home: str,
     132    config_path: str,
     133    selected: int,
     134    view: str,
     135    width: int,
     136    rows: int,
     137) -> list[str]:
     138    if view == "model":
     139        lines = _model_page_lines(config=config, selected=selected, width=width)
     140    elif view == "endpoint":
     141        lines = _endpoint_page_lines(config=config, selected=selected, width=width)
     142    elif view == "api":
     143        lines = _api_page_lines(config=config, selected=selected, width=width)
     144    elif view == "access":
     145        lines = _access_page_lines(config=config, selected=selected, width=width)
     146    elif view == "doctor":
     147        lines = _doctor_page_lines(config=config, selected=selected, width=width)
     148    else:
     149        lines = _endpoint_page_lines(config=config, selected=selected, width=width)
     150    if notices:
     151        lines = _append_notice_block(lines, notices, width=width, rows=rows)
     152    return _fit_page(lines, width=width, rows=rows)
     153
     154
     155def _model_page_lines(*, config: AppConfig, selected: int, width: int) -> list[str]:
     156    return [
     157        *_step_header("model", width=width),
     158        "",
     159        _center_ansi(_muted(_step_count_label("model")), width),
     160        _center_ansi(_bold("Enter the model id"), width),
     161        _center_ansi(_muted("This exact model powers chat replies and background workers."), width),
     162        "",
     163        *_panel(
     164            "MODEL ID",
     165            [
     166                _bold(_accent("waiting for input")),
     167                _muted(f"current config: {_one_line(config.model.model, max(16, width - 30))}"),
     168                _muted("Type the model id below. Blank input is not accepted."),
     169            ],
     170            width=min(84, width - 8),
     171            page_width=width,
     172        ),
     173        "",
     174        _center_ansi(_muted("Press Enter after typing the model. Setup moves forward automatically."), width),
     175    ]
     176
     177
     178def _endpoint_page_lines(*, config: AppConfig, selected: int, width: int) -> list[str]:
     179    return [
     180        *_step_header("endpoint", width=width),
     181        "",
     182        _center_ansi(_muted(_step_count_label("endpoint")), width),
     183        _center_ansi(_bold("Enter the endpoint first"), width),
     184        _center_ansi(_muted("Use an OpenAI-compatible /v1 endpoint. Local or hosted both work."), width),
     185        "",
     186        *_form_panel(
     187            "BASE URL",
     188            f"waiting for input · current config: {config.model.base_url}",
     189            "required",
     190            width=min(90, width - 8),
     191            page_width=width,
     192        ),
     193        "",
     194        _center_ansi(_muted("Example formats: http://localhost:8000/v1 or https://provider.example/v1"), width),
     195    ]
     196
     197
     198def _api_page_lines(*, config: AppConfig, selected: int, width: int) -> list[str]:
     199    key_state = "set" if config.model.api_key else "missing"
     200    key_color = _style(key_state, "32" if key_state == "set" else "33")
     201    return [
     202        *_step_header("api", width=width),
     203        "",
     204        _center_ansi(_muted(_step_count_label("api")), width),
     205        _center_ansi(_bold("Enter the API key"), width),
     206        _center_ansi(_muted("Hosted endpoints need a key. For a local endpoint, type skip."), width),
     207        "",
     208        *_panel(
     209            "API KEY",
     210            [
     211                f"{_muted('state')} {key_color}",
     212                f"{_muted('env')}   {_bold(config.model.api_key_env)}",
     213                _muted("Stored in the local Nipux env file, never in repository config."),
     214            ],
     215            width=min(84, width - 8),
     216            page_width=width,
     217        ),
     218        "",
     219        _center_ansi(_muted("Blank input is not accepted; type skip only when the endpoint is local."), width),
     220    ]
     221
     222
     223def _access_page_lines(*, config: AppConfig, selected: int, width: int) -> list[str]:
     224    rows = [
     225        _access_row("browser", config.tools.browser, "persistent browser automation"),
     226        _access_row("web", config.tools.web, "web search and page extraction"),
     227        _access_row("CLI", config.tools.shell, "bounded terminal commands"),
     228        _access_row("files", config.tools.files, "write deliverables into the workspace"),
     229    ]
     230    return [
     231        *_step_header("access", width=width),
     232        "",
     233        _center_ansi(_muted(_step_count_label("access")), width),
     234        _center_ansi(_bold("Choose tool access"), width),
     235        _center_ansi(_muted("These switches control the generic tools workers can call for any job."), width),
     236        "",
     237        *_panel("TOOL ACCESS", rows, width=min(90, width - 8), page_width=width),
     238        "",
     239        *_action_cards(first_run_actions("access"), selected=selected, config=config, width=width),
     240    ]
     241
     242
     243def _doctor_page_lines(*, config: AppConfig, selected: int, width: int) -> list[str]:
     244    checks = [
     245        ("state directory", "writable under ~/.nipux or NIPUX_HOME"),
     246        ("database", "SQLite state store can open"),
     247        ("model config", f"{config.model.model} at {config.model.base_url}"),
     248        (
     249            "tools",
     250            f"browser={config.tools.browser} web={config.tools.web} CLI={config.tools.shell} files={config.tools.files}",
     251        ),
     252    ]
     253    rows = [f"{_accent('✓')} {_fit_ansi(name, 18)} {_muted(detail)}" for name, detail in checks]
     254    return [
     255        *_step_header("doctor", width=width),
     256        "",
     257        _center_ansi(_muted(_step_count_label("doctor")), width),
     258        _center_ansi(_bold("Run checks"), width),
     259        _center_ansi(_muted("Doctor calls the configured model before the workspace opens."), width),
     260        "",
     261        *_panel("DOCTOR", rows, width=min(90, width - 8), page_width=width),
     262        "",
     263        _center_ansi(_muted("If a check fails, edit with /base-url, /api-key, or /model, then run Doctor again."), width),
     264        "",
     265        *_action_cards(first_run_actions("doctor"), selected=selected, config=config, width=width),
     266    ]
     267
     268
     269def _stepper_lines(view: str, *, config: AppConfig, width: int) -> list[str]:
     270    lines: list[str] = []
     271    for key, label, _detail in INSTALL_FLOW:
     272        marker = _accent("●") if key == view else _muted("○")
     273        state = _step_state(key, config=config)
     274        lines.append(_fit_ansi(f"{marker} {_fit_ansi(label, 10)} {_muted(state)}", width))
     275    return lines
     276
     277
     278def _step_header(view: str, *, width: int) -> list[str]:
     279    parts = []
     280    for index, (key, label, _detail) in enumerate(INSTALL_FLOW, start=1):
     281        marker = _accent("●") if key == view else _muted("○")
     282        text = _bold(label) if key == view else _muted(label)
     283        parts.append(f"{marker} {index} {text}")
     284    return [
     285        _center_ansi("   ".join(parts), width),
     286        _muted("─" * width),
     287    ]
     288
     289
     290def _action_cards(
     291    actions: list[tuple[str, str, str]],
     292    *,
     293    selected: int,
     294    config: AppConfig,
     295    width: int,
     296) -> list[str]:
     297    if not actions:
     298        return []
     299    gap = 2
     300    card_width = max(18, min(34, (width - (len(actions) - 1) * gap - 4) // len(actions)))
     301    cards = [_action_tile(index, action, selected=selected, config=config, width=card_width) for index, action in enumerate(actions)]
     302    rows = _join_many_cards(cards, gap=gap, width=width)
     303    return [_center_ansi(row.rstrip(), width) for row in rows]
     304
     305
     306def _action_tile(
     307    index: int,
     308    action: tuple[str, str, str],
     309    *,
     310    selected: int,
     311    config: AppConfig,
     312    width: int,
     313) -> list[str]:
     314    key, label, detail = action
     315    active = index == selected
     316    border = _accent if active else _muted
     317    marker = _accent("›") if active else _muted(" ")
     318    label_text = _bold(label) if active else label
     319    value = _action_value(key, detail, config=config)
     320    inner = max(8, width - 4)
     321    return [
     322        border("╭" + "─" * (width - 2) + "╮"),
     323        border("│ ") + _fit_ansi(f"{marker} {index + 1}. {label_text}", inner) + border(" │"),
     324        border("│ ") + _fit_ansi(_muted(_one_line(value, inner)), inner) + border(" │"),
     325        border("╰" + "─" * (width - 2) + "╯"),
     326    ]
     327
     328
     329def _panel(title: str, body: list[str], *, width: int, page_width: int | None = None) -> list[str]:
     330    width = max(32, width)
     331    inner = max(8, width - 4)
     332    title_text = f" {title} "
     333    lines = [_muted("╭─" + title_text + "─" * max(0, width - len(title_text) - 3) + "╮")]
     334    for item in body:
     335        lines.append(_muted("│ ") + _fit_ansi(item, inner) + _muted(" │"))
     336    lines.append(_muted("╰" + "─" * (width - 2) + "╯"))
     337    return [_center_ansi(line, page_width or width) for line in lines]
     338
     339
     340def _form_panel(title: str, value: str, command: str, *, width: int, page_width: int | None = None) -> list[str]:
     341    return _panel(
     342        title,
     343        [
     344            _bold(_accent(_one_line(value, max(16, width - 10)))),
     345            _muted(f"{command}; type the value in the setup input below"),
     346        ],
     347        width=width,
     348        page_width=page_width,
     349    )
     350
     351
     352def _choice_card(title: str, copy: str, value: str, *, active: bool, width: int) -> list[str]:
     353    border = _accent if active else _muted
     354    marker = _accent("● selected") if active else _muted("○ available")
     355    inner = max(8, width - 4)
     356    return [
     357        border("╭" + "─" * (width - 2) + "╮"),
     358        border("│ ") + _fit_ansi(_bold(title), inner) + border(" │"),
     359        border("│ ") + _fit_ansi(marker, inner) + border(" │"),
     360        border("│ ") + _fit_ansi(_muted(copy), inner) + border(" │"),
     361        border("│ ") + _fit_ansi(_accent(value), inner) + border(" │"),
     362        border("╰" + "─" * (width - 2) + "╯"),
     363    ]
     364
     365
     366def _join_cards(left: list[str], right: list[str], *, width: int) -> list[str]:
     367    gap = "  "
     368    rows = []
     369    for index in range(max(len(left), len(right))):
     370        left_line = left[index] if index < len(left) else " " * len(_strip_ansi(left[0]))
     371        right_line = right[index] if index < len(right) else " " * len(_strip_ansi(right[0]))
     372        rows.append(_center_ansi(left_line + gap + right_line, width))
     373    return rows
     374
     375
     376def _join_many_cards(cards: list[list[str]], *, gap: int, width: int) -> list[str]:
     377    rows: list[str] = []
     378    max_rows = max(len(card) for card in cards)
     379    gap_text = " " * gap
     380    for row_index in range(max_rows):
     381        row_parts = []
     382        for card in cards:
     383            fallback_width = len(_strip_ansi(card[0]))
     384            row_parts.append(card[row_index] if row_index < len(card) else " " * fallback_width)
     385        rows.append(gap_text.join(row_parts))
     386    return [_fit_ansi(row, width) for row in rows]
     387
     388
     389def _append_notice_block(lines: list[str], notices: list[str], *, width: int, rows: int) -> list[str]:
     390    budget = max(3, min(6, rows // 4))
     391    notice_lines = [_bold("Transcript")]
     392    for notice in notices[-budget:]:
     393        notice_lines.append(_fit_ansi(_accent("› ") + _one_line(notice, width - 4), width))
     394    if len(lines) + len(notice_lines) + 1 <= rows:
     395        return [*lines, "", *notice_lines]
     396    keep = max(0, rows - len(notice_lines) - 1)
     397    return [*lines[:keep], "", *notice_lines]
     398
     399
     400def _fit_page(lines: list[str], *, width: int, rows: int) -> list[str]:
     401    fitted = [_fit_ansi(line, width) for line in lines]
     402    if len(fitted) < rows:
     403        fitted.extend([" " * width for _ in range(rows - len(fitted))])
     404    return fitted[:rows]
     405
     406
     407def _action_line(
     408    index: int,
     409    action: tuple[str, str, str],
     410    *,
     411    selected: int,
     412    config: AppConfig,
     413    width: int,
     414) -> str:
     415    key, label, detail = action
     416    marker = _accent("›") if index == selected else _muted(" ")
     417    label_text = _bold(label) if index == selected else label
     418    value = _action_value(key, detail, config=config)
     419    return _fit_ansi(
     420        f"{marker} {index + 1}. {_fit_ansi(label_text, 15)} {_muted(_one_line(value, max(8, width - 21)))}",
     421        width,
     422    )
     423
     424
     425def _screen_value_lines(view: str, *, config: AppConfig, width: int) -> list[str]:
     426    if view == "model":
     427        return [_large_value("model", config.model.model, width=width)]
     428    if view == "endpoint":
     429        return [_large_value("endpoint", config.model.base_url, width=width)]
     430    if view == "api":
     431        key_state = "set" if config.model.api_key else "missing"
     432        return [
     433            _large_value("key", key_state, width=width),
     434            _muted(f"Stored under {config.model.api_key_env} in ~/.nipux/.env."),
     435        ]
     436    if view == "doctor":
     437        return [
     438            _large_value("check", "ready to run", width=width),
     439            _muted("Doctor verifies runtime checks, then sends a small chat request to the configured model."),
     440        ]
     441    return []
     442
     443
     444def _large_value(label: str, value: str, *, width: int) -> str:
     445    label_text = _muted(f"{label} ")
     446    return _fit_ansi(label_text + _bold(_accent(_one_line(value, max(12, width - len(label) - 2)))), width)
     447
     448
     449def _action_value(key: str, detail: str, *, config: AppConfig) -> str:
     450    if key.startswith("view:"):
     451        return detail
     452    if key.startswith("edit:"):
     453        field = key.split(":", 1)[1]
     454        return str(config_field_value(field, config))
     455    if key.startswith("toggle:"):
     456        field = key.split(":", 1)[1]
     457        return "enabled" if bool(config_field_value(field, config)) else "disabled"
     458    if key == "secret:model.api_key":
     459        return "stored in .env" if config.model.api_key else f"uses {config.model.api_key_env}"
     460    if key == "preset:local":
     461        return "http://localhost:8000/v1"
     462    return detail
     463
     464
     465def _step_state(key: str, *, config: AppConfig) -> str:
     466    if key == "model":
     467        return _one_line(config.model.model, 20)
     468    if key == "endpoint":
     469        return _one_line(config.model.base_url, 20)
     470    if key == "api":
     471        return "ready" if config.model.api_key or _is_local_endpoint(config.model.base_url) else "missing"
     472    if key == "access":
     473        enabled = sum(bool(value) for value in (config.tools.browser, config.tools.web, config.tools.shell, config.tools.files))
     474        return f"{enabled}/4 enabled"
     475    if key == "doctor":
     476        return "pending"
     477    return ""
     478
     479
     480def _first_run_hint(view: str) -> str:
     481    if view == "endpoint":
     482        return "Required: type an OpenAI-compatible endpoint URL, then Enter."
     483    if view == "api":
     484        return "Required: type an API key, or type skip for a local endpoint."
     485    if view == "model":
     486        return "Required: type the model id accepted by this endpoint."
     487    if view == "access":
     488        return "Use arrows/clicks to toggle tools, then choose Continue."
     489    if view == "doctor":
     490        return "Run Doctor, then open the chat workspace."
     491    return "Complete setup before the workspace opens."
     492
     493
     494def _first_run_edit_hint(field: str, config: AppConfig) -> str:
     495    if field == "model.base_url":
     496        return "Endpoint URL required. Enter saves and advances. Blank input is blocked."
     497    if field == "secret:model.api_key":
     498        return "API key required for hosted endpoints. For local endpoints, type skip."
     499    if field == "model.name":
     500        return "Model id required. Enter saves and advances. Blank input is blocked."
     501    return edit_target_hint(field, config)
     502
     503
     504def _first_run_prompt_label(field: str) -> str:
     505    if field == "model.base_url":
     506        return "Endpoint"
     507    if field == "secret:model.api_key":
     508        return "API key"
     509    if field == "model.name":
     510        return "Model"
     511    return edit_target_label(field)
     512
     513
     514def _left_title(view: str) -> str:
     515    return _screen_heading(view)
     516
     517
     518def _screen_heading(view: str) -> str:
     519    return {
     520        "model": "Choose model",
     521        "endpoint": "Connect endpoint",
     522        "api": "Add API key",
     523        "access": "Choose tools",
     524        "doctor": "Run checks",
     525    }.get(view, "Connect endpoint")
     526
     527
     528def _screen_copy(view: str) -> str:
     529    return {
     530        "model": "The chat controller and workers use this model unless you change it later.",
     531        "endpoint": "Use any OpenAI-compatible /v1 endpoint. This stays generic and provider-neutral.",
     532        "api": "Hosted providers need a secret. Local endpoints can continue without one.",
     533        "access": "Enable the generic tools this worker can use for any job.",
     534        "doctor": "Verify the configured model, then open the main chat workspace.",
     535    }.get(view, "Nipux installs through this full-screen setup.")
     536
     537
     538def _install_summary(config: AppConfig, *, width: int) -> str:
     539    connector = "local connector" if _is_local_endpoint(config.model.base_url) else "hosted connector"
     540    text = f"{connector} · {config.model.model} · {config.model.base_url}"
     541    return _muted(_one_line(text, width))
     542
     543
     544def _normalize_first_run_view(view: str) -> str:
     545    return view if view in FIRST_RUN_ACTIONS_BY_VIEW else "endpoint"
     546
     547
     548def _step_count_label(view: str) -> str:
     549    keys = [key for key, _label, _detail in INSTALL_FLOW]
     550    try:
     551        index = keys.index(view) + 1
     552    except ValueError:
     553        index = 1
     554    return f"STEP {index} / {len(INSTALL_FLOW)}"
     555
     556
     557def _access_row(name: str, enabled: bool, detail: str) -> str:
     558    marker = _accent("on ") if enabled else _muted("off")
     559    return f"{_fit_ansi(name, 10)} {marker} {_muted(detail)}"
     560
     561
     562def _is_local_endpoint(value: str) -> bool:
     563    lowered = value.lower()
     564    return "localhost" in lowered or "127.0.0.1" in lowered or lowered.startswith("http://0.0.0.0")
     565
     566
     567def _clamp_first_run_selection(selected: int, view: str) -> int:
     568    actions = first_run_actions(view)
     569    if not actions:
     570        return 0
     571    return max(0, min(selected, len(actions) - 1))

nipux_cli/frame_snapshot.py 183 lines

       1"""Data loading contract for the interactive Nipux terminal frame."""
       2
       3from __future__ import annotations
       4
       5from typing import Any
       6
       7from nipux_cli.config import AppConfig
       8from nipux_cli.daemon import daemon_lock_status
       9from nipux_cli.db import AgentDB
      10from nipux_cli.tui_outcomes import SUMMARY_EVENT_TYPES, SUMMARY_TOOL_EVENT_TYPES, is_summary_event_candidate
      11
      12
      13WORKSPACE_CHAT_ID = "__workspace__"
      14
      15
      16def load_frame_snapshot(
      17    db: AgentDB,
      18    config: AppConfig,
      19    job_id: str,
      20    *,
      21    default_job_id: str | None = None,
      22    history_limit: int = 12,
      23    workspace_events: list[dict[str, Any]] | None = None,
      24) -> dict[str, Any]:
      25    """Return the compact state bundle rendered by the chat TUI."""
      26
      27    resolved_job_id = job_id or default_job_id
      28    if resolved_job_id == WORKSPACE_CHAT_ID:
      29        return load_workspace_frame_snapshot(
      30            db,
      31            config,
      32            focused_job_id=default_job_id,
      33            history_limit=history_limit,
      34            workspace_events=workspace_events or [],
      35        )
      36    job = db.get_job(resolved_job_id)
      37    jobs = db.list_jobs()
      38    token_usage = db.job_token_usage(resolved_job_id)
      39    token_usage["input_cost_per_million"] = config.model.input_cost_per_million
      40    token_usage["output_cost_per_million"] = config.model.output_cost_per_million
      41    summary_events = _summary_events(db, resolved_job_id, history_limit=history_limit)
      42    return {
      43        "job_id": resolved_job_id,
      44        "job": job,
      45        "jobs": jobs,
      46        "steps": db.list_steps(job_id=resolved_job_id, limit=80),
      47        "artifacts": db.list_artifacts(resolved_job_id, limit=8),
      48        "job_artifacts": {
      49            str(item["id"]): db.list_artifacts(str(item["id"]), limit=3)
      50            for item in jobs[:6]
      51            if item.get("id")
      52        },
      53        "job_summary_events": {
      54            str(item["id"]): _summary_events(db, str(item["id"]), history_limit=3)
      55            for item in jobs[:6]
      56            if item.get("id")
      57        },
      58        "job_counts": {
      59            str(item["id"]): db.job_record_counts(str(item["id"]))
      60            for item in jobs[:6]
      61            if item.get("id")
      62        },
      63        "memory_entries": db.list_memory(resolved_job_id)[:8],
      64        "events": db.list_events(job_id=resolved_job_id, limit=max(history_limit * 16, 240)),
      65        "summary_events": summary_events,
      66        "daemon": daemon_lock_status(config.runtime.home / "agentd.lock"),
      67        "model": config.model.model,
      68        "base_url": config.model.base_url,
      69        "context_length": config.model.context_length,
      70        "token_usage": token_usage,
      71        "counts": db.job_record_counts(resolved_job_id),
      72    }
      73
      74
      75def load_workspace_frame_snapshot(
      76    db: AgentDB,
      77    config: AppConfig,
      78    *,
      79    focused_job_id: str | None = None,
      80    history_limit: int = 12,
      81    workspace_events: list[dict[str, Any]] | None = None,
      82) -> dict[str, Any]:
      83    """Return the chat/control frame before any worker job is focused."""
      84
      85    jobs = db.list_jobs()
      86    events = list(workspace_events or [])[-max(history_limit * 8, 80) :]
      87    focused_job = _safe_job(db, focused_job_id)
      88    focused_id = str(focused_job.get("id") or "") if focused_job else ""
      89    token_usage = _workspace_token_usage(config)
      90    if focused_id:
      91        token_usage = db.job_token_usage(focused_id)
      92        token_usage["input_cost_per_million"] = config.model.input_cost_per_million
      93        token_usage["output_cost_per_million"] = config.model.output_cost_per_million
      94    workspace_job = {
      95        "id": WORKSPACE_CHAT_ID,
      96        "title": "Nipux",
      97        "objective": "Chat with Nipux to create, start, inspect, and steer long-running worker jobs.",
      98        "kind": "workspace",
      99        "status": "ready",
     100        "metadata": {},
     101    }
     102    right_job = focused_job or workspace_job
     103    return {
     104        "job_id": WORKSPACE_CHAT_ID,
     105        "job": workspace_job,
     106        "right_job": right_job,
     107        "right_job_id": focused_id or WORKSPACE_CHAT_ID,
     108        "jobs": jobs,
     109        "steps": db.list_steps(job_id=focused_id, limit=80) if focused_id else [],
     110        "artifacts": db.list_artifacts(focused_id, limit=8) if focused_id else [],
     111        "job_artifacts": {
     112            str(item["id"]): db.list_artifacts(str(item["id"]), limit=3)
     113            for item in jobs[:6]
     114            if item.get("id")
     115        },
     116        "job_summary_events": {
     117            str(item["id"]): _summary_events(db, str(item["id"]), history_limit=3)
     118            for item in jobs[:6]
     119            if item.get("id")
     120        },
     121        "job_counts": {
     122            str(item["id"]): db.job_record_counts(str(item["id"]))
     123            for item in jobs[:6]
     124            if item.get("id")
     125        },
     126        "memory_entries": db.list_memory(focused_id)[:8] if focused_id else [],
     127        "events": events,
     128        "right_events": db.list_events(job_id=focused_id, limit=max(history_limit * 16, 240)) if focused_id else [],
     129        "summary_events": _summary_events(db, focused_id, history_limit=history_limit) if focused_id else [],
     130        "daemon": daemon_lock_status(config.runtime.home / "agentd.lock"),
     131        "model": config.model.model,
     132        "base_url": config.model.base_url,
     133        "context_length": config.model.context_length,
     134        "token_usage": token_usage,
     135        "counts": db.job_record_counts(focused_id) if focused_id else {"steps": 0, "artifacts": 0, "memory": 0, "events": len(events)},
     136    }
     137
     138
     139def _safe_job(db: AgentDB, job_id: str | None) -> dict[str, Any] | None:
     140    if not job_id:
     141        return None
     142    try:
     143        return db.get_job(job_id)
     144    except Exception:
     145        return None
     146
     147
     148def _workspace_token_usage(config: AppConfig) -> dict[str, Any]:
     149    return {
     150        "prompt_tokens": 0,
     151        "completion_tokens": 0,
     152        "total_tokens": 0,
     153        "cost": 0.0,
     154        "calls": 0,
     155        "has_cost": False,
     156        "input_cost_per_million": config.model.input_cost_per_million,
     157        "output_cost_per_million": config.model.output_cost_per_million,
     158    }
     159
     160
     161def _summary_events(db: AgentDB, job_id: str, *, history_limit: int) -> list[dict[str, Any]]:
     162    durable_events = db.list_events(
     163        job_id=job_id,
     164        limit=max(history_limit * 24, 360),
     165        event_types=SUMMARY_EVENT_TYPES,
     166    )
     167    tool_events = [
     168        event
     169        for event in db.list_events(
     170            job_id=job_id,
     171            limit=max(history_limit * 10, 160),
     172            event_types=SUMMARY_TOOL_EVENT_TYPES,
     173        )
     174        if is_summary_event_candidate(event)
     175    ]
     176    merged: dict[str, dict[str, Any]] = {}
     177    for event in [*durable_events, *tool_events]:
     178        event_id = str(event.get("id") or "")
     179        if event_id:
     180            merged[event_id] = event
     181        else:
     182            merged[f"{event.get('created_at')}-{event.get('event_type')}-{event.get('title')}-{len(merged)}"] = event
     183    return sorted(merged.values(), key=lambda event: (str(event.get("created_at") or ""), str(event.get("id") or "")))

nipux_cli/llm.py 288 lines

       1"""LLM provider adapters for one bounded worker step."""
       2
       3from __future__ import annotations
       4
       5import json
       6import urllib.error
       7import urllib.parse
       8import urllib.request
       9from dataclasses import dataclass, field
      10from typing import Any, Protocol
      11
      12from openai import OpenAI
      13
      14from nipux_cli.config import ModelConfig
      15
      16
      17@dataclass(frozen=True)
      18class ToolCall:
      19    name: str
      20    arguments: dict[str, Any] = field(default_factory=dict)
      21    id: str = ""
      22
      23
      24@dataclass(frozen=True)
      25class LLMResponse:
      26    content: str = ""
      27    tool_calls: list[ToolCall] = field(default_factory=list)
      28    usage: dict[str, Any] = field(default_factory=dict)
      29    model: str = ""
      30    response_id: str = ""
      31
      32
      33class LLMResponseError(RuntimeError):
      34    """Raised when a provider returns an OpenAI-shaped response without choices."""
      35
      36    def __init__(self, message: str, *, payload: dict[str, Any] | None = None):
      37        super().__init__(message)
      38        self.payload = payload or {}
      39
      40
      41class StepLLM(Protocol):
      42    def next_action(self, *, messages: list[dict[str, Any]], tools: list[dict[str, Any]]) -> LLMResponse:
      43        ...
      44
      45
      46class OpenAIChatLLM:
      47    """OpenAI-compatible chat-completions adapter."""
      48
      49    tool_repair = True
      50
      51    def __init__(self, config: ModelConfig):
      52        self.config = config
      53        headers = {}
      54        if "openrouter.ai" in config.base_url:
      55            headers = {
      56                "HTTP-Referer": "https://github.com/nipuxx/agent-cli",
      57                "X-Title": "Nipux CLI",
      58            }
      59        self._openai = OpenAI(
      60            api_key=config.api_key or "local-no-key",
      61            base_url=config.base_url,
      62            timeout=config.request_timeout_seconds,
      63            max_retries=0,
      64            default_headers=headers or None,
      65        )
      66
      67    def next_action(self, *, messages: list[dict[str, Any]], tools: list[dict[str, Any]]) -> LLMResponse:
      68        request: dict[str, Any] = {
      69            "model": self.config.model,
      70            "messages": messages,
      71            "tools": tools,
      72        }
      73        if tools:
      74            request["tool_choice"] = "required"
      75        try:
      76            response = self._openai.chat.completions.create(**request)
      77        except Exception as exc:
      78            if "tool_choice" not in request or not _is_unsupported_tool_choice_error(exc):
      79                raise
      80            fallback_request = dict(request)
      81            fallback_request.pop("tool_choice", None)
      82            response = self._openai.chat.completions.create(**fallback_request)
      83        choices = response.choices or []
      84        if not choices:
      85            payload = _response_payload(response)
      86            error = payload.get("error") if isinstance(payload.get("error"), dict) else {}
      87            detail = error.get("message") or payload.get("message") or "provider returned no choices"
      88            raise LLMResponseError(str(detail), payload=payload)
      89        message = choices[0].message
      90        calls: list[ToolCall] = []
      91        for call in message.tool_calls or []:
      92            raw_args = call.function.arguments or "{}"
      93            try:
      94                parsed = json.loads(raw_args)
      95            except json.JSONDecodeError:
      96                parsed = {}
      97            calls.append(ToolCall(name=call.function.name, arguments=parsed, id=call.id or ""))
      98        content = message.content or ""
      99        response_id = _response_id(response)
     100        usage = _response_usage(response, messages=messages, content=content, tool_calls=calls)
     101        usage = _enrich_openrouter_generation_usage(
     102            usage,
     103            response_id=response_id,
     104            base_url=self.config.base_url,
     105            api_key=self.config.api_key,
     106        )
     107        return LLMResponse(
     108            content=content,
     109            tool_calls=calls,
     110            usage=usage,
     111            model=_response_model(response),
     112            response_id=response_id,
     113        )
     114
     115    def complete(self, *, messages: list[dict[str, Any]]) -> str:
     116        return self.complete_response(messages=messages).content
     117
     118    def complete_response(self, *, messages: list[dict[str, Any]]) -> LLMResponse:
     119        response = self._openai.chat.completions.create(
     120            model=self.config.model,
     121            messages=messages,
     122        )
     123        choices = response.choices or []
     124        if not choices:
     125            payload = _response_payload(response)
     126            error = payload.get("error") if isinstance(payload.get("error"), dict) else {}
     127            detail = error.get("message") or payload.get("message") or "provider returned no choices"
     128            raise LLMResponseError(str(detail), payload=payload)
     129        content = choices[0].message.content or ""
     130        response_id = _response_id(response)
     131        usage = _response_usage(response, messages=messages, content=content, tool_calls=[])
     132        usage = _enrich_openrouter_generation_usage(
     133            usage,
     134            response_id=response_id,
     135            base_url=self.config.base_url,
     136            api_key=self.config.api_key,
     137        )
     138        return LLMResponse(
     139            content=content,
     140            usage=usage,
     141            model=_response_model(response),
     142            response_id=response_id,
     143        )
     144
     145
     146class ScriptedLLM:
     147    """Tiny deterministic LLM used by tests and CLI dry runs."""
     148
     149    def __init__(self, responses: list[LLMResponse]):
     150        self.responses = list(responses)
     151
     152    def next_action(self, *, messages: list[dict[str, Any]], tools: list[dict[str, Any]]) -> LLMResponse:
     153        del messages, tools
     154        if not self.responses:
     155            return LLMResponse(content="No scripted response left.")
     156        return self.responses.pop(0)
     157
     158
     159def _response_payload(response: Any) -> dict[str, Any]:
     160    if hasattr(response, "model_dump"):
     161        dumped = response.model_dump()
     162        return dumped if isinstance(dumped, dict) else {"response": dumped}
     163    if hasattr(response, "to_dict"):
     164        dumped = response.to_dict()
     165        return dumped if isinstance(dumped, dict) else {"response": dumped}
     166    return {"response": repr(response)}
     167
     168
     169def _response_usage(
     170    response: Any,
     171    *,
     172    messages: list[dict[str, Any]],
     173    content: str,
     174    tool_calls: list[ToolCall],
     175) -> dict[str, Any]:
     176    payload = _response_payload(response)
     177    usage = payload.get("usage")
     178    if isinstance(usage, dict):
     179        normalized = dict(usage)
     180        normalized["estimated"] = False
     181        return normalized
     182    usage_obj = getattr(response, "usage", None)
     183    if usage_obj is not None:
     184        dumped = usage_obj.model_dump() if hasattr(usage_obj, "model_dump") else getattr(usage_obj, "__dict__", {})
     185        if isinstance(dumped, dict) and dumped:
     186            normalized = dict(dumped)
     187            normalized["estimated"] = False
     188            return normalized
     189    prompt_tokens = _estimate_token_count(json.dumps(messages, ensure_ascii=False, default=str))
     190    tool_text = json.dumps([{"name": call.name, "arguments": call.arguments} for call in tool_calls], ensure_ascii=False, default=str)
     191    completion_tokens = _estimate_token_count(content + tool_text)
     192    return {
     193        "prompt_tokens": prompt_tokens,
     194        "completion_tokens": completion_tokens,
     195        "total_tokens": prompt_tokens + completion_tokens,
     196        "estimated": True,
     197    }
     198
     199
     200def _enrich_openrouter_generation_usage(
     201    usage: dict[str, Any],
     202    *,
     203    response_id: str,
     204    base_url: str,
     205    api_key: str,
     206) -> dict[str, Any]:
     207    if usage.get("cost") is not None or not response_id or not api_key:
     208        return usage
     209    if "openrouter.ai" not in base_url:
     210        return usage
     211    parsed = urllib.parse.urlparse(base_url)
     212    root = f"{parsed.scheme or 'https'}://{parsed.netloc or 'openrouter.ai'}"
     213    url = f"{root}/api/v1/generation?id={urllib.parse.quote(response_id)}"
     214    request = urllib.request.Request(url, headers={"Authorization": f"Bearer {api_key}"})
     215    try:
     216        with urllib.request.urlopen(request, timeout=5) as response:
     217            payload = json.loads(response.read().decode("utf-8", errors="replace"))
     218    except (OSError, urllib.error.URLError, urllib.error.HTTPError, json.JSONDecodeError):
     219        return usage
     220    data = payload.get("data") if isinstance(payload, dict) else None
     221    if not isinstance(data, dict):
     222        return usage
     223    enriched = dict(usage)
     224    cost = _safe_float(data.get("total_cost") or data.get("cost"))
     225    if cost is not None:
     226        enriched["cost"] = cost
     227    prompt = _safe_int(data.get("native_tokens_prompt") or data.get("tokens_prompt"))
     228    completion = _safe_int(data.get("native_tokens_completion") or data.get("tokens_completion"))
     229    total = _safe_int(data.get("native_tokens_total") or data.get("tokens_total"))
     230    if prompt is not None:
     231        enriched["prompt_tokens"] = prompt
     232    if completion is not None:
     233        enriched["completion_tokens"] = completion
     234    if total is not None:
     235        enriched["total_tokens"] = total
     236    elif prompt is not None or completion is not None:
     237        enriched["total_tokens"] = int(enriched.get("prompt_tokens") or 0) + int(enriched.get("completion_tokens") or 0)
     238    enriched["estimated"] = bool(enriched.get("estimated")) and cost is None
     239    return enriched
     240
     241
     242def _safe_float(value: Any) -> float | None:
     243    try:
     244        if value in (None, ""):
     245            return None
     246        return float(value)
     247    except (TypeError, ValueError):
     248        return None
     249
     250
     251def _safe_int(value: Any) -> int | None:
     252    try:
     253        if value in (None, ""):
     254            return None
     255        return int(float(value))
     256    except (TypeError, ValueError):
     257        return None
     258
     259
     260def _is_unsupported_tool_choice_error(exc: Exception) -> bool:
     261    text = f"{type(exc).__name__} {exc}".lower()
     262    return "tool_choice" in text and any(
     263        marker in text
     264        for marker in (
     265            "unsupported",
     266            "unknown parameter",
     267            "unrecognized",
     268            "not supported",
     269            "invalid_request",
     270            "extra inputs are not permitted",
     271        )
     272    )
     273
     274
     275def _estimate_token_count(text: str) -> int:
     276    if not text:
     277        return 0
     278    return max(1, (len(text) + 3) // 4)
     279
     280
     281def _response_model(response: Any) -> str:
     282    payload = _response_payload(response)
     283    return str(payload.get("model") or getattr(response, "model", "") or "")
     284
     285
     286def _response_id(response: Any) -> str:
     287    payload = _response_payload(response)
     288    return str(payload.get("id") or getattr(response, "id", "") or "")

nipux_cli/measurement.py 141 lines

       1"""Measurement parsing helpers for generic progress accounting."""
       2
       3from __future__ import annotations
       4
       5import re
       6from typing import Any
       7
       8
       9MEASUREMENT_PATTERN = re.compile(
      10    r"(?i)(?:"
      11    r"\b\d+(?:\.\d+)?\s*(?:%|ms|s|sec|secs|seconds|msec|us|hz|khz|mhz|ghz|kb/s|mb/s|gb/s|tb/s|"
      12    r"it/s|ops/s|req/s|qps|rps|samples/s|items/s|units/s|tokens/s|tok/s|t/s)\b"
      13    r"|(?:score|rate|speed|throughput|latency|accuracy|loss|error|duration|runtime|time|memory|cpu|gpu|ram)\D{0,40}\d+(?:\.\d+)?"
      14    r")"
      15)
      16MEASUREMENT_INTENT_PATTERN = re.compile(
      17    r"(?i)\b(bench(?:mark)?|compare|duration|eval(?:uate)?|experiment|hyperfine|latency|measure|metric|perf|"
      18    r"profile|rate|runtime|speed|test|throughput|time|trial)\b"
      19)
      20DIAGNOSTIC_MEASUREMENT_PATTERN = re.compile(r"(?i)^\s*(?:cpu|gpu|memory|mem|ram)\b")
      21ACTION_MEASUREMENT_PATTERN = re.compile(
      22    r"(?i)^\s*(?:score|rate|speed|throughput|latency|accuracy|loss|error|duration|runtime|time)\b"
      23)
      24LABELED_MEASUREMENT_PATTERN = re.compile(
      25    r"(?i)^\s*(?:score|rate|speed|throughput|latency|accuracy|loss|error|duration|runtime|time)\s*(?:=|:)\s*[-+]?\d"
      26)
      27EXPLICIT_RESULT_UNIT_PATTERN = re.compile(
      28    r"(?i)\b\d+(?:\.\d+)?\s*(?:%|ms|msec|sec|secs|seconds|it/s|ops/s|req/s|qps|rps|samples/s|items/s|units/s|"
      29    r"tokens/s|tok/s|t/s|kb/s|mb/s|gb/s|tb/s)\b"
      30)
      31TABLE_UNIT_PATTERN = re.compile(
      32    r"(?i)^(?:%|ms|msec|sec|secs|seconds|it/s|ops/s|req/s|qps|rps|samples/s|items/s|units/s|"
      33    r"tokens/s|tok/s|t/s|kb/s|mb/s|gb/s|tb/s)$"
      34)
      35TABLE_NUMBER_PATTERN = re.compile(r"[-+]?\d+(?:\.\d+)?(?:\s*(?:±|\+/-)\s*[-+]?\d+(?:\.\d+)?)?")
      36
      37
      38def measurement_candidates(output: dict[str, Any], *, command: str = "", limit: int = 8) -> list[str]:
      39    text = "\n".join(
      40        str(output.get(key) or "")
      41        for key in ("stdout", "stderr", "result", "content")
      42        if output.get(key) is not None
      43    )
      44    if not text.strip():
      45        return []
      46    command_has_measurement_intent = bool(MEASUREMENT_INTENT_PATTERN.search(command))
      47    candidates: list[str] = []
      48    for candidate in _table_measurement_candidates(text, limit=limit):
      49        if candidate not in candidates:
      50            candidates.append(candidate)
      51        if len(candidates) >= limit:
      52            return candidates
      53    for match in MEASUREMENT_PATTERN.finditer(text[:20000]):
      54        candidate = " ".join(match.group(0).split())
      55        if not EXPLICIT_RESULT_UNIT_PATTERN.search(candidate):
      56            expanded = " ".join(text[match.start() : min(len(text), match.end() + 32)].split())
      57            if EXPLICIT_RESULT_UNIT_PATTERN.search(expanded):
      58                candidate = expanded
      59        if _candidate_is_diagnostic_only(candidate, command_has_measurement_intent):
      60            continue
      61        if candidate not in candidates:
      62            candidates.append(candidate[:140])
      63        if len(candidates) >= limit:
      64            break
      65    return candidates
      66
      67
      68def _table_measurement_candidates(text: str, *, limit: int = 8) -> list[str]:
      69    candidates: list[str] = []
      70    table_lines = [line.strip() for line in str(text or "").splitlines() if line.strip().startswith("|") and "|" in line.strip()[1:]]
      71    for index, line in enumerate(table_lines):
      72        headers = _split_markdown_table_row(line)
      73        if not headers or _is_markdown_separator_row(headers):
      74            continue
      75        unit_indexes = [idx for idx, header in enumerate(headers) if TABLE_UNIT_PATTERN.search(header.strip())]
      76        if not unit_indexes:
      77            continue
      78        for row_line in table_lines[index + 1 : index + 16]:
      79            cells = _split_markdown_table_row(row_line)
      80            if not cells or _is_markdown_separator_row(cells):
      81                continue
      82            for unit_index in unit_indexes:
      83                if unit_index >= len(cells):
      84                    continue
      85                value = cells[unit_index].strip()
      86                number = TABLE_NUMBER_PATTERN.search(value)
      87                if not number:
      88                    continue
      89                unit = headers[unit_index].strip()
      90                label = _table_measurement_label(headers, cells, unit_index=unit_index)
      91                candidate = f"{label} {number.group(0).strip()} {unit}".strip()
      92                if candidate not in candidates:
      93                    candidates.append(candidate[:140])
      94                if len(candidates) >= limit:
      95                    return candidates
      96    return candidates
      97
      98
      99def _split_markdown_table_row(line: str) -> list[str]:
     100    raw = str(line or "").strip()
     101    if raw.startswith("|"):
     102        raw = raw[1:]
     103    if raw.endswith("|"):
     104        raw = raw[:-1]
     105    return [" ".join(cell.strip().split()) for cell in raw.split("|")]
     106
     107
     108def _is_markdown_separator_row(cells: list[str]) -> bool:
     109    return bool(cells) and all(re.fullmatch(r":?-{2,}:?", cell.strip()) for cell in cells if cell.strip())
     110
     111
     112def _table_measurement_label(headers: list[str], cells: list[str], *, unit_index: int) -> str:
     113    preferred_headers = {"test", "metric", "name", "case", "benchmark"}
     114    for index, header in enumerate(headers):
     115        if index >= len(cells) or index == unit_index:
     116            continue
     117        if header.strip().lower() in preferred_headers and cells[index].strip():
     118            return cells[index].strip()
     119    for index in range(min(unit_index, len(cells)) - 1, -1, -1):
     120        cell = cells[index].strip()
     121        if cell and not TABLE_NUMBER_PATTERN.fullmatch(cell):
     122            return cell
     123    return "measurement"
     124
     125
     126def measurement_candidates_are_diagnostic_only(candidates: list[Any], *, command: str = "") -> bool:
     127    command_has_measurement_intent = bool(MEASUREMENT_INTENT_PATTERN.search(command))
     128    return all(_candidate_is_diagnostic_only(str(candidate), command_has_measurement_intent) for candidate in candidates)
     129
     130
     131def _candidate_is_diagnostic_only(candidate: str, command_has_measurement_intent: bool) -> bool:
     132    has_structured_metric = bool(EXPLICIT_RESULT_UNIT_PATTERN.search(candidate) or LABELED_MEASUREMENT_PATTERN.search(candidate))
     133    if command_has_measurement_intent:
     134        return not has_structured_metric
     135    if DIAGNOSTIC_MEASUREMENT_PATTERN.search(candidate):
     136        return True
     137    if EXPLICIT_RESULT_UNIT_PATTERN.search(candidate) and not re.search(r"(?i)\b(?:cpu|gpu|ram|mem|memory)\b", candidate):
     138        return False
     139    if ACTION_MEASUREMENT_PATTERN.search(candidate):
     140        return not bool(LABELED_MEASUREMENT_PATTERN.search(candidate))
     141    return True

nipux_cli/memory_graph.py 302 lines

       1"""Job-local memory graph helpers for long-running workers."""
       2
       3from __future__ import annotations
       4
       5import re
       6from datetime import datetime
       7from typing import Any
       8
       9from nipux_cli.worker_prompt_format import clip_text
      10
      11
      12NODE_KINDS = {
      13    "artifact",
      14    "constraint",
      15    "decision",
      16    "episode",
      17    "experiment",
      18    "fact",
      19    "milestone",
      20    "question",
      21    "skill",
      22    "source",
      23    "strategy",
      24    "task",
      25}
      26NODE_STATUSES = {"active", "blocked", "deprecated", "open", "resolved", "stable"}
      27DEFAULT_NODE_KIND = "fact"
      28DEFAULT_NODE_STATUS = "active"
      29NEGATIVE_MEMORY_MARKERS = (
      30    "0 files",
      31    "0 results",
      32    "blocked until",
      33    "cannot access",
      34    "critical blocker",
      35    "does not exist",
      36    "failed to find",
      37    "missing",
      38    "must be downloaded",
      39    "no ",
      40    "no such",
      41    "none",
      42    "not available",
      43    "not detected",
      44    "not downloaded",
      45    "not found",
      46    "not installed",
      47    "prevents",
      48    "unavailable",
      49    "without",
      50)
      51
      52
      53def memory_graph_from_job(job: dict[str, Any]) -> dict[str, Any]:
      54    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
      55    graph = metadata.get("memory_graph") if isinstance(metadata.get("memory_graph"), dict) else {}
      56    nodes = graph.get("nodes") if isinstance(graph.get("nodes"), list) else []
      57    edges = graph.get("edges") if isinstance(graph.get("edges"), list) else []
      58    return {
      59        "nodes": [node for node in nodes if isinstance(node, dict)],
      60        "edges": [edge for edge in edges if isinstance(edge, dict)],
      61        "updated_at": graph.get("updated_at") or "",
      62    }
      63
      64
      65def memory_graph_for_prompt(job: dict[str, Any], *, limit: int = 10, stale_tokens: list[str] | None = None) -> str:
      66    graph = memory_graph_from_job(job)
      67    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
      68    stale_tokens = [str(token) for token in stale_tokens or [] if str(token).strip()]
      69    stale_node_ids = {
      70        str(record.get("record_id") or "")
      71        for record in metadata.get("stale_negative_records", [])
      72        if isinstance(record, dict) and str(record.get("kind") or "") == "memory_node"
      73    } if isinstance(metadata.get("stale_negative_records"), list) else set()
      74    stale_nodes = [
      75        node
      76        for node in graph["nodes"]
      77        if _node_contains_stale_token(node, stale_tokens) or _node_has_stale_id(node, stale_node_ids)
      78    ]
      79    active_nodes = [node for node in graph["nodes"] if node not in stale_nodes]
      80    nodes = rank_memory_nodes(active_nodes, limit=limit)
      81    durable_count = _durable_signal_count(job)
      82    if not nodes:
      83        hint = (
      84            "No memory graph yet. When a branch produces reusable knowledge, use record_memory_graph "
      85            "to save connected episode, fact, strategy, skill, question, decision, or constraint nodes."
      86        )
      87        if durable_count:
      88            hint += (
      89                f" Durable ledgers already contain {durable_count} reusable item(s); consolidate the most important "
      90                "ones into graph nodes before raw history grows further."
      91            )
      92        return hint
      93    edge_index = _edge_index(graph["edges"])
      94    lines = [
      95        f"Memory graph: nodes={len(graph['nodes'])} edges={len(graph['edges'])}",
      96    ]
      97    if stale_nodes:
      98        lines.append(
      99            f"Suppressed {len(stale_nodes)} stale memory node(s) matching unsupported tokens; "
     100            "do not use them as facts until observed again."
     101        )
     102    if durable_count >= 8 and len(graph["nodes"]) < max(4, durable_count // 4):
     103        lines.append(
     104            "Consolidation pressure: durable ledgers are growing faster than the memory graph. "
     105            "After the next meaningful checkpoint, use record_memory_graph to add or update connected nodes."
     106        )
     107    for node in nodes:
     108        key = str(node.get("key") or "")
     109        title = str(node.get("title") or key or "memory")
     110        kind = str(node.get("kind") or DEFAULT_NODE_KIND)
     111        status = str(node.get("status") or DEFAULT_NODE_STATUS)
     112        summary = str(node.get("summary") or "")
     113        tags = _clean_string_list(node.get("tags"))[:5]
     114        refs = _clean_string_list(node.get("evidence_refs"))[:4]
     115        parent = str(node.get("parent_key") or "")
     116        line = f"- {status} {kind}: {title}"
     117        if parent:
     118            line += f" | parent={parent}"
     119        if tags:
     120            line += f" | tags={', '.join(tags)}"
     121        if refs:
     122            line += f" | evidence={', '.join(refs)}"
     123        if summary:
     124            line += f" | {summary}"
     125        lines.append(clip_text(line, 620))
     126        related = edge_index.get(key, [])[:3]
     127        if related:
     128            lines.append("  links: " + clip_text("; ".join(related), 420))
     129    return "\n".join(lines)
     130
     131
     132def _node_contains_stale_token(node: dict[str, Any], stale_tokens: list[str]) -> bool:
     133    if not stale_tokens:
     134        return False
     135    text_parts = [
     136        node.get("key"),
     137        node.get("title"),
     138        node.get("kind"),
     139        node.get("status"),
     140        node.get("summary"),
     141        " ".join(_clean_string_list(node.get("tags"))),
     142    ]
     143    text = " ".join(str(part or "") for part in text_parts)
     144    text_lower = text.lower()
     145    negative_node = _node_has_negative_memory_marker(text_lower)
     146    for token in stale_tokens:
     147        pattern = r"(?<![A-Za-z0-9])" + re.escape(token) + r"(?![A-Za-z0-9])"
     148        if re.search(pattern, text, flags=re.IGNORECASE):
     149            return True
     150        if negative_node and token.startswith("."):
     151            bare = token[1:].strip()
     152            if bare and re.search(r"(?<![A-Za-z0-9])" + re.escape(bare) + r"(?![A-Za-z0-9])", text_lower):
     153                return True
     154    return False
     155
     156
     157def _node_has_negative_memory_marker(text_lower: str) -> bool:
     158    return any(marker in text_lower for marker in NEGATIVE_MEMORY_MARKERS)
     159
     160
     161def _node_has_stale_id(node: dict[str, Any], stale_node_ids: set[str]) -> bool:
     162    if not stale_node_ids:
     163        return False
     164    for key in ("key", "event_id", "id"):
     165        value = str(node.get(key) or "").strip()
     166        if value and value in stale_node_ids:
     167            return True
     168    return False
     169
     170
     171def search_memory_graph(graph: dict[str, Any], query: str, *, limit: int = 10) -> dict[str, Any]:
     172    nodes = graph.get("nodes") if isinstance(graph.get("nodes"), list) else []
     173    edges = graph.get("edges") if isinstance(graph.get("edges"), list) else []
     174    ranked = rank_memory_nodes([node for node in nodes if isinstance(node, dict)], query=query, limit=limit)
     175    keys = {str(node.get("key") or "") for node in ranked}
     176    related_edges = [
     177        edge
     178        for edge in edges
     179        if isinstance(edge, dict)
     180        and (str(edge.get("from_key") or "") in keys or str(edge.get("to_key") or "") in keys)
     181    ][: max(limit * 2, 10)]
     182    return {"nodes": ranked, "edges": related_edges}
     183
     184
     185def rank_memory_nodes(nodes: list[dict[str, Any]], *, query: str = "", limit: int = 10) -> list[dict[str, Any]]:
     186    tokens = _tokens(query)
     187    ranked = sorted(nodes, key=lambda node: _node_score(node, tokens), reverse=True)
     188    if tokens:
     189        ranked = [node for node in ranked if _node_score(node, tokens) > 0]
     190    return ranked[: max(0, limit)]
     191
     192
     193def _node_score(node: dict[str, Any], query_tokens: set[str]) -> float:
     194    haystack = " ".join(
     195        str(value or "")
     196        for value in [
     197            node.get("key"),
     198            node.get("title"),
     199            node.get("kind"),
     200            node.get("status"),
     201            node.get("summary"),
     202            " ".join(_clean_string_list(node.get("tags"))),
     203        ]
     204    ).lower()
     205    score = 0.0
     206    for token in query_tokens:
     207        if token in haystack:
     208            score += 4.0 if token in str(node.get("title") or "").lower() else 2.0
     209    score += _float_between(node.get("salience"), 0.0, 1.0) * 3.0
     210    score += _float_between(node.get("confidence"), 0.0, 1.0)
     211    status = str(node.get("status") or DEFAULT_NODE_STATUS)
     212    if status in {"active", "open"}:
     213        score += 1.2
     214    elif status == "stable":
     215        score += 0.7
     216    elif status == "deprecated":
     217        score -= 1.5
     218    kind = str(node.get("kind") or DEFAULT_NODE_KIND)
     219    if kind in {"strategy", "skill", "decision", "constraint", "question"}:
     220        score += 0.5
     221    score += min(int(node.get("use_count") or 0), 8) * 0.08
     222    score += _recency_score(str(node.get("updated_at") or node.get("created_at") or ""))
     223    return score
     224
     225
     226def _edge_index(edges: list[dict[str, Any]]) -> dict[str, list[str]]:
     227    index: dict[str, list[str]] = {}
     228    for edge in edges:
     229        from_key = str(edge.get("from_key") or "")
     230        to_key = str(edge.get("to_key") or "")
     231        if not from_key or not to_key:
     232            continue
     233        relation = str(edge.get("relation") or "related_to")
     234        index.setdefault(from_key, []).append(f"{relation} -> {to_key}")
     235        index.setdefault(to_key, []).append(f"{relation} <- {from_key}")
     236    return index
     237
     238
     239def _tokens(value: str) -> set[str]:
     240    return {token for token in re.findall(r"[a-z0-9][a-z0-9_-]{1,}", value.lower()) if token not in _STOPWORDS}
     241
     242
     243def _float_between(value: Any, low: float, high: float) -> float:
     244    try:
     245        number = float(value)
     246    except (TypeError, ValueError):
     247        return low
     248    return min(high, max(low, number))
     249
     250
     251def _recency_score(value: str) -> float:
     252    if not value:
     253        return 0.0
     254    try:
     255        parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
     256    except ValueError:
     257        return 0.0
     258    age_seconds = max(0.0, (datetime.now(parsed.tzinfo) - parsed).total_seconds())
     259    if age_seconds < 3600:
     260        return 0.8
     261    if age_seconds < 86_400:
     262        return 0.4
     263    if age_seconds < 604_800:
     264        return 0.15
     265    return 0.0
     266
     267
     268def _clean_string_list(value: Any) -> list[str]:
     269    if not isinstance(value, list):
     270        return []
     271    return [" ".join(str(item).split()) for item in value if str(item).strip()]
     272
     273
     274def _durable_signal_count(job: dict[str, Any]) -> int:
     275    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     276    count = 0
     277    for key in (
     278        "experiment_ledger",
     279        "finding_ledger",
     280        "lessons",
     281        "source_ledger",
     282        "task_queue",
     283    ):
     284        values = metadata.get(key)
     285        if isinstance(values, list):
     286            count += sum(1 for value in values if isinstance(value, dict))
     287    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
     288    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
     289    count += sum(1 for milestone in milestones if isinstance(milestone, dict))
     290    return count
     291
     292
     293_STOPWORDS = {
     294    "and",
     295    "for",
     296    "from",
     297    "into",
     298    "that",
     299    "the",
     300    "this",
     301    "with",
     302}

nipux_cli/memory_graph_view.py 299 lines

       1"""Self-contained HTML view for a job-local memory graph."""
       2
       3from __future__ import annotations
       4
       5import json
       6from html import escape
       7from typing import Any
       8
       9from nipux_cli.memory_graph import memory_graph_from_job
      10
      11
      12def render_memory_graph_html(job: dict[str, Any]) -> str:
      13    """Return a standalone clickable graph page for a job's memory graph."""
      14
      15    graph = memory_graph_from_job(job)
      16    nodes = [_view_node(node) for node in graph["nodes"]]
      17    edges = [_view_edge(edge) for edge in graph["edges"]]
      18    data = json.dumps(
      19        {
      20            "job": {
      21                "id": str(job.get("id") or ""),
      22                "title": str(job.get("title") or ""),
      23                "objective": str(job.get("objective") or ""),
      24            },
      25            "updated_at": graph.get("updated_at") or "",
      26            "nodes": nodes,
      27            "edges": edges,
      28        },
      29        ensure_ascii=False,
      30    ).replace("</", "<\\/")
      31    title = escape(str(job.get("title") or "Nipux memory graph"))
      32    return f"""<!doctype html>
      33<html lang="en">
      34<head>
      35<meta charset="utf-8">
      36<meta name="viewport" content="width=device-width, initial-scale=1">
      37<title>Nipux Memory Graph - {title}</title>
      38<style>
      39:root {{
      40  color-scheme: dark;
      41  --bg: #101112;
      42  --panel: #17191b;
      43  --line: #303336;
      44  --text: #eeeeea;
      45  --muted: #9a9a94;
      46  --accent: #82e6e1;
      47  --gold: #e6d06f;
      48  --purple: #c99ce8;
      49}}
      50* {{ box-sizing: border-box; }}
      51html, body {{ margin: 0; min-height: 100%; background: var(--bg); color: var(--text); font: 14px/1.45 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; }}
      52body {{ overflow: hidden; }}
      53.shell {{ display: grid; grid-template-columns: minmax(0, 1fr) 360px; height: 100vh; }}
      54.stage {{ position: relative; min-width: 0; border-right: 1px solid var(--line); }}
      55.top {{ position: absolute; top: 22px; left: 26px; right: 26px; z-index: 2; display: flex; align-items: start; justify-content: space-between; gap: 24px; }}
      56.eyebrow {{ color: var(--muted); letter-spacing: .22em; text-transform: uppercase; font-size: 12px; }}
      57h1 {{ margin: 8px 0 0; font-size: clamp(30px, 4vw, 64px); line-height: .95; letter-spacing: -.04em; }}
      58.stats {{ display: flex; gap: 18px; color: var(--muted); white-space: nowrap; }}
      59.stats b {{ color: var(--text); font-size: 20px; }}
      60canvas {{ display: block; width: 100%; height: 100%; }}
      61.help {{ position: absolute; left: 26px; bottom: 22px; color: var(--muted); z-index: 2; }}
      62aside {{ min-width: 0; background: var(--panel); padding: 26px; overflow: auto; }}
      63.card {{ border: 1px solid var(--line); border-radius: 18px; padding: 18px; margin-top: 18px; background: rgba(255,255,255,.018); }}
      64.label {{ color: var(--muted); font-size: 12px; letter-spacing: .18em; text-transform: uppercase; }}
      65.node-title {{ margin-top: 10px; font-size: 22px; line-height: 1.12; }}
      66.row {{ display: grid; grid-template-columns: 88px minmax(0, 1fr); gap: 12px; margin-top: 12px; }}
      67.row span:first-child {{ color: var(--muted); }}
      68.pill {{ display: inline-block; margin: 5px 6px 0 0; padding: 3px 8px; border: 1px solid var(--line); border-radius: 999px; color: var(--accent); }}
      69.list {{ margin: 8px 0 0; padding-left: 18px; color: var(--muted); }}
      70.empty {{ color: var(--muted); margin-top: 12px; }}
      71.search {{ width: 100%; margin-top: 18px; padding: 12px 14px; border-radius: 12px; border: 1px solid var(--line); background: #0d0e0f; color: var(--text); font: inherit; outline: none; }}
      72.search:focus {{ border-color: var(--accent); }}
      73@media (max-width: 900px) {{
      74  .shell {{ grid-template-columns: 1fr; grid-template-rows: 62vh 38vh; }}
      75  .stage {{ border-right: 0; border-bottom: 1px solid var(--line); }}
      76}}
      77</style>
      78</head>
      79<body>
      80<main class="shell">
      81  <section class="stage">
      82    <div class="top">
      83      <div>
      84        <div class="eyebrow">Nipux memory graph</div>
      85        <h1>{title}</h1>
      86      </div>
      87      <div class="stats">
      88        <div><b id="node-count">0</b><br>nodes</div>
      89        <div><b id="edge-count">0</b><br>links</div>
      90      </div>
      91    </div>
      92    <canvas id="graph"></canvas>
      93    <div class="help">drag to rotate · scroll to zoom · click a node</div>
      94  </section>
      95  <aside>
      96    <div class="label">inspect</div>
      97    <input id="search" class="search" placeholder="search nodes">
      98    <div id="details" class="card">
      99      <div class="label">selected node</div>
     100      <div class="empty">Click a node to inspect its summary, evidence, and links.</div>
     101    </div>
     102    <div id="results" class="card">
     103      <div class="label">visible nodes</div>
     104      <div class="empty">No nodes yet. The worker can create graph memory with record_memory_graph.</div>
     105    </div>
     106  </aside>
     107</main>
     108<script id="graph-data" type="application/json">{data}</script>
     109<script>
     110const data = JSON.parse(document.getElementById("graph-data").textContent);
     111const canvas = document.getElementById("graph");
     112const ctx = canvas.getContext("2d");
     113const details = document.getElementById("details");
     114const results = document.getElementById("results");
     115const search = document.getElementById("search");
     116document.getElementById("node-count").textContent = data.nodes.length;
     117document.getElementById("edge-count").textContent = data.edges.length;
     118
     119let width = 0, height = 0, zoom = 1, rotX = -0.35, rotY = 0.65, dragging = false, last = [0, 0], selected = null;
     120let lastResultsSignature = "";
     121const nodeByKey = new Map(data.nodes.map((node, index) => [node.key, {{ ...node, index }}]));
     122const nodes = data.nodes.map((node, index) => {{
     123  const a = index * 2.399963229728653;
     124  const r = 110 + (index % 7) * 24;
     125  const z = ((index * 53) % 240) - 120;
     126  return {{ ...node, x: Math.cos(a) * r, y: Math.sin(a) * r, z, vx: 0, vy: 0, vz: 0, screen: [0, 0], visible: true }};
     127}});
     128const nodeLookup = new Map(nodes.map(node => [node.key, node]));
     129const edges = data.edges.map(edge => ({{ ...edge, from: nodeLookup.get(edge.from_key), to: nodeLookup.get(edge.to_key) }})).filter(edge => edge.from && edge.to);
     130
     131function resize() {{
     132  const ratio = window.devicePixelRatio || 1;
     133  width = canvas.clientWidth;
     134  height = canvas.clientHeight;
     135  canvas.width = Math.max(1, width * ratio);
     136  canvas.height = Math.max(1, height * ratio);
     137  ctx.setTransform(ratio, 0, 0, ratio, 0, 0);
     138}}
     139window.addEventListener("resize", resize);
     140resize();
     141
     142function project(node) {{
     143  const cy = Math.cos(rotY), sy = Math.sin(rotY), cx = Math.cos(rotX), sx = Math.sin(rotX);
     144  let x = node.x * cy - node.z * sy;
     145  let z = node.x * sy + node.z * cy;
     146  let y = node.y * cx - z * sx;
     147  z = node.y * sx + z * cx;
     148  const scale = zoom * 520 / (520 + z);
     149  return [width / 2 + x * scale, height / 2 + y * scale, scale, z];
     150}}
     151
     152function color(node) {{
     153  if (node.status === "deprecated") return "#7d7d77";
     154  if (node.kind === "question") return "#e6d06f";
     155  if (node.kind === "skill" || node.kind === "strategy") return "#82e6e1";
     156  if (node.kind === "decision" || node.kind === "constraint") return "#c99ce8";
     157  return "#eeeeea";
     158}}
     159
     160function draw() {{
     161  ctx.clearRect(0, 0, width, height);
     162  ctx.fillStyle = "#101112";
     163  ctx.fillRect(0, 0, width, height);
     164  const q = search.value.trim().toLowerCase();
     165  for (const node of nodes) {{
     166    const hay = [node.key, node.title, node.kind, node.status, node.summary, ...(node.tags || [])].join(" ").toLowerCase();
     167    node.visible = !q || hay.includes(q);
     168  }}
     169  for (const edge of edges) {{
     170    if (!edge.from.visible || !edge.to.visible) continue;
     171    const a = project(edge.from), b = project(edge.to);
     172    ctx.strokeStyle = "rgba(154,154,148,.26)";
     173    ctx.lineWidth = 1;
     174    ctx.beginPath();
     175    ctx.moveTo(a[0], a[1]);
     176    ctx.lineTo(b[0], b[1]);
     177    ctx.stroke();
     178  }}
     179  const sorted = [...nodes].map(node => [node, project(node)]).sort((a, b) => a[1][3] - b[1][3]);
     180  for (const [node, p] of sorted) {{
     181    node.screen = p;
     182    if (!node.visible) continue;
     183    const radius = Math.max(5, 9 * p[2]);
     184    ctx.fillStyle = color(node);
     185    ctx.globalAlpha = selected && selected.key !== node.key ? .55 : 1;
     186    ctx.beginPath();
     187    ctx.arc(p[0], p[1], radius, 0, Math.PI * 2);
     188    ctx.fill();
     189    if (selected && selected.key === node.key) {{
     190      ctx.strokeStyle = "#ffffff";
     191      ctx.lineWidth = 2;
     192      ctx.stroke();
     193    }}
     194  }}
     195  ctx.globalAlpha = 1;
     196  renderResults(q);
     197  requestAnimationFrame(draw);
     198}}
     199
     200function renderResults(query) {{
     201  const visible = nodes.filter(node => node.visible).slice(0, 18);
     202  const signature = query + "|" + visible.map(node => node.key).join(",");
     203  if (signature === lastResultsSignature) return;
     204  lastResultsSignature = signature;
     205  results.innerHTML = '<div class="label">visible nodes</div>' + (
     206    visible.length
     207      ? visible.map(node => `<div class="row"><span>${{escapeHtml(node.kind)}}</span><a href="#" data-key="${{escapeHtml(node.key)}}">${{escapeHtml(node.title || node.key)}}</a></div>`).join("")
     208      : '<div class="empty">No nodes match the current search.</div>'
     209  );
     210  for (const link of results.querySelectorAll("a[data-key]")) {{
     211    link.addEventListener("click", event => {{
     212      event.preventDefault();
     213      selectNode(nodeLookup.get(link.dataset.key));
     214    }});
     215  }}
     216}}
     217
     218function selectNode(node) {{
     219  selected = node;
     220  if (!node) return;
     221  const linked = edges.filter(edge => edge.from.key === node.key || edge.to.key === node.key).slice(0, 12);
     222  details.innerHTML = `
     223    <div class="label">${{escapeHtml(node.kind)}} · ${{escapeHtml(node.status)}}</div>
     224    <div class="node-title">${{escapeHtml(node.title || node.key)}}</div>
     225    <div class="row"><span>key</span><div>${{escapeHtml(node.key)}}</div></div>
     226    <div class="row"><span>summary</span><div>${{escapeHtml(node.summary || "No summary recorded.")}}</div></div>
     227    <div class="row"><span>score</span><div>salience ${{node.salience ?? "n/a"}} · confidence ${{node.confidence ?? "n/a"}}</div></div>
     228    <div class="row"><span>tags</span><div>${{(node.tags || []).map(tag => `<span class="pill">${{escapeHtml(tag)}}</span>`).join("") || "none"}}</div></div>
     229    <div class="row"><span>evidence</span><ul class="list">${{(node.evidence_refs || []).map(ref => `<li>${{escapeHtml(ref)}}</li>`).join("") || "<li>none</li>"}}</ul></div>
     230    <div class="row"><span>links</span><ul class="list">${{linked.map(edge => `<li>${{escapeHtml(edge.from.key === node.key ? edge.relation + " → " + edge.to.key : edge.relation + " ← " + edge.from.key)}}</li>`).join("") || "<li>none</li>"}}</ul></div>
     231  `;
     232}}
     233
     234function escapeHtml(value) {{
     235  return String(value ?? "").replace(/[&<>"']/g, char => ({{ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;" }}[char]));
     236}}
     237
     238canvas.addEventListener("mousedown", event => {{ dragging = true; last = [event.clientX, event.clientY]; }});
     239window.addEventListener("mouseup", () => dragging = false);
     240window.addEventListener("mousemove", event => {{
     241  if (!dragging) return;
     242  rotY += (event.clientX - last[0]) * 0.006;
     243  rotX += (event.clientY - last[1]) * 0.006;
     244  last = [event.clientX, event.clientY];
     245}});
     246canvas.addEventListener("wheel", event => {{
     247  event.preventDefault();
     248  zoom = Math.max(.35, Math.min(3.2, zoom * (event.deltaY > 0 ? .92 : 1.08)));
     249}}, {{ passive: false }});
     250canvas.addEventListener("click", event => {{
     251  const rect = canvas.getBoundingClientRect();
     252  const x = event.clientX - rect.left, y = event.clientY - rect.top;
     253  let best = null, bestDistance = 18;
     254  for (const node of nodes) {{
     255    if (!node.visible) continue;
     256    const dx = node.screen[0] - x, dy = node.screen[1] - y;
     257    const distance = Math.hypot(dx, dy);
     258    if (distance < bestDistance) {{ best = node; bestDistance = distance; }}
     259  }}
     260  if (best) selectNode(best);
     261}});
     262search.addEventListener("input", () => {{}});
     263if (nodes[0]) selectNode(nodes[0]);
     264draw();
     265</script>
     266</body>
     267</html>
     268"""
     269
     270
     271def _view_node(node: dict[str, Any]) -> dict[str, Any]:
     272    return {
     273        "key": str(node.get("key") or node.get("title") or "memory"),
     274        "title": str(node.get("title") or node.get("key") or "memory"),
     275        "kind": str(node.get("kind") or "fact"),
     276        "status": str(node.get("status") or "active"),
     277        "summary": str(node.get("summary") or ""),
     278        "salience": node.get("salience"),
     279        "confidence": node.get("confidence"),
     280        "tags": _string_list(node.get("tags")),
     281        "evidence_refs": _string_list(node.get("evidence_refs")),
     282        "created_at": str(node.get("created_at") or ""),
     283        "updated_at": str(node.get("updated_at") or ""),
     284    }
     285
     286
     287def _view_edge(edge: dict[str, Any]) -> dict[str, Any]:
     288    return {
     289        "from_key": str(edge.get("from_key") or ""),
     290        "to_key": str(edge.get("to_key") or ""),
     291        "relation": str(edge.get("relation") or "related_to"),
     292        "summary": str(edge.get("summary") or ""),
     293    }
     294
     295
     296def _string_list(value: Any) -> list[str]:
     297    if not isinstance(value, list):
     298        return []
     299    return [" ".join(str(item).split()) for item in value if str(item).strip()]

nipux_cli/metric_format.py 17 lines

       1"""Small formatting helpers for measured worker results."""
       2
       3from __future__ import annotations
       4
       5from typing import Any
       6
       7
       8def format_metric_value(name: Any, value: Any, unit: Any = "") -> str:
       9    """Return a readable metric string such as ``score=0.82`` or ``tokens=4200 tokens``."""
      10
      11    metric_name = str(name or "metric").strip() or "metric"
      12    metric_value = str(value).strip()
      13    metric_unit = str(unit or "").strip()
      14    if not metric_unit:
      15        return f"{metric_name}={metric_value}"
      16    separator = "" if metric_unit.startswith(("%", "/", "°")) else " "
      17    return f"{metric_name}={metric_value}{separator}{metric_unit}"

nipux_cli/operator_context.py 83 lines

       1"""Generic filtering for operator messages that enter worker context."""
       2
       3from __future__ import annotations
       4
       5import re
       6from typing import Any
       7
       8
       9CONVERSATION_ONLY_PATTERNS = [
      10    re.compile(r"(?i)^\s*(hi|hello|hey|yo|thanks|thank you|ok|okay|cool|nice|great|hello\?)\s*[.!?]*\s*$"),
      11    re.compile(r"(?i)^\s*(how('?s| is) it going|what('?s| is) going on|any updates?|status|jobs|ls|help|clear|exit|quit)\s*[.!?]*\s*$"),
      12    re.compile(r"(?i)^\s*(how('?s| is) it going)\??\s*(have you got|any)\s+(any\s+)?(improvements?|updates?|results?)\s*(yet)?\s*[.!?]*\s*$"),
      13    re.compile(r"(?i)^\s*(run|start|stop|pause|resume|cancel|work|status|jobs|clear|exit|quit|help)\s+\d+\s*$"),
      14]
      15
      16ACTIONABLE_PATTERNS = [
      17    re.compile(
      18        r"(?i)\b("
      19        r"avoid|because|benchmark|change|constraint|correct|do not|don't|dont|fix|focus|instead|instruction|"
      20        r"measure|must|need|never|only|prefer|priority|remember|should|target|use|wrong"
      21        r"|prioriti[sz]e)\b"
      22    ),
      23    re.compile(r"[\"'`][^\"'`]{2,}[\"'`]"),
      24]
      25
      26
      27def operator_entry_is_active(entry: dict[str, Any]) -> bool:
      28    mode = str(entry.get("mode") or "steer").strip().lower().replace("-", "_")
      29    return (
      30        mode in {"steer", "follow_up"}
      31        and not entry.get("acknowledged_at")
      32        and not entry.get("superseded_at")
      33    )
      34
      35
      36def operator_entry_is_prompt_relevant(entry: dict[str, Any]) -> bool:
      37    mode = str(entry.get("mode") or "steer").strip().lower().replace("-", "_")
      38    message = str(entry.get("message") or "").strip()
      39    if not message:
      40        return False
      41    if mode == "note":
      42        return not _conversation_only(message)
      43    if mode not in {"steer", "follow_up"}:
      44        return False
      45    if entry.get("acknowledged_at") or entry.get("superseded_at"):
      46        return False
      47    return _actionable(message)
      48
      49
      50def active_prompt_operator_entries(messages: list[Any]) -> list[dict[str, Any]]:
      51    return [
      52        entry
      53        for entry in messages
      54        if isinstance(entry, dict)
      55        and operator_entry_is_prompt_relevant(entry)
      56    ]
      57
      58
      59def inactive_prompt_operator_ids(messages: list[Any]) -> list[str]:
      60    ids: list[str] = []
      61    for entry in messages:
      62        if not isinstance(entry, dict):
      63            continue
      64        if not operator_entry_is_active(entry):
      65            continue
      66        if operator_entry_is_prompt_relevant(entry):
      67            continue
      68        event_id = str(entry.get("event_id") or "")
      69        if event_id:
      70            ids.append(event_id)
      71    return ids
      72
      73
      74def _conversation_only(message: str) -> bool:
      75    text = " ".join(message.split())
      76    return any(pattern.search(text) for pattern in CONVERSATION_ONLY_PATTERNS)
      77
      78
      79def _actionable(message: str) -> bool:
      80    text = " ".join(message.split())
      81    if _conversation_only(text):
      82        return False
      83    return any(pattern.search(text) for pattern in ACTIONABLE_PATTERNS)

nipux_cli/parser_builder.py 356 lines

       1"""Argparse construction for Nipux CLI commands."""
       2
       3from __future__ import annotations
       4
       5import argparse
       6from collections.abc import Callable, Mapping
       7
       8
       9CommandHandler = Callable[[argparse.Namespace], None]
      10CommandHandlers = Mapping[str, CommandHandler]
      11
      12
      13def _handler(handlers: CommandHandlers, name: str) -> CommandHandler:
      14    return handlers[name]
      15
      16
      17def build_arg_parser(
      18    *,
      19    handlers: CommandHandlers,
      20    version: str,
      21    default_context_length: int,
      22) -> argparse.ArgumentParser:
      23    parser = argparse.ArgumentParser(prog="nipux")
      24    parser.add_argument("--version", action="version", version=f"nipux {version}")
      25    sub = parser.add_subparsers(dest="command", required=True)
      26
      27    init = sub.add_parser("init")
      28    init.add_argument("--path")
      29    init.add_argument("--force", action="store_true")
      30    init.add_argument("--openrouter", action="store_true", help="Write an OpenRouter config that reads OPENROUTER_API_KEY")
      31    init.add_argument("--model", help="Model name to write into config.yaml")
      32    init.add_argument("--base-url", help="OpenAI-compatible API base URL")
      33    init.add_argument("--api-key-env", help="Environment variable that stores the API key")
      34    init.add_argument("--context-length", type=int, default=default_context_length)
      35    init.set_defaults(func=_handler(handlers, "init"))
      36
      37    update = sub.add_parser("update")
      38    update.add_argument("--path", help="Git checkout to update. Defaults to the current Nipux install.")
      39    update.add_argument("--allow-dirty", action="store_true", help="Attempt git pull even when local changes exist")
      40    update.add_argument("--no-restart", action="store_true", help="Do not restart a running daemon after updating")
      41    update.set_defaults(func=_handler(handlers, "update"))
      42
      43    uninstall = sub.add_parser("uninstall")
      44    uninstall.add_argument("--yes", action="store_true", help="Confirm removal without an interactive prompt")
      45    uninstall.add_argument("--dry-run", action="store_true", help="Show what would be removed")
      46    uninstall.add_argument("--keep-legacy", action="store_true", help="Keep legacy ~/.kneepucks state if present")
      47    uninstall.add_argument("--keep-tool", action="store_true", help="Keep the installed nipux command")
      48    uninstall.add_argument("--remove-tool", action="store_true", help=argparse.SUPPRESS)
      49    uninstall.add_argument("--wait", type=float, default=5.0, help="Seconds to wait for daemon shutdown")
      50    uninstall.set_defaults(func=_handler(handlers, "uninstall"))
      51
      52    create = sub.add_parser("create", aliases=["new"])
      53    create.add_argument("objective")
      54    create.add_argument("--title")
      55    create.add_argument("--kind", default="generic")
      56    create.add_argument("--cadence")
      57    create.set_defaults(func=_handler(handlers, "create"))
      58
      59    jobs = sub.add_parser("jobs")
      60    jobs.set_defaults(func=_handler(handlers, "jobs"))
      61
      62    ls_cmd = sub.add_parser("ls")
      63    ls_cmd.set_defaults(func=_handler(handlers, "jobs"))
      64
      65    focus = sub.add_parser("focus")
      66    focus.add_argument("query", nargs="*")
      67    focus.set_defaults(func=_handler(handlers, "focus"))
      68
      69    rename = sub.add_parser("rename")
      70    rename.add_argument("job_id", nargs="*")
      71    rename.add_argument("--title", nargs="+", required=True)
      72    rename.set_defaults(func=_handler(handlers, "rename"))
      73
      74    delete = sub.add_parser("delete", aliases=["rm"])
      75    delete.add_argument("job_id", nargs="*")
      76    delete.add_argument("--keep-files", action="store_true")
      77    delete.set_defaults(func=_handler(handlers, "delete"))
      78
      79    chat = sub.add_parser("chat")
      80    chat.add_argument("job_id", nargs="*")
      81    chat.add_argument("--history-limit", type=int, default=12)
      82    chat.add_argument("--no-history", action="store_true")
      83    chat.set_defaults(func=_handler(handlers, "chat"))
      84
      85    shell = sub.add_parser("shell")
      86    shell.add_argument("--status", action="store_true", help="Render the full dashboard when the shell opens")
      87    shell.add_argument("--no-status", action="store_true", help=argparse.SUPPRESS)
      88    shell.add_argument("--limit", type=int, default=8)
      89    shell.add_argument("--chars", type=int, default=180)
      90    shell.set_defaults(func=_handler(handlers, "shell"))
      91
      92    steer = sub.add_parser("steer", aliases=["say"])
      93    steer.add_argument("--job", dest="job_id")
      94    steer.add_argument("message", nargs="+")
      95    steer.set_defaults(func=_handler(handlers, "steer"))
      96
      97    pause = sub.add_parser("pause")
      98    pause.add_argument("parts", nargs="*", help="Optional job title/id followed by an optional note")
      99    pause.set_defaults(func=_handler(handlers, "pause"))
     100
     101    resume = sub.add_parser("resume")
     102    resume.add_argument("job_id", nargs="*")
     103    resume.set_defaults(func=_handler(handlers, "resume"))
     104
     105    cancel = sub.add_parser("cancel")
     106    cancel.add_argument("parts", nargs="*", help="Optional job title/id followed by an optional note")
     107    cancel.set_defaults(func=_handler(handlers, "cancel"))
     108
     109    status = sub.add_parser("status")
     110    status.add_argument("job_id", nargs="*")
     111    status.add_argument("--limit", type=int, default=8)
     112    status.add_argument("--chars", type=int, default=180)
     113    status.add_argument("--full", action="store_true", help="Render the full dashboard")
     114    status.add_argument("--json", action="store_true")
     115    status.set_defaults(func=_handler(handlers, "status"))
     116
     117    health = sub.add_parser("health")
     118    health.add_argument("--limit", type=int, default=8)
     119    health.add_argument("--chars", type=int, default=180)
     120    health.set_defaults(func=_handler(handlers, "health"))
     121
     122    history = sub.add_parser("history")
     123    history.add_argument("job_id", nargs="*")
     124    history.add_argument("--limit", type=int, default=80)
     125    history.add_argument("--chars", type=int, default=260)
     126    history.add_argument("--full", action="store_true")
     127    history.add_argument("--json", action="store_true")
     128    history.set_defaults(func=_handler(handlers, "history"))
     129
     130    events = sub.add_parser("events")
     131    events.add_argument("job_id", nargs="*")
     132    events.add_argument("--limit", type=int, default=80)
     133    events.add_argument("--chars", type=int, default=260)
     134    events.add_argument("--full", action="store_true")
     135    events.add_argument("--follow", action="store_true")
     136    events.add_argument("--interval", type=float, default=2.0)
     137    events.add_argument("--json", action="store_true")
     138    events.set_defaults(func=_handler(handlers, "events"))
     139
     140    dashboard = sub.add_parser("dashboard", aliases=["dash"])
     141    dashboard.add_argument("job_id", nargs="*")
     142    dashboard.add_argument("--interval", type=float, default=2.0)
     143    dashboard.add_argument("--limit", type=int, default=12)
     144    dashboard.add_argument("--chars", type=int, default=260)
     145    dashboard.add_argument("--no-follow", dest="follow", action="store_false")
     146    dashboard.add_argument("--no-clear", dest="clear", action="store_false")
     147    dashboard.set_defaults(func=_handler(handlers, "dashboard"), follow=True, clear=True)
     148
     149    start = sub.add_parser("start")
     150    start.add_argument("--poll-seconds", type=float, default=0.0)
     151    start.add_argument("--fake", action="store_true", help="Use deterministic fake model responses")
     152    start.add_argument("--quiet", action="store_true", help="Write fewer daemon log lines")
     153    start.add_argument("--log-file")
     154    start.set_defaults(func=_handler(handlers, "start"))
     155
     156    stop = sub.add_parser("stop")
     157    stop.add_argument("job_id", nargs="*", help="Optional job title/id to pause instead of stopping the daemon")
     158    stop.add_argument("--wait", type=float, default=5.0)
     159    stop.set_defaults(func=_handler(handlers, "stop"))
     160
     161    restart = sub.add_parser("restart")
     162    restart.add_argument("--poll-seconds", type=float, default=0.0)
     163    restart.add_argument("--wait", type=float, default=5.0)
     164    restart.add_argument("--fake", action="store_true", help="Use deterministic fake model responses")
     165    restart.add_argument("--quiet", action="store_true", help="Write fewer daemon log lines")
     166    restart.add_argument("--log-file")
     167    restart.set_defaults(func=_handler(handlers, "restart"))
     168
     169    browser_dashboard = sub.add_parser("browser-dashboard")
     170    browser_dashboard.add_argument("--port", type=int, default=4848)
     171    browser_dashboard.add_argument("--foreground", action="store_true")
     172    browser_dashboard.add_argument("--stop", action="store_true")
     173    browser_dashboard.add_argument("--log-file")
     174    browser_dashboard.set_defaults(func=_handler(handlers, "browser_dashboard"))
     175
     176    autostart = sub.add_parser("autostart")
     177    autostart.add_argument("action", choices=["install", "status", "uninstall"])
     178    autostart.add_argument("--poll-seconds", type=float, default=5.0)
     179    autostart.add_argument("--quiet", action="store_true")
     180    autostart.set_defaults(func=_handler(handlers, "autostart"))
     181
     182    service = sub.add_parser("service")
     183    service.add_argument("action", choices=["install", "status", "uninstall"])
     184    service.add_argument("--poll-seconds", type=float, default=0.0)
     185    service.add_argument("--quiet", action="store_true")
     186    service.set_defaults(func=_handler(handlers, "service"))
     187
     188    artifacts = sub.add_parser("artifacts")
     189    artifacts.add_argument("job_id", nargs="*")
     190    artifacts.add_argument("--limit", type=int, default=25)
     191    artifacts.add_argument("--chars", type=int, default=220)
     192    artifacts.add_argument("--paths", action="store_true", help="Show full artifact paths")
     193    artifacts.set_defaults(func=_handler(handlers, "artifacts"))
     194
     195    artifact = sub.add_parser("artifact")
     196    artifact.add_argument("artifact_id_or_path", nargs="+")
     197    artifact.add_argument("--job", dest="job_id")
     198    artifact.add_argument("--chars", type=int, default=12000)
     199    artifact.set_defaults(func=_handler(handlers, "artifact"))
     200
     201    lessons = sub.add_parser("lessons")
     202    lessons.add_argument("job_id", nargs="*")
     203    lessons.add_argument("--limit", type=int, default=25)
     204    lessons.add_argument("--chars", type=int, default=220)
     205    lessons.set_defaults(func=_handler(handlers, "lessons"))
     206
     207    learn = sub.add_parser("learn")
     208    learn.add_argument("--job", dest="job_id")
     209    learn.add_argument("--category", default="operator_preference")
     210    learn.add_argument("--chars", type=int, default=220)
     211    learn.add_argument("lesson", nargs="+")
     212    learn.set_defaults(func=_handler(handlers, "learn"))
     213
     214    findings = sub.add_parser("findings")
     215    findings.add_argument("job_id", nargs="*")
     216    findings.add_argument("--limit", type=int, default=25)
     217    findings.add_argument("--chars", type=int, default=220)
     218    findings.add_argument("--json", action="store_true")
     219    findings.set_defaults(func=_handler(handlers, "findings"))
     220
     221    tasks = sub.add_parser("tasks")
     222    tasks.add_argument("job_id", nargs="*")
     223    tasks.add_argument("--limit", type=int, default=25)
     224    tasks.add_argument("--chars", type=int, default=220)
     225    tasks.add_argument("--status", nargs="+")
     226    tasks.add_argument("--json", action="store_true")
     227    tasks.set_defaults(func=_handler(handlers, "tasks"))
     228
     229    roadmap = sub.add_parser("roadmap")
     230    roadmap.add_argument("job_id", nargs="*")
     231    roadmap.add_argument("--limit", type=int, default=25)
     232    roadmap.add_argument("--features", type=int, default=3)
     233    roadmap.add_argument("--chars", type=int, default=220)
     234    roadmap.add_argument("--json", action="store_true")
     235    roadmap.set_defaults(func=_handler(handlers, "roadmap"))
     236
     237    experiments = sub.add_parser("experiments")
     238    experiments.add_argument("job_id", nargs="*")
     239    experiments.add_argument("--limit", type=int, default=25)
     240    experiments.add_argument("--chars", type=int, default=220)
     241    experiments.add_argument("--status", nargs="+")
     242    experiments.add_argument("--json", action="store_true")
     243    experiments.set_defaults(func=_handler(handlers, "experiments"))
     244
     245    sources = sub.add_parser("sources")
     246    sources.add_argument("job_id", nargs="*")
     247    sources.add_argument("--limit", type=int, default=25)
     248    sources.add_argument("--chars", type=int, default=220)
     249    sources.add_argument("--json", action="store_true")
     250    sources.set_defaults(func=_handler(handlers, "sources"))
     251
     252    memory = sub.add_parser("memory")
     253    memory.add_argument("job_id", nargs="*")
     254    memory.add_argument("--limit", type=int, default=10)
     255    memory.add_argument("--chars", type=int, default=260)
     256    memory.add_argument("--json", action="store_true", help="Print memory graph JSON")
     257    memory.add_argument("--graph", action="store_true", help="Write a clickable HTML memory graph")
     258    memory.add_argument("--output", help="Path for --graph HTML output")
     259    memory.set_defaults(func=_handler(handlers, "memory"))
     260
     261    metrics = sub.add_parser("metrics")
     262    metrics.add_argument("job_id", nargs="*")
     263    metrics.add_argument("--chars", type=int, default=220)
     264    metrics.set_defaults(func=_handler(handlers, "metrics"))
     265
     266    usage = sub.add_parser("usage")
     267    usage.add_argument("job_id", nargs="*")
     268    usage.add_argument("--json", action="store_true")
     269    usage.set_defaults(func=_handler(handlers, "usage"))
     270
     271    logs = sub.add_parser("logs", aliases=["outputs", "output"])
     272    logs.add_argument("job_id", nargs="*")
     273    logs.add_argument("--limit", type=int, default=25)
     274    logs.add_argument("--verbose", action="store_true")
     275    logs.add_argument("--chars", type=int, default=4000)
     276    logs.set_defaults(func=_handler(handlers, "logs"))
     277
     278    activity = sub.add_parser("activity", aliases=["feed", "tail"])
     279    activity.add_argument("job_id", nargs="*")
     280    activity.add_argument("--limit", type=int, default=20)
     281    activity.add_argument("--chars", type=int, default=180)
     282    activity.add_argument("--follow", action="store_true")
     283    activity.add_argument("--interval", type=float, default=2.0)
     284    activity.add_argument("--verbose", action="store_true")
     285    activity.add_argument("--paths", action="store_true", help="Show full artifact paths")
     286    activity.set_defaults(func=_handler(handlers, "activity"))
     287
     288    updates = sub.add_parser("updates", aliases=["outcomes", "outcome"])
     289    updates.add_argument("job_id", nargs="*")
     290    updates.add_argument("--all", action="store_true", help="Show durable outcome summaries for every job")
     291    updates.add_argument("--limit", type=int, default=5)
     292    updates.add_argument("--chars", type=int, default=180)
     293    updates.add_argument("--paths", action="store_true", help="Show full artifact paths")
     294    updates.set_defaults(func=_handler(handlers, "updates"))
     295
     296    watch = sub.add_parser("watch")
     297    watch.add_argument("job_id", nargs="+")
     298    watch.add_argument("--interval", type=float, default=2.0)
     299    watch.add_argument("--limit", type=int, default=20)
     300    watch.add_argument("--verbose", action="store_true")
     301    watch.add_argument("--chars", type=int, default=4000)
     302    watch.add_argument("--no-follow", dest="follow", action="store_false")
     303    watch.set_defaults(func=_handler(handlers, "watch"), follow=True)
     304
     305    run_one = sub.add_parser("run-one")
     306    run_one.add_argument("job_id", nargs="+")
     307    run_one.add_argument("--fake", action="store_true", help="Use a deterministic fake model response")
     308    run_one.set_defaults(func=_handler(handlers, "run_one"))
     309
     310    work = sub.add_parser("work")
     311    work.add_argument("job_id", nargs="*")
     312    work.add_argument("--steps", type=int, default=5)
     313    work.add_argument("--poll-seconds", type=float, default=0.5)
     314    work.add_argument("--fake", action="store_true", help="Use deterministic fake model responses")
     315    work.add_argument("--verbose", action="store_true", help="Print step inputs and outputs")
     316    work.add_argument("--dashboard", action="store_true", help="Render a dashboard snapshot after each step")
     317    work.add_argument("--limit", type=int, default=12)
     318    work.add_argument("--chars", type=int, default=4000)
     319    work.add_argument("--continue-on-error", action="store_true")
     320    work.set_defaults(func=_handler(handlers, "work"))
     321
     322    run = sub.add_parser("run")
     323    run.add_argument("job_id", nargs="*")
     324    run.add_argument("--poll-seconds", type=float, default=0.0)
     325    run.add_argument("--interval", type=float, default=2.0)
     326    run.add_argument("--limit", type=int, default=20)
     327    run.add_argument("--chars", type=int, default=180)
     328    run.add_argument("--verbose", action="store_true")
     329    run.add_argument("--paths", action="store_true")
     330    run.add_argument("--fake", action="store_true", help="Use deterministic fake model responses")
     331    run.add_argument("--quiet", action="store_true", help="Write fewer daemon log lines")
     332    run.add_argument("--log-file")
     333    run.add_argument("--no-follow", action="store_true", help="Start daemon and return without tailing activity")
     334    run.set_defaults(func=_handler(handlers, "run"))
     335
     336    digest = sub.add_parser("digest")
     337    digest.add_argument("job_id", nargs="+")
     338    digest.set_defaults(func=_handler(handlers, "digest"))
     339
     340    daily_digest = sub.add_parser("daily-digest")
     341    daily_digest.add_argument("--day", help="YYYY-MM-DD. Defaults to today.")
     342    daily_digest.set_defaults(func=_handler(handlers, "daily_digest"))
     343
     344    daemon = sub.add_parser("daemon")
     345    daemon.add_argument("--once", action="store_true", help="Run at most one job step and exit")
     346    daemon.add_argument("--fake", action="store_true", help="Use deterministic fake model responses")
     347    daemon.add_argument("--poll-seconds", type=float, default=0.0)
     348    daemon.add_argument("--quiet", action="store_true", help="Do not print foreground progress lines")
     349    daemon.add_argument("--verbose", action="store_true", help="Print model-visible job state and step results")
     350    daemon.set_defaults(func=_handler(handlers, "daemon"))
     351
     352    doctor = sub.add_parser("doctor")
     353    doctor.add_argument("--check-model", action="store_true", help="Also call the local model /models endpoint")
     354    doctor.set_defaults(func=_handler(handlers, "doctor"))
     355
     356    return parser

nipux_cli/planning.py 384 lines

       1"""Generic initial planning primitives for long-running jobs."""
       2
       3from __future__ import annotations
       4
       5import re
       6from typing import Any
       7
       8
       9_PROFILE_TERMS: dict[str, set[str]] = {
      10    "measured": {
      11        "accelerate",
      12        "benchmark",
      13        "compare",
      14        "decrease",
      15        "faster",
      16        "improve",
      17        "increase",
      18        "latency",
      19        "measure",
      20        "metric",
      21        "optimize",
      22        "performance",
      23        "reduce",
      24        "score",
      25        "speed",
      26        "test",
      27        "throughput",
      28    },
      29    "deliverable": {
      30        "article",
      31        "artifact",
      32        "checklist",
      33        "create",
      34        "deck",
      35        "doc",
      36        "document",
      37        "draft",
      38        "file",
      39        "generate",
      40        "guide",
      41        "manual",
      42        "memo",
      43        "outline",
      44        "paper",
      45        "produce",
      46        "presentation",
      47        "report",
      48        "spec",
      49        "template",
      50        "write",
      51    },
      52    "monitor": {
      53        "alert",
      54        "check",
      55        "observe",
      56        "periodic",
      57        "reporting",
      58        "track",
      59        "watch",
      60        "monitor",
      61    },
      62    "implementation": {
      63        "automate",
      64        "build",
      65        "change",
      66        "code",
      67        "debug",
      68        "deploy",
      69        "fix",
      70        "implement",
      71        "install",
      72        "repair",
      73        "run",
      74        "setup",
      75    },
      76    "research": {
      77        "analyze",
      78        "compare",
      79        "explore",
      80        "find",
      81        "investigate",
      82        "learn",
      83        "map",
      84        "research",
      85        "review",
      86        "summarize",
      87        "survey",
      88    },
      89}
      90_PROFILE_PRIORITY = {
      91    "measured": 0,
      92    "monitor": 1,
      93    "implementation": 2,
      94    "deliverable": 3,
      95    "research": 4,
      96}
      97
      98
      99def objective_profiles(objective: str) -> list[str]:
     100    """Infer generic work profiles from an objective without binding to a domain."""
     101
     102    tokens = set(re.findall(r"[a-z][a-z0-9_-]+", objective.lower()))
     103    scores: list[tuple[int, str]] = []
     104    for profile, terms in _PROFILE_TERMS.items():
     105        score = len(tokens & terms)
     106        if score:
     107            scores.append((score, profile))
     108    if not scores:
     109        return ["general"]
     110    scores.sort(key=lambda item: (-item[0], _PROFILE_PRIORITY.get(item[1], 99), item[1]))
     111    profiles = [profile for _score, profile in scores[:2]]
     112    return profiles or ["general"]
     113
     114
     115def initial_plan_for_objective(objective: str) -> dict[str, Any]:
     116    objective_text = " ".join(objective.split())
     117    profiles = objective_profiles(objective_text)
     118    tasks = _initial_tasks_for_profiles(profiles)
     119    questions = _initial_questions_for_profiles(profiles)
     120    return {
     121        "status": "needs_operator_review",
     122        "summary": _initial_summary_for_profiles(profiles),
     123        "profile": profiles[0],
     124        "profiles": profiles,
     125        "tasks": tasks,
     126        "questions": questions,
     127        "objective": objective_text,
     128    }
     129
     130
     131def initial_task_contract(task_title: str) -> dict[str, str]:
     132    lowered = task_title.lower()
     133    if any(word in lowered for word in ("baseline", "benchmark", "compare", "experiment", "measure", "metric", "test")):
     134        return {
     135            "output_contract": "experiment",
     136            "acceptance_criteria": "A baseline, result, comparison, or explicit blocked measurement is recorded.",
     137            "evidence_needed": "Experiment record with metric, environment or inputs, result direction, and next action.",
     138            "stall_behavior": "Record why measurement is blocked and create the smallest follow-up task that can obtain it.",
     139        }
     140    if any(
     141        word in lowered
     142        for word in (
     143            "article",
     144            "checklist",
     145            "draft",
     146            "deliverable",
     147            "document",
     148            "file",
     149            "generate",
     150            "guide",
     151            "manual",
     152            "paper",
     153            "produce",
     154            "report",
     155            "template",
     156            "write",
     157        )
     158    ):
     159        return {
     160            "output_contract": "report",
     161            "acceptance_criteria": "A durable draft, report, or deliverable section is saved with its evidence status.",
     162            "evidence_needed": "Saved output plus cited evidence, assumptions, gaps, or review notes.",
     163            "stall_behavior": "Save the partial output, record the gap, and create the next evidence or revision task.",
     164        }
     165    if any(word in lowered for word in ("validate", "review", "decide", "criteria", "constraint", "success")):
     166        return {
     167            "output_contract": "decision",
     168            "acceptance_criteria": "The decision, validation result, or success criteria are explicit.",
     169            "evidence_needed": "Operator context, durable notes, milestone validation, or task/roadmap updates.",
     170            "stall_behavior": "Ask for the missing constraint or record a reversible assumption and continue.",
     171        }
     172    if any(word in lowered for word in ("monitor", "watch", "track", "check", "observe")):
     173        return {
     174            "output_contract": "monitor",
     175            "acceptance_criteria": "A check cadence, signal, state change, or next observation time is recorded.",
     176            "evidence_needed": "Monitor result, status update, deferred follow-up, or recorded blocker.",
     177            "stall_behavior": "Defer the job until the next useful check or pivot to a diagnostic task.",
     178        }
     179    if any(word in lowered for word in ("act", "apply", "build", "change", "deploy", "fix", "implement", "install", "run")):
     180        return {
     181            "output_contract": "action",
     182            "acceptance_criteria": "The action produces an observable durable change or a clear blocker.",
     183            "evidence_needed": "Tool result plus file, artifact, ledger, task, roadmap, or experiment update.",
     184            "stall_behavior": "Record the blocker and open a smaller follow-up action.",
     185        }
     186    if "clarify" in lowered or "criteria" in lowered or "constraint" in lowered:
     187        return {
     188            "output_contract": "decision",
     189            "acceptance_criteria": "Success criteria, constraints, and first branches are explicit.",
     190            "evidence_needed": "Operator context, durable notes, or an updated roadmap/task queue.",
     191            "stall_behavior": "Ask for the missing constraint or record a decision with the best current assumption.",
     192        }
     193    if "map" in lowered or "research" in lowered or "branch" in lowered or "explore" in lowered or "source" in lowered:
     194        return {
     195            "output_contract": "research",
     196            "acceptance_criteria": "At least one viable branch is selected and low-value branches are avoided.",
     197            "evidence_needed": "Source notes, branch rationale, source ledger entries, or saved research output.",
     198            "stall_behavior": "Record a low-yield lesson and pivot to a different branch.",
     199        }
     200    if "collect" in lowered or "evidence" in lowered or "save" in lowered or "output" in lowered:
     201        return {
     202            "output_contract": "artifact",
     203            "acceptance_criteria": "A durable output is saved and linked to the task or ledger.",
     204            "evidence_needed": "Artifact, file output, finding record, source record, or experiment record.",
     205            "stall_behavior": "Record what evidence is missing and create the next evidence-producing task.",
     206        }
     207    if "reflect" in lowered or "memory" in lowered or "continue" in lowered:
     208        return {
     209            "output_contract": "monitor",
     210            "acceptance_criteria": "Progress is evaluated from durable deltas and the next branch is chosen.",
     211            "evidence_needed": "Reflection, lesson, task update, roadmap validation, or experiment comparison.",
     212            "stall_behavior": "Record a blocker or pivot when no durable delta was produced.",
     213        }
     214    return {
     215        "output_contract": "action",
     216        "acceptance_criteria": "The task produces an observable durable change.",
     217        "evidence_needed": "Tool result plus artifact, ledger, task, roadmap, or experiment update.",
     218        "stall_behavior": "Record a blocker and open a smaller follow-up task.",
     219    }
     220
     221
     222def initial_roadmap_for_objective(*, title: str, objective: str) -> dict[str, Any]:
     223    profiles = objective_profiles(objective)
     224    execute_contract = _primary_execution_contract(profiles)
     225    return {
     226        "title": title,
     227        "status": "planned",
     228        "objective": objective,
     229        "scope": (
     230            "Initial roadmap generated from the objective and inferred generic work profile "
     231            f"({', '.join(profiles)}). Refine this as evidence and operator context arrive."
     232        ),
     233        "current_milestone": "Clarify and frame the work",
     234        "validation_contract": (
     235            "Each milestone needs observable evidence that its acceptance criteria were met, "
     236            "or a recorded blocker plus follow-up tasks."
     237        ),
     238        "milestones": [
     239            {
     240                "title": "Clarify and frame the work",
     241                "status": "planned",
     242                "priority": 10,
     243                "goal": "Turn the objective into concrete success criteria and constraints.",
     244                "acceptance_criteria": "Success criteria and first branches are explicit.",
     245                "evidence_needed": "Operator context, planning notes, or a recorded task queue.",
     246                "features": [{"title": "Capture success criteria", "status": "planned", "output_contract": "decision"}],
     247            },
     248            {
     249                "title": "Execute first durable branches",
     250                "status": "planned",
     251                "priority": 8,
     252                "goal": "Produce artifacts, findings, actions, or measurements that advance the objective.",
     253                "acceptance_criteria": "At least one branch produces durable evidence.",
     254                "evidence_needed": "Saved outputs, ledger updates, action results, or experiment records.",
     255                "features": [
     256                    {
     257                        "title": "Run the first evidence-producing branch",
     258                        "status": "planned",
     259                        "output_contract": execute_contract,
     260                    }
     261                ],
     262            },
     263            {
     264                "title": "Validate and continue",
     265                "status": "planned",
     266                "priority": 6,
     267                "goal": "Check results against acceptance criteria and create follow-up work.",
     268                "acceptance_criteria": "Validation is passed, failed, or blocked with a next action.",
     269                "evidence_needed": "record_milestone_validation entry and follow-up tasks if needed.",
     270                "features": [{"title": "Validate the checkpoint", "status": "planned", "output_contract": "decision"}],
     271            },
     272        ],
     273        "metadata": {"phase": "initial_plan"},
     274    }
     275
     276
     277def _initial_summary_for_profiles(profiles: list[str]) -> str:
     278    primary = profiles[0] if profiles else "general"
     279    if primary == "measured":
     280        return "I will start by defining the measurable baseline, then iterate on branches that can prove improvement."
     281    if primary == "deliverable":
     282        return "I will start by framing the deliverable, collecting evidence, and saving drafts that can be improved."
     283    if primary == "monitor":
     284        return "I will start by defining the watched signals, first check, cadence, and durable update format."
     285    if primary == "implementation":
     286        return "I will start by inspecting the current state, planning a small action, and validating the result."
     287    if primary == "research":
     288        return "I will start by mapping source branches, collecting evidence, and saving concise findings."
     289    return "I will turn this objective into a durable long-running job before starting tool work."
     290
     291
     292def _initial_tasks_for_profiles(profiles: list[str]) -> list[str]:
     293    tasks: list[str] = ["Clarify success criteria, constraints, and review/report cadence."]
     294    primary = profiles[0] if profiles else "general"
     295    if primary == "measured":
     296        tasks.extend(
     297            [
     298                "Record the baseline metric and measurement method.",
     299                "Run the first measurable branch and record an experiment.",
     300                "Compare the result with the best known baseline and choose the next branch.",
     301            ]
     302        )
     303    elif primary == "deliverable":
     304        tasks.extend(
     305            [
     306                "Map the outline, audience, evidence gaps, and acceptance criteria.",
     307                "Collect evidence for the first section or deliverable unit.",
     308                "Save a durable draft or report checkpoint.",
     309                "Review and revise the latest draft against evidence gaps and acceptance criteria.",
     310            ]
     311        )
     312    elif primary == "monitor":
     313        tasks.extend(
     314            [
     315                "Define watched signals, check cadence, and alert conditions.",
     316                "Run the first status check and save the observation.",
     317                "Defer or continue based on the next useful check time.",
     318            ]
     319        )
     320    elif primary == "implementation":
     321        tasks.extend(
     322            [
     323                "Inspect current state and identify the smallest safe action.",
     324                "Apply one change or execute one action with observable output.",
     325                "Validate the result and record any follow-up branch.",
     326            ]
     327        )
     328    else:
     329        tasks.extend(
     330            [
     331                "Map the first research or execution branches.",
     332                "Collect evidence and save outputs as files.",
     333                "Reflect on what worked, update memory, and continue with the next branch.",
     334            ]
     335        )
     336    return tasks
     337
     338
     339def _initial_questions_for_profiles(profiles: list[str]) -> list[str]:
     340    questions = [
     341        "What result would make this job successful?",
     342        "Are there constraints, risks, or approaches I should avoid?",
     343    ]
     344    primary = profiles[0] if profiles else "general"
     345    if primary == "measured":
     346        questions.insert(1, "What metric should be treated as the primary measure of progress?")
     347    elif primary == "deliverable":
     348        questions.insert(1, "Who is the audience, and what quality bar should the deliverable meet?")
     349    elif primary == "monitor":
     350        questions.insert(1, "How often should I check, and what change should trigger a report?")
     351    elif primary == "implementation":
     352        questions.insert(1, "Which environment or files are in scope, and what requires approval?")
     353    else:
     354        questions.insert(1, "Which sources, artifacts, or signals should I trust most?")
     355    questions.append("Should this run aggressively in the background or wait for review between branches?")
     356    return questions
     357
     358
     359def _primary_execution_contract(profiles: list[str]) -> str:
     360    if "measured" in profiles:
     361        return "experiment"
     362    if "deliverable" in profiles:
     363        return "report"
     364    if "monitor" in profiles:
     365        return "monitor"
     366    if "implementation" in profiles:
     367        return "action"
     368    if "research" in profiles:
     369        return "research"
     370    return "artifact"
     371
     372
     373def format_initial_plan(plan: dict[str, Any]) -> str:
     374    tasks = plan.get("tasks") if isinstance(plan.get("tasks"), list) else []
     375    questions = plan.get("questions") if isinstance(plan.get("questions"), list) else []
     376    lines = [str(plan.get("summary") or "Initial plan created.")]
     377    if tasks:
     378        lines.append("Plan:")
     379        lines.extend(f"- {task}" for task in tasks)
     380    if questions:
     381        lines.append("Questions:")
     382        lines.extend(f"- {question}" for question in questions)
     383    lines.append("Reply with answers, or use the right-side Run control when this plan is good enough to start.")
     384    return "\n".join(lines)

nipux_cli/progress.py 213 lines

       1"""Generic progress summaries for long-running jobs."""
       2
       3from __future__ import annotations
       4
       5from dataclasses import dataclass
       6from typing import Any
       7
       8
       9@dataclass(frozen=True)
      10class ProgressCheckpoint:
      11    message: str
      12    category: str
      13    counts: dict[str, int]
      14    deltas: dict[str, int]
      15    updates: dict[str, int]
      16    resolutions: dict[str, int]
      17    recent: str
      18
      19
      20LEDGER_KEYS = ("findings", "sources", "tasks", "experiments", "lessons", "milestones")
      21
      22
      23def build_progress_checkpoint(
      24    metadata: dict[str, Any],
      25    *,
      26    previous_counts: dict[str, Any] | None = None,
      27    step_no: int,
      28    tool_name: str | None,
      29    artifact_id: str = "",
      30    is_finding_output: bool = False,
      31) -> ProgressCheckpoint:
      32    """Create the operator-facing checkpoint text from durable ledger deltas."""
      33    counts = ledger_counts(metadata)
      34    previous = previous_counts or {}
      35    deltas = {key: counts[key] - _as_int(previous.get(key)) for key in LEDGER_KEYS}
      36    updates = ledger_update_counts(metadata, since=str(metadata.get("last_checkpoint_at") or ""))
      37    resolutions = ledger_resolution_counts(metadata, since=str(metadata.get("last_checkpoint_at") or ""))
      38    recent = recent_progress_bits(metadata)
      39    if is_finding_output:
      40        message = (
      41            f"Saved output {artifact_id}; ledgers now have {counts['findings']} findings, "
      42            f"{counts['sources']} sources, {counts['tasks']} tasks, and {counts['experiments']} experiments."
      43        )
      44        category = "finding"
      45    else:
      46        changed_parts = [_count_phrase(value, key, prefix="+") for key, value in deltas.items() if value > 0]
      47        changed_parts.extend(
      48            _count_phrase(value, key, prefix="~", suffix="updated") for key, value in updates.items() if value > 0
      49        )
      50        changed_parts.extend(
      51            _count_phrase(value, key, suffix="resolved") for key, value in resolutions.items() if value > 0
      52        )
      53        changed = ", ".join(changed_parts)
      54        made_progress = bool(changed)
      55        if not changed:
      56            changed = "no new durable ledger entries"
      57        message = (
      58            f"Checkpoint step #{step_no}: {changed}. Totals: {counts['findings']} findings, "
      59            f"{counts['sources']} sources, {counts['tasks']} tasks, {counts['experiments']} experiments, "
      60            f"{counts['lessons']} lessons."
      61        )
      62        category = "progress" if made_progress else "activity"
      63    if recent:
      64        message = f"{message} Recent: {recent}."
      65    return ProgressCheckpoint(
      66        message=message,
      67        category=category,
      68        counts=counts,
      69        deltas=deltas,
      70        updates=updates,
      71        resolutions=resolutions,
      72        recent=recent,
      73    )
      74
      75
      76def ledger_counts(metadata: dict[str, Any]) -> dict[str, int]:
      77    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
      78    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
      79    return {
      80        "findings": len(_metadata_list(metadata, "finding_ledger")),
      81        "sources": len(_metadata_list(metadata, "source_ledger")),
      82        "tasks": len(_metadata_list(metadata, "task_queue")),
      83        "experiments": len(_metadata_list(metadata, "experiment_ledger")),
      84        "lessons": len(_metadata_list(metadata, "lessons")),
      85        "milestones": len(milestones),
      86    }
      87
      88
      89def ledger_update_counts(metadata: dict[str, Any], *, since: str = "") -> dict[str, int]:
      90    """Count durable ledger updates that do not increase ledger size."""
      91    counts = {key: 0 for key in LEDGER_KEYS}
      92    record_map = {
      93        "findings": "last_finding_record",
      94        "sources": "last_source_record",
      95        "tasks": "last_task_record",
      96        "experiments": "last_experiment_record",
      97    }
      98    for key, metadata_key in record_map.items():
      99        record = metadata.get(metadata_key)
     100        if _updated_existing_record(record, since=since):
     101            counts[key] += 1
     102    roadmap = metadata.get("last_roadmap_record")
     103    if isinstance(roadmap, dict) and _record_after_checkpoint(roadmap, since=since):
     104        updated = _as_int(roadmap.get("updated_milestones")) + _as_int(roadmap.get("updated_features"))
     105        if roadmap.get("roadmap_updated"):
     106            updated += 1
     107        added = _as_int(roadmap.get("added_milestones")) + _as_int(roadmap.get("added_features"))
     108        if updated > 0 and added <= 0:
     109            counts["milestones"] += 1
     110    validation = metadata.get("last_milestone_validation")
     111    if isinstance(validation, dict) and _record_after_checkpoint(validation, since=since):
     112        counts["milestones"] += 1
     113    return counts
     114
     115
     116def ledger_resolution_counts(metadata: dict[str, Any], *, since: str = "") -> dict[str, int]:
     117    """Count durable branch resolutions so task updates do not look like empty churn."""
     118    counts = {key: 0 for key in LEDGER_KEYS}
     119    task = metadata.get("last_task_record")
     120    if _updated_existing_record(task, since=since):
     121        status = str(task.get("status") or "").lower() if isinstance(task, dict) else ""
     122        if status in {"done", "blocked", "skipped"} and (task.get("result") or task.get("evidence_needed")):
     123            counts["tasks"] += 1
     124    experiment = metadata.get("last_experiment_record")
     125    if _updated_existing_record(experiment, since=since):
     126        status = str(experiment.get("status") or "").lower() if isinstance(experiment, dict) else ""
     127        if status in {"measured", "failed", "blocked", "skipped"} or experiment.get("metric_value") is not None:
     128            counts["experiments"] += 1
     129    validation = metadata.get("last_milestone_validation")
     130    if isinstance(validation, dict) and _record_after_checkpoint(validation, since=since):
     131        counts["milestones"] += 1
     132    return counts
     133
     134
     135def recent_progress_bits(metadata: dict[str, Any]) -> str:
     136    bits: list[str] = []
     137    findings = _metadata_list(metadata, "finding_ledger")
     138    if findings:
     139        finding = findings[-1]
     140        bits.append(f"finding={_clip_text(str(finding.get('name') or finding.get('title') or 'finding'), 80)}")
     141    active_tasks = [
     142        task
     143        for task in _metadata_list(metadata, "task_queue")
     144        if str(task.get("status") or "open").lower() in {"active", "open", "blocked"}
     145    ]
     146    if active_tasks:
     147        task = sorted(active_tasks, key=lambda entry: -_as_int(entry.get("priority")))[0]
     148        bits.append(f"task={_clip_text(str(task.get('title') or 'task'), 80)}")
     149    measured = [
     150        experiment
     151        for experiment in _metadata_list(metadata, "experiment_ledger")
     152        if experiment.get("metric_value") is not None
     153    ]
     154    if measured:
     155        experiment = measured[-1]
     156        metric = f"{experiment.get('metric_name') or 'metric'}={experiment.get('metric_value')}{experiment.get('metric_unit') or ''}"
     157        bits.append(f"measurement={_clip_text(metric, 80)}")
     158    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
     159    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
     160    active_milestones = [
     161        milestone
     162        for milestone in milestones
     163        if isinstance(milestone, dict)
     164        and str(milestone.get("status") or "planned").lower() in {"active", "validating", "blocked"}
     165    ]
     166    if active_milestones:
     167        bits.append(f"milestone={_clip_text(str(active_milestones[-1].get('title') or 'milestone'), 80)}")
     168    return "; ".join(bits)
     169
     170
     171def _metadata_list(metadata: dict[str, Any], key: str) -> list[dict[str, Any]]:
     172    value = metadata.get(key)
     173    if not isinstance(value, list):
     174        return []
     175    return [item for item in value if isinstance(item, dict)]
     176
     177
     178def _clip_text(value: str, limit: int) -> str:
     179    text = " ".join(value.split())
     180    if len(text) <= limit:
     181        return text
     182    return text[: max(0, limit - 1)].rstrip() + "..."
     183
     184
     185def _updated_existing_record(record: Any, *, since: str) -> bool:
     186    return (
     187        isinstance(record, dict)
     188        and record.get("created") is False
     189        and record.get("substantive_update") is not False
     190        and _record_after_checkpoint(record, since=since)
     191    )
     192
     193
     194def _record_after_checkpoint(record: dict[str, Any], *, since: str) -> bool:
     195    if not since:
     196        return True
     197    updated_at = str(record.get("updated_at") or record.get("validated_at") or record.get("last_seen") or record.get("at") or "")
     198    return bool(updated_at and updated_at > since)
     199
     200
     201def _count_phrase(value: int, key: str, *, prefix: str = "", suffix: str = "") -> str:
     202    label = key[:-1] if value == 1 and key.endswith("s") else key
     203    bits = [f"{prefix}{value} {label}"]
     204    if suffix:
     205        bits.append(suffix)
     206    return " ".join(bits)
     207
     208
     209def _as_int(value: Any) -> int:
     210    try:
     211        return int(value)
     212    except (TypeError, ValueError):
     213        return 0

nipux_cli/provider_errors.py 64 lines

       1"""Generic model-provider error classification."""
       2
       3from __future__ import annotations
       4
       5import json
       6from typing import Any
       7
       8
       9PROVIDER_ACTION_MARKERS = (
      10    "authenticationerror",
      11    "permissiondeniederror",
      12    "authentication failed",
      13    "permission denied",
      14    "invalid api key",
      15    "incorrect api key",
      16    "user not found",
      17    "key limit exceeded",
      18    "insufficient_quota",
      19    "insufficient quota",
      20    "insufficient credits",
      21    "billing",
      22    "payment required",
      23    "credit limit",
      24    "quota exceeded",
      25    "401",
      26    "403",
      27)
      28
      29RATE_LIMIT_MARKERS = (
      30    "429",
      31    "rate limit",
      32    "ratelimit",
      33    "too many requests",
      34    "temporarily over capacity",
      35)
      36
      37PROVIDER_ACTION_REQUIRED_NOTE = (
      38    "Model provider requires operator action: authentication, permission, billing, or quota is blocking calls. "
      39    "Paused this job so the daemon does not repeat failing model requests. Update credentials/model access, then resume."
      40)
      41
      42
      43def provider_error_text(error: Any) -> str:
      44    if isinstance(error, str):
      45        return error.lower()
      46    parts = [type(error).__name__, str(error)]
      47    payload = getattr(error, "payload", None)
      48    if isinstance(payload, dict) and payload:
      49        parts.append(json.dumps(payload, ensure_ascii=False, default=str))
      50    return " ".join(parts).lower()
      51
      52
      53def provider_action_required(text_or_error: Any) -> bool:
      54    text = provider_error_text(text_or_error)
      55    return any(marker in text for marker in PROVIDER_ACTION_MARKERS)
      56
      57
      58def provider_action_required_note(text_or_error: Any) -> str:
      59    return PROVIDER_ACTION_REQUIRED_NOTE if provider_action_required(text_or_error) else ""
      60
      61
      62def provider_rate_limited(text_or_error: Any) -> bool:
      63    text = provider_error_text(text_or_error)
      64    return any(marker in text for marker in RATE_LIMIT_MARKERS)

nipux_cli/record_commands.py 542 lines

       1"""Read-only CLI commands for job records, ledgers, memory, and usage."""
       2
       3from __future__ import annotations
       4
       5import json
       6from dataclasses import dataclass
       7from pathlib import Path
       8from typing import Any, Callable
       9
      10from nipux_cli.artifacts import ArtifactStore, sha256_text
      11from nipux_cli.cli_render import job_ref_text as default_job_ref_text
      12from nipux_cli.cli_render import json_default, rule
      13from nipux_cli.daemon import daemon_lock_status
      14from nipux_cli.memory_graph import memory_graph_from_job
      15from nipux_cli.memory_graph_view import render_memory_graph_html
      16from nipux_cli.tui_status import active_operator_messages, worker_label
      17from nipux_cli.tui_style import _one_line
      18from nipux_cli.usage import format_usage_report
      19
      20
      21@dataclass(frozen=True)
      22class RecordCommandDeps:
      23    db_factory: Callable[[], tuple[Any, Any]]
      24    resolve_job_id: Callable[[Any, Any], str | None]
      25    job_ref_text: Callable[[Any], str] = default_job_ref_text
      26
      27
      28def cmd_findings_impl(args: Any, deps: RecordCommandDeps) -> None:
      29    db, _ = deps.db_factory()
      30    try:
      31        job_id = _resolve_or_print(db, args, deps)
      32        if not job_id:
      33            return
      34        job = db.get_job(job_id)
      35        findings = _metadata_records(job, "finding_ledger")
      36        if args.json:
      37            print(json.dumps(findings, ensure_ascii=False, indent=2, default=json_default))
      38            return
      39        print(f"findings {job['title']} | {len(findings)} unique")
      40        print(rule("="))
      41        if not findings:
      42            print("none yet")
      43            return
      44        ranked = sorted(findings, key=lambda finding: float(finding.get("score") or 0), reverse=True)
      45        for index, finding in enumerate(ranked[: args.limit], start=1):
      46            score = finding.get("score")
      47            score_text = f" score={score:g}" if isinstance(score, (int, float)) else ""
      48            print(f"{index:>2}. {_one_line(finding.get('name') or 'unknown', 54)}{score_text}")
      49            details = " | ".join(
      50                value
      51                for value in [
      52                    str(finding.get("location") or "").strip(),
      53                    str(finding.get("category") or "").strip(),
      54                    str(finding.get("status") or "").strip(),
      55                ]
      56                if value
      57            )
      58            if details:
      59                print(f"    {details}")
      60            if finding.get("url") or finding.get("source_url"):
      61                print(f"    {finding.get('url') or finding.get('source_url')}")
      62            if finding.get("reason"):
      63                print(f"    {_one_line(finding['reason'], args.chars)}")
      64    finally:
      65        db.close()
      66
      67
      68def cmd_tasks_impl(args: Any, deps: RecordCommandDeps) -> None:
      69    db, _ = deps.db_factory()
      70    try:
      71        job_id = _resolve_or_print(db, args, deps)
      72        if not job_id:
      73            return
      74        job = db.get_job(job_id)
      75        tasks = _metadata_records(job, "task_queue")
      76        if args.status:
      77            wanted = {status.strip().lower() for status in args.status}
      78            tasks = [task for task in tasks if str(task.get("status") or "open").lower() in wanted]
      79        if args.json:
      80            print(json.dumps(tasks, ensure_ascii=False, indent=2, default=json_default))
      81            return
      82        status_order = {"active": 0, "open": 1, "blocked": 2, "done": 3, "skipped": 4}
      83        ranked = sorted(
      84            tasks,
      85            key=lambda task: (
      86                status_order.get(str(task.get("status") or "open"), 9),
      87                -int(task.get("priority") or 0),
      88                str(task.get("title") or ""),
      89            ),
      90        )
      91        print(f"tasks {job['title']} | {len(ranked)} tracked")
      92        print(rule("="))
      93        if not ranked:
      94            print("none yet")
      95            return
      96        for index, task in enumerate(ranked[: args.limit], start=1):
      97            status = str(task.get("status") or "open")
      98            priority = int(task.get("priority") or 0)
      99            print(f"{index:>2}. {status:<7} p={priority:<3} {_one_line(task.get('title') or 'untitled', 54)}")
     100            details = " | ".join(
     101                value
     102                for value in [
     103                    f"contract={task.get('output_contract')}" if task.get("output_contract") else "",
     104                    f"accept={task.get('acceptance_criteria')}" if task.get("acceptance_criteria") else "",
     105                    f"evidence={task.get('evidence_needed')}" if task.get("evidence_needed") else "",
     106                    f"stall={task.get('stall_behavior')}" if task.get("stall_behavior") else "",
     107                    str(task.get("goal") or "").strip(),
     108                    str(task.get("source_hint") or "").strip(),
     109                    str(task.get("result") or "").strip(),
     110                ]
     111                if value
     112            )
     113            if details:
     114                print(f"    {_one_line(details, args.chars)}")
     115    finally:
     116        db.close()
     117
     118
     119def cmd_roadmap_impl(args: Any, deps: RecordCommandDeps) -> None:
     120    db, _ = deps.db_factory()
     121    try:
     122        job_id = _resolve_or_print(db, args, deps)
     123        if not job_id:
     124            return
     125        job = db.get_job(job_id)
     126        metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     127        roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
     128        if args.json:
     129            print(json.dumps(roadmap, ensure_ascii=False, indent=2, default=json_default))
     130            return
     131        print(f"roadmap {job['title']}")
     132        print(rule("="))
     133        if not roadmap:
     134            print("none yet")
     135            print("the worker can create one with record_roadmap when broad work needs milestones")
     136            return
     137        milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
     138        print(f"title: {roadmap.get('title') or 'Roadmap'}")
     139        print(f"status: {roadmap.get('status') or 'planned'} | milestones: {len(milestones)}")
     140        if roadmap.get("current_milestone"):
     141            print(f"current: {_one_line(roadmap.get('current_milestone') or '', args.chars)}")
     142        if roadmap.get("scope"):
     143            print(f"scope: {_one_line(roadmap.get('scope') or '', args.chars)}")
     144        if roadmap.get("validation_contract"):
     145            print(f"validation: {_one_line(roadmap.get('validation_contract') or '', args.chars)}")
     146        _print_milestones(milestones, limit=args.limit, features=args.features, chars=args.chars)
     147    finally:
     148        db.close()
     149
     150
     151def cmd_experiments_impl(args: Any, deps: RecordCommandDeps) -> None:
     152    db, _ = deps.db_factory()
     153    try:
     154        job_id = _resolve_or_print(db, args, deps)
     155        if not job_id:
     156            return
     157        job = db.get_job(job_id)
     158        experiments = _metadata_records(job, "experiment_ledger")
     159        if args.status:
     160            wanted = {status.strip().lower() for status in args.status}
     161            experiments = [
     162                experiment for experiment in experiments if str(experiment.get("status") or "planned").lower() in wanted
     163            ]
     164        if args.json:
     165            print(json.dumps(experiments, ensure_ascii=False, indent=2, default=json_default))
     166            return
     167        status_order = {"running": 0, "planned": 1, "measured": 2, "blocked": 3, "failed": 4, "skipped": 5}
     168        ranked = sorted(
     169            experiments,
     170            key=lambda experiment: (
     171                not bool(experiment.get("best_observed")),
     172                status_order.get(str(experiment.get("status") or "planned"), 9),
     173                str(experiment.get("updated_at") or experiment.get("created_at") or ""),
     174            ),
     175        )
     176        print(f"experiments {job['title']} | {len(ranked)} tracked")
     177        print(rule("="))
     178        if not ranked:
     179            print("none yet")
     180            return
     181        for index, experiment in enumerate(ranked[: args.limit], start=1):
     182            status = str(experiment.get("status") or "planned")
     183            best = " *best*" if experiment.get("best_observed") else ""
     184            metric = ""
     185            if experiment.get("metric_value") is not None:
     186                metric = (
     187                    f" {experiment.get('metric_name') or 'metric'}="
     188                    f"{experiment.get('metric_value')}{experiment.get('metric_unit') or ''}"
     189                )
     190            print(f"{index:>2}. {status:<8} {_one_line(experiment.get('title') or 'experiment', 54)}{metric}{best}")
     191            details = " | ".join(
     192                value
     193                for value in [
     194                    str(experiment.get("result") or "").strip(),
     195                    f"next: {experiment.get('next_action')}" if experiment.get("next_action") else "",
     196                    f"delta: {experiment.get('delta_from_previous_best')}"
     197                    if experiment.get("delta_from_previous_best") is not None
     198                    else "",
     199                ]
     200                if value
     201            )
     202            if details:
     203                print(f"    {_one_line(details, args.chars)}")
     204    finally:
     205        db.close()
     206
     207
     208def cmd_sources_impl(args: Any, deps: RecordCommandDeps) -> None:
     209    db, _ = deps.db_factory()
     210    try:
     211        job_id = _resolve_or_print(db, args, deps)
     212        if not job_id:
     213            return
     214        job = db.get_job(job_id)
     215        sources = _metadata_records(job, "source_ledger")
     216        if args.json:
     217            print(json.dumps(sources, ensure_ascii=False, indent=2, default=json_default))
     218            return
     219        ranked = sorted(
     220            sources,
     221            key=lambda source: (float(source.get("usefulness_score") or 0), int(source.get("yield_count") or 0)),
     222            reverse=True,
     223        )
     224        print(f"sources {job['title']} | {len(sources)} scored")
     225        print(rule("="))
     226        if not ranked:
     227            print("none yet")
     228            return
     229        for index, source in enumerate(ranked[: args.limit], start=1):
     230            score = float(source.get("usefulness_score") or 0)
     231            print(
     232                f"{index:>2}. {_one_line(source.get('source') or 'unknown', 58)} "
     233                f"score={score:g} findings={source.get('yield_count') or 0} fails={source.get('fail_count') or 0}"
     234            )
     235            detail = " | ".join(
     236                value
     237                for value in [
     238                    str(source.get("source_type") or "").strip(),
     239                    str(source.get("last_outcome") or "").strip(),
     240                ]
     241                if value
     242            )
     243            if detail:
     244                print(f"    {_one_line(detail, args.chars)}")
     245            warnings = source.get("warnings") if isinstance(source.get("warnings"), list) else []
     246            if warnings:
     247                print(f"    warnings: {_one_line(', '.join(str(item) for item in warnings[-3:]), args.chars)}")
     248    finally:
     249        db.close()
     250
     251
     252def cmd_memory_impl(args: Any, deps: RecordCommandDeps) -> None:
     253    db, config = deps.db_factory()
     254    try:
     255        job_id = _resolve_or_print(db, args, deps)
     256        if not job_id:
     257            return
     258        job = db.get_job(job_id)
     259        metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     260        lessons = _metadata_records(job, "lessons")
     261        reflections = _metadata_records(job, "reflections")
     262        compact = db.list_memory(job_id)
     263        active_operator = active_operator_messages(metadata)
     264        graph = memory_graph_from_job(job)
     265        if bool(getattr(args, "json", False)):
     266            print(json.dumps(graph, ensure_ascii=False, indent=2, default=json_default))
     267            return
     268        if bool(getattr(args, "graph", False)):
     269            _write_memory_graph_view(db=db, config=config, job=job, graph=graph, output=getattr(args, "output", None))
     270            return
     271        pending_measurement = (
     272            metadata.get("pending_measurement_obligation")
     273            if isinstance(metadata.get("pending_measurement_obligation"), dict)
     274            else {}
     275        )
     276        print(f"memory {job['title']}")
     277        print(rule("="))
     278        print(
     279            f"lessons={len(lessons)} reflections={len(reflections)} compact_entries={len(compact)} "
     280            f"graph_nodes={len(graph['nodes'])} graph_edges={len(graph['edges'])}"
     281        )
     282        _print_memory_sections(
     283            active_operator=active_operator,
     284            pending_measurement=pending_measurement,
     285            graph=graph,
     286            reflections=reflections,
     287            lessons=lessons,
     288            compact=compact,
     289            limit=args.limit,
     290            chars=args.chars,
     291        )
     292    finally:
     293        db.close()
     294
     295
     296def _write_memory_graph_view(
     297    *,
     298    db: Any,
     299    config: Any,
     300    job: dict[str, Any],
     301    graph: dict[str, Any],
     302    output: str | None,
     303) -> None:
     304    html = render_memory_graph_html(job)
     305    summary = f"Clickable memory graph with {len(graph['nodes'])} nodes and {len(graph['edges'])} links."
     306    metadata = {
     307        "memory_graph": True,
     308        "node_count": len(graph["nodes"]),
     309        "edge_count": len(graph["edges"]),
     310    }
     311    if output:
     312        path = Path(output).expanduser()
     313        path.parent.mkdir(parents=True, exist_ok=True)
     314        path.write_text(html, encoding="utf-8")
     315        artifact_id = db.add_artifact(
     316            job_id=str(job["id"]),
     317            path=path,
     318            sha256=sha256_text(html),
     319            artifact_type="html",
     320            title="Memory Graph",
     321            summary=summary,
     322            metadata=metadata,
     323        )
     324        print(f"memory graph written: {path}")
     325        print(f"artifact: {artifact_id}")
     326        return
     327    store = ArtifactStore(config.runtime.home, db)
     328    stored = store.write_text(
     329        job_id=str(job["id"]),
     330        content=html,
     331        title="Memory Graph",
     332        summary=summary,
     333        artifact_type="html",
     334        metadata=metadata,
     335    )
     336    print(f"memory graph written: {stored.path}")
     337    print(f"artifact: {stored.id}")
     338
     339
     340def cmd_metrics_impl(args: Any, deps: RecordCommandDeps) -> None:
     341    db, config = deps.db_factory()
     342    try:
     343        job_id = _resolve_or_print(db, args, deps)
     344        if not job_id:
     345            return
     346        job = db.get_job(job_id)
     347        steps = db.list_steps(job_id=job_id)
     348        artifacts = db.list_artifacts(job_id, limit=1000)
     349        findings = _metadata_records(job, "finding_ledger")
     350        sources = _metadata_records(job, "source_ledger")
     351        tasks = _metadata_records(job, "task_queue")
     352        experiments = _metadata_records(job, "experiment_ledger")
     353        lessons = _metadata_records(job, "lessons")
     354        reflections = _metadata_records(job, "reflections")
     355        metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     356        roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
     357        milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
     358        daemon = daemon_lock_status(config.runtime.home / "agentd.lock")
     359        finding_batches = [
     360            artifact
     361            for artifact in artifacts
     362            if "finding" in str(artifact.get("title") or artifact.get("summary") or "").lower()
     363        ]
     364        blocked = [step for step in steps if step.get("status") == "blocked"]
     365        failed = [step for step in steps if step.get("status") == "failed"]
     366        print(f"metrics {job['title']}")
     367        print(rule("="))
     368        print(f"daemon: {'running' if daemon['running'] else 'stopped'} | worker: {worker_label(job, bool(daemon['running']))}")
     369        print(f"steps: {_step_count(steps)} | failed: {len(failed)} | blocked/recovered: {len(blocked)}")
     370        print(f"artifacts: {len(artifacts)} | finding_batches: {len(finding_batches)}")
     371        print(
     372            f"findings: {len(findings)} | sources: {len(sources)} | tasks: {len(tasks)} | "
     373            f"milestones: {len(milestones)} | experiments: {len(experiments)} | "
     374            f"lessons: {len(lessons)} | reflections: {len(reflections)}"
     375        )
     376        _print_best_records(sources=sources, findings=findings, experiments=experiments, chars=args.chars)
     377    finally:
     378        db.close()
     379
     380
     381def cmd_usage_impl(args: Any, deps: RecordCommandDeps) -> None:
     382    db, config = deps.db_factory()
     383    try:
     384        job_id = _resolve_or_print(db, args, deps)
     385        if not job_id:
     386            return
     387        job = db.get_job(job_id)
     388        usage = db.job_token_usage(job_id)
     389        usage["input_cost_per_million"] = config.model.input_cost_per_million
     390        usage["output_cost_per_million"] = config.model.output_cost_per_million
     391        usage["max_job_cost_usd"] = config.runtime.max_job_cost_usd
     392        if args.json:
     393            print(json.dumps(usage, ensure_ascii=False, indent=2, sort_keys=True))
     394            return
     395        lines = format_usage_report(
     396            title=str(job.get("title") or job_id),
     397            usage=usage,
     398            context_length=int(config.model.context_length or 0),
     399            model=str(config.model.model),
     400            base_url=str(config.model.base_url),
     401        )
     402        print("\n".join(lines))
     403    finally:
     404        db.close()
     405
     406
     407def _resolve_or_print(db: Any, args: Any, deps: RecordCommandDeps) -> str | None:
     408    job_id = deps.resolve_job_id(db, args.job_id)
     409    if job_id:
     410        return job_id
     411    ref = deps.job_ref_text(args.job_id)
     412    print(f"No job matched: {ref}" if ref else "No jobs found.")
     413    return None
     414
     415
     416def _metadata_records(job: dict[str, Any], key: str) -> list[dict[str, Any]]:
     417    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     418    values = metadata.get(key)
     419    if not isinstance(values, list):
     420        return []
     421    return [value for value in values if isinstance(value, dict)]
     422
     423
     424def _print_milestones(milestones: list[Any], *, limit: int, features: int, chars: int) -> None:
     425    if not milestones:
     426        return
     427    print()
     428    status_order = {"active": 0, "validating": 1, "planned": 2, "blocked": 3, "done": 4, "skipped": 5}
     429    ranked = sorted(
     430        [milestone for milestone in milestones if isinstance(milestone, dict)],
     431        key=lambda milestone: (
     432            status_order.get(str(milestone.get("status") or "planned"), 9),
     433            -int(milestone.get("priority") or 0),
     434            str(milestone.get("title") or ""),
     435        ),
     436    )
     437    for index, milestone in enumerate(ranked[:limit], start=1):
     438        status = str(milestone.get("status") or "planned")
     439        validation = str(milestone.get("validation_status") or "not_started")
     440        milestone_features = milestone.get("features") if isinstance(milestone.get("features"), list) else []
     441        open_features = sum(
     442            1 for feature in milestone_features
     443            if isinstance(feature, dict) and str(feature.get("status") or "planned") in {"planned", "active"}
     444        )
     445        print(
     446            f"{index:>2}. {status:<10} validation={validation:<11} "
     447            f"p={int(milestone.get('priority') or 0):<3} {_one_line(milestone.get('title') or 'milestone', 54)}"
     448        )
     449        details = " | ".join(
     450            value
     451            for value in [
     452                f"features={len(milestone_features)}/{open_features} open" if milestone_features else "",
     453                f"accept={milestone.get('acceptance_criteria')}" if milestone.get("acceptance_criteria") else "",
     454                f"evidence={milestone.get('evidence_needed')}" if milestone.get("evidence_needed") else "",
     455                f"result={milestone.get('validation_result')}" if milestone.get("validation_result") else "",
     456                f"next={milestone.get('next_action')}" if milestone.get("next_action") else "",
     457            ]
     458            if value
     459        )
     460        if details:
     461            print(f"    {_one_line(details, chars)}")
     462        for feature in milestone_features[: min(3, features)]:
     463            if isinstance(feature, dict):
     464                print(f"    - {str(feature.get('status') or 'planned'):<7} {_one_line(feature.get('title') or 'feature', max(30, chars - 16))}")
     465
     466
     467def _print_memory_sections(
     468    *,
     469    active_operator: list[dict[str, Any]],
     470    pending_measurement: dict[str, Any],
     471    graph: dict[str, Any],
     472    reflections: list[dict[str, Any]],
     473    lessons: list[dict[str, Any]],
     474    compact: list[dict[str, Any]],
     475    limit: int,
     476    chars: int,
     477) -> None:
     478    if active_operator:
     479        print()
     480        print("active operator context:")
     481        for entry in active_operator[-min(limit, 8) :]:
     482            marker = entry.get("event_id") or "operator"
     483            print(f"  {marker}: {_one_line(entry.get('message') or '', chars)}")
     484    if pending_measurement:
     485        print()
     486        print(f"pending measurement: step #{pending_measurement.get('source_step_no') or '?'}")
     487        candidates = pending_measurement.get("metric_candidates") if isinstance(pending_measurement.get("metric_candidates"), list) else []
     488        if candidates:
     489            print(f"  candidates: {_one_line(', '.join(str(item) for item in candidates[:5]), chars)}")
     490    nodes = graph.get("nodes") if isinstance(graph.get("nodes"), list) else []
     491    if nodes:
     492        print()
     493        print("memory graph:")
     494        for node in nodes[: min(limit, 8)]:
     495            print(
     496                f"  {node.get('kind') or 'fact'}:{node.get('status') or 'active'} "
     497                f"{_one_line(node.get('title') or node.get('key') or 'memory', chars)}"
     498            )
     499        print("  view: memory --graph")
     500    if reflections:
     501        print()
     502        print("latest reflection:")
     503        reflection = reflections[-1]
     504        print(f"  {_one_line(reflection.get('summary') or '', chars)}")
     505        if reflection.get("strategy"):
     506            print(f"  strategy: {_one_line(reflection['strategy'], chars)}")
     507    if lessons:
     508        print()
     509        print("latest lessons:")
     510        for lesson in lessons[-min(limit, 8) :]:
     511            print(f"  {lesson.get('category') or 'memory'}: {_one_line(lesson.get('lesson') or '', chars)}")
     512    if compact:
     513        print()
     514        print("compact memory:")
     515        for entry in compact[: min(limit, 3)]:
     516            print(f"  {entry.get('key')}: {_one_line(entry.get('summary') or '', chars)}")
     517
     518
     519def _print_best_records(
     520    *,
     521    sources: list[dict[str, Any]],
     522    findings: list[dict[str, Any]],
     523    experiments: list[dict[str, Any]],
     524    chars: int,
     525) -> None:
     526    if sources:
     527        best = max(sources, key=lambda source: float(source.get("usefulness_score") or 0))
     528        print(f"best source: {_one_line(best.get('source') or '', chars)} score={best.get('usefulness_score')}")
     529    if findings:
     530        best_finding = max(findings, key=lambda finding: float(finding.get("score") or 0))
     531        print(f"best finding: {_one_line(best_finding.get('name') or '', chars)} score={best_finding.get('score')}")
     532    measured = [experiment for experiment in experiments if experiment.get("metric_value") is not None]
     533    best_experiments = [experiment for experiment in measured if experiment.get("best_observed")]
     534    if best_experiments:
     535        best_experiment = best_experiments[-1]
     536        metric = f"{best_experiment.get('metric_name') or 'metric'}={best_experiment.get('metric_value')}{best_experiment.get('metric_unit') or ''}"
     537        print(f"best experiment: {_one_line(best_experiment.get('title') or '', chars)} {metric}")
     538
     539
     540def _step_count(steps: list[dict[str, Any]]) -> int:
     541    numbers = [int(step.get("step_no") or 0) for step in steps]
     542    return max(numbers, default=0)

nipux_cli/scheduling.py 76 lines

       1"""Shared scheduling helpers for deferred long-running work."""
       2
       3from __future__ import annotations
       4
       5from datetime import datetime, timezone
       6from typing import Any
       7
       8
       9def job_deferred_until(job: dict[str, Any], *, now: datetime | None = None) -> datetime | None:
      10    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
      11    raw_until = str(metadata.get("defer_until") or "").strip()
      12    if not raw_until:
      13        return None
      14    try:
      15        until = datetime.fromisoformat(raw_until.replace("Z", "+00:00"))
      16    except ValueError:
      17        return None
      18    if until.tzinfo is None:
      19        until = until.replace(tzinfo=timezone.utc)
      20    until = until.astimezone(timezone.utc)
      21    now = (now or datetime.now(timezone.utc)).astimezone(timezone.utc)
      22    return until if until > now else None
      23
      24
      25def job_is_deferred(job: dict[str, Any], *, now: datetime | None = None) -> bool:
      26    return job_deferred_until(job, now=now) is not None
      27
      28
      29def job_provider_blocked(job: dict[str, Any]) -> bool:
      30    """Return true when provider calls need operator action before retrying."""
      31
      32    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
      33    blocked_raw = str(metadata.get("provider_blocked_at") or "").strip()
      34    if not blocked_raw:
      35        return False
      36    unblocked_raw = str(metadata.get("provider_unblocked_at") or "").strip()
      37    if not unblocked_raw:
      38        return True
      39    blocked_at = _metadata_time(blocked_raw)
      40    unblocked_at = _metadata_time(unblocked_raw)
      41    if blocked_at is None or unblocked_at is None:
      42        return False
      43    return blocked_at > unblocked_at
      44
      45
      46def provider_retry_metadata() -> dict[str, str]:
      47    """Metadata patch used when the operator explicitly retries provider work."""
      48
      49    return {
      50        "provider_blocked_at": "",
      51        "provider_unblocked_at": datetime.now(timezone.utc).isoformat(),
      52    }
      53
      54
      55def operator_resume_metadata() -> dict[str, str]:
      56    """Metadata patch used when the operator explicitly makes a job runnable."""
      57
      58    patch = provider_retry_metadata()
      59    patch.update(
      60        {
      61            "defer_until": "",
      62            "defer_reason": "",
      63            "defer_next_action": "",
      64        }
      65    )
      66    return patch
      67
      68
      69def _metadata_time(value: str) -> datetime | None:
      70    try:
      71        parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
      72    except ValueError:
      73        return None
      74    if parsed.tzinfo is None:
      75        parsed = parsed.replace(tzinfo=timezone.utc)
      76    return parsed.astimezone(timezone.utc)

nipux_cli/service_install.py 179 lines

       1"""OS service installation helpers for the Nipux daemon."""
       2
       3from __future__ import annotations
       4
       5import os
       6import shlex
       7import shutil
       8import subprocess
       9import sys
      10from argparse import Namespace
      11from pathlib import Path
      12
      13from nipux_cli.config import load_config
      14
      15
      16def launch_agent_path() -> Path:
      17    return Path.home() / "Library" / "LaunchAgents" / "com.nipux.agent.plist"
      18
      19
      20def launch_agent_plist(*, poll_seconds: float, quiet: bool) -> str:
      21    config = load_config()
      22    config.ensure_dirs()
      23    command = [
      24        sys.executable,
      25        "-m",
      26        "nipux_cli.cli",
      27        "daemon",
      28        "--poll-seconds",
      29        str(poll_seconds),
      30    ]
      31    command.append("--quiet" if quiet else "--verbose")
      32    args_xml = "\n".join(f"        <string>{xml_escape(part)}</string>" for part in command)
      33    log_path = config.runtime.logs_dir / "launchd-daemon.log"
      34    return f"""<?xml version="1.0" encoding="UTF-8"?>
      35<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
      36<plist version="1.0">
      37  <dict>
      38    <key>Label</key>
      39    <string>com.nipux.agent</string>
      40    <key>ProgramArguments</key>
      41    <array>
      42{args_xml}
      43    </array>
      44    <key>EnvironmentVariables</key>
      45    <dict>
      46      <key>NIPUX_HOME</key>
      47      <string>{xml_escape(str(config.runtime.home))}</string>
      48    </dict>
      49    <key>RunAtLoad</key>
      50    <true/>
      51    <key>KeepAlive</key>
      52    <true/>
      53    <key>StandardOutPath</key>
      54    <string>{xml_escape(str(log_path))}</string>
      55    <key>StandardErrorPath</key>
      56    <string>{xml_escape(str(log_path))}</string>
      57    <key>WorkingDirectory</key>
      58    <string>{xml_escape(str(Path.cwd()))}</string>
      59  </dict>
      60</plist>
      61"""
      62
      63
      64def systemd_service_path() -> Path:
      65    return Path.home() / ".config" / "systemd" / "user" / "nipux.service"
      66
      67
      68def systemd_service_text(*, poll_seconds: float, quiet: bool) -> str:
      69    config = load_config()
      70    config.ensure_dirs()
      71    command = [
      72        sys.executable,
      73        "-m",
      74        "nipux_cli.cli",
      75        "daemon",
      76        "--poll-seconds",
      77        str(poll_seconds),
      78    ]
      79    command.append("--quiet" if quiet else "--verbose")
      80    return "\n".join(
      81        [
      82            "[Unit]",
      83            "Description=Nipux 24/7 autonomous worker",
      84            "After=network-online.target",
      85            "Wants=network-online.target",
      86            "",
      87            "[Service]",
      88            "Type=simple",
      89            f"WorkingDirectory={Path.cwd()}",
      90            f"Environment=NIPUX_HOME={config.runtime.home}",
      91            f"ExecStart={' '.join(shlex.quote(part) for part in command)}",
      92            "Restart=always",
      93            "RestartSec=3",
      94            "",
      95            "[Install]",
      96            "WantedBy=default.target",
      97            "",
      98        ]
      99    )
     100
     101
     102def cmd_autostart(args: Namespace) -> None:
     103    path = launch_agent_path()
     104    label = "gui/" + str(os.getuid()) + "/com.nipux.agent"
     105    if args.action == "status":
     106        status = "installed" if path.exists() else "not installed"
     107        print(f"autostart: {status}")
     108        print(f"plist: {path}")
     109        if path.exists():
     110            result = subprocess.run(
     111                ["launchctl", "print", label], check=False, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
     112            )
     113            print("launchd: loaded" if result.returncode == 0 else "launchd: not loaded")
     114        return
     115    if args.action == "install":
     116        path.parent.mkdir(parents=True, exist_ok=True)
     117        path.write_text(launch_agent_plist(poll_seconds=args.poll_seconds, quiet=args.quiet), encoding="utf-8")
     118        subprocess.run(
     119            ["launchctl", "bootout", label], check=False, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
     120        )
     121        result = subprocess.run(["launchctl", "bootstrap", "gui/" + str(os.getuid()), str(path)], check=False)
     122        if result.returncode:
     123            raise SystemExit(result.returncode)
     124        subprocess.run(["launchctl", "enable", label], check=False)
     125        print(f"autostart installed: {path}")
     126        print("daemon will start at login and launchd will keep it alive")
     127        return
     128    if args.action == "uninstall":
     129        subprocess.run(
     130            ["launchctl", "bootout", label], check=False, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
     131        )
     132        if path.exists():
     133            path.unlink()
     134        print("autostart uninstalled")
     135        return
     136    raise SystemExit(f"unknown autostart action: {args.action}")
     137
     138
     139def cmd_service(args: Namespace) -> None:
     140    path = systemd_service_path()
     141    systemctl = shutil.which("systemctl")
     142    user_cmd = [systemctl, "--user"] if systemctl else None
     143    if args.action == "status":
     144        print(f"service: {'installed' if path.exists() else 'not installed'}")
     145        print(f"unit: {path}")
     146        if user_cmd:
     147            result = subprocess.run(
     148                [*user_cmd, "is-active", "nipux.service"], check=False, capture_output=True, text=True
     149            )
     150            print(f"systemd: {result.stdout.strip() or result.stderr.strip() or 'unknown'}")
     151        else:
     152            print("systemd: unavailable on this machine")
     153        return
     154    if args.action == "install":
     155        path.parent.mkdir(parents=True, exist_ok=True)
     156        path.write_text(systemd_service_text(poll_seconds=args.poll_seconds, quiet=args.quiet), encoding="utf-8")
     157        print(f"service file written: {path}")
     158        if user_cmd:
     159            subprocess.run([*user_cmd, "daemon-reload"], check=False)
     160            subprocess.run([*user_cmd, "enable", "--now", "nipux.service"], check=False)
     161            print("systemd user service enabled and started")
     162        else:
     163            print(
     164                "systemd not found; copy this service to a Linux server or run: systemctl --user enable --now nipux.service"
     165            )
     166        return
     167    if args.action == "uninstall":
     168        if user_cmd:
     169            subprocess.run([*user_cmd, "disable", "--now", "nipux.service"], check=False)
     170            subprocess.run([*user_cmd, "daemon-reload"], check=False)
     171        if path.exists():
     172            path.unlink()
     173        print("service uninstalled")
     174        return
     175    raise SystemExit(f"unknown service action: {args.action}")
     176
     177
     178def xml_escape(value: str) -> str:
     179    return value.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")

nipux_cli/settings.py 153 lines

       1"""Inline config editing helpers for Nipux slash commands."""
       2
       3from __future__ import annotations
       4
       5import os
       6from pathlib import Path
       7from typing import Any
       8
       9import yaml
      10
      11from nipux_cli.config import default_config_yaml, get_agent_home, load_config, write_private_text
      12from nipux_cli.cli_state import clear_model_setup_verified
      13from nipux_cli.tui_commands import SETTINGS_FIELD_TYPES
      14
      15
      16def config_field_value(field: str, config: Any | None = None) -> Any:
      17    config = load_config() if config is None else config
      18    values = {
      19        "model.name": config.model.model,
      20        "model.base_url": config.model.base_url,
      21        "model.api_key_env": config.model.api_key_env,
      22        "model.context_length": config.model.context_length,
      23        "model.request_timeout_seconds": config.model.request_timeout_seconds,
      24        "model.input_cost_per_million": config.model.input_cost_per_million,
      25        "model.output_cost_per_million": config.model.output_cost_per_million,
      26        "runtime.home": str(config.runtime.home),
      27        "runtime.max_step_seconds": config.runtime.max_step_seconds,
      28        "runtime.artifact_inline_char_limit": config.runtime.artifact_inline_char_limit,
      29        "runtime.daily_digest_enabled": config.runtime.daily_digest_enabled,
      30        "runtime.daily_digest_time": config.runtime.daily_digest_time,
      31        "runtime.max_job_cost_usd": config.runtime.max_job_cost_usd,
      32        "tools.browser": config.tools.browser,
      33        "tools.web": config.tools.web,
      34        "tools.shell": config.tools.shell,
      35        "tools.files": config.tools.files,
      36    }
      37    return values.get(field, "")
      38
      39
      40def save_config_field(field: str, raw_value: str) -> Any:
      41    value = _coerce_config_value(field, raw_value)
      42    data = _load_config_yaml()
      43    section, key = field.split(".", 1)
      44    target = data.setdefault(section, {})
      45    if not isinstance(target, dict):
      46        target = {}
      47        data[section] = target
      48    target[key] = value
      49    _save_config_yaml(data)
      50    return value
      51
      52
      53def inline_setting_notice(field: str, raw_value: str) -> str:
      54    value = raw_value.strip()
      55    if not value:
      56        return f"kept {field}"
      57    if field == "secret:model.api_key":
      58        config = load_config()
      59        name = config.model.api_key_env
      60        _save_env_secret(name, value)
      61        clear_model_setup_verified()
      62        return f"saved {name} in {_short_path(get_agent_home() / '.env')}"
      63    try:
      64        saved = save_config_field(field, value)
      65    except ValueError as exc:
      66        return f"{field}: {exc}"
      67    clear_model_setup_verified()
      68    return f"saved {field} = {saved}"
      69
      70
      71def edit_target_label(field: str) -> str:
      72    if field == "secret:model.api_key":
      73        return "API key"
      74    return field
      75
      76
      77def edit_target_hint(field: str, config: Any | None = None) -> str:
      78    config = config or load_config()
      79    if field == "secret:model.api_key":
      80        state = "set" if config.model.api_key else "missing"
      81        return f"Editing API key ({state}). Enter saves, Esc cancels. Input is hidden."
      82    current = config_field_value(field, config)
      83    return f"Editing {field}. Current: {current}. Enter saves, Esc cancels, empty keeps current."
      84
      85
      86def edit_target_masks_input(field: str | None) -> bool:
      87    return field == "secret:model.api_key"
      88
      89
      90def _config_path() -> Path:
      91    return get_agent_home() / "config.yaml"
      92
      93
      94def _load_config_yaml() -> dict[str, Any]:
      95    path = _config_path()
      96    if not path.exists():
      97        loaded = yaml.safe_load(default_config_yaml()) or {}
      98        return loaded if isinstance(loaded, dict) else {}
      99    loaded = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
     100    return loaded if isinstance(loaded, dict) else {}
     101
     102
     103def _save_config_yaml(data: dict[str, Any]) -> None:
     104    path = _config_path()
     105    write_private_text(path, yaml.safe_dump(data, sort_keys=False))
     106
     107
     108def _save_env_secret(name: str, value: str) -> None:
     109    env_path = get_agent_home() / ".env"
     110    env_path.parent.mkdir(parents=True, exist_ok=True)
     111    existing: dict[str, str] = {}
     112    if env_path.exists():
     113        for raw in env_path.read_text(encoding="utf-8", errors="ignore").splitlines():
     114            if "=" not in raw or raw.strip().startswith("#"):
     115                continue
     116            key, current = raw.split("=", 1)
     117            if key.strip():
     118                existing[key.strip()] = current.strip()
     119    existing[name] = value
     120    write_private_text(env_path, "\n".join(f"{key}={current}" for key, current in existing.items()) + "\n")
     121    os.environ[name] = value
     122
     123
     124def _coerce_config_value(field: str, raw_value: str) -> Any:
     125    kind = SETTINGS_FIELD_TYPES.get(field, "str")
     126    value = raw_value.strip()
     127    if field == "runtime.max_job_cost_usd" and value.lower() in {"0", "none", "off", "false", "null"}:
     128        return None
     129    if kind == "int":
     130        return int(value)
     131    if kind == "float":
     132        return float(value)
     133    if kind == "bool":
     134        lowered = value.lower()
     135        if lowered in {"1", "true", "yes", "on"}:
     136            return True
     137        if lowered in {"0", "false", "no", "off"}:
     138            return False
     139        raise ValueError("use true or false")
     140    if kind == "path":
     141        return str(Path(value).expanduser())
     142    return value
     143
     144
     145def _short_path(path: Path | str, *, max_width: int = 80) -> str:
     146    text = str(path)
     147    home = str(Path.home())
     148    if text.startswith(home + os.sep):
     149        text = "~" + text[len(home) :]
     150    if len(text) <= max_width:
     151        return text
     152    keep = max(12, max_width - 4)
     153    return "..." + text[-keep:]

nipux_cli/settings_commands.py 84 lines

       1"""Slash-command handlers for inline Nipux configuration."""
       2
       3from __future__ import annotations
       4
       5import shlex
       6from contextlib import redirect_stdout
       7from io import StringIO
       8
       9from nipux_cli.config import load_config
      10from nipux_cli.settings import config_field_value, inline_setting_notice
      11from nipux_cli.tui_commands import CHAT_SETTING_COMMANDS
      12
      13
      14def handle_chat_setting_command(command: str, rest: list[str]) -> bool:
      15    if command == "config":
      16        print("\n".join(config_summary_lines()))
      17        return True
      18    if command in {"key", "api-key"}:
      19        if not rest:
      20            config = load_config()
      21            state = "set" if config.model.api_key else "missing"
      22            print(f"API key is {state} via {config.model.api_key_env}. Use /api-key KEY to save a new one.")
      23            return True
      24        print(inline_setting_notice("secret:model.api_key", " ".join(rest)))
      25        return True
      26    if command not in CHAT_SETTING_COMMANDS:
      27        return False
      28    field, placeholder = CHAT_SETTING_COMMANDS[command]
      29    if not rest:
      30        current = config_field_value(field)
      31        print(f"{field} = {current}")
      32        print(f"usage: /{command} {placeholder}")
      33        return True
      34    print(inline_setting_notice(field, " ".join(rest)))
      35    return True
      36
      37
      38def config_summary_lines() -> list[str]:
      39    config = load_config()
      40    key_state = "set" if config.model.api_key else "missing"
      41    input_cost = _rate_text(config.model.input_cost_per_million)
      42    output_cost = _rate_text(config.model.output_cost_per_million)
      43    return [
      44        "config",
      45        f"model: {config.model.model}",
      46        f"endpoint: {config.model.base_url}",
      47        f"key: {key_state} ({config.model.api_key_env})",
      48        f"context: {config.model.context_length}",
      49        f"request timeout: {config.model.request_timeout_seconds}s",
      50        f"cost rates: input {input_cost} / output {output_cost} per 1M tokens",
      51        (
      52            "tools: "
      53            f"browser {config.tools.browser}, web {config.tools.web}, "
      54            f"CLI {config.tools.shell}, files {config.tools.files}"
      55        ),
      56        f"home: {config.runtime.home}",
      57        f"step timeout: {config.runtime.max_step_seconds}s",
      58        f"output preview: {config.runtime.artifact_inline_char_limit} chars",
      59        f"job cost limit: {_cost_limit_text(config.runtime.max_job_cost_usd)}",
      60        f"daily digest: {config.runtime.daily_digest_enabled} at {config.runtime.daily_digest_time}",
      61    ]
      62
      63
      64def _rate_text(value: float | None) -> str:
      65    return "provider-reported" if value is None else f"${value:g}"
      66
      67
      68def _cost_limit_text(value: float | None) -> str:
      69    return "none" if value is None else f"${value:g}"
      70
      71
      72def capture_setting_command(line: str) -> list[str]:
      73    try:
      74        parts = shlex.split(line[1:] if line.startswith("/") else line)
      75    except ValueError as exc:
      76        return [f"parse error: {exc}"]
      77    if not parts:
      78        return []
      79    stream = StringIO()
      80    with redirect_stdout(stream):
      81        if not handle_chat_setting_command(parts[0], parts[1:]):
      82            print(f"unknown config command: /{parts[0]}")
      83    lines = [" ".join(item.split()) for item in stream.getvalue().splitlines() if item.strip()]
      84    return lines[-12:] or ["done"]

nipux_cli/shell_tools.py 348 lines

       1"""Shell and workspace file tools for Nipux workers."""
       2
       3from __future__ import annotations
       4
       5import contextlib
       6import json
       7import os
       8import re
       9import signal
      10import subprocess
      11import time
      12from datetime import datetime, timezone
      13from pathlib import Path
      14from typing import Any
      15
      16
      17def write_file(args: dict[str, Any], ctx: Any) -> str:
      18    del ctx
      19    raw_path = str(args.get("path") or "").strip()
      20    if not raw_path:
      21        return _json({"success": False, "error": "path is required"})
      22    if "content" not in args:
      23        return _json({"success": False, "error": "content is required"})
      24    mode = str(args.get("mode") or "overwrite").strip().lower()
      25    if mode not in {"overwrite", "append"}:
      26        return _json({"success": False, "error": f"invalid mode: {mode}"})
      27    path = Path(raw_path).expanduser()
      28    if not path.is_absolute():
      29        path = Path.cwd() / path
      30    if path.exists() and path.is_dir():
      31        return _json({"success": False, "error": f"path is a directory: {path}"})
      32    create_parents = bool(args.get("create_parents", True))
      33    if create_parents:
      34        path.parent.mkdir(parents=True, exist_ok=True)
      35    content = str(args.get("content") or "")
      36    write_mode = "a" if mode == "append" else "w"
      37    with path.open(write_mode, encoding="utf-8") as fh:
      38        fh.write(content)
      39    return _json({
      40        "success": True,
      41        "path": str(path),
      42        "mode": mode,
      43        "bytes": path.stat().st_size,
      44    })
      45
      46
      47def shell_exec(args: dict[str, Any], ctx: Any) -> str:
      48    command = str(args.get("command") or "").strip()
      49    if not command:
      50        return _json({"success": False, "error": "command is required"})
      51    cwd_raw = str(args.get("cwd") or "").strip()
      52    cwd = cwd_raw or None
      53    if cwd and not Path(cwd).expanduser().exists():
      54        return _json({"success": False, "error": f"cwd does not exist: {cwd}"})
      55    timeout_raw = args.get("timeout_seconds")
      56    timeout = float(timeout_raw) if isinstance(timeout_raw, (int, float)) else 60.0
      57    timeout = max(1.0, min(timeout, 900.0))
      58    max_chars_raw = args.get("max_output_chars")
      59    max_chars = int(max_chars_raw) if isinstance(max_chars_raw, (int, float)) else 12000
      60    max_chars = max(1000, min(max_chars, 50000))
      61    shell = "/bin/zsh" if Path("/bin/zsh").exists() else None
      62    env = dict(os.environ)
      63    env["NIPUX_JOB_ID"] = ctx.job_id
      64    if ctx.run_id:
      65        env["NIPUX_RUN_ID"] = ctx.run_id
      66    started = time.monotonic()
      67    process: subprocess.Popen[str] | None = None
      68    try:
      69        process = subprocess.Popen(
      70            command,
      71            shell=True,
      72            executable=shell,
      73            cwd=str(Path(cwd).expanduser()) if cwd else None,
      74            env=env,
      75            stdout=subprocess.PIPE,
      76            stderr=subprocess.PIPE,
      77            text=True,
      78            start_new_session=True,
      79        )
      80        _register_shell_process(ctx, process, command=command, cwd=cwd or os.getcwd(), timeout_seconds=timeout)
      81        stdout, stderr = process.communicate(timeout=timeout)
      82    except subprocess.TimeoutExpired:
      83        assert process is not None
      84        _terminate_process_group(process)
      85        try:
      86            stdout, stderr = process.communicate(timeout=2)
      87        except subprocess.TimeoutExpired:
      88            _kill_process_group(process)
      89            stdout, stderr = process.communicate()
      90        return _json({
      91            "success": False,
      92            "error": f"command timed out after {timeout:.1f}s",
      93            "timed_out": True,
      94            "command": command,
      95            "cwd": cwd or os.getcwd(),
      96            "timeout_seconds": timeout,
      97            "duration_seconds": round(time.monotonic() - started, 3),
      98            "returncode": None,
      99            "stdout": _truncate_output(stdout, max_chars),
     100            "stderr": _truncate_output(stderr, max_chars),
     101        })
     102    except BaseException:
     103        if process is not None and process.poll() is None:
     104            _terminate_process_group(process)
     105            try:
     106                process.wait(timeout=2)
     107            except subprocess.TimeoutExpired:
     108                _kill_process_group(process)
     109        raise
     110    finally:
     111        if process is not None:
     112            _unregister_shell_process(ctx, process.pid)
     113    error = _shell_error(process.returncode, stdout, stderr, command=command)
     114    return _json({
     115        "success": process.returncode == 0 and not error,
     116        "error": error,
     117        "command": command,
     118        "cwd": cwd or os.getcwd(),
     119        "duration_seconds": round(time.monotonic() - started, 3),
     120        "returncode": process.returncode,
     121        "stdout": _truncate_output(stdout, max_chars),
     122        "stderr": _truncate_output(stderr, max_chars),
     123    })
     124
     125
     126def cleanup_registered_shell_processes(home: str | Path) -> list[dict[str, Any]]:
     127    path = _shell_process_registry_path(home)
     128    records = _read_shell_process_registry(path)
     129    if not records:
     130        return []
     131    cleaned: list[dict[str, Any]] = []
     132    survivors: list[dict[str, Any]] = []
     133    for record in records:
     134        pid = _as_int(record.get("pid"))
     135        if pid <= 0:
     136            continue
     137        if not _pid_exists(pid):
     138            continue
     139        try:
     140            os.killpg(pid, signal.SIGTERM)
     141        except ProcessLookupError:
     142            continue
     143        except PermissionError:
     144            survivors.append(record)
     145            continue
     146        time.sleep(0.05)
     147        if _pid_exists(pid):
     148            with contextlib.suppress(ProcessLookupError, PermissionError):
     149                os.killpg(pid, signal.SIGKILL)
     150        record = dict(record)
     151        record["cleaned_at"] = datetime.now(timezone.utc).isoformat()
     152        cleaned.append(record)
     153    _write_shell_process_registry(path, survivors)
     154    return cleaned
     155
     156
     157def _shell_error(returncode: int | None, stdout: str, stderr: str, *, command: str = "") -> str:
     158    if returncode == 0:
     159        return _shell_success_anomaly(stdout, stderr, command=command)
     160    combined = "\n".join(part.strip() for part in (stderr, stdout) if part and part.strip())
     161    lowered = combined.lower()
     162    if "sudo:" in lowered and ("password" in lowered or "terminal is required" in lowered):
     163        return "command requires interactive sudo/password; configure non-interactive privileges or choose a non-sudo path"
     164    if "permission denied" in lowered:
     165        return "command failed with permission denied"
     166    missing_probe = _missing_executable_probe(command, combined)
     167    if missing_probe:
     168        return f"command probe found no executable: {missing_probe}"
     169    excerpt = " ".join(combined.split())[:500] if combined else "no output"
     170    return f"command exited with status {returncode}: {excerpt}"
     171
     172
     173def _shell_success_anomaly(stdout: str, stderr: str, *, command: str = "") -> str:
     174    combined = "\n".join(part.strip() for part in (stderr, stdout) if part and part.strip())
     175    if not combined:
     176        empty_probe = _empty_observation_probe(command)
     177        if empty_probe:
     178            return f"command probe produced no output despite exit status 0: {empty_probe}"
     179        return ""
     180    lowered = combined.lower()
     181    auth_markers = (
     182        "401 unauthorized",
     183        "403 forbidden",
     184        "authentication failed",
     185        "username/password authentication failed",
     186        "invalid username or password",
     187        "permission denied",
     188    )
     189    if _shell_sudo_password_anomaly(lowered):
     190        return "command output indicates interactive sudo/password requirement despite exit status 0"
     191    if any(marker in lowered for marker in auth_markers):
     192        excerpt = " ".join(combined.split())[:500]
     193        return f"command output indicates authentication or authorization failure despite exit status 0: {excerpt}"
     194    missing_probe = _missing_executable_probe(command, combined)
     195    if missing_probe:
     196        return f"command probe found no executable despite exit status 0: {missing_probe}"
     197    command_missing_match = _shell_missing_command_anomaly(combined)
     198    if command_missing_match:
     199        excerpt = " ".join(combined.split())[:500]
     200        return f"command output indicates missing command despite exit status 0: {excerpt}"
     201    build_error_match = _shell_build_error_anomaly(combined)
     202    if build_error_match:
     203        excerpt = " ".join(combined.split())[:500]
     204        return f"command output indicates build/tool failure despite exit status 0: {excerpt}"
     205    http_error_match = _shell_http_error_anomaly(lowered)
     206    if http_error_match:
     207        excerpt = " ".join(combined.split())[:500]
     208        return f"command output indicates HTTP failure despite exit status 0: {excerpt}"
     209    return ""
     210
     211
     212def _missing_executable_probe(command: str, combined_output: str) -> str:
     213    text = str(command or "").strip()
     214    match = re.match(r"^(?:which|command\s+-v)\s+([A-Za-z0-9_.+-]+)(?:\s|$)", text)
     215    if not match:
     216        return ""
     217    if not combined_output.strip() or "not found" in combined_output.lower():
     218        return match.group(1)
     219    return ""
     220
     221
     222def _empty_observation_probe(command: str) -> str:
     223    text = str(command or "").strip()
     224    if re.match(r"^(?:which|command\s+-v)\s+([A-Za-z0-9_.+-]+)(?:\s|$)", text):
     225        return "probe found no executable: executable lookup returned no path"
     226    if re.match(r"^(?:find|ls|stat|file)\b", text):
     227        return "read-only filesystem probe returned no observation"
     228    return ""
     229
     230
     231def _shell_missing_command_anomaly(text: str) -> bool:
     232    return bool(
     233        re.search(
     234            r"(?im)(?:^|\n)(?:/bin/sh:\s*\d+:\s*)?(?:(?:/|~)[^\s:'\"]+|[A-Za-z0-9_.+-]+):\s*(?:command not found|not found)\s*$",
     235            text,
     236        )
     237    )
     238
     239
     240def _shell_sudo_password_anomaly(text: str) -> bool:
     241    return "sudo:" in text and ("password" in text or "terminal is required" in text)
     242
     243
     244def _shell_build_error_anomaly(text: str) -> bool:
     245    lowered = text.lower()
     246    if "no rule to make target" in lowered:
     247        return True
     248    if "***" in text and "stop." in lowered:
     249        return True
     250    return bool(re.search(r"(?im)^\s*(?:make(?:\[\d+\])?:\s*)?\*\*\* .*\bstop\.\s*$", text))
     251
     252
     253def _shell_http_error_anomaly(text: str) -> bool:
     254    return any(f" {code} " in f" {text} " for code in ("400", "401", "403", "404", "429", "500", "502", "503", "504")) and any(
     255        marker in text for marker in ("http", "error", "unauthorized", "forbidden", "not found", "too many requests")
     256    )
     257
     258
     259def _terminate_process_group(process: subprocess.Popen[str]) -> None:
     260    try:
     261        os.killpg(process.pid, signal.SIGTERM)
     262    except ProcessLookupError:
     263        return
     264
     265
     266def _kill_process_group(process: subprocess.Popen[str]) -> None:
     267    try:
     268        os.killpg(process.pid, signal.SIGKILL)
     269    except ProcessLookupError:
     270        return
     271
     272
     273def _register_shell_process(ctx: Any, process: subprocess.Popen[str], *, command: str, cwd: str, timeout_seconds: float) -> None:
     274    path = _shell_process_registry_path(ctx.config.runtime.home)
     275    path.parent.mkdir(parents=True, exist_ok=True)
     276    record = {
     277        "pid": process.pid,
     278        "pgid": process.pid,
     279        "job_id": getattr(ctx, "job_id", ""),
     280        "run_id": getattr(ctx, "run_id", "") or "",
     281        "step_id": getattr(ctx, "step_id", "") or "",
     282        "command": command[:1000],
     283        "cwd": cwd,
     284        "timeout_seconds": timeout_seconds,
     285        "started_at": datetime.now(timezone.utc).isoformat(),
     286    }
     287    with path.open("a", encoding="utf-8") as handle:
     288        handle.write(json.dumps(record, ensure_ascii=False) + "\n")
     289
     290
     291def _unregister_shell_process(ctx: Any, pid: int) -> None:
     292    path = _shell_process_registry_path(ctx.config.runtime.home)
     293    records = [record for record in _read_shell_process_registry(path) if _as_int(record.get("pid")) != pid]
     294    _write_shell_process_registry(path, records)
     295
     296
     297def _shell_process_registry_path(home: str | Path) -> Path:
     298    return Path(home).expanduser() / "runtime" / "shell_processes.jsonl"
     299
     300
     301def _read_shell_process_registry(path: Path) -> list[dict[str, Any]]:
     302    if not path.exists():
     303        return []
     304    records: list[dict[str, Any]] = []
     305    for line in path.read_text(encoding="utf-8").splitlines():
     306        try:
     307            record = json.loads(line)
     308        except json.JSONDecodeError:
     309            continue
     310        if isinstance(record, dict):
     311            records.append(record)
     312    return records
     313
     314
     315def _write_shell_process_registry(path: Path, records: list[dict[str, Any]]) -> None:
     316    path.parent.mkdir(parents=True, exist_ok=True)
     317    if not records:
     318        with contextlib.suppress(FileNotFoundError):
     319            path.unlink()
     320        return
     321    path.write_text("".join(json.dumps(record, ensure_ascii=False) + "\n" for record in records), encoding="utf-8")
     322
     323
     324def _pid_exists(pid: int) -> bool:
     325    try:
     326        os.kill(pid, 0)
     327        return True
     328    except ProcessLookupError:
     329        return False
     330
     331
     332def _as_int(value: Any) -> int:
     333    try:
     334        return int(value)
     335    except (TypeError, ValueError):
     336        return 0
     337
     338
     339def _truncate_output(value: Any, max_chars: int) -> str:
     340    text = value.decode("utf-8", errors="replace") if isinstance(value, bytes) else str(value or "")
     341    if len(text) <= max_chars:
     342        return text
     343    omitted = len(text) - max_chars
     344    return text[:max_chars] + f"\n... truncated {omitted} chars ..."
     345
     346
     347def _json(value: Any) -> str:
     348    return json.dumps(value, ensure_ascii=False)

nipux_cli/source_quality.py 32 lines

       1"""Source quality checks for web and browser tools."""
       2
       3from __future__ import annotations
       4
       5ANTI_BOT_MARKERS = (
       6    "performing security verification",
       7    "cloudflare security challenge",
       8    "verifies you are not a bot",
       9    "verify you are not a bot",
      10    "enable javascript and cookies",
      11    "checking your browser before accessing",
      12    "just a moment...",
      13    "you have been blocked",
      14    "browsing and clicking at a speed much faster than expected",
      15    "there is a robot on the same network",
      16)
      17
      18
      19def anti_bot_reason(*parts: str) -> str | None:
      20    """Return a short reason if text looks like an anti-bot interstitial."""
      21
      22    text = " ".join(part for part in parts if part).lower()
      23    if not text:
      24        return None
      25    for marker in ANTI_BOT_MARKERS:
      26        if marker in text:
      27            if "cloudflare" in text:
      28                return "cloudflare anti-bot challenge"
      29            if "captcha" in text or "you have been blocked" in text:
      30                return "captcha/anti-bot block"
      31            return "anti-bot challenge"
      32    return None

nipux_cli/task_match.py 102 lines

       1"""Task title matching helpers for long-running job queues."""
       2
       3from __future__ import annotations
       4
       5import re
       6from typing import Any
       7
       8TASK_MATCH_STOPWORDS = {
       9    "a",
      10    "an",
      11    "and",
      12    "as",
      13    "at",
      14    "by",
      15    "for",
      16    "from",
      17    "in",
      18    "into",
      19    "of",
      20    "on",
      21    "or",
      22    "the",
      23    "then",
      24    "to",
      25    "via",
      26    "with",
      27}
      28
      29
      30def task_key(parent: str, title: str) -> str:
      31    return re.sub(r"[^a-z0-9]+", "-", f"{parent}|{title}".lower()).strip("-")[:120]
      32
      33
      34def find_semantic_task_match(
      35    *,
      36    title: str,
      37    parent: str,
      38    tasks: list[dict[str, Any]],
      39    statuses: set[str] | None = None,
      40    min_score: float = 0.55,
      41) -> dict[str, Any] | None:
      42    incoming_title = str(title or "").strip()
      43    if not incoming_title:
      44        return None
      45    incoming_parent = str(parent or "").strip()
      46    incoming_key = task_key(incoming_parent, incoming_title)
      47    incoming_tokens = _task_tokens(incoming_title)
      48    if len(incoming_tokens) < 2:
      49        return None
      50    allowed_statuses = statuses or {"active", "open", "blocked"}
      51    best: dict[str, Any] | None = None
      52    best_score = 0.0
      53    for task in tasks:
      54        if not isinstance(task, dict):
      55            continue
      56        candidate_title = str(task.get("title") or "").strip()
      57        if not candidate_title:
      58            continue
      59        candidate_parent = str(task.get("parent") or "").strip()
      60        if incoming_parent and candidate_parent and incoming_parent != candidate_parent:
      61            continue
      62        candidate_key = str(task.get("key") or task_key(candidate_parent, candidate_title))
      63        if candidate_key == incoming_key:
      64            return None
      65        status = str(task.get("status") or "open").strip().lower().replace(" ", "_")
      66        if status not in allowed_statuses:
      67            continue
      68        candidate_tokens = _task_tokens(candidate_title)
      69        if len(candidate_tokens) < 2:
      70            continue
      71        score, overlap = _task_similarity(incoming_tokens, candidate_tokens)
      72        if overlap < 2 or score < min_score or score <= best_score:
      73            continue
      74        best_score = score
      75        best = {
      76            "task": task,
      77            "key": candidate_key,
      78            "title": candidate_title,
      79            "parent": candidate_parent,
      80            "status": status,
      81            "score": round(score, 3),
      82            "overlap": overlap,
      83        }
      84    return best
      85
      86
      87def _task_tokens(text: str) -> set[str]:
      88    tokens = {
      89        token
      90        for token in re.findall(r"[a-z0-9]+", str(text or "").lower())
      91        if len(token) > 1 and token not in TASK_MATCH_STOPWORDS
      92    }
      93    return tokens
      94
      95
      96def _task_similarity(left: set[str], right: set[str]) -> tuple[float, int]:
      97    overlap = len(left & right)
      98    if overlap <= 0:
      99        return 0.0, 0
     100    jaccard = overlap / max(1, len(left | right))
     101    containment = overlap / max(1, min(len(left), len(right)))
     102    return max(jaccard, containment), overlap

nipux_cli/templates.py 67 lines

       1"""Program templates for generic long-running jobs."""
       2
       3from __future__ import annotations
       4
       5
       6def program_for_job(*, kind: str, title: str, objective: str) -> str:
       7    kind = (kind or "generic").strip().lower()
       8    body = _TEMPLATES.get(kind, _generic_template)
       9    return body(title=title, objective=objective).strip() + "\n"
      10
      11
      12def _generic_template(*, title: str, objective: str) -> str:
      13    return f"""# {title}
      14
      15## Objective
      16
      17{objective}
      18
      19## Operating Rules
      20
      21- Work forever in bounded, resumable steps until the operator explicitly cancels or pauses the job.
      22- Treat useful results as checkpoints, not endings: save the result, create the next branch, and continue.
      23- Save important observations as artifacts.
      24- Use report_update for short progress notes or blocked-state notes.
      25- Use record_lesson when a source, mistake, operator preference, or strategy should affect future steps.
      26- Use record_source and record_findings when those tools are available so the job improves its ledgers over time.
      27- Use record_roadmap for broad work that needs milestones, feature groups, validation contracts, and roadmap-level checkpoints.
      28- Use record_milestone_validation to validate milestones from evidence and create follow-up tasks when validation fails or blocks.
      29- Use record_tasks to split broad objectives into durable branches with output contracts, acceptance criteria, required evidence, and stall behavior.
      30- Use record_experiment whenever a branch produces measured results, comparisons, benchmarks, scores, or optimization data.
      31- Use acknowledge_operator_context after incorporating or superseding active operator steering.
      32- Use browser and web tools first. Do not assume memory is exact unless it points to an artifact.
      33- Prefer quantity of attempts over one giant plan.
      34"""
      35
      36
      37def _research_paper_template(*, title: str, objective: str) -> str:
      38    return f"""# {title}
      39
      40## Objective
      41
      42{objective}
      43
      44## Research Rules
      45
      46- Save exact source URLs and extracted text snippets as artifacts.
      47- Keep a rolling citation map with claims, evidence, and open questions.
      48- Separate facts from hypotheses.
      49- Produce drafts only after evidence artifacts exist.
      50- Use report_update for brief progress, gap, or blocked-source notes.
      51- Use record_roadmap for the paper outline, evidence milestones, draft milestones, and validation checkpoints.
      52- Use record_milestone_validation when a section or draft milestone has enough evidence to judge.
      53- Use record_tasks to track source clusters, sections, and unresolved evidence gaps with output contracts and acceptance criteria.
      54- Use acknowledge_operator_context after incorporating or superseding active operator steering.
      55
      56## Step Loop
      57
      581. Search for one source cluster.
      592. Extract and save relevant evidence.
      603. Update the citation/evidence map.
      614. Write or improve one section when enough evidence exists.
      62"""
      63
      64
      65_TEMPLATES = {
      66    "research_paper": _research_paper_template,
      67}

nipux_cli/tools.py 2229 lines

       1"""Static tool registry for the Nipux agent."""
       2
       3from __future__ import annotations
       4
       5import json
       6import re
       7import time
       8from dataclasses import dataclass
       9from datetime import datetime, timedelta, timezone
      10from typing import Any, Callable
      11
      12from nipux_cli.artifacts import ArtifactStore
      13from nipux_cli.config import AppConfig
      14from nipux_cli.db import AgentDB
      15from nipux_cli.metric_format import format_metric_value
      16from nipux_cli.digest import send_digest_email
      17from nipux_cli.memory_graph import memory_graph_from_job, search_memory_graph
      18from nipux_cli.planning import initial_task_contract
      19from nipux_cli.shell_tools import shell_exec as _shell_exec
      20from nipux_cli.shell_tools import write_file as _write_file
      21from nipux_cli.task_match import find_semantic_task_match, task_key
      22
      23
      24@dataclass(frozen=True)
      25class ToolContext:
      26    config: AppConfig
      27    db: AgentDB
      28    artifacts: ArtifactStore
      29    job_id: str
      30    run_id: str | None = None
      31    step_id: str | None = None
      32    task_id: str | None = None
      33
      34
      35Handler = Callable[[dict[str, Any], ToolContext], str]
      36
      37EVIDENCE_OUTPUT_TERMS = {
      38    "audit",
      39    "checkpoint",
      40    "evidence",
      41    "extract",
      42    "extracted",
      43    "notes",
      44    "source",
      45    "sources",
      46}
      47DELIVERABLE_OUTPUT_TERMS = {
      48    "compiled",
      49    "deliverable",
      50    "draft",
      51    "final",
      52    "revision",
      53    "updated",
      54}
      55
      56
      57@dataclass(frozen=True)
      58class ToolSpec:
      59    name: str
      60    description: str
      61    parameters: dict[str, Any]
      62    handler: Handler
      63
      64    def as_openai_tool(self) -> dict[str, Any]:
      65        return {
      66            "type": "function",
      67            "function": {
      68                "name": self.name,
      69                "description": self.description,
      70                "parameters": self.parameters,
      71            },
      72        }
      73
      74
      75def _missing_argument(value: Any) -> bool:
      76    if value is None:
      77        return True
      78    if isinstance(value, str):
      79        stripped = value.strip()
      80        if not stripped:
      81            return True
      82        lowered = stripped.lower()
      83        if lowered in {"...", "…", "<...>", "{...}", "{{...}}", "placeholder", "todo", "tbd"}:
      84            return True
      85        if re.fullmatch(r"[.<{\[(\s]*\.{3,}[\s>}\])]*", stripped):
      86            return True
      87        return False
      88    if isinstance(value, (list, dict, tuple, set)):
      89        return not value
      90    return False
      91
      92
      93REFERENCE_LIKE_FIELD_PATTERN = re.compile(r"(?i)(?:^|_)(?:artifact|id|path|ref|source|url)(?:$|_)")
      94
      95
      96def _placeholder_argument(value: Any) -> bool:
      97    if _missing_argument(value):
      98        return True
      99    if not isinstance(value, str):
     100        return False
     101    stripped = value.strip().strip("'\"")
     102    if not stripped:
     103        return True
     104    if re.search(r"\s", stripped):
     105        return False
     106    return bool(re.search(r"(?:\.{3,}|…)$", stripped))
     107
     108
     109def _schema_placeholder_arguments(schema: dict[str, Any], value: Any, *, path: str = "") -> list[str]:
     110    schema_type = schema.get("type")
     111    placeholders: list[str] = []
     112    if schema_type == "object" and isinstance(value, dict):
     113        properties = schema.get("properties") if isinstance(schema.get("properties"), dict) else {}
     114        for name, child_schema in properties.items():
     115            if name not in value or not isinstance(child_schema, dict):
     116                continue
     117            child_path = f"{path}.{name}" if path else str(name)
     118            if REFERENCE_LIKE_FIELD_PATTERN.search(str(name)) and _placeholder_argument(value.get(name)):
     119                placeholders.append(child_path)
     120                continue
     121            placeholders.extend(_schema_placeholder_arguments(child_schema, value.get(name), path=child_path))
     122    elif schema_type == "array" and isinstance(value, list):
     123        item_schema = schema.get("items") if isinstance(schema.get("items"), dict) else {}
     124        for index, item in enumerate(value[:50]):
     125            placeholders.extend(_schema_placeholder_arguments(item_schema, item, path=f"{path}[{index}]"))
     126    return placeholders
     127
     128
     129def _schema_missing_arguments(schema: dict[str, Any], value: Any, *, path: str = "") -> list[str]:
     130    schema_type = schema.get("type")
     131    missing: list[str] = []
     132    if schema_type == "object" and isinstance(value, dict):
     133        properties = schema.get("properties") if isinstance(schema.get("properties"), dict) else {}
     134        for required in schema.get("required") or []:
     135            name = str(required)
     136            if _missing_argument(value.get(name)):
     137                missing.append(f"{path}.{name}" if path else name)
     138        for name, child_schema in properties.items():
     139            if name in value and isinstance(child_schema, dict) and not _missing_argument(value.get(name)):
     140                child_path = f"{path}.{name}" if path else str(name)
     141                missing.extend(_schema_missing_arguments(child_schema, value.get(name), path=child_path))
     142    elif schema_type == "array" and isinstance(value, list):
     143        item_schema = schema.get("items") if isinstance(schema.get("items"), dict) else {}
     144        for index, item in enumerate(value[:50]):
     145            missing.extend(_schema_missing_arguments(item_schema, item, path=f"{path}[{index}]"))
     146    return missing
     147
     148
     149REQUIRED_ARGUMENT_ALIASES: dict[str, dict[str, tuple[str, ...]]] = {
     150    "record_experiment": {
     151        "title": ("title", "name", "metric_name", "hypothesis", "result", "outcome"),
     152    },
     153}
     154
     155REQUIRED_ARGUMENT_GROUPS: dict[str, tuple[tuple[str, tuple[str, ...]], ...]] = {
     156    "read_artifact": (("artifact reference", ("artifact_id", "path", "title", "ref")),),
     157    "record_memory_graph": (("nodes or edges", ("nodes", "edges")),),
     158}
     159
     160
     161def _json(value: Any) -> str:
     162    return json.dumps(value, ensure_ascii=False)
     163
     164
     165def _write_artifact(args: dict[str, Any], ctx: ToolContext) -> str:
     166    content = str(args.get("content") or "")
     167    if not content:
     168        return _json({"success": False, "error": "content is required"})
     169    stored = ctx.artifacts.write_text(
     170        job_id=ctx.job_id,
     171        run_id=ctx.run_id,
     172        step_id=ctx.step_id,
     173        content=content,
     174        title=args.get("title"),
     175        summary=args.get("summary"),
     176        artifact_type=str(args.get("type") or "text"),
     177        metadata=args.get("metadata") if isinstance(args.get("metadata"), dict) else None,
     178    )
     179    return _json({
     180        "success": True,
     181        "artifact_id": stored.id,
     182        "path": str(stored.path),
     183        "sha256": stored.sha256,
     184    })
     185
     186
     187def _read_artifact(args: dict[str, Any], ctx: ToolContext) -> str:
     188    artifact_ref = str(args.get("artifact_id") or args.get("path") or args.get("title") or args.get("ref") or "")
     189    if not artifact_ref:
     190        return _json({"success": False, "error": "artifact_id is required"})
     191    resolved = _resolve_artifact_ref(ctx, artifact_ref)
     192    if not resolved:
     193        recent = _recent_artifact_refs(ctx)
     194        return _json({
     195            "success": False,
     196            "recoverable": True,
     197            "error": f"artifact not found: {artifact_ref}",
     198            "guidance": (
     199                "The requested artifact reference does not exist. Use one of the recent_artifacts refs, "
     200                "call search_artifacts with a concrete query, or continue from already observed evidence."
     201            ),
     202            "recent_artifacts": recent,
     203        })
     204    try:
     205        content = ctx.artifacts.read_text(resolved["id"])
     206    except (OSError, KeyError, ValueError) as exc:
     207        return _json({"success": False, "artifact_id": resolved["id"], "error": str(exc)})
     208    return _json({"success": True, "artifact_id": resolved["id"], "title": resolved.get("title"), "path": resolved.get("path"), "content": content})
     209
     210
     211def _recent_artifact_refs(ctx: ToolContext, limit: int = 8) -> list[dict[str, str]]:
     212    artifacts = ctx.db.list_artifacts(ctx.job_id, limit=limit)
     213    refs: list[dict[str, str]] = []
     214    for index, artifact in enumerate(artifacts, start=1):
     215        refs.append({
     216            "number": str(index),
     217            "id": str(artifact.get("id") or ""),
     218            "title": str(artifact.get("title") or ""),
     219            "path": str(artifact.get("path") or ""),
     220        })
     221    return refs
     222
     223
     224def _resolve_artifact_ref(ctx: ToolContext, artifact_ref: str) -> dict[str, Any] | None:
     225    ref = artifact_ref.strip().strip("'\"")
     226    if not ref:
     227        return None
     228    artifacts = ctx.db.list_artifacts(ctx.job_id, limit=250)
     229    for artifact in artifacts:
     230        if ref == artifact.get("id") or ref == str(artifact.get("path") or ""):
     231            return artifact
     232    if ref.isdigit():
     233        index = int(ref) - 1
     234        if 0 <= index < len(artifacts):
     235            return artifacts[index]
     236    lowered = ref.lower()
     237    for artifact in artifacts:
     238        title = str(artifact.get("title") or "").lower()
     239        if lowered == title:
     240            return artifact
     241    for artifact in artifacts:
     242        haystack = " ".join(str(artifact.get(key) or "") for key in ("title", "summary", "path")).lower()
     243        if lowered and lowered in haystack:
     244            return artifact
     245    return None
     246
     247
     248def _search_artifacts(args: dict[str, Any], ctx: ToolContext) -> str:
     249    query = str(args.get("query") or "")
     250    limit = int(args.get("limit") or 10)
     251    return _json({"success": True, "results": ctx.artifacts.search_text(job_id=ctx.job_id, query=query, limit=limit)})
     252
     253
     254def _update_job_state(args: dict[str, Any], ctx: ToolContext) -> str:
     255    status = str(args.get("status") or "").strip().lower()
     256    if status in {"paused", "cancelled", "completed", "failed"}:
     257        note = str(args.get("note") or "")
     258        follow_up_task = None
     259        if status == "completed":
     260            follow_up_task = _append_completion_audit_task(
     261                ctx,
     262                source="update_job_state",
     263                requested_status=status,
     264                claimed_message=note,
     265            )
     266        metadata = {"requested_status": status, "kept_running": True}
     267        if follow_up_task is not None:
     268            metadata["follow_up_task"] = follow_up_task.get("key")
     269        ctx.db.append_agent_update(
     270            ctx.job_id,
     271            f"Worker requested {status}; job remains running. {note}".strip(),
     272            category="progress" if status == "completed" else "blocked",
     273            metadata=metadata,
     274        )
     275        result = {
     276            "success": True,
     277            "job_id": ctx.job_id,
     278            "status": "running",
     279            "requested_status": status,
     280            "kept_running": True,
     281            "guidance": (
     282                "Jobs are perpetual by default. Do not mark the job complete or failed. "
     283                "Save the current result, create follow-up tasks, report a checkpoint, and continue."
     284            ),
     285        }
     286        if follow_up_task is not None:
     287            result["follow_up_task"] = follow_up_task
     288        return _json(result)
     289    if status not in {"queued", "running"}:
     290        return _json({"success": False, "error": f"invalid status: {status}"})
     291    note = str(args.get("note") or "")
     292    patch = {"last_note": note} if note else None
     293    ctx.db.update_job_status(ctx.job_id, status, metadata_patch=patch)
     294    return _json({"success": True, "job_id": ctx.job_id, "status": status})
     295
     296
     297def _defer_job(args: dict[str, Any], ctx: ToolContext) -> str:
     298    until = _defer_until(args)
     299    reason = str(args.get("reason") or "").strip()
     300    next_action = str(args.get("next_action") or "").strip()
     301    patch = {
     302        "defer_until": until.isoformat(),
     303        "defer_reason": reason,
     304        "defer_next_action": next_action,
     305    }
     306    job = ctx.db.get_job(ctx.job_id)
     307    status = str(job.get("status") or "queued")
     308    if status not in {"queued", "running"}:
     309        status = "queued"
     310    ctx.db.update_job_status(ctx.job_id, status, metadata_patch=patch)
     311    message = f"Deferred until {until.isoformat()}"
     312    if reason:
     313        message += f": {reason}"
     314    if next_action:
     315        message += f" Next: {next_action}"
     316    ctx.db.append_agent_update(
     317        ctx.job_id,
     318        message,
     319        category="progress",
     320        metadata={"defer_until": until.isoformat(), "reason": reason, "next_action": next_action},
     321    )
     322    return _json({
     323        "success": True,
     324        "job_id": ctx.job_id,
     325        "status": status,
     326        "defer_until": until.isoformat(),
     327        "reason": reason,
     328        "next_action": next_action,
     329    })
     330
     331
     332def _defer_until(args: dict[str, Any]) -> datetime:
     333    raw_until = str(args.get("until") or "").strip()
     334    if raw_until:
     335        try:
     336            parsed = datetime.fromisoformat(raw_until.replace("Z", "+00:00"))
     337        except ValueError:
     338            parsed = datetime.now(timezone.utc)
     339        if parsed.tzinfo is None:
     340            parsed = parsed.replace(tzinfo=timezone.utc)
     341        return parsed.astimezone(timezone.utc)
     342    seconds = args.get("seconds", args.get("delay_seconds", 300))
     343    try:
     344        delay = max(1.0, float(seconds))
     345    except (TypeError, ValueError):
     346        delay = 300.0
     347    return datetime.now(timezone.utc) + timedelta(seconds=delay)
     348
     349
     350def _report_update(args: dict[str, Any], ctx: ToolContext) -> str:
     351    message = str(args.get("message") or args.get("summary") or "").strip()
     352    if not message:
     353        return _json({"success": False, "error": "message is required"})
     354    category = str(args.get("category") or "progress").strip().lower()
     355    metadata = args.get("metadata") if isinstance(args.get("metadata"), dict) else {}
     356    normalized_message = _perpetual_checkpoint_message(message)
     357    if normalized_message != message:
     358        metadata = {**metadata, "original_message": message, "rewritten_completion_claim": True}
     359        message = normalized_message
     360        follow_up_task = _append_completion_audit_task(
     361            ctx,
     362            source="report_update",
     363            requested_status="completed",
     364            claimed_message=str(metadata.get("original_message") or ""),
     365        )
     366        metadata["follow_up_task"] = follow_up_task.get("key")
     367    entry = ctx.db.append_agent_update(ctx.job_id, message, category=category, metadata=metadata)
     368    return _json({"success": True, "job_id": ctx.job_id, "update": entry})
     369
     370
     371def _append_completion_audit_task(
     372    ctx: ToolContext,
     373    *,
     374    source: str,
     375    requested_status: str,
     376    claimed_message: str = "",
     377) -> dict[str, Any]:
     378    return ctx.db.append_task_record(
     379        ctx.job_id,
     380        title="Audit latest checkpoint against objective",
     381        status="open",
     382        priority=7,
     383        goal=(
     384            "Before treating the latest checkpoint as sufficient, compare the objective and operator context "
     385            "against concrete artifacts, files, findings, measurements, validations, and task results."
     386        ),
     387        output_contract="decision",
     388        acceptance_criteria=(
     389            "A prompt-to-artifact checklist maps explicit requirements to evidence, identifies uncovered gaps, "
     390            "and opens or continues the next branch from those gaps."
     391        ),
     392        evidence_needed=(
     393            "Objective text, active operator context, latest durable outputs, recent tool/test results, "
     394            "task queue state, roadmap validations, and measured results when applicable."
     395        ),
     396        stall_behavior=(
     397            "If evidence is missing, mark the checkpoint incomplete, record the gap, and create the smallest "
     398            "follow-up task instead of claiming completion."
     399        ),
     400        metadata={
     401            "source": source,
     402            "requested_status": requested_status,
     403            "completion_audit_required": True,
     404            "claimed_message": claimed_message[:1000],
     405        },
     406    )
     407
     408
     409def _perpetual_checkpoint_message(message: str) -> str:
     410    """Keep worker reports checkpoint-oriented without hiding the underlying audit trail."""
     411
     412    text = " ".join(str(message or "").split())
     413    if not text:
     414        return ""
     415    leading_claim = re.compile(
     416        r"(?i)^\s*(?:the\s+)?(?:job|objective|run|work)\s+"
     417        r"(?:is\s+|was\s+)?(?:complete|completed|done|finished)\b[.!:,\-\s]*"
     418    )
     419    if leading_claim.search(text):
     420        rest = leading_claim.sub("", text, count=1).strip()
     421        if rest:
     422            return f"Checkpoint reported; continuing work. {rest}"
     423        return "Checkpoint reported; continuing work."
     424    whole_job_claim = re.compile(
     425        r"(?i)\b(?:completed|finished|done\s+with)\s+(?:the\s+)?(?:job|objective|run|work)\b"
     426    )
     427    if whole_job_claim.search(text):
     428        return "Checkpoint reported; continuing work. " + whole_job_claim.sub("reached a checkpoint for the work", text, count=1)
     429    return text
     430
     431
     432def _record_lesson(args: dict[str, Any], ctx: ToolContext) -> str:
     433    lesson = str(args.get("lesson") or args.get("memory") or "").strip()
     434    if not lesson:
     435        return _json({"success": False, "error": "lesson is required"})
     436    category = str(args.get("category") or "memory").strip().lower()
     437    confidence_arg = args.get("confidence")
     438    confidence = float(confidence_arg) if isinstance(confidence_arg, (int, float)) else None
     439    metadata = args.get("metadata") if isinstance(args.get("metadata"), dict) else {}
     440    pending_measurement = _pending_measurement(ctx)
     441    measurement_resolution_category = category in {"constraint", "mistake", "strategy", "memory"}
     442    if pending_measurement and measurement_resolution_category and not _lesson_explains_measurement_obligation(lesson, metadata):
     443        return _json({
     444            "success": False,
     445            "error": "measurement explanation required",
     446            "message": (
     447                "A pending measurement must be resolved with record_experiment, a follow-up task, "
     448                "or a lesson that explicitly explains why the output is not a valid measurement."
     449            ),
     450            "pending_measurement_obligation": pending_measurement,
     451        })
     452    entry = ctx.db.append_lesson(ctx.job_id, lesson, category=category, confidence=confidence, metadata=metadata)
     453    if pending_measurement and measurement_resolution_category:
     454        _resolve_measurement_obligation(
     455            ctx,
     456            status="explained",
     457            reason=lesson,
     458            via_tool="record_lesson",
     459        )
     460    return _json({"success": True, "job_id": ctx.job_id, "lesson": entry})
     461
     462
     463def _lesson_explains_measurement_obligation(lesson: str, metadata: dict[str, Any]) -> bool:
     464    parts = [lesson]
     465    for value in metadata.values():
     466        if isinstance(value, (str, int, float, bool)):
     467            parts.append(str(value))
     468    text = " ".join(parts).lower()
     469    measurement_terms = (
     470        "measure",
     471        "measured",
     472        "measurement",
     473        "metric",
     474        "experiment",
     475        "trial",
     476        "benchmark",
     477        "result",
     478        "obligation",
     479    )
     480    accounting_terms = (
     481        "invalid",
     482        "valid",
     483        "not valid",
     484        "diagnostic",
     485        "missing",
     486        "no metric",
     487        "without metric",
     488        "blocked",
     489        "failed",
     490        "failure",
     491        "timeout",
     492        "permission",
     493        "auth",
     494        "quota",
     495        "rate limit",
     496        "unavailable",
     497        "unable",
     498        "cannot",
     499        "can't",
     500        "could not",
     501        "stale",
     502        "incomplete",
     503        "not comparable",
     504        "not enough",
     505        "rerun",
     506        "re-run",
     507        "retry",
     508    )
     509    return any(term in text for term in measurement_terms) and any(term in text for term in accounting_terms)
     510
     511
     512def _record_memory_graph(args: dict[str, Any], ctx: ToolContext) -> str:
     513    nodes = args.get("nodes") if isinstance(args.get("nodes"), list) else []
     514    edges = args.get("edges") if isinstance(args.get("edges"), list) else []
     515    if not nodes and not edges:
     516        return _json({"success": False, "error": "nodes or edges are required"})
     517    record = ctx.db.append_memory_graph_records(ctx.job_id, nodes=nodes, edges=edges)
     518    ctx.db.append_agent_update(
     519        ctx.job_id,
     520        (
     521            "Memory graph updated: "
     522            f"{record.get('added_nodes')} new nodes, {record.get('updated_nodes')} updated, "
     523            f"{record.get('added_edges')} new links."
     524        ),
     525        category="progress",
     526        metadata={
     527            "memory_graph_event_id": record.get("event_id"),
     528            "added_nodes": record.get("added_nodes"),
     529            "updated_nodes": record.get("updated_nodes"),
     530            "added_edges": record.get("added_edges"),
     531        },
     532    )
     533    return _json({"success": True, "job_id": ctx.job_id, **record})
     534
     535
     536def _search_memory_graph(args: dict[str, Any], ctx: ToolContext) -> str:
     537    query = str(args.get("query") or "")
     538    limit = int(args.get("limit") or 10)
     539    job = ctx.db.get_job(ctx.job_id)
     540    graph = memory_graph_from_job(job)
     541    results = search_memory_graph(graph, query=query, limit=limit)
     542    return _json({"success": True, "job_id": ctx.job_id, "query": query, **results})
     543
     544
     545def _acknowledge_operator_context(args: dict[str, Any], ctx: ToolContext) -> str:
     546    raw_ids = args.get("message_ids")
     547    message_ids = [str(item) for item in raw_ids] if isinstance(raw_ids, list) else []
     548    summary = str(args.get("summary") or args.get("reason") or "").strip()
     549    status = str(args.get("status") or "acknowledged").strip().lower()
     550    pending = _acknowledgeable_operator_messages(ctx.db.get_job(ctx.job_id), message_ids=message_ids)
     551    if not pending:
     552        return _json({
     553            "success": False,
     554            "recoverable": True,
     555            "error": "no active operator context to acknowledge",
     556            "message_ids": message_ids,
     557            "guidance": "Use acknowledge_operator_context only after incorporating claimed operator steering. Use report_update, record_lesson, record_tasks, or record_experiment for ordinary progress.",
     558        })
     559    result = ctx.db.acknowledge_operator_messages(
     560        ctx.job_id,
     561        message_ids=message_ids,
     562        summary=summary,
     563        status=status,
     564    )
     565    message = summary or f"Operator context {result.get('status')}."
     566    ctx.db.append_agent_update(
     567        ctx.job_id,
     568        message,
     569        category="progress",
     570        metadata={
     571            "operator_context_status": result.get("status"),
     572            "operator_message_count": result.get("count"),
     573            "operator_message_ids": [
     574                entry.get("event_id")
     575                for entry in result.get("messages", [])
     576                if isinstance(entry, dict) and entry.get("event_id")
     577            ],
     578        },
     579    )
     580    return _json({"success": True, "job_id": ctx.job_id, **result})
     581
     582
     583def _acknowledgeable_operator_messages(job: dict[str, Any], *, message_ids: list[str]) -> list[dict[str, Any]]:
     584    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     585    messages = metadata.get("operator_messages") if isinstance(metadata.get("operator_messages"), list) else []
     586    wanted = {str(message_id).strip() for message_id in message_ids if str(message_id).strip()}
     587    pending = []
     588    for entry in messages:
     589        if not isinstance(entry, dict):
     590            continue
     591        mode = str(entry.get("mode") or "steer").strip().lower().replace("-", "_")
     592        if mode not in {"steer", "follow_up"}:
     593            continue
     594        event_id = str(entry.get("event_id") or "")
     595        if wanted and event_id not in wanted:
     596            continue
     597        if not wanted and not entry.get("claimed_at"):
     598            continue
     599        if entry.get("acknowledged_at") or entry.get("superseded_at"):
     600            continue
     601        pending.append(entry)
     602    return pending
     603
     604
     605def _record_source(args: dict[str, Any], ctx: ToolContext) -> str:
     606    source = str(args.get("source") or args.get("url") or args.get("domain") or "").strip()
     607    if not source:
     608        return _json({"success": False, "error": "source is required"})
     609    warnings_raw = args.get("warnings")
     610    warnings = [str(item) for item in warnings_raw] if isinstance(warnings_raw, list) else []
     611    score_arg = args.get("usefulness_score")
     612    usefulness_score = float(score_arg) if isinstance(score_arg, (int, float)) else None
     613    yield_count = int(args.get("yield_count") or 0)
     614    fail_count_delta = int(args.get("fail_count_delta") or 0)
     615    metadata = args.get("metadata") if isinstance(args.get("metadata"), dict) else {}
     616    source_type = str(args.get("source_type") or "")
     617    outcome = str(args.get("outcome") or "")
     618    if not _source_has_assessment(
     619        source_type=source_type,
     620        usefulness_score=usefulness_score,
     621        yield_count=yield_count,
     622        fail_count_delta=fail_count_delta,
     623        warnings=warnings,
     624        outcome=outcome,
     625        metadata=metadata,
     626    ):
     627        return _json({
     628            "success": False,
     629            "error": "source assessment is required",
     630            "guidance": (
     631                "record_source must say why the source matters: include usefulness_score, "
     632                "yield_count, fail_count_delta, warnings, outcome, or evidence metadata."
     633            ),
     634        })
     635    entry = ctx.db.append_source_record(
     636        ctx.job_id,
     637        source,
     638        source_type=source_type,
     639        usefulness_score=usefulness_score,
     640        yield_count=yield_count,
     641        fail_count_delta=fail_count_delta,
     642        warnings=warnings,
     643        outcome=outcome,
     644        metadata=metadata,
     645    )
     646    return _json({"success": True, "job_id": ctx.job_id, "source": entry})
     647
     648
     649def _source_has_assessment(
     650    *,
     651    source_type: str,
     652    usefulness_score: float | None,
     653    yield_count: int,
     654    fail_count_delta: int,
     655    warnings: list[str],
     656    outcome: str,
     657    metadata: dict[str, Any],
     658) -> bool:
     659    return bool(
     660        usefulness_score is not None
     661        or yield_count
     662        or fail_count_delta
     663        or warnings
     664        or outcome.strip()
     665        or any(str(value).strip() for value in metadata.values())
     666    )
     667
     668
     669def _record_findings(args: dict[str, Any], ctx: ToolContext) -> str:
     670    raw_findings = args.get("findings")
     671    if isinstance(raw_findings, list):
     672        findings = [item for item in raw_findings if isinstance(item, dict)]
     673    else:
     674        findings = [args]
     675    if not findings:
     676        return _json({"success": False, "error": "findings are required"})
     677    evidence_artifact = str(args.get("evidence_artifact") or args.get("artifact_id") or "")
     678    stored = []
     679    added = 0
     680    updated = 0
     681    unchanged = 0
     682    rejected = []
     683    source_yields: dict[str, int] = {}
     684    for finding in findings[:50]:
     685        name = str(finding.get("name") or finding.get("title") or "").strip()
     686        if not name:
     687            continue
     688        source_url = str(finding.get("source_url") or finding.get("source") or args.get("source_url") or args.get("source") or "")
     689        reason = str(finding.get("reason") or finding.get("rationale") or "")
     690        finding_evidence_artifact = str(finding.get("evidence_artifact") or evidence_artifact)
     691        metadata = finding.get("metadata") if isinstance(finding.get("metadata"), dict) else {}
     692        if not _finding_has_evidence(
     693            url=str(finding.get("url") or ""),
     694            source_url=source_url,
     695            reason=reason,
     696            evidence_artifact=finding_evidence_artifact,
     697            metadata=metadata,
     698        ):
     699            rejected.append({"name": name, "reason": "missing_evidence"})
     700            continue
     701        score_arg = finding.get("score")
     702        score = float(score_arg) if isinstance(score_arg, (int, float)) else None
     703        entry = ctx.db.append_finding_record(
     704            ctx.job_id,
     705            name=name,
     706            url=str(finding.get("url") or ""),
     707            source_url=source_url,
     708            category=str(finding.get("category") or finding.get("type") or ""),
     709            location=str(finding.get("location") or ""),
     710            contact=str(finding.get("contact") or ""),
     711            reason=reason,
     712            status=str(finding.get("status") or "new"),
     713            score=score,
     714            evidence_artifact=finding_evidence_artifact,
     715            metadata=metadata,
     716        )
     717        if entry.get("created"):
     718            added += 1
     719            if source_url:
     720                source_yields[source_url] = source_yields.get(source_url, 0) + 1
     721        elif entry.get("substantive_update"):
     722            updated += 1
     723        else:
     724            unchanged += 1
     725        stored.append(entry)
     726    if not stored:
     727        return _json({
     728            "success": False,
     729            "error": "no valid finding with name/title and evidence was provided",
     730            "rejected": rejected,
     731            "guidance": (
     732                "Each finding must include an evidence anchor such as source_url/url, reason/rationale, "
     733                "evidence_artifact, or evidence metadata. Use record_tasks or record_source for unevidenced candidates."
     734            ),
     735        })
     736    source_records = []
     737    for source_url, count in source_yields.items():
     738        score = round(min(1.0, 0.55 + min(count, 10) * 0.04), 2)
     739        source_records.append(
     740            ctx.db.append_source_record(
     741                ctx.job_id,
     742                source_url,
     743                source_type=str(args.get("source_type") or "finding_source"),
     744                usefulness_score=score,
     745                yield_count=count,
     746                outcome=f"record_findings yielded {count} new candidate(s)",
     747                metadata={"auto_from_record_findings": True, "evidence_artifact": evidence_artifact},
     748            )
     749        )
     750    if added or updated or source_records:
     751        message = f"Finding ledger updated: {added} new, {updated} changed. Source ledger updated: {len(source_records)}."
     752        if unchanged:
     753            message += f" {unchanged} unchanged."
     754        ctx.db.append_agent_update(
     755            ctx.job_id,
     756            message,
     757            category="finding",
     758            metadata={
     759                "added": added,
     760                "updated": updated,
     761                "unchanged": unchanged,
     762                "rejected": len(rejected),
     763                "sources_updated": len(source_records),
     764            },
     765        )
     766    return _json({
     767        "success": True,
     768        "job_id": ctx.job_id,
     769        "added": added,
     770        "updated": updated,
     771        "unchanged": unchanged,
     772        "rejected": rejected,
     773        "sources_updated": len(source_records),
     774        "sources": source_records,
     775        "findings": stored,
     776    })
     777
     778
     779def _finding_has_evidence(
     780    *,
     781    url: str,
     782    source_url: str,
     783    reason: str,
     784    evidence_artifact: str,
     785    metadata: dict[str, Any],
     786) -> bool:
     787    if url.strip() or source_url.strip() or reason.strip() or evidence_artifact.strip():
     788        return True
     789    evidence_keys = {
     790        "artifact_id",
     791        "evidence_artifact",
     792        "experiment_key",
     793        "file_path",
     794        "output_path",
     795        "source_id",
     796        "source_url",
     797        "step_id",
     798    }
     799    return any(str(metadata.get(key) or "").strip() for key in evidence_keys)
     800
     801
     802def _record_tasks(args: dict[str, Any], ctx: ToolContext) -> str:
     803    raw_tasks = args.get("tasks")
     804    if isinstance(raw_tasks, list):
     805        tasks = [item for item in raw_tasks if isinstance(item, dict)]
     806    else:
     807        tasks = [args]
     808    if not tasks:
     809        return _json({"success": False, "error": "tasks are required"})
     810
     811    pending_measurement = _pending_measurement(ctx)
     812    task_queue_pressure = _task_queue_pressure_active(ctx)
     813    prepared_tasks: list[dict[str, Any]] = []
     814    for task in tasks[:50]:
     815        title = str(task.get("title") or task.get("name") or "").strip()
     816        if not title:
     817            continue
     818        parent = str(task.get("parent") or "")
     819        metadata = task.get("metadata") if isinstance(task.get("metadata"), dict) else {}
     820        if task_queue_pressure:
     821            match = _semantic_task_match_under_pressure(ctx, title=title, parent=parent)
     822            if match:
     823                metadata = dict(metadata)
     824                metadata.setdefault("original_title", title)
     825                metadata.setdefault(
     826                    "matched_existing_task",
     827                    {
     828                        "key": match.get("key"),
     829                        "title": match.get("title"),
     830                        "score": match.get("score"),
     831                        "overlap": match.get("overlap"),
     832                    },
     833                )
     834                title = str(match.get("title") or title)
     835                parent = str(match.get("parent") or parent)
     836        output_contract = str(
     837            task.get("output_contract")
     838            or task.get("contract")
     839            or metadata.get("output_contract")
     840            or metadata.get("contract")
     841            or ""
     842        )
     843        acceptance_criteria = str(task.get("acceptance_criteria") or "")
     844        evidence_needed = str(task.get("evidence_needed") or "")
     845        stall_behavior = str(task.get("stall_behavior") or "")
     846        output_contract, acceptance_criteria, evidence_needed, stall_behavior, metadata = _complete_task_contract(
     847            title=title,
     848            output_contract=output_contract,
     849            acceptance_criteria=acceptance_criteria,
     850            evidence_needed=evidence_needed,
     851            stall_behavior=stall_behavior,
     852            metadata=metadata,
     853        )
     854        goal = str(task.get("goal") or task.get("description") or "")
     855        source_hint = str(task.get("source_hint") or task.get("source") or "")
     856        result_text = str(task.get("result") or task.get("outcome") or "")
     857        priority_arg = task.get("priority")
     858        priority = int(priority_arg) if isinstance(priority_arg, (int, float)) else 0
     859        status = str(task.get("status") or "open")
     860        if pending_measurement and not _task_targets_measurement_obligation(
     861            title=title,
     862            goal=goal,
     863            source_hint=source_hint,
     864            result=result_text,
     865            output_contract=output_contract,
     866            acceptance_criteria=acceptance_criteria,
     867            evidence_needed=evidence_needed,
     868            stall_behavior=stall_behavior,
     869            metadata=metadata,
     870        ) and not _task_would_be_unchanged(
     871            ctx,
     872            title=title,
     873            status=status,
     874            priority=priority,
     875            goal=goal,
     876            source_hint=source_hint,
     877            result=result_text,
     878            parent=parent,
     879            output_contract=output_contract,
     880            acceptance_criteria=acceptance_criteria,
     881            evidence_needed=evidence_needed,
     882            stall_behavior=stall_behavior,
     883            metadata=metadata,
     884        ):
     885            return _json({
     886                "success": False,
     887                "error": "measurement task required",
     888                "message": (
     889                    "A pending measurement can only be deferred by a task that explicitly obtains, "
     890                    "repairs, validates, or accounts for that measurement."
     891                ),
     892                "rejected_task": title,
     893                "pending_measurement_obligation": pending_measurement,
     894            })
     895        prepared_tasks.append({
     896            "task": task,
     897            "title": title,
     898            "goal": goal,
     899            "source_hint": source_hint,
     900            "result_text": result_text,
     901            "parent": parent,
     902            "priority": priority,
     903            "status": status,
     904            "metadata": metadata,
     905            "output_contract": output_contract,
     906            "acceptance_criteria": acceptance_criteria,
     907            "evidence_needed": evidence_needed,
     908            "stall_behavior": stall_behavior,
     909        })
     910
     911    stored = []
     912    added = 0
     913    updated = 0
     914    unchanged = 0
     915    for prepared in prepared_tasks:
     916        task = prepared["task"]
     917        title = prepared["title"]
     918        status = prepared["status"]
     919        metadata = prepared["metadata"]
     920        output_contract = prepared["output_contract"]
     921        acceptance_criteria = prepared["acceptance_criteria"]
     922        evidence_needed = prepared["evidence_needed"]
     923        stall_behavior = prepared["stall_behavior"]
     924        result_text = prepared["result_text"]
     925        status, metadata = _validated_task_status(
     926            ctx,
     927            status=status,
     928            output_contract=output_contract,
     929            result=result_text,
     930            metadata=metadata,
     931        )
     932        entry = ctx.db.append_task_record(
     933            ctx.job_id,
     934            title=title,
     935            status=status,
     936            priority=prepared["priority"],
     937            goal=prepared["goal"],
     938            source_hint=prepared["source_hint"],
     939            result=result_text,
     940            parent=prepared["parent"],
     941            output_contract=output_contract,
     942            acceptance_criteria=acceptance_criteria,
     943            evidence_needed=evidence_needed,
     944            stall_behavior=stall_behavior,
     945            metadata=metadata,
     946        )
     947        if entry.get("created"):
     948            added += 1
     949        elif entry.get("substantive_update"):
     950            updated += 1
     951        else:
     952            unchanged += 1
     953        stored.append(entry)
     954    if not stored:
     955        return _json({"success": False, "error": "no valid task with title/name was provided"})
     956
     957    if added or updated:
     958        message = f"Task queue updated: {added} new, {updated} changed."
     959        if unchanged:
     960            message += f" {unchanged} unchanged."
     961        ctx.db.append_agent_update(
     962            ctx.job_id,
     963            message,
     964            category="plan",
     965            metadata={"added": added, "updated": updated, "unchanged": unchanged},
     966        )
     967    if (added or updated) and pending_measurement:
     968        _resolve_measurement_obligation(
     969            ctx,
     970            status="deferred",
     971            reason="Created or updated task branch to obtain or handle the pending measurement.",
     972            via_tool="record_tasks",
     973        )
     974    return _json({"success": True, "job_id": ctx.job_id, "added": added, "updated": updated, "unchanged": unchanged, "tasks": stored})
     975
     976
     977def _task_targets_measurement_obligation(
     978    *,
     979    title: str,
     980    goal: str,
     981    source_hint: str,
     982    result: str,
     983    output_contract: str,
     984    acceptance_criteria: str,
     985    evidence_needed: str,
     986    stall_behavior: str,
     987    metadata: dict[str, Any],
     988) -> bool:
     989    text = " ".join([
     990        title,
     991        goal,
     992        source_hint,
     993        result,
     994        output_contract,
     995        acceptance_criteria,
     996        evidence_needed,
     997        stall_behavior,
     998        _metadata_scalar_text(metadata),
     999    ]).lower()
    1000    contract = output_contract.strip().lower()
    1001    if contract in {"experiment", "monitor"} and _text_mentions_measurement(text):
    1002        return True
    1003    return _text_mentions_measurement(text) and _text_mentions_measurement_accounting(text)
    1004
    1005
    1006def _task_would_be_unchanged(
    1007    ctx: ToolContext,
    1008    *,
    1009    title: str,
    1010    status: str,
    1011    priority: int,
    1012    goal: str,
    1013    source_hint: str,
    1014    result: str,
    1015    parent: str,
    1016    output_contract: str,
    1017    acceptance_criteria: str,
    1018    evidence_needed: str,
    1019    stall_behavior: str,
    1020    metadata: dict[str, Any],
    1021) -> bool:
    1022    try:
    1023        job = ctx.db.get_job(ctx.job_id)
    1024    except KeyError:
    1025        return False
    1026    job_metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    1027    tasks = job_metadata.get("task_queue") if isinstance(job_metadata.get("task_queue"), list) else []
    1028    key = task_key(parent, title)
    1029    current = next(
    1030        (
    1031            entry
    1032            for entry in tasks
    1033            if isinstance(entry, dict)
    1034            and (
    1035                entry.get("key") == key
    1036                or (not entry.get("key") and task_key(str(entry.get("parent") or ""), str(entry.get("title") or "")) == key)
    1037            )
    1038        ),
    1039        None,
    1040    )
    1041    if not current:
    1042        return False
    1043    fields = (
    1044        "status",
    1045        "priority",
    1046        "goal",
    1047        "source_hint",
    1048        "result",
    1049        "parent",
    1050        "output_contract",
    1051        "acceptance_criteria",
    1052        "evidence_needed",
    1053        "stall_behavior",
    1054        "metadata",
    1055    )
    1056    before = _task_change_fingerprint(current, fields)
    1057    after = dict(current)
    1058    cleaned_status = (status.strip().lower() or "open").replace(" ", "_")
    1059    after["status"] = cleaned_status if cleaned_status in {"open", "active", "done", "blocked", "skipped"} else "open"
    1060    after["priority"] = int(priority)
    1061    for field, value in {
    1062        "goal": goal.strip(),
    1063        "source_hint": source_hint.strip(),
    1064        "result": result.strip(),
    1065        "parent": parent.strip(),
    1066        "output_contract": output_contract.strip().lower().replace(" ", "_"),
    1067        "acceptance_criteria": acceptance_criteria.strip(),
    1068        "evidence_needed": evidence_needed.strip(),
    1069        "stall_behavior": stall_behavior.strip(),
    1070    }.items():
    1071        if value:
    1072            after[field] = value
    1073    if metadata:
    1074        merged_metadata = after.get("metadata") if isinstance(after.get("metadata"), dict) else {}
    1075        merged_metadata = dict(merged_metadata)
    1076        merged_metadata.update(metadata)
    1077        after["metadata"] = merged_metadata
    1078    return before == _task_change_fingerprint(after, fields)
    1079
    1080
    1081def _task_change_fingerprint(entry: dict[str, Any], fields: tuple[str, ...]) -> str:
    1082    return json.dumps({field: entry.get(field) for field in fields}, sort_keys=True, separators=(",", ":"))
    1083
    1084
    1085def _task_queue_pressure_active(ctx: ToolContext) -> bool:
    1086    try:
    1087        job = ctx.db.get_job(ctx.job_id)
    1088    except KeyError:
    1089        return False
    1090    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    1091    tasks = metadata.get("task_queue") if isinstance(metadata.get("task_queue"), list) else []
    1092    objective_tasks = [task for task in tasks if not _task_is_guard_recovery(task)]
    1093    open_tasks = [
    1094        task
    1095        for task in objective_tasks
    1096        if str(task.get("status") or "open").strip().lower().replace(" ", "_") in {"open", "active"}
    1097    ]
    1098    return len(objective_tasks) > 80 or len(open_tasks) >= 40
    1099
    1100
    1101def _semantic_task_match_under_pressure(ctx: ToolContext, *, title: str, parent: str) -> dict[str, Any] | None:
    1102    try:
    1103        job = ctx.db.get_job(ctx.job_id)
    1104    except KeyError:
    1105        return None
    1106    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    1107    tasks = metadata.get("task_queue") if isinstance(metadata.get("task_queue"), list) else []
    1108    return find_semantic_task_match(
    1109        title=title,
    1110        parent=parent,
    1111        tasks=[task for task in tasks if not _task_is_guard_recovery(task)],
    1112    )
    1113
    1114
    1115def _task_is_guard_recovery(task: dict[str, Any]) -> bool:
    1116    metadata = task.get("metadata") if isinstance(task.get("metadata"), dict) else {}
    1117    return bool(metadata.get("guard_recovery")) or str(task.get("title") or "").strip().lower().startswith("resolve guard:")
    1118
    1119
    1120def _text_mentions_measurement(text: str) -> bool:
    1121    terms = (
    1122        "measure",
    1123        "measured",
    1124        "measurement",
    1125        "metric",
    1126        "experiment",
    1127        "trial",
    1128        "benchmark",
    1129        "result",
    1130        "obligation",
    1131    )
    1132    return any(term in text for term in terms)
    1133
    1134
    1135def _text_mentions_measurement_accounting(text: str) -> bool:
    1136    terms = (
    1137        "account",
    1138        "accounting",
    1139        "obtain",
    1140        "repair",
    1141        "validate",
    1142        "valid",
    1143        "invalid",
    1144        "diagnostic",
    1145        "missing",
    1146        "no metric",
    1147        "without metric",
    1148        "blocked",
    1149        "failed",
    1150        "failure",
    1151        "timeout",
    1152        "permission",
    1153        "auth",
    1154        "quota",
    1155        "rate limit",
    1156        "unavailable",
    1157        "unable",
    1158        "cannot",
    1159        "can't",
    1160        "could not",
    1161        "stale",
    1162        "incomplete",
    1163        "not comparable",
    1164        "not enough",
    1165        "rerun",
    1166        "re-run",
    1167        "retry",
    1168        "next measured",
    1169    )
    1170    return any(term in text for term in terms)
    1171
    1172
    1173def _metadata_scalar_text(metadata: dict[str, Any]) -> str:
    1174    parts = []
    1175    for value in metadata.values():
    1176        if isinstance(value, (str, int, float, bool)):
    1177            parts.append(str(value))
    1178    return " ".join(parts)
    1179
    1180
    1181def _complete_task_contract(
    1182    *,
    1183    title: str,
    1184    output_contract: str,
    1185    acceptance_criteria: str,
    1186    evidence_needed: str,
    1187    stall_behavior: str,
    1188    metadata: dict[str, Any],
    1189) -> tuple[str, str, str, str, dict[str, Any]]:
    1190    defaults = initial_task_contract(title)
    1191    inferred = []
    1192    if not output_contract.strip():
    1193        output_contract = defaults["output_contract"]
    1194        inferred.append("output_contract")
    1195    if not acceptance_criteria.strip():
    1196        acceptance_criteria = defaults["acceptance_criteria"]
    1197        inferred.append("acceptance_criteria")
    1198    if not evidence_needed.strip():
    1199        evidence_needed = defaults["evidence_needed"]
    1200        inferred.append("evidence_needed")
    1201    if not stall_behavior.strip():
    1202        stall_behavior = defaults["stall_behavior"]
    1203        inferred.append("stall_behavior")
    1204    if inferred:
    1205        metadata = dict(metadata)
    1206        existing = metadata.get("contract_inferred_fields")
    1207        existing_fields = [str(item) for item in existing] if isinstance(existing, list) else []
    1208        metadata["contract_inferred_fields"] = sorted(set(existing_fields + inferred))
    1209    return output_contract, acceptance_criteria, evidence_needed, stall_behavior, metadata
    1210
    1211
    1212def _validated_task_status(
    1213    ctx: ToolContext,
    1214    *,
    1215    status: str,
    1216    output_contract: str,
    1217    result: str,
    1218    metadata: dict[str, Any],
    1219) -> tuple[str, dict[str, Any]]:
    1220    normalized_status = status.strip().lower().replace(" ", "_") or "open"
    1221    contract = output_contract.strip().lower().replace(" ", "_")
    1222    if normalized_status == "done" and not result.strip() and not _task_metadata_has_completion_evidence(metadata, contract=contract):
    1223        updated = dict(metadata)
    1224        updated["completion_validation"] = "missing_result_evidence"
    1225        return "active", updated
    1226    if normalized_status != "done":
    1227        return status, metadata
    1228    if contract in {"artifact", "report"}:
    1229        if _recent_deliverable_evidence(ctx):
    1230            return status, metadata
    1231        updated = dict(metadata)
    1232        updated["completion_validation"] = "missing_recent_deliverable_evidence"
    1233        if result:
    1234            updated["claimed_result"] = result
    1235        return "active", updated
    1236    if _task_contract_completion_has_evidence(ctx, contract=contract, metadata=metadata):
    1237        return status, metadata
    1238    updated = dict(metadata)
    1239    updated["completion_validation"] = f"missing_{contract}_evidence" if contract else "missing_contract_evidence"
    1240    if result:
    1241        updated["claimed_result"] = result
    1242    return "active", updated
    1243
    1244
    1245def _task_contract_completion_has_evidence(ctx: ToolContext, *, contract: str, metadata: dict[str, Any]) -> bool:
    1246    if not contract or contract == "decision":
    1247        return True
    1248    if _task_metadata_has_completion_evidence(metadata, contract=contract):
    1249        return True
    1250    recent_evidence_tools = {
    1251        "research": {"record_source", "record_findings"},
    1252        "experiment": {"record_experiment"},
    1253        "action": {"shell_exec", "write_file", "write_artifact"},
    1254        "monitor": {"defer_job"},
    1255        "validation": {"record_milestone_validation", "shell_exec"},
    1256    }
    1257    tools = recent_evidence_tools.get(contract)
    1258    if not tools:
    1259        return True
    1260    for step in reversed(ctx.db.list_steps(job_id=ctx.job_id, limit=12)):
    1261        if step.get("id") == ctx.step_id or step.get("status") != "completed":
    1262            continue
    1263        tool_name = str(step.get("tool_name") or "")
    1264        if contract == "action" and tool_name == "shell_exec":
    1265            input_data = step.get("input") if isinstance(step.get("input"), dict) else {}
    1266            args = input_data.get("arguments") if isinstance(input_data.get("arguments"), dict) else {}
    1267            if _shell_command_counts_as_action_evidence(str(args.get("command") or "")):
    1268                return True
    1269            continue
    1270        if tool_name in tools:
    1271            return True
    1272    return False
    1273
    1274
    1275def _task_metadata_has_completion_evidence(metadata: dict[str, Any], *, contract: str = "") -> bool:
    1276    evidence_keys = {
    1277        "artifact_id",
    1278        "evidence_artifact",
    1279        "experiment_key",
    1280        "file_path",
    1281        "output_path",
    1282        "validation_event_id",
    1283    }
    1284    if contract == "research":
    1285        evidence_keys.update({"finding_key", "source_key", "source_url", "finding_id", "source_id"})
    1286    elif contract == "experiment":
    1287        evidence_keys.update({"metric_name", "metric_value", "measurement_id"})
    1288    elif contract == "action":
    1289        evidence_keys.update({"step_id", "command", "action_id"})
    1290    elif contract == "monitor":
    1291        evidence_keys.update({"defer_until", "monitor_id", "check_at"})
    1292    return any(str(metadata.get(key) or "").strip() for key in evidence_keys)
    1293
    1294
    1295def _recent_deliverable_evidence(ctx: ToolContext, *, limit: int = 12) -> bool:
    1296    for step in reversed(ctx.db.list_steps(job_id=ctx.job_id, limit=limit)):
    1297        if step.get("id") == ctx.step_id:
    1298            continue
    1299        if step.get("status") != "completed":
    1300            continue
    1301        tool_name = str(step.get("tool_name") or "")
    1302        input_data = step.get("input") if isinstance(step.get("input"), dict) else {}
    1303        args = input_data.get("arguments") if isinstance(input_data.get("arguments"), dict) else {}
    1304        if tool_name == "write_artifact" and _artifact_args_look_like_deliverable(args):
    1305            return True
    1306        if tool_name == "write_file":
    1307            return True
    1308        if tool_name == "shell_exec" and _shell_command_looks_like_write(str(args.get("command") or "")):
    1309            return True
    1310    return False
    1311
    1312
    1313def _artifact_args_look_like_deliverable(args: dict[str, Any]) -> bool:
    1314    text = " ".join(str(args.get(key) or "") for key in ("title", "summary", "type")).lower()
    1315    if not text:
    1316        return False
    1317    evidence_like = any(term in text for term in EVIDENCE_OUTPUT_TERMS)
    1318    deliverable_like = any(term in text for term in DELIVERABLE_OUTPUT_TERMS)
    1319    return deliverable_like and not evidence_like
    1320
    1321
    1322def _shell_command_looks_like_write(command: str) -> bool:
    1323    text = command.strip()
    1324    if not text:
    1325        return False
    1326    write_patterns = [
    1327        r"(?<!\d)>>?\s*[^&]",
    1328        r"\b1>>?\s*[^&]",
    1329        r"\btee\b",
    1330        r"\bcat\s+>\b",
    1331        r"\bpython[0-9.]*\b.*\bwrite_text\b",
    1332        r"\bpython[0-9.]*\b.*\bopen\([^)]*,\s*['\"]w",
    1333        r"\bsed\s+-i\b",
    1334    ]
    1335    return any(re.search(pattern, text, flags=re.IGNORECASE | re.DOTALL) for pattern in write_patterns)
    1336
    1337
    1338def _shell_command_counts_as_action_evidence(command: str) -> bool:
    1339    text = command.strip()
    1340    if not text:
    1341        return False
    1342    if _shell_command_looks_like_write(text):
    1343        return True
    1344    read_only = re.compile(
    1345        r"(?is)^\s*(?:"
    1346        r"awk\b|cat\b|df\b|du\b|echo\b|find\b|git\s+(?:diff|grep|log|ls-files|show|status)\b|"
    1347        r"grep\b|head\b|ls\b|pwd\b|rg\b|sed\s+-n\b|stat\b|tail\b|tree\b|wc\b"
    1348        r")"
    1349    )
    1350    if read_only.search(text):
    1351        return False
    1352    if re.match(r"(?is)^curl\b", text):
    1353        mutating_flags = (
    1354            r"\b-X\s*(?:POST|PUT|PATCH|DELETE)\b|--request\s+(?:POST|PUT|PATCH|DELETE)\b|"
    1355            r"(?:^|\s)(?:-d|--data|--form|-F|-T|--upload-file)\b"
    1356        )
    1357        return bool(re.search(mutating_flags, text))
    1358    return True
    1359
    1360
    1361def _record_roadmap(args: dict[str, Any], ctx: ToolContext) -> str:
    1362    title = str(args.get("title") or args.get("name") or "").strip()
    1363    if not title:
    1364        return _json({"success": False, "error": "title is required"})
    1365    milestones_arg = args.get("milestones")
    1366    milestones = [item for item in milestones_arg if isinstance(item, dict)] if isinstance(milestones_arg, list) else []
    1367    roadmap = ctx.db.append_roadmap_record(
    1368        ctx.job_id,
    1369        title=title,
    1370        status=str(args.get("status") or "planned"),
    1371        objective=str(args.get("objective") or ""),
    1372        scope=str(args.get("scope") or ""),
    1373        current_milestone=str(args.get("current_milestone") or ""),
    1374        validation_contract=str(args.get("validation_contract") or ""),
    1375        milestones=milestones,
    1376        metadata=args.get("metadata") if isinstance(args.get("metadata"), dict) else {},
    1377    )
    1378    ctx.db.append_agent_update(
    1379        ctx.job_id,
    1380        (
    1381            f"Roadmap updated: {roadmap.get('status')} with "
    1382            f"{len(roadmap.get('milestones') or [])} milestones."
    1383        ),
    1384        category="plan",
    1385        metadata={
    1386            "roadmap_title": roadmap.get("title"),
    1387            "roadmap_status": roadmap.get("status"),
    1388            "milestone_count": len(roadmap.get("milestones") or []),
    1389            "current_milestone": roadmap.get("current_milestone"),
    1390        },
    1391    )
    1392    return _json({"success": True, "job_id": ctx.job_id, "roadmap": roadmap})
    1393
    1394
    1395def _record_milestone_validation(args: dict[str, Any], ctx: ToolContext) -> str:
    1396    milestone = str(args.get("milestone") or args.get("milestone_title") or "").strip()
    1397    if not milestone:
    1398        return _json({"success": False, "error": "milestone is required"})
    1399    raw_issues = args.get("issues")
    1400    issues = [str(item) for item in raw_issues if str(item).strip()] if isinstance(raw_issues, list) else []
    1401    follow_up_items = args.get("follow_up_tasks") if isinstance(args.get("follow_up_tasks"), list) else []
    1402    validation_status = str(args.get("validation_status") or args.get("status") or "pending").strip().lower().replace(" ", "_")
    1403    if validation_status not in {"pending", "passed", "failed", "blocked"}:
    1404        validation_status = "pending"
    1405    result_text = str(args.get("result") or args.get("summary") or "").strip()
    1406    evidence_text = str(args.get("evidence") or args.get("evidence_artifact") or "").strip()
    1407    next_action = str(args.get("next_action") or "").strip()
    1408    metadata = args.get("metadata") if isinstance(args.get("metadata"), dict) else {}
    1409    if validation_status == "passed" and not _validation_has_positive_evidence(result_text, evidence_text, metadata):
    1410        return _json({
    1411            "success": False,
    1412            "error": "passed milestone validation requires evidence or result",
    1413            "guidance": (
    1414                "Use validation_status=passed only after concrete evidence or a validation result proves the milestone. "
    1415                "Use pending, failed, or blocked when validation is incomplete or missing evidence."
    1416            ),
    1417        })
    1418    if validation_status in {"failed", "blocked"} and not (
    1419        result_text or evidence_text or issues or follow_up_items or next_action
    1420    ):
    1421        return _json({
    1422            "success": False,
    1423            "error": f"{validation_status} milestone validation requires a gap, issue, evidence, next_action, or follow-up task",
    1424            "guidance": (
    1425                "Failed or blocked validation must say what is missing or what should happen next, "
    1426                "so the worker can continue from a concrete gap instead of logging a vague checkpoint."
    1427            ),
    1428        })
    1429    validation = ctx.db.append_milestone_validation_record(
    1430        ctx.job_id,
    1431        milestone=milestone,
    1432        validation_status=validation_status,
    1433        result=result_text,
    1434        evidence=evidence_text,
    1435        issues=issues,
    1436        next_action=next_action,
    1437        metadata=metadata,
    1438    )
    1439    follow_up_tasks = []
    1440    for task in follow_up_items[:25]:
    1441        if not isinstance(task, dict):
    1442            continue
    1443        title = str(task.get("title") or task.get("name") or "").strip()
    1444        if not title:
    1445            continue
    1446        priority_arg = task.get("priority")
    1447        priority = int(priority_arg) if isinstance(priority_arg, (int, float)) else 0
    1448        follow_up_tasks.append(ctx.db.append_task_record(
    1449            ctx.job_id,
    1450            title=title,
    1451            status=str(task.get("status") or "open"),
    1452            priority=priority,
    1453            goal=str(task.get("goal") or task.get("description") or ""),
    1454            source_hint=str(task.get("source_hint") or task.get("source") or ""),
    1455            result=str(task.get("result") or task.get("outcome") or ""),
    1456            parent=str(task.get("parent") or milestone),
    1457            output_contract=str(
    1458                task.get("output_contract")
    1459                or task.get("contract")
    1460                or (task.get("metadata") if isinstance(task.get("metadata"), dict) else {}).get("output_contract")
    1461                or (task.get("metadata") if isinstance(task.get("metadata"), dict) else {}).get("contract")
    1462                or "action"
    1463            ),
    1464            acceptance_criteria=str(task.get("acceptance_criteria") or ""),
    1465            evidence_needed=str(task.get("evidence_needed") or ""),
    1466            stall_behavior=str(task.get("stall_behavior") or ""),
    1467            metadata=task.get("metadata") if isinstance(task.get("metadata"), dict) else {"source": "milestone_validation"},
    1468        ))
    1469    ctx.db.append_agent_update(
    1470        ctx.job_id,
    1471        (
    1472            f"Milestone validation {validation.get('validation_status')}: "
    1473            f"{validation.get('title') or milestone}; follow-up tasks {len(follow_up_tasks)}."
    1474        ),
    1475        category="plan",
    1476        metadata={
    1477            "milestone": validation.get("title") or milestone,
    1478            "validation_status": validation.get("validation_status"),
    1479            "follow_up_tasks": len(follow_up_tasks),
    1480        },
    1481    )
    1482    return _json({
    1483        "success": True,
    1484        "job_id": ctx.job_id,
    1485        "validation": validation,
    1486        "follow_up_tasks": follow_up_tasks,
    1487    })
    1488
    1489
    1490def _validation_has_positive_evidence(result: str, evidence: str, metadata: dict[str, Any]) -> bool:
    1491    if result.strip() or evidence.strip():
    1492        return True
    1493    evidence_keys = {
    1494        "artifact_id",
    1495        "evidence_artifact",
    1496        "experiment_key",
    1497        "file_path",
    1498        "output_path",
    1499        "source_url",
    1500        "finding_key",
    1501        "validation_event_id",
    1502    }
    1503    return any(str(metadata.get(key) or "").strip() for key in evidence_keys)
    1504
    1505
    1506def _record_experiment(args: dict[str, Any], ctx: ToolContext) -> str:
    1507    title = str(
    1508        args.get("title")
    1509        or args.get("name")
    1510        or args.get("metric_name")
    1511        or args.get("hypothesis")
    1512        or args.get("result")
    1513        or args.get("outcome")
    1514        or "Experiment checkpoint"
    1515    ).strip()
    1516    metric_name = str(args.get("metric_name") or "").strip()
    1517    metric_value = _optional_float(args.get("metric_value"))
    1518    baseline_value = _optional_float(args.get("baseline_value"))
    1519    status = str(args.get("status") or "planned").strip().lower() or "planned"
    1520    next_action = str(args.get("next_action") or "").strip()
    1521    if status == "measured" and (not metric_name or metric_value is None):
    1522        return _json({
    1523            "success": False,
    1524            "error": "measured experiments require metric_name and numeric metric_value",
    1525            "guidance": (
    1526                "Use status=measured only for a real measurement with a metric name and numeric value. "
    1527                "Use status=failed, blocked, skipped, running, or planned when the trial did not produce a valid metric."
    1528            ),
    1529        })
    1530    if status in {"measured", "failed", "blocked", "skipped"} and not next_action:
    1531        return _json({
    1532            "success": False,
    1533            "error": "next_action is required for measured, failed, blocked, or skipped experiments",
    1534            "guidance": (
    1535                "Experiment records that close out a trial must leave a concrete next action, "
    1536                "such as the next experiment, action branch, monitor branch, pivot, or blocked condition."
    1537            ),
    1538        })
    1539    if status in {"failed", "blocked", "skipped"} and not _experiment_has_closed_trial_context(args):
    1540        return _json({
    1541            "success": False,
    1542            "error": f"{status} experiments require result, evidence, config, or metadata",
    1543            "guidance": (
    1544                "Closed non-measured trials must record what happened or what was attempted. "
    1545                "Include result/outcome, evidence_artifact, config, or metadata with the blocker/context."
    1546            ),
    1547        })
    1548    record = ctx.db.append_experiment_record(
    1549        ctx.job_id,
    1550        title=title,
    1551        hypothesis=str(args.get("hypothesis") or ""),
    1552        status=status,
    1553        metric_name=metric_name,
    1554        metric_value=metric_value,
    1555        metric_unit=str(args.get("metric_unit") or ""),
    1556        higher_is_better=bool(args.get("higher_is_better", True)),
    1557        baseline_value=baseline_value,
    1558        config=args.get("config") if isinstance(args.get("config"), dict) else {},
    1559        result=str(args.get("result") or args.get("outcome") or ""),
    1560        evidence_artifact=str(args.get("evidence_artifact") or args.get("artifact_id") or ""),
    1561        next_action=next_action,
    1562        metadata=args.get("metadata") if isinstance(args.get("metadata"), dict) else {},
    1563    )
    1564    metric = ""
    1565    if record.get("metric_value") is not None:
    1566        metric = " " + format_metric_value(
    1567            record.get("metric_name") or "metric",
    1568            record.get("metric_value"),
    1569            record.get("metric_unit") or "",
    1570        )
    1571    best = " best" if record.get("best_observed") else ""
    1572    ctx.db.append_agent_update(
    1573        ctx.job_id,
    1574        f"Experiment {record.get('status')}: {record.get('title')}{metric}{best}.",
    1575        category="progress",
    1576        metadata={
    1577            "experiment_key": record.get("key"),
    1578            "metric_name": record.get("metric_name"),
    1579            "metric_value": record.get("metric_value"),
    1580            "best_observed": record.get("best_observed"),
    1581            "delta_from_previous_best": record.get("delta_from_previous_best"),
    1582        },
    1583    )
    1584    if record.get("metric_value") is not None or str(record.get("status") or "") in {"measured", "failed", "blocked", "skipped"}:
    1585        _resolve_measurement_obligation(
    1586            ctx,
    1587            status="recorded",
    1588            reason=f"Recorded experiment {record.get('title')}.",
    1589            via_tool="record_experiment",
    1590            experiment_key=str(record.get("key") or ""),
    1591        )
    1592    return _json({"success": True, "job_id": ctx.job_id, "experiment": record})
    1593
    1594
    1595def _experiment_has_closed_trial_context(args: dict[str, Any]) -> bool:
    1596    if str(args.get("result") or args.get("outcome") or "").strip():
    1597        return True
    1598    if str(args.get("evidence_artifact") or args.get("artifact_id") or "").strip():
    1599        return True
    1600    config = args.get("config")
    1601    if isinstance(config, dict) and any(str(value).strip() for value in config.values()):
    1602        return True
    1603    metadata = args.get("metadata")
    1604    if isinstance(metadata, dict) and any(str(value).strip() for value in metadata.values()):
    1605        return True
    1606    return False
    1607
    1608
    1609def _optional_float(value: Any) -> float | None:
    1610    if isinstance(value, bool) or value is None:
    1611        return None
    1612    if isinstance(value, (int, float)):
    1613        return float(value)
    1614    if isinstance(value, str):
    1615        text = value.strip().replace(",", "")
    1616        if not text:
    1617            return None
    1618        try:
    1619            return float(text)
    1620        except ValueError:
    1621            return None
    1622    return None
    1623
    1624
    1625def _pending_measurement(ctx: ToolContext) -> dict[str, Any] | None:
    1626    try:
    1627        job = ctx.db.get_job(ctx.job_id)
    1628    except KeyError:
    1629        return None
    1630    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    1631    obligation = metadata.get("pending_measurement_obligation")
    1632    if isinstance(obligation, dict) and not obligation.get("resolved_at"):
    1633        return obligation
    1634    return None
    1635
    1636
    1637def _resolve_measurement_obligation(
    1638    ctx: ToolContext,
    1639    *,
    1640    status: str,
    1641    reason: str,
    1642    via_tool: str,
    1643    experiment_key: str = "",
    1644) -> None:
    1645    obligation = _pending_measurement(ctx)
    1646    if not obligation:
    1647        return
    1648    resolved = dict(obligation)
    1649    resolved.update({
    1650        "resolved_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
    1651        "resolution_status": status,
    1652        "resolution_reason": reason[:1000],
    1653        "resolution_tool": via_tool,
    1654    })
    1655    if experiment_key:
    1656        resolved["experiment_key"] = experiment_key
    1657    ctx.db.update_job_metadata(
    1658        ctx.job_id,
    1659        {
    1660            "pending_measurement_obligation": {},
    1661            "last_measurement_obligation": resolved,
    1662        },
    1663    )
    1664    ctx.db.append_agent_update(
    1665        ctx.job_id,
    1666        f"Measurement obligation {status}: {reason[:220]}",
    1667        category="progress" if status == "recorded" else "blocked",
    1668        metadata={"measurement_obligation": resolved},
    1669    )
    1670
    1671
    1672def _send_digest_email(args: dict[str, Any], ctx: ToolContext) -> str:
    1673    subject = str(args.get("subject") or "Agent digest")
    1674    body = str(args.get("body") or "")
    1675    if not body:
    1676        return _json({"success": False, "error": "body is required"})
    1677    result = send_digest_email(ctx.config.email, subject=subject, body=body, to_addr=args.get("to_addr"))
    1678    stored = ctx.artifacts.write_text(
    1679        job_id=ctx.job_id,
    1680        run_id=ctx.run_id,
    1681        step_id=ctx.step_id,
    1682        content=body,
    1683        title=subject,
    1684        summary="Digest email body",
    1685        artifact_type="digest",
    1686        metadata={"email": result},
    1687    )
    1688    return _json({"success": True, "email": result, "artifact_id": stored.id, "path": str(stored.path)})
    1689
    1690
    1691def _browser_call(name: str, args: dict[str, Any], ctx: ToolContext) -> str:
    1692    from nipux_cli import browser
    1693
    1694    task_id = ctx.task_id or ctx.job_id
    1695    if name == "browser_navigate":
    1696        return _json(browser.navigate(ctx.config, task_id=task_id, url=str(args.get("url") or "")))
    1697    if name == "browser_snapshot":
    1698        return _json(browser.snapshot(ctx.config, task_id=task_id, full=bool(args.get("full", False))))
    1699    if name == "browser_click":
    1700        return _json(browser.click(ctx.config, task_id=task_id, ref=str(args.get("ref") or "")))
    1701    if name == "browser_type":
    1702        return _json(browser.fill(ctx.config, task_id=task_id, ref=str(args.get("ref") or ""), text=str(args.get("text") or "")))
    1703    if name == "browser_scroll":
    1704        return _json(browser.scroll(ctx.config, task_id=task_id, direction=str(args.get("direction") or "down")))
    1705    if name == "browser_back":
    1706        return _json(browser.back(ctx.config, task_id=task_id))
    1707    if name == "browser_press":
    1708        return _json(browser.press(ctx.config, task_id=task_id, key=str(args.get("key") or "")))
    1709    if name == "browser_console":
    1710        return _json(browser.console(ctx.config, task_id=task_id, clear=bool(args.get("clear", False)), expression=args.get("expression")))
    1711    raise KeyError(name)
    1712
    1713
    1714def _web_call(name: str, args: dict[str, Any], ctx: ToolContext) -> str:
    1715    del ctx
    1716    from nipux_cli.web import web_extract, web_search
    1717
    1718    if name == "web_search":
    1719        return _json(web_search(str(args.get("query") or ""), limit=int(args.get("limit") or 5)))
    1720    if name == "web_extract":
    1721        urls = args.get("urls") if isinstance(args.get("urls"), list) else []
    1722        return _json(web_extract(urls[:5]))
    1723    raise KeyError(name)
    1724
    1725
    1726def _browser_handler(name: str) -> Handler:
    1727    return lambda args, ctx: _browser_call(name, args, ctx)
    1728
    1729
    1730def _web_handler(name: str) -> Handler:
    1731    return lambda args, ctx: _web_call(name, args, ctx)
    1732
    1733
    1734BROWSER_SCHEMAS: list[ToolSpec] = [
    1735    ToolSpec("browser_navigate", "Navigate to a URL and return a compact browser snapshot.", {
    1736        "type": "object",
    1737        "properties": {"url": {"type": "string"}},
    1738        "required": ["url"],
    1739    }, _browser_handler("browser_navigate")),
    1740    ToolSpec("browser_snapshot", "Refresh the current page accessibility snapshot.", {
    1741        "type": "object",
    1742        "properties": {"full": {"type": "boolean", "default": False}},
    1743        "required": [],
    1744    }, _browser_handler("browser_snapshot")),
    1745    ToolSpec("browser_click", "Click an element by snapshot ref, for example @e5.", {
    1746        "type": "object",
    1747        "properties": {"ref": {"type": "string"}},
    1748        "required": ["ref"],
    1749    }, _browser_handler("browser_click")),
    1750    ToolSpec("browser_type", "Fill an input by snapshot ref.", {
    1751        "type": "object",
    1752        "properties": {"ref": {"type": "string"}, "text": {"type": "string"}},
    1753        "required": ["ref", "text"],
    1754    }, _browser_handler("browser_type")),
    1755    ToolSpec("browser_scroll", "Scroll the current page up or down.", {
    1756        "type": "object",
    1757        "properties": {"direction": {"type": "string", "enum": ["up", "down"]}},
    1758        "required": ["direction"],
    1759    }, _browser_handler("browser_scroll")),
    1760    ToolSpec("browser_back", "Navigate back in browser history.", {"type": "object", "properties": {}, "required": []}, _browser_handler("browser_back")),
    1761    ToolSpec("browser_press", "Press a keyboard key in the browser.", {
    1762        "type": "object",
    1763        "properties": {"key": {"type": "string"}},
    1764        "required": ["key"],
    1765    }, _browser_handler("browser_press")),
    1766    ToolSpec("browser_console", "Read console errors or evaluate JavaScript in the current page.", {
    1767        "type": "object",
    1768        "properties": {"clear": {"type": "boolean", "default": False}, "expression": {"type": "string"}},
    1769        "required": [],
    1770    }, _browser_handler("browser_console")),
    1771]
    1772
    1773
    1774SUPPORT_SCHEMAS: list[ToolSpec] = [
    1775    ToolSpec("web_search", "Search the web for candidate sources.", {
    1776        "type": "object",
    1777        "properties": {"query": {"type": "string"}, "limit": {"type": "integer", "default": 5}},
    1778        "required": ["query"],
    1779    }, _web_handler("web_search")),
    1780    ToolSpec("web_extract", "Extract markdown text from up to five URLs.", {
    1781        "type": "object",
    1782        "properties": {"urls": {"type": "array", "items": {"type": "string"}, "maxItems": 5}},
    1783        "required": ["urls"],
    1784    }, _web_handler("web_extract")),
    1785    ToolSpec("shell_exec", "Run a local shell command for CLI work. Use small read-only probes first. For long downloads, builds, training, crawls, or benchmarks, set a meaningful timeout, prefer resumable commands, and record or defer monitoring instead of repeatedly restarting short timed-out commands. Do not run destructive or high-risk cyber commands.", {
    1786        "type": "object",
    1787        "properties": {
    1788            "command": {"type": "string"},
    1789            "cwd": {"type": "string"},
    1790            "timeout_seconds": {"type": "number", "default": 60},
    1791            "max_output_chars": {"type": "integer", "default": 12000},
    1792        },
    1793        "required": ["command"],
    1794    }, _shell_exec),
    1795    ToolSpec("write_file", "Create, overwrite, or append a concrete workspace/local file for deliverables, code, documents, configs, or other file outputs.", {
    1796        "type": "object",
    1797        "properties": {
    1798            "path": {"type": "string"},
    1799            "content": {"type": "string"},
    1800            "mode": {"type": "string", "enum": ["overwrite", "append"], "default": "overwrite"},
    1801            "create_parents": {"type": "boolean", "default": True},
    1802        },
    1803        "required": ["path", "content"],
    1804    }, _write_file),
    1805    ToolSpec("write_artifact", "Persist important findings, evidence, reports, or checkpoints to the job artifact store.", {
    1806        "type": "object",
    1807        "properties": {
    1808            "title": {"type": "string"},
    1809            "type": {"type": "string", "default": "text"},
    1810            "summary": {"type": "string"},
    1811            "content": {"type": "string"},
    1812            "metadata": {"type": "object"},
    1813        },
    1814        "required": ["content"],
    1815    }, _write_artifact),
    1816    ToolSpec("read_artifact", "Read a saved artifact by artifact_id, visible number, exact saved path, or title.", {
    1817        "type": "object",
    1818        "properties": {
    1819            "artifact_id": {"type": "string", "description": "Artifact id, visible number, saved path, or title."},
    1820            "path": {"type": "string"},
    1821            "title": {"type": "string"},
    1822            "ref": {"type": "string"},
    1823        },
    1824        "required": [],
    1825    }, _read_artifact),
    1826    ToolSpec("search_artifacts", "Search stored artifacts for exact evidence from prior steps.", {
    1827        "type": "object",
    1828        "properties": {"query": {"type": "string"}, "limit": {"type": "integer", "default": 10}},
    1829        "required": ["query"],
    1830    }, _search_artifacts),
    1831    ToolSpec("update_job_state", "Keep the current job runnable. Completion, failure, pausing, and cancellation are operator-only; workers should report checkpoints and continue.", {
    1832        "type": "object",
    1833        "properties": {
    1834            "status": {"type": "string", "enum": ["queued", "running"]},
    1835            "note": {"type": "string"},
    1836        },
    1837        "required": ["status"],
    1838    }, _update_job_state),
    1839    ToolSpec("defer_job", "Wait before the next worker turn for this job. Use for long external processes, monitor/check-later tasks, or scheduled follow-up without completing or pausing the job.", {
    1840        "type": "object",
    1841        "properties": {
    1842            "seconds": {"type": "number", "description": "Delay in seconds before this job is runnable again.", "default": 300},
    1843            "until": {"type": "string", "description": "Optional ISO timestamp to resume after."},
    1844            "reason": {"type": "string"},
    1845            "next_action": {"type": "string"},
    1846        },
    1847        "required": [],
    1848    }, _defer_job),
    1849    ToolSpec("report_update", "Leave a short operator-readable progress note. Do not use this instead of write_artifact for durable evidence.", {
    1850        "type": "object",
    1851        "properties": {
    1852            "message": {"type": "string"},
    1853            "category": {"type": "string", "enum": ["progress", "finding", "blocked", "plan"], "default": "progress"},
    1854            "metadata": {"type": "object"},
    1855        },
    1856        "required": ["message"],
    1857    }, _report_update),
    1858    ToolSpec("record_lesson", "Save durable learning for this job: bad source patterns, success criteria, strategy changes, mistakes to avoid, or operator preferences.", {
    1859        "type": "object",
    1860        "properties": {
    1861            "lesson": {"type": "string"},
    1862            "category": {
    1863                "type": "string",
    1864                "enum": [
    1865                    "source_quality",
    1866                    "task_profile",
    1867                    "strategy",
    1868                    "mistake",
    1869                    "constraint",
    1870                    "operator_preference",
    1871                    "memory",
    1872                ],
    1873                "default": "memory",
    1874            },
    1875            "confidence": {"type": "number"},
    1876            "metadata": {"type": "object"},
    1877        },
    1878        "required": ["lesson"],
    1879    }, _record_lesson),
    1880    ToolSpec("record_memory_graph", "Create or update the job's connected memory graph: reusable episodes, facts, strategies, skills, questions, decisions, constraints, and links between them. Use this to build a durable brain for long-running work instead of relying on raw history.", {
    1881        "type": "object",
    1882        "properties": {
    1883            "nodes": {
    1884                "type": "array",
    1885                "maxItems": 50,
    1886                "items": {
    1887                    "type": "object",
    1888                    "properties": {
    1889                        "key": {"type": "string"},
    1890                        "title": {"type": "string"},
    1891                        "kind": {
    1892                            "type": "string",
    1893                            "enum": [
    1894                                "artifact",
    1895                                "constraint",
    1896                                "decision",
    1897                                "episode",
    1898                                "experiment",
    1899                                "fact",
    1900                                "milestone",
    1901                                "question",
    1902                                "skill",
    1903                                "source",
    1904                                "strategy",
    1905                                "task",
    1906                            ],
    1907                        },
    1908                        "status": {
    1909                            "type": "string",
    1910                            "enum": ["active", "blocked", "deprecated", "open", "resolved", "stable"],
    1911                            "default": "active",
    1912                        },
    1913                        "summary": {"type": "string"},
    1914                        "salience": {"type": "number"},
    1915                        "confidence": {"type": "number"},
    1916                        "tags": {"type": "array", "items": {"type": "string"}},
    1917                        "parent_key": {"type": "string"},
    1918                        "links": {"type": "array", "items": {"type": "string"}},
    1919                        "evidence_refs": {"type": "array", "items": {"type": "string"}},
    1920                        "metadata": {"type": "object"},
    1921                    },
    1922                    "required": ["title"],
    1923                },
    1924            },
    1925            "edges": {
    1926                "type": "array",
    1927                "maxItems": 100,
    1928                "items": {
    1929                    "type": "object",
    1930                    "properties": {
    1931                        "from_key": {"type": "string"},
    1932                        "to_key": {"type": "string"},
    1933                        "relation": {"type": "string"},
    1934                        "evidence_refs": {"type": "array", "items": {"type": "string"}},
    1935                        "metadata": {"type": "object"},
    1936                    },
    1937                    "required": ["from_key", "to_key", "relation"],
    1938                },
    1939            },
    1940        },
    1941        "required": [],
    1942    }, _record_memory_graph),
    1943    ToolSpec("search_memory_graph", "Search the job's connected memory graph for reusable facts, decisions, strategies, skills, questions, constraints, and related links.", {
    1944        "type": "object",
    1945        "properties": {
    1946            "query": {"type": "string"},
    1947            "limit": {"type": "integer", "default": 10},
    1948        },
    1949        "required": ["query"],
    1950    }, _search_memory_graph),
    1951    ToolSpec("acknowledge_operator_context", "Acknowledge that durable operator steering has been incorporated or superseded. Use this after acting on a chat correction so it can leave the active context while remaining in history.", {
    1952        "type": "object",
    1953        "properties": {
    1954            "message_ids": {"type": "array", "items": {"type": "string"}},
    1955            "summary": {"type": "string"},
    1956            "status": {"type": "string", "enum": ["acknowledged", "superseded"], "default": "acknowledged"},
    1957        },
    1958        "required": ["summary"],
    1959    }, _acknowledge_operator_context),
    1960    ToolSpec("record_source", "Update the source ledger with source quality, finding yield, failures, warnings, and last outcome.", {
    1961        "type": "object",
    1962        "properties": {
    1963            "source": {"type": "string"},
    1964            "source_type": {"type": "string"},
    1965            "usefulness_score": {"type": "number"},
    1966            "yield_count": {"type": "integer", "default": 0},
    1967            "fail_count_delta": {"type": "integer", "default": 0},
    1968            "warnings": {"type": "array", "items": {"type": "string"}},
    1969            "outcome": {"type": "string"},
    1970            "metadata": {"type": "object"},
    1971        },
    1972        "required": ["source"],
    1973    }, _record_source),
    1974    ToolSpec("record_findings", "Update the finding ledger with evidence-backed useful results. Each finding needs an evidence anchor such as source_url/url, reason, evidence_artifact, or evidence metadata.", {
    1975        "type": "object",
    1976        "properties": {
    1977            "evidence_artifact": {"type": "string"},
    1978            "findings": {
    1979                "type": "array",
    1980                "maxItems": 50,
    1981                "items": {
    1982                    "type": "object",
    1983                    "properties": {
    1984                        "name": {"type": "string"},
    1985                        "url": {"type": "string"},
    1986                        "source_url": {"type": "string"},
    1987                        "category": {"type": "string"},
    1988                        "location": {"type": "string"},
    1989                        "contact": {"type": "string"},
    1990                        "reason": {"type": "string"},
    1991                        "status": {"type": "string"},
    1992                        "score": {"type": "number"},
    1993                        "evidence_artifact": {"type": "string"},
    1994                        "metadata": {"type": "object"},
    1995                    },
    1996                    "required": ["name"],
    1997                },
    1998            },
    1999        },
    2000        "required": ["findings"],
    2001    }, _record_findings),
    2002    ToolSpec("record_tasks", "Create or update a durable queue of objective-neutral work branches. Use this to split long jobs into next actions, mark blocked branches, and keep the agent from cycling on one path. Missing task contract fields are filled with generic defaults from the task title. When the queue is saturated, near-duplicate task titles are folded into the matching existing task instead of creating another branch.", {
    2003        "type": "object",
    2004        "properties": {
    2005            "tasks": {
    2006                "type": "array",
    2007                "maxItems": 50,
    2008                "items": {
    2009                    "type": "object",
    2010                    "properties": {
    2011                        "title": {"type": "string"},
    2012                        "status": {"type": "string", "enum": ["open", "active", "done", "blocked", "skipped"], "default": "open"},
    2013                        "priority": {"type": "integer", "default": 0},
    2014                        "goal": {"type": "string"},
    2015                        "source_hint": {"type": "string"},
    2016                        "result": {"type": "string"},
    2017                        "parent": {"type": "string"},
    2018                        "output_contract": {
    2019                            "type": "string",
    2020                            "enum": ["research", "artifact", "experiment", "action", "monitor", "decision", "report"],
    2021                        },
    2022                        "acceptance_criteria": {"type": "string"},
    2023                        "evidence_needed": {"type": "string"},
    2024                        "stall_behavior": {"type": "string"},
    2025                        "metadata": {"type": "object"},
    2026                    },
    2027                    "required": ["title"],
    2028                },
    2029            },
    2030        },
    2031        "required": ["tasks"],
    2032    }, _record_tasks),
    2033    ToolSpec("record_roadmap", "Create or update a generic roadmap for broad work: milestones, features, success criteria, validation contract, scope, and current roadmap state. Use this before or during long-running work when task lists need higher-level structure.", {
    2034        "type": "object",
    2035        "properties": {
    2036            "title": {"type": "string"},
    2037            "status": {"type": "string", "enum": ["planned", "active", "validating", "done", "blocked", "paused"], "default": "planned"},
    2038            "objective": {"type": "string"},
    2039            "scope": {"type": "string"},
    2040            "current_milestone": {"type": "string"},
    2041            "validation_contract": {"type": "string"},
    2042            "milestones": {
    2043                "type": "array",
    2044                "maxItems": 100,
    2045                "items": {
    2046                    "type": "object",
    2047                    "properties": {
    2048                        "key": {"type": "string"},
    2049                        "title": {"type": "string"},
    2050                        "status": {"type": "string", "enum": ["planned", "active", "validating", "done", "blocked", "skipped"], "default": "planned"},
    2051                        "priority": {"type": "integer", "default": 0},
    2052                        "goal": {"type": "string"},
    2053                        "acceptance_criteria": {"type": "string"},
    2054                        "evidence_needed": {"type": "string"},
    2055                        "validation_status": {"type": "string", "enum": ["not_started", "pending", "passed", "failed", "blocked"], "default": "not_started"},
    2056                        "validation_result": {"type": "string"},
    2057                        "next_action": {"type": "string"},
    2058                        "features": {
    2059                            "type": "array",
    2060                            "maxItems": 100,
    2061                            "items": {
    2062                                "type": "object",
    2063                                "properties": {
    2064                                    "key": {"type": "string"},
    2065                                    "title": {"type": "string"},
    2066                                    "status": {"type": "string", "enum": ["planned", "active", "done", "blocked", "skipped"], "default": "planned"},
    2067                                    "goal": {"type": "string"},
    2068                                    "output_contract": {"type": "string", "enum": ["research", "artifact", "experiment", "action", "monitor", "decision", "report", "validation"]},
    2069                                    "acceptance_criteria": {"type": "string"},
    2070                                    "evidence_needed": {"type": "string"},
    2071                                    "result": {"type": "string"},
    2072                                    "metadata": {"type": "object"},
    2073                                },
    2074                                "required": ["title"],
    2075                            },
    2076                        },
    2077                        "metadata": {"type": "object"},
    2078                    },
    2079                    "required": ["title"],
    2080                },
    2081            },
    2082            "metadata": {"type": "object"},
    2083        },
    2084        "required": ["title"],
    2085    }, _record_roadmap),
    2086    ToolSpec("record_milestone_validation", "Record validation for a roadmap milestone and optionally create follow-up tasks for gaps. Use fresh evidence, acceptance criteria, and clear pass/fail/blocker reasons.", {
    2087        "type": "object",
    2088        "properties": {
    2089            "milestone": {"type": "string"},
    2090            "validation_status": {"type": "string", "enum": ["pending", "passed", "failed", "blocked"], "default": "pending"},
    2091            "result": {"type": "string"},
    2092            "evidence": {"type": "string"},
    2093            "issues": {"type": "array", "items": {"type": "string"}},
    2094            "next_action": {"type": "string"},
    2095            "follow_up_tasks": {
    2096                "type": "array",
    2097                "maxItems": 25,
    2098                "items": {
    2099                    "type": "object",
    2100                    "properties": {
    2101                        "title": {"type": "string"},
    2102                        "status": {"type": "string", "enum": ["open", "active", "done", "blocked", "skipped"], "default": "open"},
    2103                        "priority": {"type": "integer", "default": 0},
    2104                        "goal": {"type": "string"},
    2105                        "source_hint": {"type": "string"},
    2106                        "result": {"type": "string"},
    2107                        "parent": {"type": "string"},
    2108                        "output_contract": {"type": "string", "enum": ["research", "artifact", "experiment", "action", "monitor", "decision", "report"]},
    2109                        "acceptance_criteria": {"type": "string"},
    2110                        "evidence_needed": {"type": "string"},
    2111                        "stall_behavior": {"type": "string"},
    2112                        "metadata": {"type": "object"},
    2113                    },
    2114                    "required": ["title"],
    2115                },
    2116            },
    2117            "metadata": {"type": "object"},
    2118        },
    2119        "required": ["milestone", "validation_status"],
    2120    }, _record_milestone_validation),
    2121    ToolSpec("record_experiment", "Track a measurable trial, benchmark, comparison, hypothesis test, or optimization attempt. Use this after any command or source produces a concrete result so future steps compare against the best observed result instead of treating notes as progress. Closed trials must include next_action so long-running work can continue from the result.", {
    2122        "type": "object",
    2123        "properties": {
    2124            "title": {"type": "string"},
    2125            "hypothesis": {"type": "string"},
    2126            "status": {"type": "string", "enum": ["planned", "running", "measured", "failed", "blocked", "skipped"], "default": "planned"},
    2127            "metric_name": {"type": "string"},
    2128            "metric_value": {"type": "number"},
    2129            "metric_unit": {"type": "string"},
    2130            "higher_is_better": {"type": "boolean", "default": True},
    2131            "baseline_value": {"type": "number"},
    2132            "config": {"type": "object"},
    2133            "result": {"type": "string"},
    2134            "evidence_artifact": {"type": "string"},
    2135            "next_action": {"type": "string", "description": "Concrete next experiment, action, monitor branch, pivot, or blocked condition. Required when status is measured, failed, blocked, or skipped."},
    2136            "metadata": {"type": "object"},
    2137        },
    2138        "required": ["title"],
    2139    }, _record_experiment),
    2140    ToolSpec("send_digest_email", "Send or dry-run a digest email and save the body as an artifact.", {
    2141        "type": "object",
    2142        "properties": {"subject": {"type": "string"}, "body": {"type": "string"}, "to_addr": {"type": "string"}},
    2143        "required": ["body"],
    2144    }, _send_digest_email),
    2145]
    2146
    2147
    2148APPROVED_TOOL_NAMES = tuple(spec.name for spec in [*BROWSER_SCHEMAS, *SUPPORT_SCHEMAS])
    2149
    2150
    2151class ToolRegistry:
    2152    def __init__(self, specs: list[ToolSpec] | None = None):
    2153        self._specs = {spec.name: spec for spec in (specs or [*BROWSER_SCHEMAS, *SUPPORT_SCHEMAS])}
    2154
    2155    def names(self) -> list[str]:
    2156        return sorted(self._specs)
    2157
    2158    def openai_tools(self, config: AppConfig | None = None) -> list[dict[str, Any]]:
    2159        return [self._specs[name].as_openai_tool() for name in self.names() if _tool_enabled(name, config)]
    2160
    2161    def validate_arguments(self, name: str, args: dict[str, Any], config: AppConfig | None = None) -> dict[str, Any] | None:
    2162        if name not in self._specs or not _tool_enabled(name, config):
    2163            return None
    2164        args = args if isinstance(args, dict) else {}
    2165        spec = self._specs[name]
    2166        missing: list[str] = []
    2167        aliases = REQUIRED_ARGUMENT_ALIASES.get(name, {})
    2168        for required in spec.parameters.get("required") or []:
    2169            candidates = aliases.get(str(required), (str(required),))
    2170            if all(_missing_argument(args.get(candidate)) for candidate in candidates):
    2171                missing.append(" or ".join(candidates))
    2172        for label, fields in REQUIRED_ARGUMENT_GROUPS.get(name, ()):
    2173            if all(_missing_argument(args.get(field)) for field in fields):
    2174                missing.append(label)
    2175        nested_schema = dict(spec.parameters)
    2176        nested_schema["required"] = []
    2177        missing.extend(item for item in _schema_missing_arguments(nested_schema, args) if item not in missing)
    2178        placeholders = [] if missing else [item for item in _schema_placeholder_arguments(nested_schema, args) if item not in missing]
    2179        if not missing and not placeholders:
    2180            return None
    2181        concrete_fields = [*missing, *placeholders]
    2182        return {
    2183            "success": True,
    2184            "recoverable": True,
    2185            "error": "missing required tool arguments" if missing else "placeholder tool arguments",
    2186            "missing_arguments": missing,
    2187            "placeholder_arguments": placeholders,
    2188            "blocked_tool": name,
    2189            "guidance": (
    2190                f"Retry {name} with concrete values for: {', '.join(concrete_fields)}. "
    2191                "Do not call a tool with placeholder or empty arguments."
    2192            ),
    2193        }
    2194
    2195    def handle(self, name: str, args: dict[str, Any], ctx: ToolContext) -> str:
    2196        if name not in self._specs:
    2197            return _json({"success": False, "error": f"unknown tool: {name}"})
    2198        if not _tool_enabled(name, ctx.config):
    2199            group = _tool_access_group(name) or "tool"
    2200            return _json({
    2201                "success": False,
    2202                "error": f"{name} is disabled by tool access config",
    2203                "tool_access": group,
    2204            })
    2205        return self._specs[name].handler(args, ctx)
    2206
    2207
    2208def _tool_enabled(name: str, config: AppConfig | None) -> bool:
    2209    if config is None:
    2210        return True
    2211    group = _tool_access_group(name)
    2212    if group is None:
    2213        return True
    2214    return bool(getattr(config.tools, group))
    2215
    2216
    2217def _tool_access_group(name: str) -> str | None:
    2218    if name.startswith("browser_"):
    2219        return "browser"
    2220    if name.startswith("web_"):
    2221        return "web"
    2222    if name == "shell_exec":
    2223        return "shell"
    2224    if name == "write_file":
    2225        return "files"
    2226    return None
    2227
    2228
    2229DEFAULT_REGISTRY = ToolRegistry()

nipux_cli/tui_commands.py 296 lines

       1"""Slash command metadata and command-palette helpers for the TUI."""
       2
       3from __future__ import annotations
       4
       5from nipux_cli.tui_style import _accent, _bold, _fit_ansi, _muted
       6
       7
       8FIRST_RUN_SLASH_COMMANDS = [
       9    ("/model", "set model"),
      10    ("/base-url", "set endpoint"),
      11    ("/api-key", "save key"),
      12    ("/api-key-env", "key env var"),
      13    ("/config", "runtime config"),
      14    ("/context", "token budget"),
      15    ("/input-cost", "input $/1M"),
      16    ("/timeout", "request timeout"),
      17    ("/browser", "browser on/off"),
      18    ("/web", "web on/off"),
      19    ("/cli-access", "CLI on/off"),
      20    ("/file-access", "files on/off"),
      21    ("/home", "state directory"),
      22    ("/step-limit", "worker timeout"),
      23    ("/output-chars", "output preview size"),
      24    ("/output-cost", "output $/1M"),
      25    ("/max-cost", "job cost limit"),
      26    ("/daily-digest", "daily digest on/off"),
      27    ("/digest-time", "digest time"),
      28    ("/doctor", "check setup"),
      29    ("/init", "write config"),
      30    ("/help", "show commands"),
      31    ("/clear", "clear notices"),
      32    ("/exit", "quit"),
      33]
      34
      35CHAT_SLASH_COMMANDS = [
      36    ("/new", "create and start a worker"),
      37    ("/run", "resume focused work"),
      38    ("/jobs", "switch or inspect jobs"),
      39    ("/settings", "configure provider/tools"),
      40    ("/status", "focused job state"),
      41    ("/help", "core commands"),
      42    ("/outcomes", "durable work"),
      43    ("/artifacts", "saved files"),
      44    ("/activity", "tool calls"),
      45    ("/work", "one step"),
      46    ("/work-verbose", "verbose step"),
      47    ("/focus", "set focus"),
      48    ("/switch", "set focus"),
      49    ("/history", "timeline"),
      50    ("/events", "event feed"),
      51    ("/updates", "durable work"),
      52    ("/outputs", "raw runs"),
      53    ("/artifact", "open output"),
      54    ("/findings", "finding ledger"),
      55    ("/tasks", "task queue"),
      56    ("/roadmap", "milestones"),
      57    ("/experiments", "measurements"),
      58    ("/sources", "source ledger"),
      59    ("/memory", "learning"),
      60    ("/metrics", "counts"),
      61    ("/lessons", "lessons"),
      62    ("/usage", "tokens/cost"),
      63    ("/config", "runtime config"),
      64    ("/health", "daemon health"),
      65    ("/start", "start daemon"),
      66    ("/restart", "restart daemon"),
      67    ("/model", "set model"),
      68    ("/base-url", "set endpoint"),
      69    ("/api-key", "save key"),
      70    ("/api-key-env", "key env var"),
      71    ("/context", "token budget"),
      72    ("/input-cost", "input $/1M"),
      73    ("/timeout", "request timeout"),
      74    ("/browser", "browser on/off"),
      75    ("/web", "web on/off"),
      76    ("/cli-access", "CLI on/off"),
      77    ("/file-access", "files on/off"),
      78    ("/home", "state directory"),
      79    ("/step-limit", "worker timeout"),
      80    ("/output-chars", "output preview size"),
      81    ("/output-cost", "output $/1M"),
      82    ("/daily-digest", "daily digest on/off"),
      83    ("/digest-time", "digest time"),
      84    ("/doctor", "check setup"),
      85    ("/init", "write config"),
      86    ("/pause", "pause job"),
      87    ("/resume", "resume job"),
      88    ("/stop", "pause job"),
      89    ("/cancel", "cancel job"),
      90    ("/delete", "delete job"),
      91    ("/learn", "save lesson"),
      92    ("/note", "save note"),
      93    ("/follow", "queue follow-up"),
      94    ("/digest", "digest"),
      95    ("/clear", "clear notices"),
      96    ("/exit", "quit"),
      97]
      98
      99SETTINGS_FIELD_TYPES = {
     100    "model.name": "str",
     101    "model.base_url": "str",
     102    "model.api_key_env": "str",
     103    "model.context_length": "int",
     104    "model.request_timeout_seconds": "float",
     105    "model.input_cost_per_million": "float",
     106    "model.output_cost_per_million": "float",
     107    "runtime.home": "path",
     108    "runtime.max_step_seconds": "int",
     109    "runtime.artifact_inline_char_limit": "int",
     110    "runtime.daily_digest_enabled": "bool",
     111    "runtime.daily_digest_time": "str",
     112    "runtime.max_job_cost_usd": "float",
     113    "tools.browser": "bool",
     114    "tools.web": "bool",
     115    "tools.shell": "bool",
     116    "tools.files": "bool",
     117}
     118
     119CHAT_SETTING_COMMANDS = {
     120    "model": ("model.name", "MODEL"),
     121    "base-url": ("model.base_url", "URL"),
     122    "api-key-env": ("model.api_key_env", "ENV_NAME"),
     123    "context": ("model.context_length", "TOKENS"),
     124    "input-cost": ("model.input_cost_per_million", "DOLLARS_PER_1M_INPUT_TOKENS"),
     125    "output-cost": ("model.output_cost_per_million", "DOLLARS_PER_1M_OUTPUT_TOKENS"),
     126    "max-cost": ("runtime.max_job_cost_usd", "DOLLARS_OR_0"),
     127    "timeout": ("model.request_timeout_seconds", "SECONDS"),
     128    "browser": ("tools.browser", "true|false"),
     129    "web": ("tools.web", "true|false"),
     130    "cli-access": ("tools.shell", "true|false"),
     131    "file-access": ("tools.files", "true|false"),
     132    "home": ("runtime.home", "PATH"),
     133    "step-limit": ("runtime.max_step_seconds", "SECONDS"),
     134    "output-chars": ("runtime.artifact_inline_char_limit", "CHARS"),
     135    "daily-digest": ("runtime.daily_digest_enabled", "true|false"),
     136    "digest-time": ("runtime.daily_digest_time", "HH:MM"),
     137}
     138
     139SLASH_ARGUMENT_HINTS = {
     140    "new": "OBJECTIVE",
     141    "focus": "JOB_TITLE",
     142    "switch": "JOB_TITLE",
     143    "delete": "JOB_TITLE",
     144    "history": "LIMIT",
     145    "events": "LIMIT",
     146    "outputs": "LIMIT",
     147    "outcomes": "all",
     148    "updates": "all",
     149    "artifact": "QUERY_OR_ID",
     150    "work": "N",
     151    "work-verbose": "N",
     152    "learn": "LESSON",
     153    "note": "MESSAGE",
     154    "follow": "MESSAGE",
     155    **{command: placeholder for command, (_field, placeholder) in CHAT_SETTING_COMMANDS.items()},
     156    "api-key": "KEY",
     157    "key": "KEY",
     158}
     159
     160REQUIRED_SLASH_ARGUMENTS = {"new", "artifact", "learn", "note", "follow"}
     161
     162
     163def slash_suggestion_lines(
     164    input_buffer: str,
     165    commands: list[tuple[str, str]],
     166    *,
     167    width: int,
     168    limit: int = 5,
     169) -> list[str]:
     170    if not input_buffer.startswith("/"):
     171        return []
     172    parts = input_buffer[1:].split(maxsplit=1)
     173    token = parts[0].lower() if parts else ""
     174    if " " in input_buffer[1:]:
     175        hint = SLASH_ARGUMENT_HINTS.get(token)
     176        description = next((desc for cmd, desc in commands if cmd == f"/{token}"), "")
     177        if not hint:
     178            return []
     179        body = f"{_accent('/' + token)} {_muted(hint)}"
     180        if description:
     181            body += f"  {_muted(description)}"
     182        return [
     183            _muted("╭─ command " + "─" * max(0, width - 11)),
     184            _fit_ansi(_muted("│ ") + body, width),
     185            _fit_ansi(_muted(_slash_argument_footer(parts, hint)), width),
     186        ]
     187    command_names = [cmd for cmd, _desc in commands]
     188    selected_command = f"/{token}" if token else ""
     189    exact_selection = selected_command in command_names and input_buffer.rstrip() == selected_command
     190    if exact_selection:
     191        all_matches = commands
     192    else:
     193        all_matches = [(cmd, desc) for cmd, desc in commands if cmd[1:].startswith(token)]
     194        if not all_matches and token:
     195            all_matches = [(cmd, desc) for cmd, desc in commands if token in cmd[1:]]
     196    if exact_selection:
     197        selected_index = next((index for index, (cmd, _desc) in enumerate(all_matches) if cmd == selected_command), 0)
     198        start = max(0, min(selected_index - max(0, limit // 2), max(0, len(all_matches) - limit)))
     199        matches = all_matches[start : start + limit]
     200    else:
     201        matches = all_matches[:limit]
     202    if not matches:
     203        return [
     204            _muted("╭─ commands " + "─" * max(0, width - 12)),
     205            _fit_ansi(_muted("│ no matches"), width),
     206            _muted("╰" + "─" * max(0, width - 1)),
     207        ]
     208    cmd_width = min(14, max(len(cmd) for cmd, _ in matches) + 2)
     209    lines = [_fit_ansi(_muted("╭─ commands " + "─" * max(0, width - 12)), width)]
     210    for index, (cmd, desc) in enumerate(matches):
     211        active = cmd == selected_command if exact_selection else index == 0
     212        marker = _accent("›") if active else _muted(" ")
     213        hint = SLASH_ARGUMENT_HINTS.get(cmd[1:])
     214        command_text = cmd if not hint else f"{cmd} {hint}"
     215        command_width = cmd_width + (len(hint) + 1 if hint else 0)
     216        name = _bold(_accent(command_text)) if active else _accent(command_text)
     217        body = f"{_muted('│')} {marker} {_fit_ansi(name, command_width)} {_muted(desc)}"
     218        lines.append(_fit_ansi(body, width))
     219    hidden = max(0, len(all_matches) - len(matches))
     220    if hidden:
     221        lines.append(_fit_ansi(_muted("╰─ type to filter · enter selects first match"), width))
     222    else:
     223        lines.append(_fit_ansi(_muted("╰─ enter selects · tab fills · ↑↓ moves"), width))
     224    return lines
     225
     226
     227def autocomplete_slash(input_buffer: str, commands: list[tuple[str, str]]) -> str:
     228    if not input_buffer.startswith("/") or " " in input_buffer[1:]:
     229        return input_buffer
     230    matches = _slash_command_matches(input_buffer, commands)
     231    if not matches:
     232        return input_buffer
     233    return matches[0] + " "
     234
     235
     236def slash_completion_for_submit(input_buffer: str, commands: list[tuple[str, str]]) -> tuple[str, bool]:
     237    """Return the buffer to use and whether Enter should submit it now."""
     238
     239    if not input_buffer.startswith("/"):
     240        return input_buffer, True
     241    if " " in input_buffer[1:]:
     242        command = input_buffer[1:].split(maxsplit=1)[0].lower()
     243        if command in REQUIRED_SLASH_ARGUMENTS and not _slash_argument_text(input_buffer).strip():
     244            return input_buffer, False
     245        return input_buffer, True
     246    current = input_buffer.rstrip()
     247    if not current:
     248        return input_buffer, True
     249    command_names = {cmd for cmd, _desc in commands}
     250    token = current[1:].lower()
     251    exact = current in command_names
     252    if exact and token not in REQUIRED_SLASH_ARGUMENTS:
     253        return input_buffer, True
     254    matches = _slash_command_matches(input_buffer, commands)
     255    if not matches:
     256        return input_buffer, True
     257    selected = current if exact else matches[0]
     258    if selected[1:] not in REQUIRED_SLASH_ARGUMENTS:
     259        return selected, True
     260    suffix = " "
     261    completed = selected + suffix
     262    return completed, completed == input_buffer
     263
     264
     265def _slash_argument_text(input_buffer: str) -> str:
     266    parts = input_buffer[1:].split(maxsplit=1)
     267    return parts[1] if len(parts) > 1 else ""
     268
     269
     270def _slash_argument_footer(parts: list[str], hint: str) -> str:
     271    if len(parts) == 1:
     272        return f"╰─ type {hint}, then enter"
     273    return "╰─ enter sends"
     274
     275
     276def cycle_slash(input_buffer: str, commands: list[tuple[str, str]], *, direction: int) -> str:
     277    if not input_buffer.startswith("/") or " " in input_buffer[1:]:
     278        return input_buffer
     279    current = input_buffer.rstrip()
     280    command_names = [cmd for cmd, _desc in commands]
     281    matches = command_names if current in command_names else _slash_command_matches(input_buffer, commands)
     282    if not matches:
     283        return input_buffer
     284    try:
     285        index = matches.index(current)
     286    except ValueError:
     287        return matches[0] if direction >= 0 else matches[-1]
     288    return matches[(index + direction) % len(matches)]
     289
     290
     291def _slash_command_matches(input_buffer: str, commands: list[tuple[str, str]]) -> list[str]:
     292    token = input_buffer.strip()[1:].lower()
     293    matches = [cmd for cmd, _desc in commands if cmd[1:].startswith(token)]
     294    if not matches:
     295        matches = [cmd for cmd, _desc in commands if token in cmd[1:]]
     296    return matches

nipux_cli/tui_event_format.py 262 lines

       1"""Shared event formatting helpers for Nipux terminal renderers."""
       2
       3from __future__ import annotations
       4
       5import os
       6import re
       7import shlex
       8from pathlib import Path
       9from typing import Any
      10
      11from nipux_cli.metric_format import format_metric_value
      12from nipux_cli.tui_style import _one_line
      13
      14
      15def event_tool_args(metadata: dict[str, Any]) -> dict[str, Any]:
      16    input_data = metadata.get("input") if isinstance(metadata.get("input"), dict) else {}
      17    args = input_data.get("arguments") if isinstance(input_data.get("arguments"), dict) else {}
      18    return args
      19
      20
      21def shell_write_target(command: str) -> str:
      22    if not command.strip():
      23        return ""
      24    redirect = re.search(r"(?:^|\s)(?:1?>|>>)\s*([^\s;&|]+)", command)
      25    if redirect:
      26        target = redirect.group(1).strip("'\"")
      27        if target and not target.startswith("&"):
      28            return target
      29    try:
      30        parts = shlex.split(command)
      31    except ValueError:
      32        parts = command.split()
      33    for index, part in enumerate(parts):
      34        if part != "tee":
      35            continue
      36        for candidate in parts[index + 1 :]:
      37            if candidate.startswith("-"):
      38                continue
      39            return candidate
      40    return ""
      41
      42
      43def event_title_body(title: str, body: str, *, fallback: str) -> str:
      44    if title and body and title not in body:
      45        return f"{title} - {body}"
      46    return title or body or fallback
      47
      48
      49def experiment_metric_text(metadata: dict[str, Any]) -> str:
      50    value = metadata.get("metric_value")
      51    if value in (None, ""):
      52        return ""
      53    name = metadata.get("metric_name") or "metric"
      54    unit = metadata.get("metric_unit") or ""
      55    direction = metadata.get("result_direction") or metadata.get("decision") or ""
      56    return " ".join(part for part in [format_metric_value(name, value, unit), str(direction)] if part)
      57
      58
      59def event_clock(event: dict[str, Any]) -> str:
      60    compact = _compact_time(str(event.get("created_at") or ""))
      61    if len(compact) >= 16 and compact[10:11] == " ":
      62        return compact[11:16]
      63    return "" if compact == "?" else _one_line(compact, 5)
      64
      65
      66def event_hour(event: dict[str, Any]) -> str:
      67    compact = _compact_time(str(event.get("created_at") or ""))
      68    if len(compact) >= 13 and compact[10:11] == " ":
      69        return f"{compact[:13]}:00"
      70    if len(compact) >= 2:
      71        return compact
      72    return "recent"
      73
      74
      75def friendly_error_text(text: str) -> str:
      76    lowered = text.lower()
      77    if "key limit exceeded" in lowered:
      78        return "Provider key limit exceeded. Update the key limit or switch models."
      79    if "authenticationerror" in lowered or "user not found" in lowered or "401" in lowered:
      80        return "Model authentication failed. Update the API key with /api-key, then try again."
      81    if "permissiondeniederror" in lowered or "403" in lowered:
      82        return "Provider permission denied. Check model access or key limits."
      83    if (
      84        "apiconnectionerror" in lowered
      85        or "connection error" in lowered
      86        or "connection refused" in lowered
      87        or "failed to establish a new connection" in lowered
      88    ):
      89        return "Model endpoint is unreachable. Check /base-url or start the configured model server, then run /doctor."
      90    if "timeout" in lowered or "timed out" in lowered:
      91        return "Model request timed out. Check the endpoint/model or adjust /timeout, then run /doctor."
      92    return _one_line(clean_step_summary(text), 220)
      93
      94
      95def brief_reflection_text(text: str) -> str:
      96    clean = clean_step_summary(text)
      97    match = re.search(r"Reflection through step #?([0-9]+):\s*(.*?)(?:\. Best |\.\s*$|$)", clean)
      98    if match:
      99        counts = match.group(2)
     100        counts = counts.replace(", 0 active operator messages", "")
     101        counts = counts.replace(", 0 recent finding artifacts", "")
     102        return _one_line(f"reflected #{match.group(1)}: {counts}", 140)
     103    return _one_line(clean, 140)
     104
     105
     106def generic_display_text(value: Any) -> str:
     107    return " ".join(str(value).split())
     108
     109
     110def clean_step_summary(summary: Any) -> str:
     111    text = " ".join(str(summary).split())
     112    if text.startswith("write_artifact saved ") and " at /" in text:
     113        return text.split(" at /", 1)[0]
     114    return text
     115
     116
     117def chat_message_paragraphs(value: Any) -> list[str]:
     118    text = str(value).replace("\r\n", "\n").replace("\r", "\n")
     119    text = re.sub(r"\*\*([^*]+)\*\*", r"\1", text)
     120    text = re.sub(r"`([^`]+)`", r"\1", text)
     121    text = re.sub(r"(?<!^)\s(?=(?:[0-9]+\.|[-*])\s+)", "\n", text)
     122    paragraphs: list[str] = []
     123    for raw in text.splitlines():
     124        line = " ".join(raw.strip().split())
     125        if line:
     126            paragraphs.append(line)
     127    return paragraphs or [""]
     128
     129
     130def chat_agent_message_text(title: str, body: str) -> str:
     131    lowered = title.lower()
     132    if lowered == "chat":
     133        return body
     134    if lowered in {"plan", "planning"}:
     135        plan_body = body.split("Questions:", 1)[0]
     136        tasks = len(re.findall(r"(?:^|\s)- ", plan_body))
     137        if tasks:
     138            return f"Plan drafted with {tasks} items. Reply with changes or start work from the controls."
     139        return "Plan drafted. Reply with changes or start work from the controls."
     140    if lowered in {"progress", "update", "report"}:
     141        return _one_line(clean_step_summary(body), 220)
     142    return ""
     143
     144
     145def tool_live_summary(tool: str, metadata: dict[str, Any], body: str) -> str:
     146    args = event_tool_args(metadata)
     147    clean_body = clean_step_summary(body)
     148    if tool == "web_search":
     149        query = str(args.get("query") or _regex_group(r"query='([^']+)'", clean_body) or "")
     150        return f"search {query}" if query else "search web"
     151    if tool == "web_extract":
     152        urls = args.get("urls") if isinstance(args.get("urls"), list) else []
     153        count = len(urls)
     154        fetched = _regex_group(r"fetched ([0-9]+/[0-9]+ pages)", clean_body)
     155        return f"extract {fetched or (str(count) + ' pages' if count else 'pages')}"
     156    if tool == "shell_exec":
     157        command = str(args.get("command") or _regex_group(r"cmd='([^']+)'", clean_body) or "")
     158        prefix = f"shell {_short_command(command)}" if command else "shell command"
     159        rc = metadata.get("output", {}).get("returncode") if isinstance(metadata.get("output"), dict) else None
     160        return f"{prefix} rc={rc}" if rc is not None else prefix
     161    if tool == "browser_navigate":
     162        url = str(args.get("url") or _regex_group(r"<([^>]+)>", clean_body) or "")
     163        return f"open {_short_url(url)}" if url else "open page"
     164    if tool == "browser_snapshot":
     165        return "snapshot page"
     166    if tool == "browser_click":
     167        ref = str(args.get("ref") or "")
     168        return f"click {ref}" if ref else "click page"
     169    if tool == "browser_scroll":
     170        return f"scroll {args.get('direction') or 'page'}"
     171    if tool == "write_artifact":
     172        return "save output"
     173    if tool == "write_file":
     174        args_path = str(args.get("path") or "")
     175        output = metadata.get("output") if isinstance(metadata.get("output"), dict) else {}
     176        path = str(output.get("path") or args_path)
     177        return f"update {short_path(path, max_width=36)}" if path else "update file"
     178    if tool == "defer_job":
     179        seconds = args.get("seconds") or args.get("delay_seconds")
     180        until = args.get("until")
     181        if until:
     182            return f"wait until {until}"
     183        return f"wait {seconds}s" if seconds else "wait before next check"
     184    if tool == "record_lesson":
     185        return "learn memory"
     186    if tool == "record_memory_graph":
     187        return "map memory"
     188    if tool == "search_memory_graph":
     189        return "search memory"
     190    if tool == "record_source":
     191        return "score source"
     192    if tool == "record_findings":
     193        return "record findings"
     194    if tool == "record_tasks":
     195        return "update tasks"
     196    if tool == "record_roadmap":
     197        return "update roadmap"
     198    if tool == "record_milestone_validation":
     199        return "validate roadmap"
     200    if tool == "record_experiment":
     201        return "record experiment"
     202    if tool == "acknowledge_operator_context":
     203        return "ack operator"
     204    if tool == "report_update":
     205        return "report update"
     206    if tool == "read_artifact":
     207        return "read output"
     208    if tool == "search_artifacts":
     209        return "search outputs"
     210    return tool or clean_body or "step"
     211
     212
     213def short_path(path: Path | str, *, max_width: int = 80) -> str:
     214    text = str(path)
     215    home = str(Path.home())
     216    if text.startswith(home + os.sep):
     217        text = "~" + text[len(home) :]
     218    if len(text) <= max_width:
     219        return text
     220    keep = max(12, max_width - 4)
     221    return "..." + text[-keep:]
     222
     223
     224def _regex_group(pattern: str, text: str) -> str:
     225    match = re.search(pattern, text)
     226    return match.group(1) if match else ""
     227
     228
     229def _short_url(url: str) -> str:
     230    if not url:
     231        return ""
     232    stripped = url.replace("https://", "").replace("http://", "")
     233    return stripped.split("/", 1)[0] or stripped
     234
     235
     236def _short_command(command: str) -> str:
     237    if not command:
     238        return ""
     239    try:
     240        parts = shlex.split(command)
     241    except ValueError:
     242        parts = command.split()
     243    if not parts:
     244        return ""
     245    if parts[0] == "ssh":
     246        host = next((part for part in parts[1:] if not part.startswith("-") and "=" not in part), "")
     247        remote = " ".join(parts[parts.index(host) + 1 :]) if host in parts else ""
     248        if remote:
     249            remote_parts = remote.split()
     250            remote_head = remote_parts[0] if remote_parts else "remote"
     251            return f"ssh {host} {remote_head}"
     252        return f"ssh {host}".strip()
     253    if parts[0] in {"python", "python3", "uv", "npm", "pnpm", "yarn", "node"} and len(parts) > 1:
     254        return " ".join(parts[:3])
     255    return " ".join(parts[:2])
     256
     257
     258def _compact_time(value: str) -> str:
     259    text = value.replace("T", " ")
     260    if len(text) >= 16 and text[4:5] == "-" and text[13:14] == ":":
     261        return text[:16]
     262    return _one_line(text, 16)

nipux_cli/tui_events.py 371 lines

       1"""Compact event rendering helpers for the Nipux terminal UI."""
       2
       3from __future__ import annotations
       4
       5import re
       6import textwrap
       7from typing import Any
       8
       9from nipux_cli.tui_event_format import (
      10    brief_reflection_text,
      11    chat_message_paragraphs,
      12    event_clock,
      13    event_title_body,
      14    friendly_error_text,
      15    generic_display_text,
      16    tool_live_summary,
      17)
      18from nipux_cli.tui_style import (
      19    _accent,
      20    _bold,
      21    _center_ansi,
      22    _fit_ansi,
      23    _muted,
      24    _one_line,
      25    _style,
      26)
      27
      28THINKING_NOTICE_PREFIX = "__nipux_thinking__:"
      29WAITING_NOTICE_PREFIX = "__nipux_waiting__:"
      30
      31LOW_VALUE_CHAT_NOTICES = (
      32    "sent; waiting for model",
      33    "sent, waiting for model",
      34    "waiting for model",
      35    "waiting for the next worker step",
      36)
      37
      38NIPUX_HERO = [
      39    "███╗   ██╗██╗██████╗ ██╗   ██╗██╗  ██╗",
      40    "████╗  ██║██║██╔══██╗██║   ██║╚██╗██╔╝",
      41    "██╔██╗ ██║██║██████╔╝██║   ██║ ╚███╔╝ ",
      42    "██║╚██╗██║██║██╔═══╝ ██║   ██║ ██╔██╗ ",
      43    "██║ ╚████║██║██║     ╚██████╔╝██╔╝ ██╗",
      44    "╚═╝  ╚═══╝╚═╝╚═╝      ╚═════╝ ╚═╝  ╚═╝",
      45]
      46
      47LOW_SIGNAL_FRAME_TOOLS = {
      48    "acknowledge_operator_context",
      49    "read_artifact",
      50    "record_experiment",
      51    "record_findings",
      52    "record_lesson",
      53    "record_milestone_validation",
      54    "record_roadmap",
      55    "record_source",
      56    "record_tasks",
      57    "reflect",
      58    "report_update",
      59    "search_artifacts",
      60    "update_job_state",
      61    "write_artifact",
      62}
      63
      64GENERIC_CHAT_NOTICE_PREFIXES = (
      65    "opened ",
      66    "focus set",
      67    "paused ",
      68    "resumed ",
      69    "cancelled ",
      70    "deleted ",
      71)
      72
      73
      74def chat_event_parts(event: dict[str, Any]) -> tuple[str, str, str] | None:
      75    kind = str(event.get("event_type") or "")
      76    title = str(event.get("title") or "").strip()
      77    body = str(event.get("body") or "")
      78    metadata = event.get("metadata") if isinstance(event.get("metadata"), dict) else {}
      79    clock = event_clock(event)
      80    if kind == "operator_message":
      81        return "YOU", body, clock
      82    if kind == "agent_message" and title == "chat":
      83        if metadata.get("error"):
      84            body = friendly_error_text(body)
      85        if _is_low_value_chat_notice(_normalized_chat_body(body)) or _is_waiting_notice(_normalized_chat_body(body)):
      86            return None
      87        return "AGENT", body, clock
      88    return None
      89
      90
      91def append_chat_output(lines: list[str], label: str, body: Any, *, clock: str, width: int) -> None:
      92    label_text = _chat_label(label)
      93    meta = f"{label_text} {_muted(clock)}" if clock else label_text
      94    lines.append(_fit_ansi(meta, width))
      95    prefix = "  "
      96    available = max(18, width - len(prefix))
      97    for paragraph in chat_message_paragraphs(body):
      98        wrapped = textwrap.wrap(paragraph, width=available) or [""]
      99        for part in wrapped:
     100            lines.append(_fit_ansi(prefix + part, width))
     101    if len(lines) == 1:
     102        lines.append(_fit_ansi(prefix, width))
     103
     104
     105def chat_pane_lines(events: list[dict[str, Any]], notices: list[str], *, width: int, rows: int) -> list[str]:
     106    items: list[tuple[str, str, str]] = []
     107    seen_chat_bodies: set[tuple[str, str]] = set()
     108    seen_bodies: set[str] = set()
     109    for event in events:
     110        rendered = chat_event_parts(event)
     111        if not rendered:
     112            continue
     113        label, body, clock = rendered
     114        items.append((label, body, clock))
     115        normalized = _normalized_chat_body(body)
     116        seen_chat_bodies.add((label, normalized))
     117        seen_bodies.add(normalized)
     118    for notice in notices:
     119        if notice.startswith(THINKING_NOTICE_PREFIX):
     120            body = notice.removeprefix(THINKING_NOTICE_PREFIX).strip() or "thinking"
     121            items.append(("THINKING", body, ""))
     122        elif notice.startswith(WAITING_NOTICE_PREFIX):
     123            body = notice.removeprefix(WAITING_NOTICE_PREFIX).strip() or "waiting"
     124            items.append(("WAITING", body, ""))
     125        elif notice.startswith("> "):
     126            body = notice[2:]
     127            normalized = _normalized_chat_body(body)
     128            if normalized and normalized not in seen_bodies and ("YOU", normalized) not in seen_chat_bodies:
     129                items.append(("YOU", body, ""))
     130                seen_bodies.add(normalized)
     131        else:
     132            body = notice
     133            normalized = _normalized_chat_body(body)
     134            if _is_low_value_chat_notice(normalized) or _is_waiting_notice(normalized) or _is_generic_chat_notice(normalized):
     135                continue
     136            if normalized and normalized not in seen_bodies and ("AGENT", normalized) not in seen_chat_bodies:
     137                items.append(("NIPUX", body, ""))
     138                seen_bodies.add(normalized)
     139    if not items:
     140        return chat_empty_state_lines(width=width, rows=rows)
     141    rendered_items = [_chat_item_lines(label, body, clock=clock, width=width) for label, body, clock in items[-max(4, rows) :]]
     142    output_rows = _flatten_chat_blocks(rendered_items)
     143    if len(output_rows) <= rows:
     144        return output_rows
     145    if rows <= 1:
     146        return output_rows[-rows:]
     147    newest = rendered_items[-1]
     148    if len(newest) >= rows:
     149        if rows <= 3:
     150            visible = newest[: rows - 1]
     151            hidden = len(newest) - len(visible)
     152            marker = _fit_ansi(_muted(f"... {hidden} more lines in /history."), width)
     153            return [*visible, marker]
     154        head = max(1, min(4, rows // 3))
     155        tail = max(1, rows - head - 1)
     156        hidden = max(0, len(newest) - head - tail)
     157        marker = _fit_ansi(_muted(f"... {hidden} middle lines hidden; /history shows all."), width)
     158        return [*newest[:head], marker, *newest[-tail:]]
     159    visible_blocks: list[list[str]] = [newest]
     160    used = len(newest)
     161    hidden_lines = 0
     162    for block in reversed(rendered_items[:-1]):
     163        if used + len(block) + 1 <= rows:
     164            visible_blocks.insert(0, block)
     165            used += len(block)
     166        else:
     167            hidden_lines += len(block)
     168    marker = _fit_ansi(_muted(f"... {hidden_lines} older chat lines hidden; /history shows all."), width)
     169    return [marker, *_flatten_chat_blocks(visible_blocks)][:rows]
     170
     171
     172def _chat_item_lines(label: str, body: Any, *, clock: str, width: int) -> list[str]:
     173    if label == "THINKING":
     174        return [_fit_ansi(f"{_style('AGENT', '36;1')}  {_accent(str(body))}", width)]
     175    if label == "WAITING":
     176        return [_fit_ansi(f"{_style('AGENT', '36;1')}  {_muted(str(body))}", width)]
     177    lines: list[str] = []
     178    append_chat_output(lines, label, body, clock=clock, width=width)
     179    return lines
     180
     181
     182def _flatten_chat_blocks(blocks: list[list[str]]) -> list[str]:
     183    rows: list[str] = []
     184    for block in blocks:
     185        if rows:
     186            rows.append("")
     187        rows.extend(block)
     188    return rows
     189
     190
     191def _chat_label(label: str) -> str:
     192    if label == "YOU":
     193        return _style("you", "35;1")
     194    if label == "AGENT":
     195        return _style("nipux", "36;1")
     196    if label == "THINKING":
     197        return _style("thinking", "36")
     198    if label == "WAITING":
     199        return _style("waiting", "36")
     200    return _muted(label.lower())
     201
     202
     203def _normalized_chat_body(value: Any) -> str:
     204    return " ".join(str(value or "").split())
     205
     206
     207def _is_low_value_chat_notice(normalized: str) -> bool:
     208    lowered = normalized.lower()
     209    return any(phrase in lowered for phrase in LOW_VALUE_CHAT_NOTICES)
     210
     211
     212def _is_waiting_notice(normalized: str) -> bool:
     213    lowered = normalized.lower()
     214    return (
     215        lowered.startswith("waiting:")
     216        or lowered.startswith("waiting for ")
     217        or "message saved for the worker" in lowered
     218    )
     219
     220
     221def _is_generic_chat_notice(normalized: str) -> bool:
     222    lowered = normalized.lower()
     223    return any(lowered.startswith(prefix) for prefix in GENERIC_CHAT_NOTICE_PREFIXES)
     224
     225
     226def chat_empty_state_lines(*, width: int, rows: int) -> list[str]:
     227    content = [
     228        _center_ansi(_bold(_accent("NIPUX")), width),
     229        "",
     230        *_centered_wrapped_hint("Type a goal in plain English to start a worker.", width=width),
     231        *_centered_wrapped_hint("Or use /new OBJECTIVE.", width=width),
     232        *_centered_wrapped_hint("/settings configures provider/tools.", width=width),
     233    ]
     234    top_pad = max(0, (rows - len(content)) // 2)
     235    return ([""] * top_pad + content)[:rows]
     236
     237
     238def _centered_wrapped_hint(text: str, *, width: int) -> list[str]:
     239    available = max(24, min(68, width - 4))
     240    return [_center_ansi(_muted(part), width) for part in textwrap.wrap(text, width=available) or [text]]
     241
     242
     243def worker_activity_lines(events: list[dict[str, Any]], *, width: int, limit: int) -> list[str]:
     244    items: list[dict[str, Any]] = []
     245    for event in events:
     246        line = minimal_live_event_line(event, chars=max(16, width - 12))
     247        if not line:
     248            continue
     249        if items and items[-1].get("key") == line:
     250            items[-1]["count"] = int(items[-1].get("count") or 1) + 1
     251            continue
     252        items.append({"line": line, "count": 1, "key": line})
     253    rendered = []
     254    for item in items[-limit:]:
     255        line = str(item["line"])
     256        count = int(item.get("count") or 1)
     257        rendered.append(f"{live_badge(line)} {_one_line(live_display_text(line, count=count), max(16, width - 9))}")
     258    return rendered
     259
     260
     261def minimal_live_event_line(event: dict[str, Any], *, chars: int = 92) -> str:
     262    kind = str(event.get("event_type") or "")
     263    title = str(event.get("title") or "").strip()
     264    body = generic_display_text(event.get("body") or "")
     265    metadata = event.get("metadata") if isinstance(event.get("metadata"), dict) else {}
     266    status = str(metadata.get("status") or "")
     267    if kind == "operator_message":
     268        return ""
     269    if kind == "agent_message" and title == "chat":
     270        return ""
     271    if kind == "operator_context":
     272        return _one_line(f"operator {title or body}", chars)
     273    if kind == "tool_call":
     274        if title in LOW_SIGNAL_FRAME_TOOLS:
     275            return ""
     276        return _one_line("start " + tool_live_summary(title, metadata, body), chars)
     277    if kind == "tool_result":
     278        if title in LOW_SIGNAL_FRAME_TOOLS and status == "completed":
     279            return ""
     280        if title == "llm" and status == "failed":
     281            return ""
     282        prefix = "blocked" if status == "blocked" else "failed" if status == "failed" else "done"
     283        detail = friendly_error_text(body or title) if status in {"blocked", "failed"} else tool_live_summary(title, metadata, body)
     284        return _one_line(f"{prefix} {detail}", chars)
     285    if kind == "error":
     286        detail = friendly_error_text(body or title or "error")
     287        return _one_line(f"error {detail}", chars)
     288    if kind == "artifact":
     289        return _one_line(f"saved {title or body or 'output'}", chars)
     290    if kind == "finding":
     291        return _one_line(f"finding {title or body}", chars)
     292    if kind == "source":
     293        return _one_line(f"source {title or body}", chars)
     294    if kind == "task":
     295        return _one_line(f"task {title or body}", chars)
     296    if kind == "roadmap":
     297        return _one_line(f"roadmap {title or body}", chars)
     298    if kind == "milestone_validation":
     299        validation = str(metadata.get("validation_status") or metadata.get("status") or "")
     300        return _one_line(f"validate {validation} {title or body}".strip(), chars)
     301    if kind == "experiment":
     302        return _one_line(f"experiment {title or body}", chars)
     303    if kind == "lesson":
     304        detail = event_title_body(title, body, fallback="lesson")
     305        return _one_line(f"learned {detail}", chars)
     306    if kind == "reflection":
     307        return _one_line(f"reflect {brief_reflection_text(body or title)}", chars)
     308    if kind == "agent_message":
     309        if title not in {"chat", "progress", "update", "report"}:
     310            return ""
     311        return _one_line(f"update {body or title}", chars)
     312    if kind in {"daemon", "loop"}:
     313        return ""
     314    return ""
     315
     316
     317def live_badge(text: str) -> str:
     318    badge_text = re.sub(r"^x[0-9]+\s+", "", text)
     319    if badge_text.startswith("error") or badge_text.startswith("failed"):
     320        return _style("FAIL", "31")
     321    if badge_text.startswith("blocked"):
     322        return _style("BLOCK", "33")
     323    if badge_text.startswith("start"):
     324        return _style("run ", "36")
     325    if badge_text.startswith("done"):
     326        return _style("done", "32")
     327    if badge_text.startswith("saved"):
     328        return _style("save", "32")
     329    if badge_text.startswith("finding"):
     330        return _style("find", "32")
     331    if badge_text.startswith("source"):
     332        return _style("src ", "36")
     333    if badge_text.startswith("experiment"):
     334        return _style("test", "33")
     335    if badge_text.startswith("task"):
     336        return _style("task", "33")
     337    if badge_text.startswith("learned"):
     338        return _style("mem ", "36")
     339    if badge_text.startswith("reflect"):
     340        return _style("plan", "35")
     341    if badge_text.startswith("update"):
     342        return _style("note", "35")
     343    return _style("info", "2")
     344
     345
     346def live_display_text(text: str, *, count: int = 1) -> str:
     347    if count > 1 and (
     348        text.startswith("error")
     349        or text.startswith("failed")
     350        or text.startswith("blocked")
     351    ):
     352        return f"x{count} {text}"
     353    base = text
     354    for prefix in (
     355        "start ",
     356        "done ",
     357        "saved ",
     358        "finding ",
     359        "source ",
     360        "experiment ",
     361        "task ",
     362        "learned ",
     363        "reflect ",
     364        "update ",
     365    ):
     366        if text.startswith(prefix):
     367            base = text[len(prefix) :]
     368            break
     369    if count > 1:
     370        return f"{base} x{count}"
     371    return base

nipux_cli/tui_input.py 80 lines

       1"""Terminal input helpers for full-screen Nipux frames."""
       2
       3from __future__ import annotations
       4
       5import os
       6import re
       7import select
       8import sys
       9import time
      10
      11
      12def read_terminal_char(fd: int) -> str:
      13    data = os.read(fd, 1)
      14    return data.decode("latin1", errors="ignore")
      15
      16
      17def read_escape_sequence(first: str, *, fd: int | None = None) -> str:
      18    fd = sys.stdin.fileno() if fd is None else fd
      19    sequence = first
      20    deadline = time.monotonic() + 0.12
      21    while len(sequence) < 96:
      22        timeout = max(0.0, min(0.04, deadline - time.monotonic()))
      23        if timeout <= 0:
      24            break
      25        readable, _, _ = select.select([fd], [], [], timeout)
      26        if not readable:
      27            break
      28        sequence += read_terminal_char(fd)
      29        if terminal_escape_complete(sequence):
      30            break
      31    return sequence
      32
      33
      34def terminal_escape_complete(sequence: str) -> bool:
      35    if sequence in {"\x1b[A", "\x1b[B", "\x1b[C", "\x1b[D", "\x1bOA", "\x1bOB", "\x1bOC", "\x1bOD"}:
      36        return True
      37    if re.match(r"^\x1b\[[0-9;?]*[ABCD]$", sequence):
      38        return True
      39    if re.match(r"^\x1b\[<\d+;\d+;\d+[mM]$", sequence):
      40        return True
      41    if sequence.startswith("\x1b[M") and len(sequence) >= 6:
      42        return True
      43    return False
      44
      45
      46def decode_terminal_escape(sequence: str) -> tuple[str, tuple[int, int] | None]:
      47    arrows = {
      48        "\x1b[A": "up",
      49        "\x1b[B": "down",
      50        "\x1b[C": "right",
      51        "\x1b[D": "left",
      52        "\x1bOA": "up",
      53        "\x1bOB": "down",
      54        "\x1bOC": "right",
      55        "\x1bOD": "left",
      56    }
      57    if sequence in arrows:
      58        return arrows[sequence], None
      59    csi_arrow = re.match(r"^\x1b\[[0-9;?]*([ABCD])$", sequence)
      60    if csi_arrow:
      61        return {"A": "up", "B": "down", "C": "right", "D": "left"}[csi_arrow.group(1)], None
      62    match = re.match(r"^\x1b\[<(\d+);(\d+);(\d+)([mM])$", sequence)
      63    if match and match.group(4) == "M":
      64        button = int(match.group(1))
      65        if button == 0:
      66            return "click", (int(match.group(2)), int(match.group(3)))
      67    if sequence.startswith("\x1b[M") and len(sequence) >= 6:
      68        button = ord(sequence[3]) - 32
      69        if button == 0:
      70            return "click", (ord(sequence[4]) - 32, ord(sequence[5]) - 32)
      71    return "unknown", None
      72
      73
      74def drain_pending_input(fd: int | None = None) -> None:
      75    fd = sys.stdin.fileno() if fd is None else fd
      76    while True:
      77        readable, _, _ = select.select([fd], [], [], 0)
      78        if not readable:
      79            return
      80        os.read(fd, 1)

nipux_cli/tui_layout.py 234 lines

       1"""Reusable terminal layout primitives for Nipux frames."""
       2
       3from __future__ import annotations
       4
       5from typing import Any
       6
       7from nipux_cli.tui_style import (
       8    _accent,
       9    _bold,
      10    _fit_ansi,
      11    _muted,
      12    _one_line,
      13    _strip_ansi,
      14    _style,
      15)
      16
      17
      18def _top_bar(
      19    width: int,
      20    *,
      21    state: str,
      22    daemon: str,
      23    model: str,
      24    token_usage: dict[str, Any] | None = None,
      25    context_length: int = 0,
      26    base_url: str = "",
      27) -> list[str]:
      28    del state, daemon
      29    title = _style("NIPUX", "38;5;123;1")
      30    usage_text = _token_usage_topline(token_usage or {}, context_length=context_length, model=model, base_url=base_url)
      31    model_line = f"{_muted('model')} {_style(_one_line(model, max(16, width // 3)), '36')}"
      32    if width >= 118:
      33        compact_model = f"{_muted('model')} {_style(_one_line(model, max(14, width // 5)), '36')}"
      34        return [
      35            _edge_line(title, f"{compact_model}  {usage_text}", width=width),
      36            _muted("━" * width),
      37        ]
      38    first = _edge_line(title, model_line, width=width)
      39    second = _edge_line("", usage_text, width=width)
      40    return [
      41        first,
      42        second,
      43        _muted("━" * width),
      44    ]
      45
      46
      47def _two_col_title(left_width: int, right_width: int, left: str, right: str) -> str:
      48    left_title = _style(left.upper(), "38;5;252;1")
      49    right_title = _style(right.upper(), "38;5;252;1")
      50    return _fit_ansi(left_title, left_width) + _muted(" │ ") + _fit_ansi(right_title, right_width)
      51
      52
      53def _two_col_line(left: str, right: str, *, left_width: int, right_width: int) -> str:
      54    return _fit_ansi(left, left_width) + _muted(" │ ") + _fit_ansi(right, right_width)
      55
      56
      57def _edge_line(left: str, right: str, *, width: int) -> str:
      58    right_len = len(_strip_ansi(right))
      59    left_width = max(0, width - right_len - 2)
      60    left_text = _fit_ansi(left, left_width)
      61    gap = max(1, width - len(_strip_ansi(left_text)) - right_len)
      62    return _fit_ansi(left_text + " " * gap + right, width)
      63
      64
      65def _triple_line(left: str, center: str, right: str, *, width: int) -> str:
      66    right_len = len(_strip_ansi(right))
      67    center_len = len(_strip_ansi(center))
      68    left_len = len(_strip_ansi(left))
      69    center_start = max(left_len + 2, (width - center_len) // 2)
      70    right_start = max(center_start + center_len + 1, width - right_len)
      71    if right_start >= width:
      72        return _edge_line(center, right, width=width)
      73    parts = [
      74        left,
      75        " " * max(1, center_start - left_len),
      76        center,
      77        " " * max(1, right_start - center_start - center_len),
      78        right,
      79    ]
      80    return _fit_ansi("".join(parts), width)
      81
      82
      83def _compose_bar(
      84    input_buffer: str,
      85    *,
      86    width: int,
      87    hint: str | None = None,
      88    suggestions: list[str] | None = None,
      89    prompt_label: str = "❯",
      90    title: str = "message",
      91    mask_input: bool = False,
      92) -> list[str]:
      93    if mask_input:
      94        visible_input = "•" * min(len(input_buffer), max(8, width - 8))
      95    else:
      96        visible_input = input_buffer[-max(8, width - 8) :]
      97    hint = _muted(hint or "Enter send  ·  / commands  ·  arrows navigate")
      98    label = _accent(prompt_label) if prompt_label == "❯" else _muted(prompt_label)
      99    prompt = f"{label} {visible_input}{_accent('▌')}"
     100    lines = []
     101    if suggestions:
     102        lines.extend(suggestions)
     103    title = f" {title.strip()} "
     104    lines.extend([
     105        _muted("╭─" + title + "─" * max(0, width - len(title) - 2)),
     106        _fit_ansi(_muted("│ ") + prompt, width),
     107        _fit_ansi(_muted("╰─ ") + hint, width),
     108    ])
     109    return lines
     110
     111
     112def _metric_strip(items: list[tuple[str, Any]], *, width: int) -> str:
     113    parts = [f"{_muted(label)} {_bold(value)}" for label, value in items]
     114    text = "  ".join(parts)
     115    if len(_strip_ansi(text)) <= width:
     116        return text
     117    compact = [f"{label}:{value}" for label, value in items]
     118    return _one_line("  ".join(compact), width)
     119
     120
     121def _pill(label: str, value: Any) -> str:
     122    value_text = str(value)
     123    color = "36"
     124    lowered = value_text.lower()
     125    if any(term in lowered for term in ("running", "active", "advancing", "ok")):
     126        color = "32"
     127    elif any(term in lowered for term in ("paused", "idle", "queued", "planning")):
     128        color = "33"
     129    elif any(term in lowered for term in ("failed", "cancelled", "error", "stopped")):
     130        color = "31"
     131    return f"{_muted(label)} {_style(value_text, color)}"
     132
     133
     134def _token_usage_topline(
     135    usage: dict[str, Any],
     136    *,
     137    context_length: int,
     138    model: str,
     139    base_url: str,
     140) -> str:
     141    calls = _safe_int(usage.get("calls"))
     142    if calls <= 0:
     143        return (
     144            f"{_muted('ctx')} {_style('0', '36')}  "
     145            f"{_muted('out')} {_style('0', '36')}  "
     146            f"{_muted('tok')} {_style('0', '36')}  "
     147            f"{_muted('cost')} {_style('$0.00', '36')}"
     148        )
     149    latest_prompt = _safe_int(usage.get("latest_prompt_tokens"))
     150    completion = _safe_int(usage.get("completion_tokens"))
     151    total = _safe_int(usage.get("total_tokens")) or latest_prompt + completion
     152    ctx_text = _format_compact_count(latest_prompt)
     153    if context_length > 0:
     154        ctx_text = f"{ctx_text}/{_format_compact_count(context_length)}"
     155    cost_text = _format_usage_cost(usage, model=model, base_url=base_url)
     156    return (
     157        f"{_muted('ctx')} {_style(ctx_text, '36')}  "
     158        f"{_muted('out')} {_style(_format_compact_count(completion), '36')}  "
     159        f"{_muted('tok')} {_style(_format_compact_count(total), '36')}  "
     160        f"{_muted('cost')} {_style(cost_text, '36')}"
     161    )
     162
     163
     164def _model_cost_is_zero(*, model: str, base_url: str) -> bool:
     165    lowered_model = model.lower()
     166    lowered_url = base_url.lower()
     167    return (
     168        lowered_model.endswith(":free")
     169        or lowered_model in {"local-model", "fake", "test"}
     170        or "localhost" in lowered_url
     171        or "127.0.0.1" in lowered_url
     172    )
     173
     174
     175def _format_usage_cost(usage: dict[str, Any], *, model: str, base_url: str) -> str:
     176    if bool(usage.get("has_cost")):
     177        return f"${_safe_float(usage.get('cost')):.4f}"
     178    if _model_cost_is_zero(model=model, base_url=base_url):
     179        return "$0.00"
     180    input_rate = _safe_optional_float(usage.get("input_cost_per_million"))
     181    output_rate = _safe_optional_float(usage.get("output_cost_per_million"))
     182    if input_rate is not None and output_rate is not None:
     183        prompt = _safe_int(usage.get("prompt_tokens"))
     184        completion = _safe_int(usage.get("completion_tokens"))
     185        if prompt > 0 or completion > 0:
     186            estimated = (prompt / 1_000_000 * input_rate) + (completion / 1_000_000 * output_rate)
     187            return f"~${estimated:.4f}"
     188    if _safe_int(usage.get("estimated_calls")):
     189        return "pending"
     190    return "pending"
     191
     192
     193def _format_compact_count(value: Any) -> str:
     194    number = _safe_int(value)
     195    if number >= 1_000_000_000:
     196        return f"{number / 1_000_000_000:.1f}B"
     197    if number >= 1_000_000:
     198        return f"{number / 1_000_000:.1f}M"
     199    if number >= 1_000:
     200        return f"{number / 1_000:.1f}K"
     201    return str(number)
     202
     203
     204def _safe_int(value: Any) -> int:
     205    try:
     206        return int(float(value))
     207    except (TypeError, ValueError):
     208        return 0
     209
     210
     211def _safe_float(value: Any) -> float:
     212    try:
     213        return float(value)
     214    except (TypeError, ValueError):
     215        return 0.0
     216
     217
     218def _safe_optional_float(value: Any) -> float | None:
     219    if value in (None, ""):
     220        return None
     221    try:
     222        return float(value)
     223    except (TypeError, ValueError):
     224        return None
     225
     226
     227def _status_dot(state: str) -> str:
     228    if state in {"advancing", "running", "active"}:
     229        return _style("●", "32")
     230    if state in {"paused", "queued", "planning", "idle"}:
     231        return _style("●", "33")
     232    if state in {"failed", "cancelled"}:
     233        return _style("●", "31")
     234    return _style("●", "36")

nipux_cli/tui_outcomes.py 508 lines

       1"""Durable outcome summaries for the Nipux terminal UI."""
       2
       3from __future__ import annotations
       4
       5import textwrap
       6from typing import Any
       7
       8from nipux_cli.tui_event_format import (
       9    brief_reflection_text,
      10    chat_agent_message_text,
      11    event_clock,
      12    event_hour,
      13    event_title_body,
      14    event_tool_args,
      15    experiment_metric_text,
      16    generic_display_text,
      17    shell_write_target,
      18    short_path,
      19    tool_live_summary,
      20)
      21from nipux_cli.tui_style import _bold, _event_badge, _fit_ansi, _muted, _one_line, _page_indicator, _strip_ansi
      22
      23
      24CHAT_RIGHT_PAGES = [("updates", "Updates"), ("status", "Jobs")]
      25
      26DURABLE_OUTCOME_LABELS = {
      27    "SAVE",
      28    "FIND",
      29    "SOURCE",
      30    "TEST",
      31    "TASK",
      32    "ROAD",
      33    "VALID",
      34    "LEARN",
      35    "FILE",
      36}
      37
      38SUMMARY_COUNT_LABELS = DURABLE_OUTCOME_LABELS | {"DONE", "FAIL"}
      39PRIMARY_OUTCOME_LABELS = DURABLE_OUTCOME_LABELS | {"FAIL"}
      40OUTCOME_LABEL_ORDER = [
      41    "SAVE",
      42    "FIND",
      43    "TEST",
      44    "FILE",
      45    "TASK",
      46    "ROAD",
      47    "VALID",
      48    "SOURCE",
      49    "LEARN",
      50    "PLAN",
      51    "UPDATE",
      52    "FAIL",
      53    "DONE",
      54]
      55
      56OUTCOME_SUMMARY_NAMES = {
      57    "SAVE": "outputs",
      58    "FIND": "findings",
      59    "SOURCE": "sources",
      60    "TEST": "measurements",
      61    "TASK": "tasks",
      62    "ROAD": "roadmap",
      63    "VALID": "validations",
      64    "LEARN": "lessons",
      65    "PLAN": "plans",
      66    "UPDATE": "updates",
      67    "FAIL": "blocks",
      68    "FILE": "files",
      69    "DONE": "research",
      70}
      71
      72SUMMARY_EVENT_TYPES = (
      73    "agent_message",
      74    "artifact",
      75    "error",
      76    "experiment",
      77    "finding",
      78    "lesson",
      79    "milestone_validation",
      80    "reflection",
      81    "roadmap",
      82    "source",
      83    "task",
      84)
      85
      86SUMMARY_TOOL_EVENT_TYPES = ("tool_result",)
      87
      88
      89def model_update_event_parts(event: dict[str, Any], *, width: int, compact: bool = True) -> tuple[str, str, str] | None:
      90    kind = str(event.get("event_type") or "")
      91    title = generic_display_text(event.get("title") or "")
      92    body = generic_display_text(event.get("body") or "")
      93    metadata = event.get("metadata") if isinstance(event.get("metadata"), dict) else {}
      94    status = str(metadata.get("status") or "")
      95    clock = event_clock(event)
      96    chars = max(24, width - 16)
      97    if kind == "error":
      98        return "FAIL", _outcome_text(event_title_body(title, body, fallback="error"), chars=chars, compact=compact), clock
      99    if kind == "artifact":
     100        detail = title or body or str(metadata.get("summary") or "") or "saved output"
     101        return "SAVE", _outcome_text(detail, chars=chars, compact=compact), clock
     102    if kind == "finding":
     103        return "FIND", _outcome_text(event_title_body(title, body, fallback="finding"), chars=chars, compact=compact), clock
     104    if kind == "source":
     105        return "SOURCE", _outcome_text(event_title_body(title, body, fallback="source"), chars=chars, compact=compact), clock
     106    if kind == "experiment":
     107        metric = experiment_metric_text(metadata)
     108        detail = event_title_body(title, body, fallback="measurement")
     109        if metric and metric not in detail:
     110            detail = f"{detail} - {metric}"
     111        return "TEST", _outcome_text(detail, chars=chars, compact=compact), clock
     112    if kind == "task":
     113        task_status = str(metadata.get("status") or "")
     114        detail = event_title_body(title, body, fallback="task")
     115        prefix = f"{task_status} " if task_status else ""
     116        return "TASK", _outcome_text(prefix + detail, chars=chars, compact=compact), clock
     117    if kind == "roadmap":
     118        return "ROAD", _outcome_text(event_title_body(title, body, fallback="roadmap"), chars=chars, compact=compact), clock
     119    if kind == "milestone_validation":
     120        validation = str(metadata.get("validation_status") or metadata.get("status") or "")
     121        detail = event_title_body(title, body, fallback="milestone")
     122        return "VALID", _outcome_text(f"{validation} {detail}".strip(), chars=chars, compact=compact), clock
     123    if kind == "lesson":
     124        return "LEARN", _outcome_text(event_title_body(title, body, fallback="lesson"), chars=chars, compact=compact), clock
     125    if kind == "reflection":
     126        return "PLAN", _outcome_text(brief_reflection_text(body or title), chars=chars, compact=compact), clock
     127    if kind == "agent_message" and title.lower() in {"error", "blocked"}:
     128        detail = body or chat_agent_message_text(title, body) or event_title_body(title, body, fallback="error")
     129        return "FAIL", _outcome_text(detail, chars=chars, compact=compact), clock
     130    if kind == "agent_message" and title.lower() in {"progress", "update", "report", "plan", "planning"}:
     131        durable_progress = _durable_progress_event_parts(metadata, body=body, chars=chars, compact=compact, clock=clock)
     132        if durable_progress:
     133            return durable_progress
     134        detail = chat_agent_message_text(title, body) or event_title_body(title, body, fallback="update")
     135        return "UPDATE", _outcome_text(detail, chars=chars, compact=compact), clock
     136    if kind == "tool_result" and status == "completed":
     137        tool = title
     138        if tool in {"web_search", "web_extract"}:
     139            return "DONE", _outcome_text(tool_live_summary(tool, metadata, body), chars=chars, compact=compact), clock
     140        if tool == "shell_exec":
     141            command = str(event_tool_args(metadata).get("command") or "")
     142            target = shell_write_target(command)
     143            if target:
     144                return "FILE", _outcome_text(f"updated {short_path(target, max_width=chars - 8)} via shell", chars=chars, compact=compact), clock
     145        if tool == "write_file":
     146            output = metadata.get("output") if isinstance(metadata.get("output"), dict) else {}
     147            path = str(output.get("path") or event_tool_args(metadata).get("path") or "")
     148            return "FILE", _outcome_text(f"updated {short_path(path, max_width=chars - 8)}", chars=chars, compact=compact), clock
     149    return None
     150
     151
     152def is_summary_event_candidate(event: dict[str, Any]) -> bool:
     153    kind = str(event.get("event_type") or "")
     154    if kind in SUMMARY_EVENT_TYPES:
     155        return True
     156    if kind != "tool_result":
     157        return False
     158    metadata = event.get("metadata") if isinstance(event.get("metadata"), dict) else {}
     159    if str(metadata.get("status") or "") != "completed":
     160        return False
     161    title = str(event.get("title") or "")
     162    if title == "write_file":
     163        return True
     164    if title == "shell_exec":
     165        command = str(event_tool_args(metadata).get("command") or "")
     166        return bool(shell_write_target(command))
     167    return False
     168
     169
     170def latest_durable_outcome_line(events: list[dict[str, Any]], *, width: int) -> str:
     171    fallback: tuple[str, str, str] | None = None
     172    for event in reversed(events):
     173        parsed = model_update_event_parts(event, width=width)
     174        if not parsed:
     175            continue
     176        label, text, _clock = parsed
     177        if label == "DONE":
     178            fallback = fallback or parsed
     179            continue
     180        if label not in PRIMARY_OUTCOME_LABELS:
     181            continue
     182        prefix = f"{_muted('Outcome')} {_event_badge(label)} "
     183        return _fit_ansi(prefix + _one_line(text, max(12, width - len(_strip_ansi(prefix)))), width)
     184    if fallback:
     185        label, text, _clock = fallback
     186        prefix = f"{_muted('Outcome')} {_event_badge(label)} "
     187        return _fit_ansi(prefix + _one_line(text, max(12, width - len(_strip_ansi(prefix)))), width)
     188    return ""
     189
     190
     191def latest_hour_outcome_summary_line(events: list[dict[str, Any]], *, width: int) -> str:
     192    """Return a single compact count summary for the newest visible activity hour."""
     193
     194    buckets: dict[str, dict[str, int]] = {}
     195    order: list[str] = []
     196    for event in events:
     197        parsed = model_update_event_parts(event, width=max(width, 180), compact=False)
     198        if not parsed:
     199            continue
     200        label, _text, _clock = parsed
     201        if label not in SUMMARY_COUNT_LABELS:
     202            continue
     203        hour = event_hour(event)
     204        if hour not in buckets:
     205            buckets[hour] = {}
     206            order.append(hour)
     207        buckets[hour][label] = int(buckets[hour].get(label) or 0) + 1
     208    if not order:
     209        return ""
     210    summary = hourly_outcome_summary(buckets[order[-1]])
     211    if not summary:
     212        return ""
     213    prefix = f"{_muted('Latest hour')} "
     214    return _fit_ansi(prefix + _bold(_one_line(summary, max(12, width - len(_strip_ansi(prefix))))), width)
     215
     216
     217def visible_outcome_summary_line(events: list[dict[str, Any]], *, width: int) -> str:
     218    """Return a stable summary of the durable outcomes available to the pane."""
     219
     220    counts = outcome_counts(events, include_research=False, include_failures=True)
     221    summary = hourly_outcome_summary(counts)
     222    if not summary:
     223        return ""
     224    prefix = f"{_muted('Visible')} "
     225    return _fit_ansi(prefix + _bold(_one_line(summary, max(12, width - len(_strip_ansi(prefix))))), width)
     226
     227
     228def job_outcome_summary(events: list[dict[str, Any]], *, width: int) -> str:
     229    """Return a short per-job durable outcome mix for compact job cards."""
     230
     231    counts = outcome_counts(events, include_research=False, include_failures=False)
     232    summary = hourly_outcome_summary(counts)
     233    if not summary:
     234        return ""
     235    return _one_line(summary, width)
     236
     237
     238def outcome_counts(
     239    events: list[dict[str, Any]],
     240    *,
     241    include_research: bool,
     242    include_failures: bool,
     243) -> dict[str, int]:
     244    counts: dict[str, int] = {}
     245    for event in events:
     246        parsed = model_update_event_parts(event, width=220, compact=False)
     247        if not parsed:
     248            continue
     249        label, _text, _clock = parsed
     250        if label == "DONE" and not include_research:
     251            continue
     252        if label == "FAIL" and not include_failures:
     253            continue
     254        if label not in SUMMARY_COUNT_LABELS:
     255            continue
     256        counts[label] = int(counts.get(label) or 0) + 1
     257    return counts
     258
     259
     260def recent_model_update_lines(
     261    events: list[dict[str, Any]],
     262    *,
     263    width: int,
     264    limit: int,
     265    include_research: bool = False,
     266    wrap: bool = True,
     267) -> list[str]:
     268    """Render recent durable worker outcomes for the compact status pane."""
     269    if limit <= 0:
     270        return []
     271    lines: list[str] = []
     272    items: list[dict[str, Any]] = []
     273    index_by_key: dict[tuple[str, str], int] = {}
     274    for event in reversed(events):
     275        parsed = model_update_event_parts(event, width=max(width, 180), compact=False)
     276        if not parsed:
     277            continue
     278        label, text, clock = parsed
     279        if label == "DONE" and not include_research:
     280            continue
     281        if label not in PRIMARY_OUTCOME_LABELS and not (include_research and label == "DONE"):
     282            continue
     283        key = (label, text)
     284        if key in index_by_key:
     285            items[index_by_key[key]]["count"] = int(items[index_by_key[key]].get("count") or 1) + 1
     286            continue
     287        index_by_key[key] = len(items)
     288        items.append({"label": label, "text": text, "clock": clock, "count": 1})
     289        if len(items) >= max(limit * 2, limit + 8):
     290            break
     291    for item in items:
     292        label = str(item["label"])
     293        text = str(item["text"])
     294        clock = str(item["clock"])
     295        count = int(item.get("count") or 1)
     296        prefix = f"{_muted(clock)} {_event_badge(label)} " if clock else f"{_event_badge(label)} "
     297        prefix_width = len(_strip_ansi(prefix))
     298        available = max(12, width - prefix_width - 2)
     299        if count > 1:
     300            text = f"{text} x{count}"
     301        if not wrap:
     302            lines.append(_fit_ansi(prefix + _one_line(text, available), width))
     303            if len(lines) >= limit:
     304                return lines
     305            continue
     306        wrapped = textwrap.wrap(text, width=available) or [""]
     307        lines.append(_fit_ansi(prefix + wrapped[0], width))
     308        if len(lines) >= limit:
     309            return lines
     310        continuation_prefix = " " * prefix_width
     311        for part in wrapped[1:]:
     312            lines.append(_fit_ansi(continuation_prefix + part, width))
     313            if len(lines) >= limit:
     314                return lines
     315        if len(lines) >= limit:
     316            return lines
     317    return lines
     318
     319
     320def chat_updates_pane_lines(
     321    *,
     322    job: dict[str, Any],
     323    events: list[dict[str, Any]],
     324    width: int,
     325    rows: int,
     326) -> list[str]:
     327    lines = [
     328        f"{_muted('Page')}   {_page_indicator('updates', CHAT_RIGHT_PAGES)}",
     329        f"{_muted('Focus')}  {_bold(_one_line(job.get('title') or 'untitled', width - 8))}",
     330    ]
     331    counts = outcome_counts(events, include_research=False, include_failures=True)
     332    summary = hourly_outcome_summary(counts)
     333    if summary:
     334        lines.extend([*_wrapped_label_line("Visible", summary, width=width), ""])
     335    update_lines = recent_model_update_lines(events, width=width, limit=max(4, rows - len(lines)), wrap=False)
     336    if update_lines:
     337        lines.extend(update_lines)
     338    else:
     339        lines.extend(["", _muted("No model updates yet.")])
     340    return [_fit_ansi(line, width) for line in lines[:rows]]
     341
     342
     343def _wrapped_label_line(label: str, text: str, *, width: int) -> list[str]:
     344    prefix = f"{_muted(label)} "
     345    prefix_width = len(_strip_ansi(prefix))
     346    available = max(12, width - prefix_width)
     347    wrapped = textwrap.wrap(text, width=available) or [""]
     348    lines = [_fit_ansi(prefix + _bold(wrapped[0]), width)]
     349    continuation = " " * prefix_width
     350    for part in wrapped[1:]:
     351        lines.append(_fit_ansi(continuation + _bold(part), width))
     352    return lines
     353
     354
     355def hourly_update_lines(events: list[dict[str, Any]], *, width: int, limit: int) -> list[str]:
     356    if limit <= 0:
     357        return []
     358    buckets: dict[str, dict[str, Any]] = {}
     359    order: list[str] = []
     360    for event in events:
     361        parsed = model_update_event_parts(event, width=max(width, 220), compact=False)
     362        if not parsed:
     363            continue
     364        label, text, clock = parsed
     365        if label not in SUMMARY_COUNT_LABELS:
     366            continue
     367        hour = event_hour(event)
     368        if hour not in buckets:
     369            buckets[hour] = {"counts": {}, "items": [], "clock": clock}
     370            order.append(hour)
     371        bucket = buckets[hour]
     372        counts = bucket["counts"]
     373        counts[label] = int(counts.get(label) or 0) + 1
     374        item = (label, text)
     375        if item not in bucket["items"]:
     376            bucket["items"].append(item)
     377    rendered: list[str] = []
     378    # Each visible hour needs a header and at least a couple durable outcomes.
     379    # Showing too many buckets makes the pane churn and can trim off the hour
     380    # label, which is harder to scan during long-running jobs.
     381    max_visible_hours = max(1, min(len(order), max(1, limit // 4)))
     382    recent_hours = order[-max_visible_hours:]
     383    available_items = max(1, limit - len(recent_hours))
     384    per_bucket = max(1, min(6, available_items // max(1, len(recent_hours))))
     385    for hour in recent_hours:
     386        bucket = buckets[hour]
     387        counts = bucket["counts"]
     388        summary = hourly_outcome_summary(counts)
     389        rendered.append(_fit_ansi(f"{_muted(hour)} {_bold(summary or 'activity')}", width))
     390        primary_items = [item for item in bucket["items"] if item[0] in PRIMARY_OUTCOME_LABELS]
     391        visible_items = primary_items or bucket["items"]
     392        for label, text in visible_items[-per_bucket:]:
     393            prefix = f"  {_event_badge(label)} "
     394            available = max(16, width - len(_strip_ansi(prefix)))
     395            parts = textwrap.wrap(text, width=available) or [""]
     396            rendered.append(_fit_ansi(prefix + parts[0], width))
     397            for part in parts[1:]:
     398                rendered.append(_fit_ansi(" " * len(_strip_ansi(prefix)) + part, width))
     399                if len(rendered) >= limit:
     400                    return rendered[:limit]
     401        if len(rendered) >= limit:
     402            return rendered[:limit]
     403    return rendered[:limit]
     404
     405
     406def hourly_outcome_summary(counts: dict[str, Any]) -> str:
     407    pieces: list[str] = []
     408    ordered = [label for label in OUTCOME_LABEL_ORDER if label in counts]
     409    ordered.extend(sorted(label for label in counts if label not in set(OUTCOME_LABEL_ORDER)))
     410    for label in ordered:
     411        count = int(counts.get(label) or 0)
     412        if count <= 0:
     413            continue
     414        name = OUTCOME_SUMMARY_NAMES.get(label, label.lower())
     415        pieces.append(f"{count} {name}")
     416    return " ".join(pieces)
     417
     418
     419def _durable_progress_event_parts(
     420    metadata: dict[str, Any],
     421    *,
     422    body: str,
     423    chars: int,
     424    compact: bool,
     425    clock: str,
     426) -> tuple[str, str, str] | None:
     427    deltas = _count_map(metadata.get("deltas"))
     428    updates = _count_map(metadata.get("updates"))
     429    resolutions = _count_map(metadata.get("resolutions"))
     430    totals = {
     431        key: int(deltas.get(key) or 0) + int(updates.get(key) or 0) + int(resolutions.get(key) or 0)
     432        for key in set(deltas) | set(updates) | set(resolutions)
     433    }
     434    if not any(value > 0 for value in totals.values()):
     435        return None
     436    key = _dominant_progress_key(totals, resolutions=resolutions)
     437    if not key:
     438        return None
     439    label = _progress_label_for_key(key, resolution=bool(resolutions.get(key)))
     440    pieces: list[str] = []
     441    for record_key in ("findings", "experiments", "sources", "tasks", "milestones", "lessons"):
     442        if deltas.get(record_key):
     443            pieces.append(_progress_count_phrase(int(deltas[record_key]), record_key, prefix="+"))
     444        if updates.get(record_key):
     445            pieces.append(_progress_count_phrase(int(updates[record_key]), record_key, prefix="~", suffix="updated"))
     446        if resolutions.get(record_key):
     447            pieces.append(_progress_count_phrase(int(resolutions[record_key]), record_key, suffix="resolved"))
     448    detail = ", ".join(pieces)
     449    if body:
     450        detail = f"{detail} - {generic_display_text(body)}"
     451    return label, _outcome_text(detail, chars=chars, compact=compact), clock
     452
     453
     454def _count_map(value: Any) -> dict[str, int]:
     455    if not isinstance(value, dict):
     456        return {}
     457    result: dict[str, int] = {}
     458    for key, raw_count in value.items():
     459        try:
     460            count = int(raw_count)
     461        except (TypeError, ValueError):
     462            continue
     463        if count > 0:
     464            result[str(key)] = count
     465    return result
     466
     467
     468def _dominant_progress_key(totals: dict[str, int], *, resolutions: dict[str, int]) -> str:
     469    for key in ("experiments", "milestones", "findings", "sources", "tasks", "lessons"):
     470        if resolutions.get(key):
     471            return key
     472    for key in ("experiments", "findings", "milestones", "sources", "tasks", "lessons"):
     473        if totals.get(key):
     474            return key
     475    return ""
     476
     477
     478def _progress_label_for_key(key: str, *, resolution: bool) -> str:
     479    if key == "findings":
     480        return "FIND"
     481    if key == "sources":
     482        return "SOURCE"
     483    if key == "experiments":
     484        return "TEST"
     485    if key == "tasks":
     486        return "TASK"
     487    if key == "milestones":
     488        return "VALID" if resolution else "ROAD"
     489    if key == "lessons":
     490        return "LEARN"
     491    return "UPDATE"
     492
     493
     494def _progress_count_phrase(value: int, key: str, *, prefix: str = "", suffix: str = "") -> str:
     495    label = OUTCOME_SUMMARY_NAMES.get(_progress_label_for_key(key, resolution=False), key)
     496    if value == 1 and label.endswith("s"):
     497        label = label[:-1]
     498    parts = [f"{prefix}{value} {label}"]
     499    if suffix:
     500        parts.append(suffix)
     501    return " ".join(parts)
     502
     503
     504def _outcome_text(text: str, *, chars: int, compact: bool) -> str:
     505    clean = generic_display_text(text)
     506    if compact:
     507        return _one_line(clean, chars)
     508    return _one_line(clean, 900)

nipux_cli/tui_status.py 540 lines

       1"""Status and work-pane renderers for the Nipux terminal UI."""
       2
       3from __future__ import annotations
       4
       5import textwrap
       6from typing import Any
       7
       8from nipux_cli.config import AppConfig
       9from nipux_cli.operator_context import active_prompt_operator_entries
      10from nipux_cli.scheduling import job_deferred_until, job_provider_blocked
      11from nipux_cli.tui_event_format import experiment_metric_text
      12from nipux_cli.tui_events import (
      13    worker_activity_lines,
      14)
      15from nipux_cli.tui_outcomes import (
      16    CHAT_RIGHT_PAGES,
      17    job_outcome_summary,
      18    latest_durable_outcome_line,
      19    latest_hour_outcome_summary_line,
      20    model_update_event_parts,
      21    recent_model_update_lines,
      22)
      23from nipux_cli.tui_layout import _format_compact_count, _metric_strip
      24from nipux_cli.tui_style import (
      25    _accent,
      26    _bold,
      27    _event_badge,
      28    _fit_ansi,
      29    _muted,
      30    _one_line,
      31    _page_indicator,
      32    _status_badge,
      33)
      34
      35
      36def worker_label(job: dict[str, Any], daemon_running: bool) -> str:
      37    status = str(job.get("status") or "")
      38    if job_provider_blocked(job):
      39        return "provider wait"
      40    if status == "planning":
      41        return "waiting"
      42    if status in {"paused", "completed", "cancelled", "failed"}:
      43        return status
      44    if job_deferred_until(job):
      45        return "waiting"
      46    return "active" if daemon_running and status in {"running", "queued"} else "idle"
      47
      48
      49def job_display_state(job: dict[str, Any], daemon_running: bool) -> str:
      50    status = str(job.get("status") or "")
      51    if job_provider_blocked(job):
      52        return "provider wait"
      53    if status in {"running", "queued"}:
      54        if job_deferred_until(job):
      55            return "waiting"
      56        return "advancing" if daemon_running else "open"
      57    return status or "unknown"
      58
      59
      60def active_operator_messages(metadata: dict[str, Any]) -> list[dict[str, Any]]:
      61    messages = metadata.get("operator_messages") if isinstance(metadata.get("operator_messages"), list) else []
      62    return [
      63        entry
      64        for entry in messages
      65        if isinstance(entry, dict)
      66        and entry in active_prompt_operator_entries(messages)
      67        and str(entry.get("mode") or "steer") in {"steer", "follow_up"}
      68    ]
      69
      70
      71def right_pane_lines(
      72    *,
      73    job: dict[str, Any],
      74    jobs: list[dict[str, Any]],
      75    job_artifacts: dict[str, list[dict[str, Any]]],
      76    job_summary_events: dict[str, list[dict[str, Any]]],
      77    job_counts: dict[str, dict[str, Any]],
      78    job_id: str,
      79    daemon_running: bool,
      80    state: str,
      81    worker: str,
      82    daemon_text: str,
      83    model: str,
      84    goal_text: str,
      85    latest_text: str,
      86    metrics: list[tuple[str, Any]],
      87    events: list[dict[str, Any]],
      88    token_usage: dict[str, Any],
      89    context_length: int,
      90    width: int,
      91    rows: int,
      92    right_view: str = "status",
      93) -> list[str]:
      94    del model, latest_text, daemon_text
      95    if _is_workspace_placeholder(job) and not jobs:
      96        return _empty_workspace_status_lines(right_view=right_view, width=width, rows=rows)
      97    info_lines = _chat_workspace_lines(
      98        right_view=right_view,
      99        job=job,
     100        state=state,
     101        worker=worker,
     102        goal_text=goal_text,
     103        token_usage=token_usage,
     104        context_length=context_length,
     105        width=width,
     106    )
     107    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     108    active_operator = active_operator_messages(metadata)
     109    pending_measurement = (
     110        metadata.get("pending_measurement_obligation")
     111        if isinstance(metadata.get("pending_measurement_obligation"), dict)
     112        else {}
     113    )
     114    if active_operator:
     115        info_lines.append(f"{_muted('Operator')} {len(active_operator)} active")
     116        info_lines.append(f"{_muted('Context')} {_one_line(active_operator[-1].get('message') or '', width - 8)}")
     117    if pending_measurement:
     118        info_lines.append(f"{_muted('Measure')} pending step #{pending_measurement.get('source_step_no') or '?'}")
     119    if job_provider_blocked(job):
     120        info_lines.append(_fit_ansi(f"{_muted('Provider')} action needed before retrying model calls", width))
     121    defer_line = _defer_status_line(job, width=width)
     122    if defer_line:
     123        info_lines.append(defer_line)
     124    spacious = rows >= 18
     125    if spacious:
     126        info_lines.append("")
     127        info_lines.append(_bold("Now"))
     128    latest_hour = latest_hour_outcome_summary_line(events, width=width) if rows >= 18 else ""
     129    if latest_hour:
     130        info_lines.append(latest_hour)
     131    latest_outcome = latest_durable_outcome_line(events, width=width)
     132    if latest_outcome:
     133        info_lines.append(latest_outcome)
     134    if spacious and not latest_hour and not latest_outcome:
     135        info_lines.append(_muted("No durable outcome yet."))
     136    if spacious:
     137        info_lines.append("")
     138        info_lines.append(_bold("Progress"))
     139        info_lines.extend(_metrics_grid_lines(metrics, width=width))
     140        yield_line = _yield_line(metrics, width=width)
     141        if yield_line:
     142            info_lines.append(yield_line)
     143    else:
     144        info_lines.append(_metric_strip(metrics[:5], width=width))
     145    info_lines.append("")
     146    info_lines.append(_bold("Jobs"))
     147    info_lines.extend(
     148        frame_jobs_lines(
     149            jobs[:5],
     150            focused_job_id=job_id,
     151            daemon_running=daemon_running,
     152            width=width,
     153            job_artifacts=job_artifacts,
     154            job_summary_events=job_summary_events,
     155            job_counts=job_counts,
     156            show_outputs=True,
     157        )
     158    )
     159    info_lines.append("")
     160    info_lines.append(_bold("Recent outcomes"))
     161    outcome_lines = recent_model_update_lines(events, width=width, limit=max(3, rows - len(info_lines)))
     162    if outcome_lines:
     163        info_lines.extend(outcome_lines)
     164    else:
     165        current_outputs = job_artifacts.get(job_id) or []
     166        if current_outputs:
     167            for artifact in current_outputs[:4]:
     168                title = _one_line(str(artifact.get("title") or artifact.get("id") or "output"), max(10, width - 8))
     169                info_lines.append(_fit_ansi(f"{_event_badge('SAVE')} {title}", width))
     170        else:
     171            info_lines.append(_muted("No durable outcomes yet."))
     172    return info_lines[:rows]
     173
     174
     175def _is_workspace_placeholder(job: dict[str, Any]) -> bool:
     176    return str(job.get("kind") or "") == "workspace"
     177
     178
     179def _empty_workspace_status_lines(*, right_view: str, width: int, rows: int) -> list[str]:
     180    lines = [
     181        f"{_muted('Page')}   {_page_indicator(right_view, CHAT_RIGHT_PAGES)}",
     182        _bold("No workers yet"),
     183        _muted("Type a goal in chat to start one."),
     184        "",
     185        f"{_muted('Start')}  {_bold('plain English goal')} or {_bold('/new OBJECTIVE')}",
     186        f"{_muted('Setup')}  {_bold('/settings')}",
     187        f"{_muted('Check')}  {_bold('/doctor')}",
     188    ]
     189    return [_fit_ansi(line, width) for line in lines[:rows]]
     190
     191
     192def chat_work_pane_lines(
     193    *,
     194    job: dict[str, Any],
     195    events: list[dict[str, Any]],
     196    tasks: list[dict[str, Any]],
     197    experiments: list[dict[str, Any]],
     198    width: int,
     199    rows: int,
     200) -> list[str]:
     201    lines = [
     202        f"{_muted('Page')}   {_page_indicator('work', CHAT_RIGHT_PAGES)}",
     203        f"{_muted('Focus')}  {_bold(_one_line(job.get('title') or 'untitled', width - 8))}",
     204    ]
     205    done_lines = recent_model_update_lines(events, width=width, limit=max(2, rows // 4))
     206    if done_lines:
     207        lines.extend(["", _bold("Done")])
     208        lines.extend(done_lines)
     209    lines.extend([
     210        "",
     211        _bold("Tool / console"),
     212    ])
     213    tool_budget = max(3, min(max(4, rows // 3), rows - len(lines) - 5))
     214    tool_lines = worker_activity_lines(events, width=width, limit=tool_budget)
     215    if tool_lines:
     216        lines.extend(tool_lines)
     217    else:
     218        lines.append(_muted("No recent tool calls."))
     219    remaining = max(0, rows - len(lines))
     220    if remaining > 4:
     221        lines.append("")
     222        lines.append(_bold("Tasks"))
     223        for task in _rank_visible_tasks(tasks)[: max(1, remaining // 2)]:
     224            status = str(task.get("status") or "open")
     225            title = _one_line(str(task.get("title") or "task"), max(10, width - 15))
     226            lines.append(_fit_ansi(f"{_status_badge(status)} {title}", width))
     227    remaining = max(0, rows - len(lines))
     228    if remaining > 3 and experiments:
     229        lines.append("")
     230        lines.append(_bold("Measurements"))
     231        for experiment in experiments[-max(1, remaining - 2) :]:
     232            metric = experiment_metric_text(experiment)
     233            title = _one_line(str(experiment.get("title") or "experiment"), max(10, width - 16))
     234            suffix = f" {_muted(metric)}" if metric else ""
     235            lines.append(_fit_ansi(f"{_event_badge('TEST')} {title}{suffix}", width))
     236    return [_fit_ansi(line, width) for line in lines[:rows]]
     237
     238
     239def chat_settings_pane_lines(
     240    *,
     241    config: AppConfig,
     242    width: int,
     243    rows: int,
     244) -> list[str]:
     245    key_state = "set" if config.model.api_key else "missing"
     246    input_cost = _rate_text(config.model.input_cost_per_million)
     247    output_cost = _rate_text(config.model.output_cost_per_million)
     248    lines = [
     249        f"{_muted('Page')}   {_page_indicator('settings', CHAT_RIGHT_PAGES)}",
     250        _bold("Model"),
     251        _setting_line("id", config.model.model, command="/model MODEL", width=width),
     252        _setting_line("endpoint", config.model.base_url, command="/base-url URL", width=width),
     253        _setting_line("key", f"{key_state} via {config.model.api_key_env}", command="/api-key KEY", width=width),
     254        _setting_line("context", str(config.model.context_length), command="/context TOKENS", width=width),
     255        "",
     256        _bold("Runtime"),
     257        _setting_line("home", str(config.runtime.home), command="/home PATH", width=width),
     258        _setting_line("step", f"{config.runtime.max_step_seconds}s", command="/step-limit SECONDS", width=width),
     259        _setting_line("preview", f"{config.runtime.artifact_inline_char_limit} chars", command="/output-chars CHARS", width=width),
     260        "",
     261        _bold("Cost"),
     262        _setting_line("input", input_cost, command="/input-cost DOLLARS", width=width),
     263        _setting_line("output", output_cost, command="/output-cost DOLLARS", width=width),
     264        "",
     265        _bold("Digest"),
     266        _setting_line(
     267            "daily",
     268            f"{config.runtime.daily_digest_enabled} at {config.runtime.daily_digest_time}",
     269            command="/daily-digest true|false",
     270            width=width,
     271        ),
     272        _muted("Type a command in the composer to edit."),
     273    ]
     274    return [_fit_ansi(line, width) for line in lines[:rows]]
     275
     276
     277def frame_jobs_lines(
     278    jobs: list[dict[str, Any]],
     279    *,
     280    focused_job_id: str,
     281    daemon_running: bool,
     282    width: int,
     283    job_artifacts: dict[str, list[dict[str, Any]]] | None = None,
     284    job_summary_events: dict[str, list[dict[str, Any]]] | None = None,
     285    job_counts: dict[str, dict[str, Any]] | None = None,
     286    show_outputs: bool = False,
     287) -> list[str]:
     288    rendered = []
     289    for index, item in enumerate(jobs[:5], start=1):
     290        item_id = str(item.get("id") or "")
     291        marker = _accent("●") if item_id == focused_job_id else _muted("○")
     292        title_width = max(14, min(30, width - 34))
     293        title = _one_line(str(item.get("title") or item.get("id") or "job"), title_width)
     294        state = _status_badge(job_display_state(item, daemon_running))
     295        worker = _status_badge(worker_label(item, daemon_running))
     296        kind = _one_line(item.get("kind") or "", max(0, width - title_width - 33))
     297        rendered.append(
     298            _fit_ansi(
     299                f"{marker} {index:<2} {_fit_ansi(title, title_width)} "
     300                f"{_fit_ansi(state, 10)} {_fit_ansi(worker, 10)} {kind}",
     301                width,
     302            )
     303        )
     304        if show_outputs:
     305            rendered.extend(_job_compact_work_lines(
     306                outputs=(job_artifacts or {}).get(item_id) or [],
     307                counts=(job_counts or {}).get(item_id) or {},
     308                events=(job_summary_events or {}).get(item_id) or [],
     309                width=width,
     310                focused=item_id == focused_job_id,
     311            ))
     312    return rendered
     313
     314
     315def _job_compact_work_lines(
     316    *,
     317    outputs: list[dict[str, Any]],
     318    counts: dict[str, Any],
     319    events: list[dict[str, Any]],
     320    width: int,
     321    focused: bool = False,
     322) -> list[str]:
     323    lines: list[str] = []
     324    summary = job_outcome_summary(events, width=max(12, width - 13))
     325    if summary:
     326        lines.append(_fit_ansi(f"   {_muted('work')} {_bold(summary)}", width))
     327    if outputs:
     328        latest = outputs[0]
     329        output_total = int(counts.get("artifacts") or len(outputs))
     330        output_count = f"{output_total} outputs" if output_total != 1 else "1 output"
     331        title_budget = max(12, width - 13 - len(output_count))
     332        output_title = _one_line(str(latest.get("title") or latest.get("id") or "saved output"), title_budget)
     333        lines.append(_fit_ansi(f"   {_muted('made')} {_bold(output_count)} · {output_title}", width))
     334        if focused and len(outputs) > 1 and width >= 42:
     335            second = outputs[1]
     336            second_title = _one_line(str(second.get("title") or second.get("id") or "saved output"), max(12, width - 10))
     337            lines.append(_fit_ansi(f"   {_muted('also')} {second_title}", width))
     338    for outcome in _job_recent_non_output_pieces(
     339        events,
     340        width=max(12, width - 10),
     341        skip_save=bool(outputs),
     342        limit=2 if focused else 1,
     343    ):
     344        lines.append(_fit_ansi(f"   {_muted('did')}  {outcome}", width))
     345    return lines
     346
     347
     348def _job_recent_non_output_pieces(
     349    events: list[dict[str, Any]],
     350    *,
     351    width: int,
     352    skip_save: bool,
     353    limit: int,
     354) -> list[str]:
     355    pieces: list[str] = []
     356    seen: set[str] = set()
     357    for event in reversed(events):
     358        parsed = model_update_event_parts(event, width=max(width, 120))
     359        if not parsed:
     360            continue
     361        label, text, _clock = parsed
     362        if label == "DONE":
     363            continue
     364        if skip_save and label == "SAVE":
     365            continue
     366        prefix = _compact_outcome_label(label)
     367        piece = f"{_muted(prefix)} {_one_line(text, max(12, width - len(prefix) - 1))}"
     368        dedupe_key = _one_line(f"{prefix} {text}", 120)
     369        if dedupe_key in seen:
     370            continue
     371        seen.add(dedupe_key)
     372        pieces.append(piece)
     373        if len(pieces) >= max(1, limit):
     374            break
     375    return pieces
     376
     377
     378def _compact_outcome_label(label: str) -> str:
     379    return {
     380        "FIND": "find",
     381        "SOURCE": "src",
     382        "TEST": "test",
     383        "TASK": "task",
     384        "ROAD": "road",
     385        "VALID": "valid",
     386        "LEARN": "learn",
     387        "FILE": "file",
     388        "SAVE": "out",
     389        "FAIL": "fail",
     390        "PLAN": "plan",
     391        "UPDATE": "note",
     392    }.get(label, label.lower())
     393
     394
     395def _defer_status_line(job: dict[str, Any], *, width: int) -> str:
     396    until = job_deferred_until(job)
     397    if not until:
     398        return ""
     399    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     400    reason = str(metadata.get("defer_reason") or metadata.get("defer_next_action") or "").strip()
     401    time_text = until.astimezone().strftime("%b %d %H:%M")
     402    detail = f"next check {time_text}"
     403    if reason:
     404        detail += f" - {reason}"
     405    return _fit_ansi(f"{_muted('Wait')}   {_one_line(detail, max(12, width - 7))}", width)
     406
     407
     408def _rank_visible_tasks(tasks: list[dict[str, Any]]) -> list[dict[str, Any]]:
     409    status_order = {"active": 0, "open": 1, "blocked": 2, "validating": 3, "done": 4, "skipped": 5}
     410    return sorted(
     411        [task for task in tasks if isinstance(task, dict)],
     412        key=lambda task: (
     413            status_order.get(str(task.get("status") or "open"), 9),
     414            -int(task.get("priority") or 0),
     415            str(task.get("title") or ""),
     416        ),
     417    )
     418
     419
     420def _chat_workspace_lines(
     421    *,
     422    right_view: str,
     423    job: dict[str, Any],
     424    state: str,
     425    worker: str,
     426    goal_text: str,
     427    token_usage: dict[str, Any],
     428    context_length: int,
     429    width: int,
     430) -> list[str]:
     431    goal_lines = textwrap.wrap(goal_text, width=max(20, width - 8))[:2] or [""]
     432    while len(goal_lines) < 2:
     433        goal_lines.append("")
     434    title = _one_line(str(job.get("title") or "untitled"), max(10, width))
     435    lines = [
     436        f"{_muted('Page')}   {_page_indicator(right_view, CHAT_RIGHT_PAGES)}",
     437        _bold(title),
     438        f"{_muted('State')}  {_status_badge(state)}  {_muted('worker')} {_status_badge(worker)}",
     439        f"{_muted('Goal')}   {goal_lines[0]}",
     440        f"{_muted('       ')}{goal_lines[1]}",
     441    ]
     442    task_line = _current_task_line(job, width=width)
     443    if task_line:
     444        lines.append(task_line)
     445    context_line = _context_pressure_line(token_usage, context_length=context_length, width=width)
     446    if context_line:
     447        lines.append(context_line)
     448    return lines
     449
     450
     451def _current_task_line(job: dict[str, Any], *, width: int) -> str:
     452    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     453    tasks = metadata.get("task_queue") if isinstance(metadata.get("task_queue"), list) else []
     454    visible = [
     455        task
     456        for task in tasks
     457        if isinstance(task, dict)
     458        and str(task.get("status") or "open") in {"active", "open", "blocked"}
     459    ]
     460    if not visible:
     461        return ""
     462    ranked = _rank_visible_tasks(visible)
     463    task = ranked[0]
     464    status = str(task.get("status") or "open")
     465    title = _one_line(str(task.get("title") or "task"), max(12, width - 16))
     466    return _fit_ansi(f"{_muted('Task')}   {_status_badge(status)} {title}", width)
     467
     468
     469def _context_pressure_line(usage: dict[str, Any], *, context_length: int, width: int) -> str:
     470    latest_prompt = _safe_int(usage.get("latest_prompt_tokens"))
     471    context_limit = _safe_int(usage.get("latest_context_length")) or context_length
     472    if latest_prompt <= 0 or context_limit <= 0:
     473        return ""
     474    fraction = latest_prompt / max(1, context_limit)
     475    if fraction < 0.65:
     476        return ""
     477    label = "high" if fraction >= 0.85 else "watch" if fraction >= 0.65 else "ok"
     478    detail = (
     479        f"{_format_compact_count(latest_prompt)}/{_format_compact_count(context_limit)} "
     480        f"{fraction:.0%} {label}"
     481    )
     482    return _fit_ansi(f"{_muted('Context')} {_one_line(detail, max(12, width - 8))}", width)
     483
     484
     485def _metrics_grid_lines(metrics: list[tuple[str, Any]], *, width: int) -> list[str]:
     486    wanted = ["actions", "outputs", "findings", "sources", "tasks", "experiments", "memory"]
     487    lookup = {label: value for label, value in metrics}
     488    items = [(label, lookup[label]) for label in wanted if label in lookup]
     489    if width < 40:
     490        return [_metric_strip(items, width=width)]
     491    lines: list[str] = []
     492    col_width = max(16, (width - 2) // 2)
     493    for index in range(0, len(items), 2):
     494        left = _metric_cell(items[index], width=col_width)
     495        right = _metric_cell(items[index + 1], width=col_width) if index + 1 < len(items) else ""
     496        lines.append(_fit_ansi(left + "  " + right, width))
     497    return lines
     498
     499
     500def _metric_cell(item: tuple[str, Any], *, width: int) -> str:
     501    label, value = item
     502    return _fit_ansi(f"{_muted(label)} {_bold(value)}", width)
     503
     504
     505def _yield_line(metrics: list[tuple[str, Any]], *, width: int) -> str:
     506    lookup = {label: value for label, value in metrics}
     507    actions = _safe_int(lookup.get("actions"))
     508    if actions < 20:
     509        return ""
     510    outputs = _safe_int(lookup.get("outputs"))
     511    findings = _safe_int(lookup.get("findings"))
     512    sources = _safe_int(lookup.get("sources"))
     513    experiments = _safe_int(lookup.get("experiments"))
     514    durable = outputs + findings + sources + experiments
     515    if durable <= 0:
     516        return _fit_ansi(f"{_muted('Yield')}  {_status_badge('blocked')} no durable outcomes after {actions} actions", width)
     517    actions_per = actions / durable
     518    label = "watch" if actions_per >= 25 else "ok"
     519    if actions_per < 8:
     520        return ""
     521    detail = f"{actions_per:.1f} actions/outcome"
     522    return _fit_ansi(f"{_muted('Yield')}  {_status_badge(label)} {detail}", width)
     523
     524
     525def _setting_line(label: str, value: str, *, command: str, width: int) -> str:
     526    left = f"{_muted(label)} {_bold(_one_line(value, max(8, width - 24)))}"
     527    if width < 46:
     528        return _fit_ansi(left, width)
     529    return _fit_ansi(left + "  " + _muted(command), width)
     530
     531
     532def _rate_text(value: float | None) -> str:
     533    return "provider-reported" if value is None else f"${value:g}/1M"
     534
     535
     536def _safe_int(value: Any) -> int:
     537    try:
     538        return int(float(value))
     539    except (TypeError, ValueError):
     540        return 0

nipux_cli/tui_style.py 153 lines

       1"""Small terminal styling helpers shared by the CLI frame renderers."""
       2
       3from __future__ import annotations
       4
       5import os
       6import re
       7import sys
       8from typing import Any
       9
      10
      11def _fancy_ui() -> bool:
      12    return (
      13        sys.stdout.isatty()
      14        and os.environ.get("NO_COLOR") is None
      15        and os.environ.get("NIPUX_PLAIN") is None
      16        and os.environ.get("TERM", "") not in {"", "dumb"}
      17    )
      18
      19
      20def _style(text: Any, code: str) -> str:
      21    value = str(text)
      22    if not _fancy_ui():
      23        return value
      24    return f"\033[{code}m{value}\033[0m"
      25
      26
      27def _accent(text: Any) -> str:
      28    return _style(text, "38;5;123")
      29
      30
      31def _muted(text: Any) -> str:
      32    return _style(text, "38;5;248")
      33
      34
      35def _bold(text: Any) -> str:
      36    return _style(text, "1")
      37
      38
      39def _one_line(value: Any, width: int) -> str:
      40    text = " ".join(str(value).split())
      41    if len(text) <= width:
      42        return text
      43    return text[: max(0, width - 3)] + "..."
      44
      45
      46def _strip_ansi(text: str) -> str:
      47    return re.sub(r"\x1b\[[0-9;]*m", "", text)
      48
      49
      50def _fit_ansi(text: Any, width: int) -> str:
      51    width = max(0, int(width))
      52    content = str(text)
      53    visible = _strip_ansi(content)
      54    if len(visible) > width:
      55        content = _one_line(visible, width)
      56        visible = content
      57    return content + " " * max(0, width - len(visible))
      58
      59
      60def _center_ansi(text: str, width: int) -> str:
      61    text_width = len(_strip_ansi(text))
      62    if text_width >= width:
      63        return _fit_ansi(text, width)
      64    left_pad = max(0, (width - text_width) // 2)
      65    return _fit_ansi(" " * left_pad + text, width)
      66
      67
      68def _themed_lines(lines: list[str], *, width: int) -> list[str]:
      69    if not _fancy_ui():
      70        return [_fit_ansi(line, width) for line in lines]
      71    bg = "\033[48;5;234m\033[38;5;252m"
      72    reset = "\033[0m"
      73    return [bg + _fit_ansi(line, width).replace(reset, reset + bg) + reset for line in lines]
      74
      75
      76def _frame_enter_sequence() -> str:
      77    theme = "\033[48;5;234m\033[38;5;252m" if _fancy_ui() else ""
      78    return f"\033[?1049h{theme}\033[2J\033[H\033[?25l\033[?1000h\033[?1002h\033[?1006h"
      79
      80
      81def _frame_exit_sequence() -> str:
      82    return "\033[?1006l\033[?1002l\033[?1000l\033[?25h\033[0m\033[?1049l"
      83
      84
      85def _page_indicator(active: str, pages: list[tuple[str, str]]) -> str:
      86    parts: list[str] = []
      87    for key, label in pages:
      88        if key == active:
      89            parts.append(f"{_accent('●')} {_bold(label)}")
      90        else:
      91            parts.append(f"{_muted('○')} {_muted(label)}")
      92    return "  ".join(parts)
      93
      94
      95def _status_badge(value: Any) -> str:
      96    text = str(value)
      97    color = {
      98        "active": "32",
      99        "advancing": "32",
     100        "running": "32",
     101        "queued": "33",
     102        "planning": "35",
     103        "waiting": "35",
     104        "open": "33",
     105        "idle": "33",
     106        "paused": "33",
     107        "cancelled": "31",
     108        "failed": "31",
     109        "completed": "36",
     110        "ok": "32",
     111        "watch": "33",
     112        "ready": "32",
     113        "switch": "36",
     114        "missing": "31",
     115        "check": "33",
     116        "next": "35",
     117        "recommended": "36",
     118    }.get(text, "37")
     119    return _style(text, color)
     120
     121
     122def _event_badge(label: str) -> str:
     123    padded = f"{label:<8}"
     124    color = {
     125        "AGENT": "36",
     126        "USER": "35",
     127        "FOLLOW": "35",
     128        "YOU": "35",
     129        "NIPUX": "36",
     130        "RUN": "34",
     131        "TOOL": "34",
     132        "DONE": "32",
     133        "FILE": "32",
     134        "SAVE": "32",
     135        "OUTPUT": "32",
     136        "FIND": "32",
     137        "SOURCE": "36",
     138        "TASK": "33",
     139        "ROAD": "35",
     140        "VALID": "35",
     141        "TEST": "33",
     142        "UPDATE": "36",
     143        "ACK": "36",
     144        "FAIL": "31",
     145        "LEARN": "36",
     146        "PLAN": "36",
     147        "DIGEST": "36",
     148        "MEMORY": "36",
     149        "SYSTEM": "2",
     150        "BLOCK": "33",
     151        "ERROR": "31",
     152    }.get(label, "37")
     153    return _style(padded, color)

nipux_cli/uninstall.py 217 lines

       1"""Uninstall helpers for local Nipux runtime state."""
       2
       3from __future__ import annotations
       4
       5import os
       6import shutil
       7import subprocess
       8from dataclasses import dataclass
       9from pathlib import Path
      10from typing import Callable, Sequence
      11
      12from nipux_cli.config import get_agent_home
      13from nipux_cli.service_install import launch_agent_path, systemd_service_path
      14
      15
      16Runner = Callable[..., subprocess.CompletedProcess[str]]
      17CommandRunner = Callable[[Sequence[str]], subprocess.CompletedProcess[str]]
      18
      19
      20@dataclass(frozen=True)
      21class UninstallPlan:
      22    paths: tuple[Path, ...]
      23    service_paths: tuple[Path, ...]
      24
      25
      26def build_uninstall_plan(*, runtime_home: Path | None = None, include_legacy: bool = True) -> UninstallPlan:
      27    """Return all local runtime paths that a full uninstall should remove."""
      28
      29    homes = [runtime_home.expanduser() if runtime_home else get_agent_home(), get_agent_home(), Path.home() / ".nipux"]
      30    if include_legacy:
      31        homes.append(Path.home() / ".kneepucks")
      32    paths = tuple(_dedupe_paths(homes))
      33    service_paths = tuple(_dedupe_paths([launch_agent_path(), systemd_service_path()]))
      34    return UninstallPlan(paths=paths, service_paths=service_paths)
      35
      36
      37def uninstall_runtime(
      38    *,
      39    runtime_home: Path | None = None,
      40    dry_run: bool = False,
      41    include_legacy: bool = True,
      42    runner: Runner = subprocess.run,
      43) -> list[str]:
      44    """Remove local Nipux state, logs, service files, and legacy state dirs."""
      45
      46    plan = build_uninstall_plan(runtime_home=runtime_home, include_legacy=include_legacy)
      47    lines: list[str] = []
      48    lines.extend(_disable_services(dry_run=dry_run, runner=runner))
      49    for path in (*plan.service_paths, *plan.paths):
      50        target = path.expanduser()
      51        _assert_safe_delete_target(target)
      52        if dry_run:
      53            lines.append(f"would remove {target}")
      54            continue
      55        if target.is_dir() and not target.is_symlink():
      56            shutil.rmtree(target)
      57            lines.append(f"removed {target}")
      58        elif target.exists() or target.is_symlink():
      59            target.unlink()
      60            lines.append(f"removed {target}")
      61        else:
      62            lines.append(f"not found {target}")
      63    return lines
      64
      65
      66def uninstall_installed_tool(
      67    *,
      68    dry_run: bool = False,
      69    runner: CommandRunner | None = None,
      70) -> tuple[int, list[str]]:
      71    """Remove the installed `nipux` command from common uv-tool locations."""
      72
      73    uv = shutil.which("uv")
      74    run = runner or _run_command
      75    lines: list[str] = []
      76    if dry_run:
      77        lines.append("would run uv tool uninstall nipux")
      78        for path in installed_tool_paths():
      79            lines.append(f"would remove installed command path {path}")
      80        return 0, lines
      81    if uv:
      82        result = run([uv, "tool", "uninstall", "nipux"])
      83        lines.extend(_process_lines(result))
      84        if result.returncode == 0:
      85            if not lines:
      86                lines.append("removed installed nipux command")
      87            return 0, lines
      88        lines.append("uv tool uninstall failed; checking safe local tool paths")
      89    else:
      90        lines.append("uv not found; checking safe local tool paths")
      91
      92    removed = False
      93    errors = 0
      94    for path in installed_tool_paths():
      95        try:
      96            if path.is_dir() and not path.is_symlink():
      97                shutil.rmtree(path)
      98                lines.append(f"removed {path}")
      99                removed = True
     100            elif path.exists() or path.is_symlink():
     101                path.unlink()
     102                lines.append(f"removed {path}")
     103                removed = True
     104        except OSError as exc:
     105            lines.append(f"failed to remove {path}: {exc}")
     106            errors += 1
     107    if removed and not errors:
     108        return 0, lines
     109    if not removed:
     110        lines.append("installed nipux command not found")
     111    return (1 if errors else 0), lines
     112
     113
     114def installed_tool_paths() -> tuple[Path, ...]:
     115    """Return safe user-level paths for uv-tool Nipux installs."""
     116
     117    home = Path.home().expanduser().resolve(strict=False)
     118    candidates = [
     119        home / ".local" / "bin" / "nipux",
     120        home / ".local" / "share" / "uv" / "tools" / "nipux",
     121    ]
     122    current = shutil.which("nipux")
     123    if current:
     124        candidates.append(Path(current))
     125    safe: list[Path] = []
     126    for path in _dedupe_paths(candidates):
     127        expanded = path.expanduser()
     128        if _is_safe_installed_tool_path(expanded, home=home):
     129            safe.append(expanded)
     130    return tuple(safe)
     131
     132
     133def _disable_services(*, dry_run: bool, runner: Runner) -> list[str]:
     134    lines: list[str] = []
     135    launch_path = launch_agent_path()
     136    label = "gui/" + str(os.getuid()) + "/com.nipux.agent"
     137    launchctl = shutil.which("launchctl")
     138    if dry_run:
     139        lines.append(f"would unload launchd {label}")
     140    elif launchctl:
     141        runner([launchctl, "bootout", label], check=False, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
     142        lines.append(f"unloaded launchd {label}")
     143    else:
     144        lines.append("launchd unavailable")
     145
     146    systemctl = shutil.which("systemctl")
     147    service_path = systemd_service_path()
     148    if systemctl and service_path.exists():
     149        if dry_run:
     150            lines.append("would disable systemd user service nipux.service")
     151        else:
     152            runner(
     153                [systemctl, "--user", "disable", "--now", "nipux.service"],
     154                check=False,
     155                stdout=subprocess.DEVNULL,
     156                stderr=subprocess.DEVNULL,
     157            )
     158            runner([systemctl, "--user", "daemon-reload"], check=False, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
     159            lines.append("disabled systemd user service nipux.service")
     160    elif service_path.exists():
     161        lines.append("systemd unavailable; removing service file only")
     162
     163    if not launch_path.exists() and not service_path.exists():
     164        lines.append("no installed service files found")
     165    return lines
     166
     167
     168def _run_command(command: Sequence[str]) -> subprocess.CompletedProcess[str]:
     169    return subprocess.run(
     170        list(command),
     171        text=True,
     172        stdout=subprocess.PIPE,
     173        stderr=subprocess.STDOUT,
     174        check=False,
     175    )
     176
     177
     178def _process_lines(process: subprocess.CompletedProcess[str]) -> list[str]:
     179    output = process.stdout if isinstance(process.stdout, str) else ""
     180    stderr = process.stderr if isinstance(process.stderr, str) else ""
     181    return [line.rstrip() for line in f"{output}\n{stderr}".splitlines() if line.strip()]
     182
     183
     184def _dedupe_paths(paths: list[Path]) -> list[Path]:
     185    seen: set[str] = set()
     186    result: list[Path] = []
     187    for path in paths:
     188        key = str(path.expanduser())
     189        if key in seen:
     190            continue
     191        seen.add(key)
     192        result.append(path)
     193    return result
     194
     195
     196def _assert_safe_delete_target(path: Path) -> None:
     197    resolved = path.expanduser().resolve(strict=False)
     198    home = Path.home().resolve(strict=False)
     199    forbidden = {Path("/").resolve(strict=False), home}
     200    if resolved in forbidden:
     201        raise ValueError(f"refusing to remove unsafe path: {path}")
     202    if len(resolved.parts) < 3:
     203        raise ValueError(f"refusing to remove broad path: {path}")
     204
     205
     206def _is_safe_installed_tool_path(path: Path, *, home: Path) -> bool:
     207    expanded = path.expanduser()
     208    resolved = expanded.resolve(strict=False)
     209    user_bin = home / ".local" / "bin" / "nipux"
     210    uv_tool_root = home / ".local" / "share" / "uv" / "tools" / "nipux"
     211    return (
     212        expanded == user_bin
     213        or resolved == user_bin
     214        or expanded == uv_tool_root
     215        or resolved == uv_tool_root
     216        or uv_tool_root in resolved.parents
     217    )

nipux_cli/updater.py 179 lines

       1"""Self-update helpers for source checkouts and installed tools."""
       2
       3from __future__ import annotations
       4
       5import os
       6import shutil
       7import subprocess
       8from collections.abc import Callable, Sequence
       9from pathlib import Path
      10
      11
      12GitRunner = Callable[[Sequence[str], Path], subprocess.CompletedProcess[str]]
      13CommandRunner = Callable[[Sequence[str]], subprocess.CompletedProcess[str]]
      14
      15DEFAULT_UPDATE_REPO = "https://github.com/nipuxx/agent-cli.git"
      16DEFAULT_UPDATE_REF = "main"
      17
      18
      19def find_checkout_root(start: str | Path | None = None) -> Path | None:
      20    """Return the nearest enclosing git checkout for the Nipux install."""
      21
      22    current = Path(start).expanduser().resolve() if start else Path(__file__).resolve()
      23    if current.is_file():
      24        current = current.parent
      25    for candidate in (current, *current.parents):
      26        if (candidate / ".git").exists():
      27            return candidate
      28    return None
      29
      30
      31def update_checkout(
      32    *,
      33    path: str | Path | None = None,
      34    allow_dirty: bool = False,
      35    runner: GitRunner | None = None,
      36    command_runner: CommandRunner | None = None,
      37) -> tuple[int, list[str]]:
      38    """Update the current Nipux install and return output lines.
      39
      40    Source checkouts are fast-forwarded with git. Installed tools are refreshed
      41    from the configured source repository so `nipux update` works from anywhere.
      42    """
      43
      44    root = Path(path).expanduser().resolve() if path else find_checkout_root()
      45    if not root or not (root / ".git").exists():
      46        prefix = []
      47        if path is not None:
      48            prefix.append(f"{_short_path(root)} is not a source checkout; updating the installed Nipux tool instead.")
      49        code, lines = _update_uv_tool_install(runner=command_runner)
      50        return code, [*prefix, *lines]
      51    run = runner or _run_git
      52    top_level = run(["git", "rev-parse", "--show-toplevel"], root)
      53    if top_level.returncode != 0:
      54        return top_level.returncode, ["Cannot update: git could not identify the checkout.", *_process_lines(top_level)]
      55    checkout = Path(top_level.stdout.strip() or root).expanduser().resolve()
      56    before = _git_text(run(["git", "rev-parse", "--short", "HEAD"], checkout), fallback="unknown")
      57    branch = _git_text(run(["git", "branch", "--show-current"], checkout), fallback="detached")
      58    dirty = run(["git", "status", "--porcelain"], checkout)
      59    if dirty.returncode != 0:
      60        return dirty.returncode, ["Cannot update: git status failed.", *_process_lines(dirty)]
      61    if dirty.stdout.strip() and not allow_dirty:
      62        return (
      63            1,
      64            [
      65                f"Cannot update: local changes exist in {_short_path(checkout)}.",
      66                "Commit or stash them first, then run `nipux update` again.",
      67            ],
      68        )
      69    lines = [f"Updating Nipux in {_short_path(checkout)}", f"Current: {branch} @ {before}"]
      70    pulled = run(["git", "pull", "--ff-only"], checkout)
      71    lines.extend(_process_lines(pulled))
      72    if pulled.returncode != 0:
      73        return pulled.returncode, ["Update failed.", *lines]
      74    after = _git_text(run(["git", "rev-parse", "--short", "HEAD"], checkout), fallback=before)
      75    if after == before:
      76        lines.append("Nipux is already up to date.")
      77    else:
      78        lines.append(f"Updated Nipux: {before} -> {after}.")
      79    lines.append("Update complete.")
      80    return 0, lines
      81
      82
      83def _update_uv_tool_install(*, runner: CommandRunner | None = None) -> tuple[int, list[str]]:
      84    uv = shutil.which("uv")
      85    if not uv:
      86        return (
      87            1,
      88            [
      89                "Cannot update automatically because `uv` was not found.",
      90                "Install uv, then run `nipux update` again.",
      91            ],
      92        )
      93    run = runner or _run_command
      94    spec = _uv_tool_update_spec()
      95    lines = [
      96        "Updating installed Nipux command.",
      97        f"Source: {spec}",
      98    ]
      99    current = shutil.which("nipux")
     100    if current:
     101        lines.append(f"Command: {current}")
     102    updated = run([uv, "tool", "install", "--force", "--upgrade", "--reinstall", "--refresh", spec])
     103    lines.extend(_process_lines(updated))
     104    if updated.returncode != 0:
     105        return updated.returncode, ["Update failed.", *lines]
     106    lines.append("Nipux command refreshed from source.")
     107    verified = _verify_updated_command(runner=run)
     108    if verified:
     109        lines.append(verified)
     110    lines.append("Update complete.")
     111    return 0, lines
     112
     113
     114def _verify_updated_command(*, runner: CommandRunner) -> str:
     115    nipux = shutil.which("nipux")
     116    if not nipux:
     117        return ""
     118    checked = runner([nipux, "--version"])
     119    version_line = " ".join(_process_lines(checked)).strip()
     120    if checked.returncode != 0 or not version_line:
     121        return ""
     122    return f"Verified: {version_line}"
     123
     124
     125def _uv_tool_update_spec() -> str:
     126    """Return the direct source uv should use for installed-tool updates."""
     127
     128    explicit = os.environ.get("NIPUX_UPDATE_SPEC", "").strip()
     129    if explicit:
     130        return explicit
     131    repo = os.environ.get("NIPUX_REPO_URL", DEFAULT_UPDATE_REPO).strip() or DEFAULT_UPDATE_REPO
     132    ref = os.environ.get("NIPUX_REF", DEFAULT_UPDATE_REF).strip() or DEFAULT_UPDATE_REF
     133    if repo.startswith(("git+", "http://", "https://", "ssh://", "file://")):
     134        prefix = repo if repo.startswith("git+") else f"git+{repo}"
     135        return f"{prefix}@{ref}"
     136    return f"git+{repo}@{ref}"
     137
     138
     139def _run_git(command: Sequence[str], cwd: Path) -> subprocess.CompletedProcess[str]:
     140    return subprocess.run(
     141        list(command),
     142        cwd=cwd,
     143        text=True,
     144        stdout=subprocess.PIPE,
     145        stderr=subprocess.STDOUT,
     146        check=False,
     147    )
     148
     149
     150def _run_command(command: Sequence[str]) -> subprocess.CompletedProcess[str]:
     151    return subprocess.run(
     152        list(command),
     153        text=True,
     154        stdout=subprocess.PIPE,
     155        stderr=subprocess.STDOUT,
     156        check=False,
     157    )
     158
     159
     160def _process_lines(process: subprocess.CompletedProcess[str]) -> list[str]:
     161    output = process.stdout if isinstance(process.stdout, str) else ""
     162    return [line.rstrip() for line in output.splitlines() if line.strip()]
     163
     164
     165def _git_text(process: subprocess.CompletedProcess[str], *, fallback: str) -> str:
     166    if process.returncode != 0:
     167        return fallback
     168    value = process.stdout.strip() if isinstance(process.stdout, str) else ""
     169    return value or fallback
     170
     171
     172def _short_path(path: Path | str, *, max_width: int = 96) -> str:
     173    text = str(path)
     174    home = str(Path.home())
     175    if text.startswith(home + os.sep):
     176        text = "~" + text[len(home) :]
     177    if len(text) <= max_width:
     178        return text
     179    return "..." + text[-max(12, max_width - 4) :]

nipux_cli/updates.py 132 lines

       1"""Readable durable progress reports for jobs."""
       2
       3from __future__ import annotations
       4
       5import shlex
       6from typing import Any
       7
       8from nipux_cli.config import AppConfig
       9from nipux_cli.daemon import daemon_lock_status
      10from nipux_cli.db import AgentDB
      11from nipux_cli.tui_outcomes import hourly_update_lines, recent_model_update_lines
      12from nipux_cli.tui_status import job_display_state
      13from nipux_cli.tui_style import _one_line
      14
      15
      16def render_updates_report(
      17    db: AgentDB,
      18    config: AppConfig,
      19    job_id: str,
      20    *,
      21    limit: int = 5,
      22    chars: int = 180,
      23    paths: bool = False,
      24) -> list[str]:
      25    job = db.get_job(job_id)
      26    artifacts = db.list_artifacts(job_id, limit=limit)
      27    events = db.list_timeline_events(job_id, limit=max(250, limit * 80))
      28    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
      29    operator_messages = _metadata_list(metadata, "operator_messages")
      30    agent_updates = _metadata_list(metadata, "agent_updates")
      31    lessons = _metadata_list(metadata, "lessons")
      32    daemon = daemon_lock_status(config.runtime.home / "agentd.lock")
      33    lines = [
      34        f"updates {job['title']} | state {job_display_state(job, bool(daemon['running']))}",
      35        "=" * 80,
      36    ]
      37    if operator_messages:
      38        latest = operator_messages[-1]
      39        lines.append(f"last steering: {_one_line(latest.get('message') or '', chars)}")
      40    lines.append("outcomes by hour:")
      41    outcome_lines = hourly_update_lines(events, width=max(72, chars), limit=max(8, limit * 4))
      42    if outcome_lines:
      43        lines.extend(f"  {line}" for line in outcome_lines)
      44    else:
      45        lines.append("  none yet")
      46    if agent_updates:
      47        lines.extend(["", "latest agent notes:"])
      48        for update in agent_updates[-min(limit, 5) :]:
      49            category = update.get("category") or "progress"
      50            lines.append(f"  {category}: {_one_line(update.get('message') or '', chars)}")
      51    if lessons:
      52        lines.extend(["", "latest lessons:"])
      53        for lesson in lessons[-min(limit, 5) :]:
      54            category = lesson.get("category") or "memory"
      55            lines.append(f"  {category}: {_one_line(lesson.get('lesson') or '', chars)}")
      56    lines.extend(["", "latest saved outputs:"])
      57    if not artifacts:
      58        lines.append("  none yet")
      59    for artifact in artifacts:
      60        title = artifact.get("title") or artifact["id"]
      61        summary = f" - {_one_line(artifact['summary'], chars)}" if artifact.get("summary") else ""
      62        lines.append(f"  {artifact['created_at']} {title}{summary}")
      63        lines.append(f"    view: artifact {shlex.quote(title)}")
      64        if paths:
      65            lines.append(f"    {artifact['path']}")
      66    lines.extend(["", "raw tool stream: activity"])
      67    return lines
      68
      69
      70def render_all_updates_report(
      71    db: AgentDB,
      72    config: AppConfig,
      73    *,
      74    limit: int = 5,
      75    chars: int = 180,
      76    paths: bool = False,
      77) -> list[str]:
      78    jobs = db.list_jobs()
      79    daemon = daemon_lock_status(config.runtime.home / "agentd.lock")
      80    lines = [
      81        f"outcomes all jobs | {len(jobs)} tracked",
      82        "=" * 80,
      83    ]
      84    if not jobs:
      85        lines.append("No jobs yet.")
      86        return lines
      87    for job in jobs[: max(1, limit)]:
      88        job_id = str(job["id"])
      89        counts = db.job_record_counts(job_id)
      90        state = job_display_state(job, bool(daemon["running"]))
      91        lines.append("")
      92        lines.append(f"{job['title']} | {state}")
      93        lines.append(
      94            "  "
      95            + " ".join(
      96                [
      97                    f"actions={counts.get('steps', 0)}",
      98                    f"outputs={counts.get('artifacts', 0)}",
      99                    f"findings={_metadata_count(job, 'finding_ledger')}",
     100                    f"tasks={_metadata_count(job, 'task_queue')}",
     101                    f"experiments={_metadata_count(job, 'experiment_ledger')}",
     102                ]
     103            )
     104        )
     105        events = db.list_events(job_id=job_id, limit=max(200, limit * 60))
     106        outcome_lines = recent_model_update_lines(events, width=max(72, chars), limit=max(2, min(4, limit)))
     107        if outcome_lines:
     108            lines.extend(f"  {line}" for line in outcome_lines)
     109        else:
     110            lines.append("  no durable outcomes yet")
     111        artifacts = db.list_artifacts(job_id, limit=2)
     112        for artifact in artifacts:
     113            title = artifact.get("title") or artifact["id"]
     114            summary = f" - {_one_line(artifact['summary'], chars)}" if artifact.get("summary") else ""
     115            lines.append(f"  output: {_one_line(title, chars)}{summary}")
     116            if paths:
     117                lines.append(f"    {artifact['path']}")
     118    if len(jobs) > limit:
     119        lines.append("")
     120        lines.append(f"... {len(jobs) - limit} more jobs hidden. Increase --limit to show more.")
     121    return lines
     122
     123
     124def _metadata_list(metadata: dict[str, Any], key: str) -> list[dict[str, Any]]:
     125    values = metadata.get(key)
     126    return [value for value in values if isinstance(value, dict)] if isinstance(values, list) else []
     127
     128
     129def _metadata_count(job: dict[str, Any], key: str) -> int:
     130    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     131    values = metadata.get(key)
     132    return len(values) if isinstance(values, list) else 0

nipux_cli/usage.py 78 lines

       1"""Formatting helpers for model token and cost usage."""
       2
       3from __future__ import annotations
       4
       5from typing import Any
       6
       7from nipux_cli.tui_layout import _format_compact_count, _format_usage_cost
       8
       9
      10def format_usage_report(
      11    *,
      12    title: str,
      13    usage: dict[str, Any],
      14    context_length: int,
      15    model: str,
      16    base_url: str,
      17) -> list[str]:
      18    calls = _safe_int(usage.get("calls"))
      19    prompt = _safe_int(usage.get("prompt_tokens"))
      20    completion = _safe_int(usage.get("completion_tokens"))
      21    total = _safe_int(usage.get("total_tokens")) or prompt + completion
      22    latest_prompt = _safe_int(usage.get("latest_prompt_tokens"))
      23    latest_completion = _safe_int(usage.get("latest_completion_tokens"))
      24    latest_total = _safe_int(usage.get("latest_total_tokens")) or latest_prompt + latest_completion
      25    estimated = _safe_int(usage.get("estimated_calls"))
      26    cached = _safe_int(usage.get("cached_tokens"))
      27    reasoning = _safe_int(usage.get("reasoning_tokens"))
      28    cost = _format_usage_cost(usage, model=model, base_url=base_url)
      29    cost_limit = _safe_optional_float(usage.get("max_job_cost_usd"))
      30    context_text = _format_compact_count(latest_prompt)
      31    if context_length > 0:
      32        context_text = f"{context_text}/{_format_compact_count(context_length)}"
      33    lines = [
      34        f"usage {title}",
      35        "=" * 80,
      36        f"model: {model}",
      37        f"calls: {calls} | estimated: {estimated}",
      38        f"tokens: total={_format_compact_count(total)} prompt={_format_compact_count(prompt)} output={_format_compact_count(completion)}",
      39        f"latest: ctx={context_text} output={_format_compact_count(latest_completion)} total={_format_compact_count(latest_total)}",
      40        f"details: cached={_format_compact_count(cached)} reasoning={_format_compact_count(reasoning)} cost={cost}",
      41    ]
      42    if cost_limit is not None and cost_limit > 0:
      43        current_cost = _safe_float(usage.get("cost"))
      44        remaining = max(0.0, cost_limit - current_cost)
      45        lines.append(f"limit: max job cost=${cost_limit:g} remaining=${remaining:.4f}")
      46    if calls <= 0:
      47        lines.append("no model usage has been recorded for this job yet")
      48    elif estimated:
      49        lines.append("some usage is estimated because the provider did not return complete token/cost metadata")
      50    elif not bool(usage.get("has_cost")):
      51        lines.append(
      52            "cost is pending unless the provider returns cost metadata, configured token rates are set, "
      53            "or the model is local/free"
      54        )
      55    return lines
      56
      57
      58def _safe_int(value: Any) -> int:
      59    try:
      60        return int(float(value))
      61    except (TypeError, ValueError):
      62        return 0
      63
      64
      65def _safe_float(value: Any) -> float:
      66    try:
      67        return float(value)
      68    except (TypeError, ValueError):
      69        return 0.0
      70
      71
      72def _safe_optional_float(value: Any) -> float | None:
      73    if value in (None, ""):
      74        return None
      75    try:
      76        return float(value)
      77    except (TypeError, ValueError):
      78        return None

nipux_cli/web.py 121 lines

       1"""Small web search/extract helpers without external web tool dependencies."""
       2
       3from __future__ import annotations
       4
       5import html
       6import re
       7import urllib.parse
       8import urllib.request
       9from html.parser import HTMLParser
      10from typing import Any
      11
      12from nipux_cli.source_quality import anti_bot_reason
      13
      14
      15class _TextExtractor(HTMLParser):
      16    def __init__(self):
      17        super().__init__()
      18        self.parts: list[str] = []
      19        self.skip_depth = 0
      20
      21    def handle_starttag(self, tag: str, attrs):
      22        del attrs
      23        if tag in {"script", "style", "noscript"}:
      24            self.skip_depth += 1
      25        if tag in {"p", "br", "div", "section", "article", "li", "h1", "h2", "h3"}:
      26            self.parts.append("\n")
      27
      28    def handle_endtag(self, tag: str):
      29        if tag in {"script", "style", "noscript"} and self.skip_depth:
      30            self.skip_depth -= 1
      31        if tag in {"p", "div", "section", "article", "li"}:
      32            self.parts.append("\n")
      33
      34    def handle_data(self, data: str):
      35        if not self.skip_depth:
      36            text = data.strip()
      37            if text:
      38                self.parts.append(text)
      39
      40    def text(self) -> str:
      41        raw = " ".join(self.parts)
      42        raw = re.sub(r"[ \t\r\f\v]+", " ", raw)
      43        raw = re.sub(r"\n\s+", "\n", raw)
      44        raw = re.sub(r"\n{3,}", "\n\n", raw)
      45        return html.unescape(raw).strip()
      46
      47
      48def _request(url: str, *, timeout: int = 20, max_bytes: int = 2_000_000) -> tuple[str, str]:
      49    request = urllib.request.Request(
      50        url,
      51        headers={
      52            "User-Agent": "Mozilla/5.0 (compatible; nipux/0.1; +https://github.com/nipuxx/agent-cli)",
      53            "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,text/plain;q=0.8,*/*;q=0.5",
      54        },
      55    )
      56    with urllib.request.urlopen(request, timeout=timeout) as response:
      57        content_type = response.headers.get("content-type", "")
      58        body = response.read(max_bytes + 1)
      59    if len(body) > max_bytes:
      60        raise ValueError(f"response exceeded {max_bytes} bytes")
      61    charset = "utf-8"
      62    match = re.search(r"charset=([\w.-]+)", content_type, re.I)
      63    if match:
      64        charset = match.group(1)
      65    return body.decode(charset, errors="replace"), content_type
      66
      67
      68def _strip_html(markup: str) -> str:
      69    parser = _TextExtractor()
      70    parser.feed(markup)
      71    return parser.text()
      72
      73
      74def _duckduckgo_link(raw: str) -> str:
      75    parsed = urllib.parse.urlparse(html.unescape(raw))
      76    query = urllib.parse.parse_qs(parsed.query)
      77    if "uddg" in query and query["uddg"]:
      78        return query["uddg"][0]
      79    return html.unescape(raw)
      80
      81
      82def web_search(query: str, *, limit: int = 5) -> dict[str, Any]:
      83    url = "https://duckduckgo.com/html/?" + urllib.parse.urlencode({"q": query})
      84    markup, _ = _request(url)
      85    pattern = re.compile(
      86        r'<a[^>]+class="result__a"[^>]+href="(?P<href>[^"]+)"[^>]*>(?P<title>.*?)</a>',
      87        re.I | re.S,
      88    )
      89    results = []
      90    for match in pattern.finditer(markup):
      91        title = re.sub(r"<[^>]+>", "", match.group("title"))
      92        results.append({"title": html.unescape(title).strip(), "url": _duckduckgo_link(match.group("href"))})
      93        if len(results) >= limit:
      94            break
      95    return {"success": True, "query": query, "results": results}
      96
      97
      98def web_extract(urls: list[str], *, limit_chars: int = 12_000) -> dict[str, Any]:
      99    pages = []
     100    for url in urls[:5]:
     101        try:
     102            body, content_type = _request(url)
     103            text = body if "text/plain" in content_type else _strip_html(body)
     104            reason = anti_bot_reason(url, text[:2000])
     105            page = {
     106                "url": url,
     107                "content_type": content_type,
     108                "text": text[:limit_chars],
     109                "truncated": len(text) > limit_chars,
     110            }
     111            if reason:
     112                page["source_warning"] = reason
     113                page["warnings"] = [{
     114                    "type": "anti_bot",
     115                    "message": reason,
     116                    "guidance": "This page may require normal human browser verification. Do not bypass protections.",
     117                }]
     118            pages.append(page)
     119        except Exception as exc:
     120            pages.append({"url": url, "error": str(exc)})
     121    return {"success": True, "pages": pages}

nipux_cli/worker.py 7538 lines

       1"""Bounded worker loop for one restartable agent step."""
       2
       3from __future__ import annotations
       4
       5import json
       6import re
       7import shlex
       8import signal
       9import threading
      10import time
      11from dataclasses import dataclass
      12from datetime import datetime, timezone
      13from pathlib import Path
      14from typing import Any
      15from urllib.parse import urlparse
      16
      17from nipux_cli.artifacts import ArtifactStore
      18from nipux_cli.config import AppConfig, load_config
      19from nipux_cli.compression import refresh_memory_index
      20from nipux_cli.context_pressure import (
      21    context_pressure_for_prompt,
      22    emit_context_pressure_update,
      23    emit_usage_pressure_update,
      24    usage_pressure_for_prompt,
      25)
      26from nipux_cli.db import AgentDB
      27from nipux_cli.llm import LLMResponse, LLMResponseError, OpenAIChatLLM, StepLLM, ToolCall
      28from nipux_cli.measurement import measurement_candidates, measurement_candidates_are_diagnostic_only
      29from nipux_cli.memory_graph import memory_graph_from_job
      30from nipux_cli.metric_format import format_metric_value
      31from nipux_cli.operator_context import (
      32    inactive_prompt_operator_ids,
      33)
      34from nipux_cli.progress import build_progress_checkpoint
      35from nipux_cli.provider_errors import provider_action_required_note
      36from nipux_cli.source_quality import anti_bot_reason
      37from nipux_cli.task_match import find_semantic_task_match, task_key
      38from nipux_cli.tools import DEFAULT_REGISTRY, ToolContext, ToolRegistry
      39from nipux_cli.worker_policy import (
      40    ACTIVITY_STAGNATION_BLOCKED_TOOLS,
      41    ACTIVITY_STAGNATION_CHECKPOINTS,
      42    ANTI_BOT_ACK_TERMS,
      43    ARTIFACT_ACCOUNTING_BLOCKED_TOOLS,
      44    ARTIFACT_ACCOUNTING_RESOLUTION_TOOLS,
      45    BRANCH_WORK_TOOLS,
      46    CHURN_TOOLS,
      47    DELIVERABLE_ARTIFACT_TERMS,
      48    DELIVERABLE_PROGRESS_BLOCKED_TOOLS,
      49    DELIVERABLE_RESEARCH_BUDGET_STEPS,
      50    EVIDENCE_ARTIFACT_TERMS,
      51    EXPERIMENT_DELIVERY_ACTION_TERMS,
      52    EXPERIMENT_INFORMATION_ACTION_TERMS,
      53    EXPERIMENT_NEXT_ACTION_BLOCKED_TOOLS,
      54    FILE_VALIDATION_BLOCKED_TOOLS,
      55    FILE_VALIDATION_RESOLUTION_TOOLS,
      56    LEDGER_PROGRESS_TOOLS,
      57    MAX_WORKER_PROMPT_CHARS,
      58    MEASURABLE_ACTION_BUDGET_STEPS,
      59    MEASURABLE_PROGRESS_PATTERN,
      60    MEASURABLE_RESEARCH_BLOCKED_TOOLS,
      61    MEASURABLE_RESEARCH_BUDGET_STEPS,
      62    MEASUREMENT_BLOCKED_TOOLS,
      63    MEASUREMENT_RESOLUTION_TOOLS,
      64    MEMORY_CONSOLIDATION_BLOCKED_TOOLS,
      65    MEMORY_ENTRY_PROMPT_CHARS,
      66    MEMORY_PROMPT_CHARS,
      67    MILESTONE_VALIDATION_BLOCKED_TOOLS,
      68    PROGRAM_PROMPT_CHARS,
      69    QUERY_STOPWORDS,
      70    READ_ONLY_SHELL_COMMAND_PATTERN,
      71    RECENT_STATE_PROMPT_CHARS,
      72    RECENT_STATE_STEPS,
      73    RECOVERABLE_GUARD_ERRORS,
      74    REFLECTION_INTERVAL_STEPS,
      75    RESEARCH_BALANCE_BLOCKED_TOOLS,
      76    ROADMAP_STALENESS_BLOCKED_TOOLS,
      77    SOURCE_YIELD_BLOCKED_TOOLS,
      78    SYSTEM_PROMPT,
      79    TASK_DELIVERABLE_ACTION_TERMS,
      80    TASK_PLANNING_STAGNATION_CHECKPOINTS,
      81    TASK_QUEUE_SATURATION_OPEN_TASKS,
      82    TASK_QUEUE_TOTAL_SOFT_LIMIT,
      83    TEXT_TOKEN_STOPWORDS,
      84)
      85from nipux_cli.worker_prompt_context import (
      86    _as_float,
      87    _as_int,
      88    _experiments_for_prompt,
      89    _ledgers_for_prompt,
      90    _lessons_for_prompt,
      91    _memory_graph_for_prompt,
      92    _memory_entries_for_prompt,
      93    _metadata_list,
      94    _operator_messages_for_prompt,
      95    _outcomes_for_prompt,
      96    _render_worker_prompt,
      97    _roadmap_for_prompt,
      98    _tasks_for_prompt,
      99    _timeline_for_prompt,
     100)
     101from nipux_cli.worker_prompt_format import (
     102    clip_text as _clip_text,
     103    format_step_for_prompt as _format_step_for_prompt,
     104    observation_for_prompt as _observation_for_prompt,
     105)
     106from nipux_cli.worker_tool_summary import summarize_tool_result as _summarize_tool_result
     107from nipux_cli.worker_usage import turn_usage_metadata
     108
     109
     110__all__ = ["MAX_WORKER_PROMPT_CHARS", "_render_worker_prompt", "build_messages", "run_one_step"]
     111
     112
     113LESSON_SPRAWL_MIN_LESSONS = 30
     114LESSON_SPRAWL_RECENT_LESSONS = 3
     115EXPERIMENT_STAGNATION_MIN_TRIALS = 6
     116EXPERIMENT_STAGNATION_NON_IMPROVING = 4
     117SOURCE_YIELD_MIN_SOURCES = 12
     118SOURCE_YIELD_MIN_RECENT_GATHERING = 5
     119
     120
     121@dataclass(frozen=True)
     122class StepExecution:
     123    job_id: str
     124    run_id: str
     125    step_id: str
     126    tool_name: str | None
     127    status: str
     128    result: dict[str, Any]
     129
     130
     131EXPERIMENT_NEXT_ACTION_VERIFY_SHELL_PATTERN = re.compile(
     132    r"(?is)^\s*(?:command\s+-v\b|which\b|type\b|test\b|ls\b|find\b|stat\b|file\b)"
     133)
     134EXPERIMENT_NEXT_ACTION_VERIFY_STOPWORDS = {
     135    "action",
     136    "after",
     137    "before",
     138    "from",
     139    "into",
     140    "next",
     141    "real",
     142    "then",
     143    "using",
     144    "with",
     145}
     146MILESTONE_MATCH_STOPWORDS = {
     147    "acceptance",
     148    "blocked",
     149    "criteria",
     150    "current",
     151    "done",
     152    "evidence",
     153    "failed",
     154    "issue",
     155    "issues",
     156    "milestone",
     157    "needed",
     158    "pending",
     159    "passed",
     160    "result",
     161    "roadmap",
     162    "status",
     163    "title",
     164    "validating",
     165    "validation",
     166    "validate",
     167}
     168
     169
     170def build_messages(
     171    job: dict[str, Any],
     172    recent_steps: list[dict[str, Any]],
     173    memory_entries: list[dict[str, Any]] | None = None,
     174    program_text: str = "",
     175    timeline_events: list[dict[str, Any]] | None = None,
     176    active_operator_messages: list[dict[str, Any]] | None = None,
     177    include_unclaimed_operator_messages: bool = True,
     178    token_usage: dict[str, Any] | None = None,
     179) -> list[dict[str, Any]]:
     180    step_lines = []
     181    for step in recent_steps[-RECENT_STATE_STEPS:]:
     182        step_lines.append(_clip_text(_format_step_for_prompt(step), 720))
     183    state = _clip_text("\n".join(step_lines), RECENT_STATE_PROMPT_CHARS) if step_lines else "No prior steps."
     184    memory_lines = []
     185    for entry in _memory_entries_for_prompt(memory_entries or []):
     186        refs = ", ".join((entry.get("artifact_refs") or [])[:8])
     187        suffix = f"\nArtifact refs: {refs}" if refs else ""
     188        memory_lines.append(
     189            _clip_text(f"### {entry.get('key') or 'memory'}\n{entry.get('summary') or ''}{suffix}", MEMORY_ENTRY_PROMPT_CHARS)
     190        )
     191    memory_text = _clip_text("\n\n".join(memory_lines), MEMORY_PROMPT_CHARS) if memory_lines else "No compact memory yet."
     192    program = _clip_text(program_text.strip(), PROGRAM_PROMPT_CHARS) if program_text else "No program.md saved yet."
     193    operator_messages = _operator_messages_for_prompt(
     194        job,
     195        active_messages=active_operator_messages or [],
     196        include_unclaimed=include_unclaimed_operator_messages,
     197    )
     198    current_execution_focus = _current_execution_focus_for_prompt(job, recent_steps)
     199    measurement_obligation = _measurement_obligation_for_prompt(job)
     200    recent_measurement_evidence = _recent_measurement_evidence_for_prompt(job, recent_steps)
     201    file_validation_obligation = _file_validation_obligation_for_prompt(job)
     202    candidate_file_discovery = _candidate_file_discovery_for_prompt(job, recent_steps)
     203    shell_path_recovery = _shell_path_recovery_for_prompt(recent_steps)
     204    shell_permission_recovery = _shell_permission_recovery_for_prompt(recent_steps)
     205    measured_progress_guard = _measured_progress_guard_for_prompt(job, recent_steps)
     206    experiment_stagnation_guard = _experiment_stagnation_guard_for_prompt(job, recent_steps)
     207    research_balance_guard = _research_balance_guard_for_prompt(job, recent_steps)
     208    source_yield_guard = _source_yield_guard_for_prompt(job, recent_steps)
     209    deliverable_progress_guard = _deliverable_progress_guard_for_prompt(job, recent_steps)
     210    progress_accounting_guard = _progress_accounting_for_prompt(recent_steps)
     211    evidence_checkpoint_guard = _evidence_checkpoint_accounting_for_prompt(job, recent_steps)
     212    activity_stagnation = _activity_stagnation_for_prompt(job)
     213    task_planning_guard = _task_planning_guard_for_prompt(job)
     214    task_queue_saturation = _task_queue_saturation_for_prompt(job, recent_steps)
     215    memory_consolidation_guard = _memory_consolidation_guard_for_prompt(job, recent_steps)
     216    lesson_consolidation_guard = _lesson_consolidation_guard_for_prompt(job, recent_steps)
     217    durable_yield = _durable_yield_for_prompt(job, recent_steps)
     218    context_pressure = context_pressure_for_prompt(job)
     219    usage_pressure = usage_pressure_for_prompt(job, token_usage)
     220    lessons = _lessons_for_prompt(job)
     221    memory_graph = _memory_graph_for_prompt(job)
     222    roadmap = _roadmap_for_prompt(job)
     223    tasks = _tasks_for_prompt(job)
     224    ledgers = _ledgers_for_prompt(job)
     225    experiments = _experiments_for_prompt(job)
     226    reflections = _reflections_for_prompt(job)
     227    timeline = _timeline_for_prompt(timeline_events or [])
     228    outcomes = _outcomes_for_prompt(timeline_events or [])
     229    next_constraint = _next_action_constraint(job, recent_steps)
     230    content = _render_worker_prompt(
     231        job,
     232        sections=[
     233            (
     234                "Workspace",
     235                "\n".join([
     236                    "- shell_exec runs on the machine hosting this Nipux worker, in the current worker directory unless the command changes it",
     237                    "- saved artifacts are separate Nipux outputs; read_artifact is only for those saved outputs",
     238                    "- use shell_exec for workspace/project files unless the file is a saved artifact",
     239                ]),
     240            ),
     241            ("Operator context", operator_messages),
     242            ("Current execution focus", current_execution_focus),
     243            ("Pending measurement obligation", measurement_obligation),
     244            ("Recent measurement evidence", recent_measurement_evidence),
     245            ("Pending file validation obligation", file_validation_obligation),
     246            ("Candidate file discovery", candidate_file_discovery),
     247            ("Shell path recovery", shell_path_recovery),
     248            ("Shell permission recovery", shell_permission_recovery),
     249            ("Measured progress guard", measured_progress_guard),
     250            ("Experiment stagnation guard", experiment_stagnation_guard),
     251            ("Research balance guard", research_balance_guard),
     252            ("Source yield guard", source_yield_guard),
     253            ("Deliverable progress guard", deliverable_progress_guard),
     254            ("Progress accounting guard", progress_accounting_guard),
     255            ("Evidence checkpoint accounting guard", evidence_checkpoint_guard),
     256            ("Activity stagnation", activity_stagnation),
     257            ("Task planning guard", task_planning_guard),
     258            ("Task queue saturation", task_queue_saturation),
     259            ("Memory consolidation guard", memory_consolidation_guard),
     260            ("Lesson consolidation guard", lesson_consolidation_guard),
     261            ("Durable progress yield", durable_yield),
     262            ("Context pressure", context_pressure),
     263            ("Usage pressure", usage_pressure),
     264            ("Program", program),
     265            ("Lessons learned", lessons),
     266            ("Memory graph", memory_graph),
     267            ("Roadmap", roadmap),
     268            ("Task queue", tasks),
     269            ("Durable outcomes", outcomes),
     270            ("Ledgers", ledgers),
     271            ("Experiment ledger", experiments),
     272            ("Reflections", reflections),
     273            ("Compact memory", memory_text),
     274            ("Recent visible timeline", timeline),
     275            ("Recent state", state),
     276            ("Next-action constraint", next_constraint),
     277        ],
     278    )
     279    return [
     280        {"role": "system", "content": SYSTEM_PROMPT},
     281        {"role": "user", "content": content},
     282    ]
     283
     284
     285def _acknowledge_non_prompt_operator_context(db: AgentDB, job_id: str) -> int:
     286    job = db.get_job(job_id)
     287    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     288    messages = metadata.get("operator_messages") if isinstance(metadata.get("operator_messages"), list) else []
     289    message_ids = inactive_prompt_operator_ids(messages)
     290    if not message_ids:
     291        return 0
     292    result = db.acknowledge_operator_messages(
     293        job_id,
     294        message_ids=message_ids,
     295        summary="conversation-only message retained in history, not used as worker constraint",
     296    )
     297    return int(result.get("count") or 0)
     298
     299
     300def _measured_progress_guard_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
     301    context = _measured_progress_guard_context(job, recent_steps)
     302    if not context:
     303        return "None."
     304    if _as_int(context.get("shell_actions_since_last_experiment")) >= _as_int(context.get("shell_action_budget")):
     305        candidate_context = _candidate_file_discovery_context(job, recent_steps)
     306        shell_guidance = "Do not call shell_exec or do more research next."
     307        if candidate_context:
     308            shell_guidance = (
     309                "Do not call broad shell_exec or do more research next. A single bounded shell_exec is allowed only "
     310                "when it validates one exact candidate path already listed in Candidate file discovery."
     311            )
     312        return (
     313            "This objective or active task is measurably framed, and the shell/action budget since the last experiment "
     314            f"is exhausted. completed_since_last_experiment={context.get('completed_since_last_experiment')} "
     315            f"shell_actions={context.get('shell_actions_since_last_experiment')} shell_budget={context.get('shell_action_budget')} "
     316            f"reason={context.get('reason')}. {shell_guidance} Use record_experiment "
     317            "for a known result, record_tasks to create a missing experiment/monitor branch, or record_lesson if the "
     318            "branch is blocked or the recent outputs were not valid measurements."
     319        )
     320    return (
     321        "This objective or active task is measurably framed, but recent work has not produced "
     322        f"new experiment records. completed_since_last_experiment={context.get('completed_since_last_experiment')} "
     323        f"research_budget={context.get('research_budget')} shell_actions={context.get('shell_actions_since_last_experiment')} "
     324        f"shell_budget={context.get('shell_action_budget')} reason={context.get('reason')}. "
     325        "Next useful actions: run a small measuring action, call record_experiment for a known result, "
     326        "or use record_tasks to create an experiment/action/monitor task with acceptance criteria and evidence."
     327    )
     328
     329
     330def _deliverable_progress_guard_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
     331    context = _deliverable_progress_guard_context(job, recent_steps)
     332    if not context:
     333        return "None."
     334    return (
     335        "This objective or active task expects a durable deliverable, but recent branch work has not produced a "
     336        "draft/report/file checkpoint. "
     337        f"completed_since_last_deliverable={context.get('completed_since_last_deliverable')} "
     338        f"research_budget={context.get('research_budget')} reason={context.get('reason')}. "
     339        "Next useful actions: write_file or write_artifact for a partial deliverable, record_tasks for a smaller "
     340        "deliverable branch, record_roadmap/record_milestone_validation for validation, or record_lesson if the "
     341        "deliverable is blocked."
     342    )
     343
     344
     345def _experiment_stagnation_guard_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
     346    context = _experiment_stagnation_context(job, recent_steps)
     347    if not context:
     348        return "None."
     349    return (
     350        "Recent measured trials have not improved the best observed result. "
     351        f"metric={context.get('metric_name')} unit={context.get('metric_unit')} "
     352        f"best={context.get('best_value')} latest={context.get('latest_value')} "
     353        f"non_improving={context.get('non_improving_count')} recent_trials={context.get('recent_trials')}. "
     354        "Before more experiments, shell execution, research, or output churn, record a decision: reject or block the "
     355        "stale branch, pivot to a materially different branch, update the roadmap/task queue, or explain why the "
     356        "stagnant measurements are still useful."
     357    )
     358
     359
     360def _measurement_obligation_for_prompt(job: dict[str, Any]) -> str:
     361    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     362    obligation = metadata.get("pending_measurement_obligation")
     363    if not isinstance(obligation, dict) or not obligation or obligation.get("resolved_at"):
     364        return "None."
     365    candidates = obligation.get("metric_candidates") if isinstance(obligation.get("metric_candidates"), list) else []
     366    lines = [
     367        f"source_step=#{obligation.get('source_step_no') or '?'} tool={obligation.get('tool') or ''}",
     368        f"summary={obligation.get('summary') or ''}",
     369    ]
     370    command = str(obligation.get("command") or "")
     371    if command:
     372        lines.append(f"command={_clip_text(command, 360)}")
     373    if candidates:
     374        lines.append("metric_candidates=" + "; ".join(str(item) for item in candidates[:6]))
     375    lines.append(
     376        "Before more research or artifact churn, call record_experiment with the measured result, "
     377        "record_lesson explaining why it is not a valid measurement, or record_tasks to create the missing measurement branch."
     378    )
     379    return "\n".join(lines)
     380
     381
     382def _recent_measurement_evidence_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]], *, window: int = 140) -> str:
     383    if _pending_measurement_obligation(job):
     384        return "Covered by the pending measurement obligation."
     385    latest_experiment_step_no = max(
     386        (
     387            _as_int(step.get("step_no"))
     388            for step in recent_steps
     389            if step.get("tool_name") == "record_experiment" and step.get("status") == "completed"
     390        ),
     391        default=0,
     392    )
     393    lines: list[str] = []
     394    for step in reversed(_completed_or_failed_recent_steps(recent_steps)[-window:]):
     395        if step.get("tool_name") != "shell_exec":
     396            continue
     397        output = step.get("output") if isinstance(step.get("output"), dict) else {}
     398        if not output:
     399            continue
     400        command = _step_command(step) or str(output.get("command") or "")
     401        candidates = measurement_candidates(output, command=command, limit=4)
     402        if not candidates or measurement_candidates_are_diagnostic_only(candidates, command=command):
     403            continue
     404        step_no = _as_int(step.get("step_no"))
     405        relation = "after last experiment" if step_no > latest_experiment_step_no else "before last experiment"
     406        prefix = f"- step #{step_no or '?'} {step.get('status') or ''} ({relation})"
     407        detail = "; ".join(str(candidate) for candidate in candidates[:3])
     408        command_detail = f" command={_clip_text(command, 180)}" if command else ""
     409        lines.append(_clip_text(f"{prefix}: {detail}.{command_detail}", 520))
     410        if len(lines) >= 6:
     411            break
     412    if not lines:
     413        return "None."
     414    return "\n".join([
     415        "Recent shell output contains measurable-looking values. Reconcile valid values with record_experiment; "
     416        "if a value is invalid, record why before treating the branch as complete.",
     417        *reversed(lines),
     418    ])
     419
     420
     421def _file_validation_obligation_for_prompt(job: dict[str, Any]) -> str:
     422    obligation = _pending_file_validation_obligation(job)
     423    if not obligation:
     424        return "None."
     425    lines = [
     426        f"path={obligation.get('path') or ''}",
     427        f"source_step=#{obligation.get('source_step_no') or '?'}",
     428        f"reason={obligation.get('reason') or 'recent file output needs validation'}",
     429    ]
     430    suggested = str(obligation.get("suggested_validation") or "").strip()
     431    if suggested:
     432        lines.append(f"suggested_validation={suggested}")
     433    lines.append(
     434        "Before more research/output churn, validate the file with shell_exec, "
     435        "corroborating any `file` output with header/signature bytes, checksum/size, or a parser/loader when the expected "
     436        "format matters, "
     437        "or use record_tasks/record_lesson/record_experiment to explain the blocked or deferred validation."
     438    )
     439    return "\n".join(lines)
     440
     441
     442def _current_execution_focus_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
     443    focus = _current_execution_focus_context(job, recent_steps)
     444    if not focus:
     445        return (
     446            "No isolated focus yet. Follow the next-action constraint, choose one bounded branch, "
     447            "and account for the result before expanding the backlog."
     448        )
     449    lines = [
     450        f"phase={focus['phase']}",
     451        "focus=" + _clip_text(str(focus.get("focus") or ""), 620),
     452        "next=" + _clip_text(str(focus.get("next") or ""), 620),
     453    ]
     454    evidence = str(focus.get("evidence") or "").strip()
     455    if evidence:
     456        lines.append("evidence=" + _clip_text(evidence, 520))
     457    task = str(focus.get("task") or "").strip()
     458    if task:
     459        lines.append("task=" + _clip_text(task, 520))
     460    backlog = focus.get("backlog") if isinstance(focus.get("backlog"), dict) else {}
     461    if backlog:
     462        lines.append(
     463            "backlog="
     464            f"{backlog.get('total')} tasks, {backlog.get('open')} runnable/open. "
     465            "Treat it as advisory until this focus is resolved; do not add new branches unless directly closing, "
     466            "merging, or relabeling existing work."
     467        )
     468    lines.append(
     469        "Boundary: do not switch to unrelated search, task creation, or stale branches unless this focus is blocked "
     470        "by fresh tool evidence. If it is blocked, record the blocker and the next concrete recovery action."
     471    )
     472    return "\n".join(lines)
     473
     474
     475def _current_execution_focus_context(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> dict[str, Any] | None:
     476    backlog = _task_backlog_pressure_context(job)
     477    measurement_obligation = _pending_measurement_obligation(job)
     478    if measurement_obligation:
     479        return {
     480            "phase": "account_measurement",
     481            "focus": f"Resolve pending measurement from step #{measurement_obligation.get('source_step_no') or '?'}",
     482            "next": "Use record_experiment for a valid measured result, or record_lesson/record_tasks if the result is invalid or missing.",
     483            "evidence": str(measurement_obligation.get("summary") or ""),
     484            "backlog": backlog,
     485        }
     486    file_validation = _pending_file_validation_obligation(job)
     487    if file_validation:
     488        return {
     489            "phase": "validate_file",
     490            "focus": f"Validate recently written file {file_validation.get('path') or ''}",
     491            "next": str(file_validation.get("suggested_validation") or "Run one bounded validation command, then account for the result."),
     492            "evidence": str(file_validation.get("reason") or ""),
     493            "backlog": backlog,
     494        }
     495    checkpoint = _auto_checkpoint_accounting_context(job, recent_steps)
     496    if checkpoint:
     497        if checkpoint.get("checkpoint_read"):
     498            next_action = "Use a durable ledger tool to account for the already-read evidence checkpoint."
     499        else:
     500            next_action = "Read the specific checkpoint artifact or account for it from existing evidence."
     501        return {
     502            "phase": "account_checkpoint",
     503            "focus": f"Resolve auto-saved evidence checkpoint {checkpoint.get('artifact_id') or checkpoint.get('title') or ''}",
     504            "next": next_action,
     505            "evidence": str(checkpoint.get("summary") or checkpoint.get("title") or ""),
     506            "backlog": backlog,
     507        }
     508    candidate_files = _candidate_file_discovery_context(job, recent_steps)
     509    if candidate_files:
     510        paths = candidate_files.get("paths") if isinstance(candidate_files.get("paths"), list) else []
     511        invalid_paths = set(str(path) for path in candidate_files.get("invalid_paths") or [])
     512        primary_path = next((str(path) for path in paths if str(path) not in invalid_paths), str(paths[0]) if paths else "")
     513        validated = _candidate_file_recently_validated(primary_path, recent_steps)
     514        if validated:
     515            return {
     516                "phase": "execute_with_validated_candidate",
     517                "focus": f"Use the recently validated candidate path: {primary_path}",
     518                "next": (
     519                    "Run the next bounded action or measurement that uses this validated path; do not repeat "
     520                    "existence checks unless a new command requires a different property."
     521                ),
     522                "evidence": validated,
     523                "task": str(candidate_files.get("task_text") or ""),
     524                "backlog": backlog,
     525            }
     526        return {
     527            "phase": "execute_candidate_validation",
     528            "focus": f"Validate the highest-confidence candidate path: {primary_path}",
     529            "next": "Run one bounded shell/file validation against the primary path, then record the measurement, finding, lesson, or blocker.",
     530            "evidence": f"Ranked candidates: {'; '.join(str(path) for path in paths[:4])}",
     531            "task": str(candidate_files.get("task_text") or ""),
     532            "backlog": backlog,
     533        }
     534    grounding_block = _latest_evidence_grounding_block(recent_steps)
     535    if grounding_block:
     536        return {
     537            "phase": "repair_record",
     538            "focus": "Rewrite or replace the blocked durable record using only observed evidence.",
     539            "next": "Use exact observed tokens/paths from recent evidence, or record why the attempted claim is invalid.",
     540            "evidence": str(grounding_block.get("error") or grounding_block.get("summary") or ""),
     541            "backlog": backlog,
     542        }
     543    experiment_next_action = _latest_experiment_next_action_context(job)
     544    if experiment_next_action:
     545        return {
     546            "phase": "execute_measured_next_action",
     547            "focus": "Continue from the latest measured experiment decision.",
     548            "next": str(experiment_next_action.get("next_action") or ""),
     549            "evidence": str(experiment_next_action.get("title") or experiment_next_action.get("summary") or ""),
     550            "backlog": backlog,
     551        }
     552    milestone_validation = _milestone_validation_needed(job)
     553    if milestone_validation:
     554        return {
     555            "phase": "validate_milestone",
     556            "focus": f"Validate milestone: {milestone_validation.get('title') or 'current milestone'}",
     557            "next": "Use record_milestone_validation with pass/fail/blocker status from observed evidence.",
     558            "evidence": str(milestone_validation.get("evidence_needed") or milestone_validation.get("acceptance_criteria") or ""),
     559            "backlog": backlog,
     560        }
     561    task = _primary_execution_task(job)
     562    if task and backlog:
     563        return {
     564            "phase": "execute_task",
     565            "focus": str(task.get("title") or "current task"),
     566            "next": str(task.get("next_action") or task.get("goal") or task.get("acceptance_criteria") or "Take one bounded action for this task."),
     567            "evidence": str(task.get("evidence_needed") or task.get("result") or ""),
     568            "task": str(task),
     569            "backlog": backlog,
     570        }
     571    return None
     572
     573
     574def _task_backlog_pressure_context(job: dict[str, Any]) -> dict[str, int] | None:
     575    tasks = [task for task in _metadata_list(job, "task_queue") if isinstance(task, dict)]
     576    if not tasks:
     577        return None
     578    runnable_statuses = {"active", "open", "waiting", "blocked"}
     579    open_count = sum(1 for task in tasks if str(task.get("status") or "open").lower() in runnable_statuses)
     580    total_count = len(tasks)
     581    if total_count < TASK_QUEUE_TOTAL_SOFT_LIMIT and open_count < TASK_QUEUE_SATURATION_OPEN_TASKS:
     582        return None
     583    return {"total": total_count, "open": open_count}
     584
     585
     586def _primary_execution_task(job: dict[str, Any]) -> dict[str, Any] | None:
     587    tasks = [task for task in _metadata_list(job, "task_queue") if isinstance(task, dict)]
     588    status_rank = {"active": 0, "open": 1, "waiting": 2, "blocked": 3}
     589    runnable = [task for task in tasks if str(task.get("status") or "open").lower() in status_rank]
     590    if not runnable:
     591        return None
     592    return sorted(
     593        runnable,
     594        key=lambda task: (
     595            status_rank.get(str(task.get("status") or "open").lower(), 9),
     596            -_as_int(task.get("priority")),
     597            str(task.get("title") or ""),
     598        ),
     599    )[0]
     600
     601
     602def _candidate_file_discovery_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
     603    context = _candidate_file_discovery_context(job, recent_steps)
     604    if not context:
     605        return "None."
     606    paths = context["paths"]
     607    source_text = context["source_text"]
     608    lines = [
     609        f"{source_text} while open work depends on file/path validation.",
     610        "Candidate paths:",
     611    ]
     612    for path in paths[:8]:
     613        lines.append(f"- {path}")
     614    primary_path = str(paths[0]) if paths else ""
     615    validation = _candidate_file_recently_validated(primary_path, recent_steps)
     616    if validation:
     617        lines.append(
     618            "Highest-ranked candidate has recent positive validation evidence. "
     619            f"Use it for the next action instead of repeating existence checks: {_clip_text(validation, 420)}"
     620        )
     621    invalid_paths = context.get("invalid_paths") if isinstance(context.get("invalid_paths"), list) else []
     622    if invalid_paths:
     623        lines.append(
     624            "Recently invalid or stub-like candidates: "
     625            + "; ".join(str(path) for path in invalid_paths[:5])
     626            + ". Prefer higher-confidence candidates before retrying these."
     627        )
     628    lines.append(
     629        "Validate likely candidates with shell_exec before recording a no-file/no-progress claim or searching for alternatives. "
     630        "Do not reject a non-empty candidate binary from `file` output alone; corroborate with header/signature bytes, "
     631        "checksum/size, or a parser/loader for the expected format, or record uncertainty. "
     632        "Treat durable-record candidates as candidates until revalidated. This supersedes stale no-candidate/no-file memory "
     633        "until validation proves those candidates are irrelevant."
     634    )
     635    lines.append(f"Relevant open work: {_clip_text(context['task_text'], 500)}")
     636    return "\n".join(lines)
     637
     638
     639def _shell_path_recovery_for_prompt(recent_steps: list[dict[str, Any]]) -> str:
     640    context = _shell_path_recovery_context(recent_steps)
     641    if not context:
     642        return "None."
     643    paths = context.get("missing_paths") if isinstance(context.get("missing_paths"), list) else []
     644    commands = context.get("missing_commands") if isinstance(context.get("missing_commands"), list) else []
     645    candidate_executables = (
     646        context.get("candidate_executables") if isinstance(context.get("candidate_executables"), dict) else {}
     647    )
     648    observed_executables = (
     649        context.get("observed_executables") if isinstance(context.get("observed_executables"), list) else []
     650    )
     651    lines = [
     652        f"Recent shell step #{context.get('step_no') or '?'} reported a missing command or path.",
     653    ]
     654    if commands:
     655        lines.append("Missing commands: " + ", ".join(str(command) for command in commands[:6]))
     656    if candidate_executables:
     657        for command, command_paths in list(candidate_executables.items())[:6]:
     658            if not isinstance(command_paths, list) or not command_paths:
     659                continue
     660            lines.append(
     661                f"Observed candidate executable for {command}: "
     662                + ", ".join(str(path) for path in command_paths[:4])
     663            )
     664        lines.append("Recovery priority: try the exact candidate path or add its directory to PATH before package-manager/install retries.")
     665    if paths:
     666        lines.append("Missing paths: " + ", ".join(str(path) for path in paths[:6]))
     667    if observed_executables:
     668        lines.append("Observed executable paths in partial shell output: " + ", ".join(str(path) for path in observed_executables[:8]))
     669    if not commands and not paths:
     670        lines.append("Missing command/path was not parsed.")
     671    command = str(context.get("command") or "")
     672    if command:
     673        lines.append(f"Failed command: {_clip_text(command, 420)}")
     674    excerpt = str(context.get("excerpt") or "")
     675    if excerpt:
     676        lines.append(f"Observed output: {_clip_text(excerpt, 360)}")
     677    lines.append(
     678        "Do not treat this output as a successful measurement or deliverable. Next, locate or verify the real "
     679        "executable/file path with a bounded shell probe such as command -v, find, ls, or an equivalent platform "
     680        "tool. Retry using only a validated path, or record the branch as blocked/skipped with the observed reason."
     681    )
     682    return "\n".join(lines)
     683
     684
     685def _shell_path_recovery_context(recent_steps: list[dict[str, Any]], *, window: int = 16) -> dict[str, Any] | None:
     686    for step in reversed(_completed_or_failed_recent_steps(recent_steps)[-window:]):
     687        if step.get("tool_name") != "shell_exec":
     688            continue
     689        output = step.get("output") if isinstance(step.get("output"), dict) else {}
     690        text = "\n".join(str(output.get(key) or "") for key in ("stdout", "stderr", "error"))
     691        if not text.strip():
     692            continue
     693        missing_paths = _missing_paths_from_shell_output(text)
     694        if not missing_paths and not _shell_output_has_missing_command(text):
     695            continue
     696        commands = _missing_commands_from_shell_output(text)
     697        return {
     698            "step_no": step.get("step_no"),
     699            "command": output.get("command"),
     700            "missing_commands": commands,
     701            "candidate_executables": _candidate_executable_paths_for_missing_commands(recent_steps, commands),
     702            "observed_executables": _observed_executable_paths_from_recent_shell(
     703                recent_steps,
     704                exclude_paths=missing_paths,
     705            ),
     706            "missing_paths": missing_paths,
     707            "excerpt": text.strip(),
     708        }
     709    return None
     710
     711
     712def _shell_permission_recovery_for_prompt(recent_steps: list[dict[str, Any]]) -> str:
     713    context = _recent_privileged_shell_failure_context(recent_steps)
     714    if not context:
     715        return "None."
     716    lines = [
     717        f"Recent shell step #{context.get('step_no') or '?'} failed because a privileged/package-manager command lacked permission.",
     718    ]
     719    command = str(context.get("command") or "")
     720    if command:
     721        lines.append(f"Failed command: {_clip_text(command, 420)}")
     722    excerpt = str(context.get("excerpt") or "")
     723    if excerpt:
     724        lines.append(f"Observed output: {_clip_text(excerpt, 360)}")
     725    lines.append("Recovery priority: try non-privileged alternatives first; record when operator credentials are required.")
     726    lines.append(
     727        "Do not retry the same privileged/package-manager path. Prefer observed executables, user-writable installs, "
     728        "existing project files, or other non-privileged alternatives; otherwise record the branch as blocked, skipped, "
     729        "or needing operator credentials."
     730    )
     731    return "\n".join(lines)
     732
     733
     734def _shell_step_failure_text(step: dict[str, Any]) -> str:
     735    output = step.get("output") if isinstance(step.get("output"), dict) else {}
     736    return "\n".join(str(output.get(key) or "") for key in ("stdout", "stderr", "error"))
     737
     738
     739def _shell_output_has_missing_command(text: str) -> bool:
     740    lowered = text.lower()
     741    return any(marker in lowered for marker in ("command not found", ": not found", "no such file or directory"))
     742
     743
     744def _missing_paths_from_shell_output(text: str) -> list[str]:
     745    patterns = [
     746        r"(?:^|\n)(?:/bin/sh:\s*\d+:\s*)?(?P<path>/[^\s:'\"]+):\s*(?:not found|No such file or directory|command not found)",
     747        r"(?:cannot access|cannot stat|can't stat|stat: cannot statx?) ['\"](?P<quoted>[^'\"]+)['\"]:\s*No such file or directory",
     748        r"(?:^|\n)(?P<plain>/[^\s:'\"]+):\s*No such file or directory",
     749    ]
     750    paths: list[str] = []
     751    seen: set[str] = set()
     752    for pattern in patterns:
     753        for match in re.finditer(pattern, text, flags=re.IGNORECASE):
     754            path = str(match.groupdict().get("path") or match.groupdict().get("quoted") or match.groupdict().get("plain") or "").strip()
     755            if not path or path in seen:
     756                continue
     757            seen.add(path)
     758            paths.append(path)
     759            if len(paths) >= 12:
     760                return paths
     761    return paths
     762
     763
     764def _missing_commands_from_shell_output(text: str) -> list[str]:
     765    patterns = [
     766        r"(?:^|\n)(?:/bin/sh:\s*\d+:\s*)?(?P<cmd>[A-Za-z0-9_.+-]+):\s*(?:not found|command not found)",
     767        r"(?:^|\n)(?:sh|bash|zsh):\s*(?P<shell_cmd>[A-Za-z0-9_.+-]+):\s*command not found",
     768        r"command not found:\s*(?P<suffix_cmd>[A-Za-z0-9_.+-]+)",
     769    ]
     770    commands: list[str] = []
     771    seen: set[str] = set()
     772    for pattern in patterns:
     773        for match in re.finditer(pattern, text, flags=re.IGNORECASE):
     774            command = str(
     775                match.groupdict().get("cmd")
     776                or match.groupdict().get("shell_cmd")
     777                or match.groupdict().get("suffix_cmd")
     778                or ""
     779            ).strip()
     780            if not command or "/" in command or command in seen:
     781                continue
     782            seen.add(command)
     783            commands.append(command)
     784            if len(commands) >= 12:
     785                return commands
     786    return commands
     787
     788
     789def _candidate_executable_paths_for_missing_commands(
     790    recent_steps: list[dict[str, Any]], missing_commands: list[str], *, window: int = 20, max_paths_per_command: int = 6
     791) -> dict[str, list[str]]:
     792    command_names = {str(command or "").strip().lower() for command in missing_commands}
     793    command_names = {command for command in command_names if command}
     794    if not command_names:
     795        return {}
     796    matches: dict[str, list[str]] = {command: [] for command in command_names}
     797    seen: set[tuple[str, str]] = set()
     798    for path in _observed_executable_paths_from_recent_shell(recent_steps, command_names=command_names, window=window):
     799        name = Path(path).name.lower()
     800        if name not in command_names:
     801            continue
     802        key = (name, path.lower())
     803        if key in seen or len(matches.get(name, [])) >= max_paths_per_command:
     804            continue
     805        seen.add(key)
     806        matches.setdefault(name, []).append(path)
     807    return {command: paths for command, paths in matches.items() if paths}
     808
     809
     810def _observed_executable_paths_from_recent_shell(
     811    recent_steps: list[dict[str, Any]],
     812    *,
     813    command_names: set[str] | None = None,
     814    exclude_paths: list[str] | None = None,
     815    window: int = 20,
     816    max_paths: int = 12,
     817) -> list[str]:
     818    excluded = {str(path or "").lower() for path in (exclude_paths or []) if path}
     819    paths: list[str] = []
     820    seen: set[str] = set()
     821    for step in _completed_or_failed_recent_steps(recent_steps)[-window:]:
     822        if step.get("tool_name") != "shell_exec":
     823            continue
     824        output = step.get("output") if isinstance(step.get("output"), dict) else {}
     825        text = "\n".join(str(output.get(key) or "") for key in ("stdout", "stderr", "error"))
     826        for line in text.splitlines():
     827            if _shell_line_reports_missing_candidate(line):
     828                continue
     829            for path in _extract_candidate_executable_paths(line, command_names):
     830                key = path.lower()
     831                if key in excluded or key in seen:
     832                    continue
     833                seen.add(key)
     834                paths.append(path)
     835                if len(paths) >= max_paths:
     836                    return paths
     837    return paths
     838
     839
     840def _shell_line_reports_missing_candidate(line: str) -> bool:
     841    lowered = str(line or "").lower()
     842    return any(
     843        marker in lowered
     844        for marker in (
     845            "not found",
     846            "no such file or directory",
     847            "cannot access",
     848            "cannot stat",
     849            "can't stat",
     850            "missing",
     851        )
     852    )
     853
     854
     855def _extract_candidate_executable_paths(text: str, command_names: set[str] | None = None) -> list[str]:
     856    commands = {command.lower() for command in (command_names or set()) if command}
     857    paths: list[str] = []
     858    seen: set[str] = set()
     859    for match in re.finditer(r"(?<![A-Za-z0-9])(?:~|/)[^\s'\"<>|;&]{2,}", text or ""):
     860        raw = _clean_candidate_file_path(match.group(0))
     861        if not _looks_like_candidate_executable_path(raw):
     862            continue
     863        name = Path(raw).name.lower()
     864        if commands and name not in commands:
     865            continue
     866        key = raw.lower()
     867        if key in seen:
     868            continue
     869        seen.add(key)
     870        paths.append(raw)
     871    return paths
     872
     873
     874def _looks_like_candidate_executable_path(value: str) -> bool:
     875    raw = str(value or "").strip()
     876    if not raw or len(raw) > 500:
     877        return False
     878    if "://" in raw or raw.startswith("//") or "..." in raw or "…" in raw or "*" in raw:
     879        return False
     880    if not raw.startswith(("/", "~")):
     881        return False
     882    name = Path(raw).name
     883    if not name or name.startswith(".") or name in {".", ".."}:
     884        return False
     885    if any(char in name for char in ("$", "{", "}", "`")):
     886        return False
     887    return True
     888
     889
     890PACKAGE_MANAGER_WRITE_COMMAND_PATTERN = re.compile(
     891    r"(?is)(?:^|[;&|]{1,2}\s*)(?:sudo\s+|doas\s+|pkexec\s+)?"
     892    r"(?:apt-get|apt|dnf|yum|apk|pacman|zypper|brew|port)\s+"
     893    r"(?:install|upgrade|update|remove|erase|add|sync|build-dep)\b"
     894)
     895PRIVILEGED_COMMAND_PATTERN = re.compile(r"(?is)(?:^|[;&|]{1,2}\s*)(?:sudo|doas|pkexec)\b")
     896
     897
     898def _shell_command_looks_privileged_or_package_manager(command: str) -> bool:
     899    text = str(command or "").strip()
     900    if not text:
     901        return False
     902    return bool(PRIVILEGED_COMMAND_PATTERN.search(text) or PACKAGE_MANAGER_WRITE_COMMAND_PATTERN.search(text))
     903
     904
     905def _shell_output_has_permission_failure(text: str) -> bool:
     906    lowered = str(text or "").lower()
     907    return any(
     908        marker in lowered
     909        for marker in (
     910            "permission denied",
     911            "not permitted",
     912            "operation not permitted",
     913            "authentication",
     914            "authorization",
     915            "are you root",
     916            "sudo:",
     917            "password is required",
     918            "unable to acquire the dpkg frontend lock",
     919            "could not open lock file",
     920        )
     921    )
     922
     923
     924def _recent_privileged_shell_failure_context(recent_steps: list[dict[str, Any]], *, window: int = 12) -> dict[str, Any] | None:
     925    accounting_tools = {"record_experiment", "record_tasks", "record_lesson", "record_roadmap", "record_milestone_validation"}
     926    latest_accounting_step = max(
     927        (
     928            _as_int(step.get("step_no"))
     929            for step in recent_steps[-window:]
     930            if step.get("status") == "completed" and step.get("tool_name") in accounting_tools
     931        ),
     932        default=0,
     933    )
     934    for step in reversed(_completed_or_failed_recent_steps(recent_steps)[-window:]):
     935        step_no = _as_int(step.get("step_no"))
     936        if latest_accounting_step and step_no <= latest_accounting_step:
     937            continue
     938        if step.get("tool_name") != "shell_exec":
     939            continue
     940        output = step.get("output") if isinstance(step.get("output"), dict) else {}
     941        command = _step_command(step) or str(output.get("command") or "")
     942        text = _shell_step_failure_text(step)
     943        if not _shell_output_has_permission_failure(text):
     944            continue
     945        if not _shell_command_looks_privileged_or_package_manager(command):
     946            continue
     947        return {
     948            "step_no": step.get("step_no"),
     949            "command": command,
     950            "excerpt": text.strip(),
     951        }
     952    return None
     953
     954
     955def _observed_candidate_recovery_required_context(recent_steps: list[dict[str, Any]], args: dict[str, Any]) -> dict[str, Any] | None:
     956    command = str(args.get("command") or "")
     957    if not command.strip():
     958        return None
     959    context = _shell_path_recovery_context(recent_steps)
     960    if not context:
     961        return None
     962    candidate_executables = (
     963        context.get("candidate_executables") if isinstance(context.get("candidate_executables"), dict) else {}
     964    )
     965    if not candidate_executables:
     966        return None
     967    for missing_command, paths in candidate_executables.items():
     968        if not isinstance(paths, list) or not paths:
     969            continue
     970        missing_name = str(missing_command or "").strip()
     971        if not missing_name:
     972            continue
     973        if not _shell_command_invokes_bare_executable(command, missing_name):
     974            continue
     975        if _shell_command_mentions_candidate_path(command, paths):
     976            continue
     977        return {
     978            "step_no": context.get("step_no"),
     979            "missing_command": missing_name,
     980            "candidate_executables": paths[:6],
     981            "blocked_command": command,
     982        }
     983    return None
     984
     985
     986def _shell_command_invokes_bare_executable(command: str, executable_name: str) -> bool:
     987    name = str(executable_name or "").strip()
     988    if not name:
     989        return False
     990    return bool(re.search(rf"(?<![A-Za-z0-9_./-]){re.escape(name)}(?![A-Za-z0-9_.-])", command))
     991
     992
     993def _shell_command_mentions_candidate_path(command: str, candidate_paths: list[Any]) -> bool:
     994    text = str(command or "")
     995    for path_value in candidate_paths:
     996        path = str(path_value or "").strip()
     997        if not path:
     998            continue
     999        if path in text:
    1000            return True
    1001        parent = str(Path(path).parent)
    1002        if parent and parent not in {".", "/"} and parent in text:
    1003            return True
    1004    return False
    1005
    1006
    1007def _candidate_file_discovery_context(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> dict[str, Any] | None:
    1008    task_text = _open_file_dependent_task_text(job)
    1009    if not task_text:
    1010        return None
    1011    recent_paths = [
    1012        *_candidate_file_paths_from_recent_shell(recent_steps),
    1013        *_candidate_file_paths_from_recent_grounding_blocks(recent_steps),
    1014    ]
    1015    durable_paths = _candidate_file_paths_from_durable_records(job)
    1016    paths: list[str] = []
    1017    seen: set[str] = set()
    1018    for path in [*recent_paths, *durable_paths]:
    1019        key = path.lower()
    1020        if key in seen:
    1021            continue
    1022        seen.add(key)
    1023        paths.append(path)
    1024    if not paths:
    1025        return None
    1026    source_text = "Recent shell output or durable records listed candidate file paths"
    1027    if recent_paths and not durable_paths:
    1028        source_text = "Recent shell output listed candidate file paths"
    1029    elif durable_paths and not recent_paths:
    1030        source_text = "Durable records mention candidate file paths"
    1031    return {
    1032        "paths": _rank_candidate_file_paths(job, task_text, paths, recent_steps=recent_steps),
    1033        "invalid_paths": _invalid_candidate_file_paths(paths, recent_steps),
    1034        "source_text": source_text,
    1035        "task_text": task_text,
    1036    }
    1037
    1038
    1039def _shell_exec_targets_candidate_file(job: dict[str, Any], recent_steps: list[dict[str, Any]], args: dict[str, Any]) -> bool:
    1040    command = str(args.get("command") or "")
    1041    if not command.strip():
    1042        return False
    1043    context = _candidate_file_discovery_context(job, recent_steps)
    1044    if not context:
    1045        return False
    1046    command_text = command.replace("\\ ", " ")
    1047    return any(path and path in command_text for path in context.get("paths", [])[:12])
    1048
    1049
    1050def _rank_candidate_file_paths(
    1051    job: dict[str, Any],
    1052    task_text: str,
    1053    paths: list[str],
    1054    *,
    1055    recent_steps: list[dict[str, Any]] | None = None,
    1056) -> list[str]:
    1057    context_tokens = _candidate_context_tokens(job, task_text)
    1058    indexed = list(enumerate(paths))
    1059    ranked = sorted(
    1060        indexed,
    1061        key=lambda item: _candidate_file_path_score(
    1062            item[1],
    1063            context_tokens,
    1064            item[0],
    1065            recent_steps=recent_steps,
    1066        ),
    1067        reverse=True,
    1068    )
    1069    return [path for _, path in ranked]
    1070
    1071
    1072def _candidate_context_tokens(job: dict[str, Any], task_text: str) -> set[str]:
    1073    text = " ".join(str(job.get(key) or "") for key in ("title", "objective", "kind")) + " " + task_text
    1074    tokens = set()
    1075    for token in re.findall(r"[A-Za-z0-9][A-Za-z0-9._-]{2,}", text.lower()):
    1076        cleaned = token.strip("._-")
    1077        if not cleaned or cleaned in QUERY_STOPWORDS or cleaned in TEXT_TOKEN_STOPWORDS:
    1078            continue
    1079        tokens.add(cleaned)
    1080        for part in re.split(r"[._-]+", cleaned):
    1081            if len(part) >= 3 and part not in QUERY_STOPWORDS and part not in TEXT_TOKEN_STOPWORDS:
    1082                tokens.add(part)
    1083    return tokens
    1084
    1085
    1086def _candidate_file_path_score(
    1087    path: str,
    1088    context_tokens: set[str],
    1089    original_index: int,
    1090    *,
    1091    recent_steps: list[dict[str, Any]] | None = None,
    1092) -> float:
    1093    lowered_path = path.lower()
    1094    name = Path(path).name.lower()
    1095    stem = Path(name).stem.lower()
    1096    path_tokens = set()
    1097    for token in re.findall(r"[a-z0-9][a-z0-9._-]{1,}", lowered_path):
    1098        path_tokens.add(token.strip("._-"))
    1099        path_tokens.update(part for part in re.split(r"[._-]+", token) if len(part) >= 2)
    1100    score = 0.0
    1101    matches = context_tokens & {token for token in path_tokens if token}
    1102    score += len(matches) * 8.0
    1103    if any(token and token in stem for token in context_tokens):
    1104        score += 6.0
    1105    if "/" in path:
    1106        score += min(path.count("/"), 8) * 0.15
    1107    auxiliary_markers = (
    1108        "vocab",
    1109        "tokenizer",
    1110        "tokeniser",
    1111        "mmproj",
    1112        "adapter",
    1113        "config",
    1114        "readme",
    1115        "license",
    1116        "metadata",
    1117        "sample",
    1118        "example",
    1119        "stub",
    1120    )
    1121    if any(marker in name for marker in auxiliary_markers):
    1122        score -= 18.0
    1123    if name.startswith("."):
    1124        score -= 20.0
    1125    suffix = Path(name).suffix.lower()
    1126    if suffix:
    1127        score += 1.0
    1128    score += _candidate_file_observation_score(path, recent_steps or [])
    1129    score -= original_index * 0.01
    1130    return score
    1131
    1132
    1133def _invalid_candidate_file_paths(paths: list[str], recent_steps: list[dict[str, Any]]) -> list[str]:
    1134    invalid: list[str] = []
    1135    for path in paths:
    1136        if _candidate_file_observation_score(path, recent_steps) <= -30:
    1137            invalid.append(path)
    1138    return invalid
    1139
    1140
    1141def _candidate_file_observation_score(path: str, recent_steps: list[dict[str, Any]], *, window: int = 12) -> float:
    1142    if not path:
    1143        return 0.0
    1144    path_key = path.lower()
    1145    score = 0.0
    1146    for step in _completed_or_failed_recent_steps(recent_steps)[-window:]:
    1147        if step.get("tool_name") != "shell_exec":
    1148            continue
    1149        output = step.get("output") if isinstance(step.get("output"), dict) else {}
    1150        text = "\n".join(str(output.get(key) or "") for key in ("stdout", "stderr", "error"))
    1151        for line in text.splitlines():
    1152            lowered = line.lower()
    1153            if path_key not in lowered:
    1154                continue
    1155            if _shell_line_reports_missing_candidate(line):
    1156                score -= 70.0
    1157            if any(marker in lowered for marker in ("ascii text", "html document", "json data", "with no line terminators")):
    1158                score -= 45.0
    1159            score += _candidate_file_size_score_from_line(line)
    1160    return score
    1161
    1162
    1163def _candidate_file_recently_validated(path: str, recent_steps: list[dict[str, Any]], *, window: int = 12) -> str:
    1164    if not path or _candidate_file_observation_score(path, recent_steps, window=window) < 30:
    1165        return ""
    1166    path_key = path.lower()
    1167    evidence_lines: list[str] = []
    1168    for step in _completed_or_failed_recent_steps(recent_steps)[-window:]:
    1169        if step.get("tool_name") != "shell_exec":
    1170            continue
    1171        output = step.get("output") if isinstance(step.get("output"), dict) else {}
    1172        text = "\n".join(str(output.get(key) or "") for key in ("stdout", "stderr", "error"))
    1173        for line in text.splitlines():
    1174            if path_key not in line.lower():
    1175                continue
    1176            if _shell_line_reports_missing_candidate(line):
    1177                continue
    1178            evidence_lines.append(" ".join(line.split()))
    1179            if len(evidence_lines) >= 3:
    1180                return " | ".join(evidence_lines)
    1181    return "recent shell evidence showed a non-trivial candidate file size or positive file metadata"
    1182
    1183
    1184def _candidate_file_size_score_from_line(line: str) -> float:
    1185    lowered = str(line or "").lower()
    1186    if re.search(r"\b\d+(?:\.\d+)?\s*(?:t|tb|tib|g|gb|gib)\b", lowered):
    1187        return 55.0
    1188    if re.search(r"\b\d+(?:\.\d+)?\s*(?:m|mb|mib)\b", lowered):
    1189        return 18.0
    1190    integers = [int(match) for match in re.findall(r"(?<![\w.])\d{1,15}(?![\w.])", lowered)]
    1191    if any(value >= 1_000_000_000 for value in integers):
    1192        return 55.0
    1193    if any(value >= 1_000_000 for value in integers):
    1194        return 18.0
    1195    return 0.0
    1196
    1197
    1198def _open_file_dependent_task_text(job: dict[str, Any]) -> str:
    1199    tasks = _metadata_list(job, "task_queue")
    1200    parts: list[str] = []
    1201    for task in tasks:
    1202        if not isinstance(task, dict):
    1203            continue
    1204        status = str(task.get("status") or "open").lower()
    1205        if status not in {"open", "active", "waiting", "blocked"}:
    1206            continue
    1207        text = " ".join(
    1208            str(task.get(key) or "")
    1209            for key in ("title", "description", "acceptance_criteria", "evidence_needed", "stall_behavior", "contract")
    1210        )
    1211        lowered = text.lower()
    1212        if any(term in lowered for term in ("file", "path", "download", "artifact", "validate", "benchmark", "script", "config")):
    1213            parts.append(" ".join(text.split()))
    1214        if len(parts) >= 4:
    1215            break
    1216    return " | ".join(parts)
    1217
    1218
    1219def _candidate_file_paths_from_recent_shell(
    1220    recent_steps: list[dict[str, Any]], *, window: int = 8, max_paths: int = 80
    1221) -> list[str]:
    1222    paths: list[str] = []
    1223    seen: set[str] = set()
    1224    for step in _completed_or_failed_recent_steps(recent_steps)[-window:]:
    1225        if step.get("tool_name") != "shell_exec":
    1226            continue
    1227        output = step.get("output") if isinstance(step.get("output"), dict) else {}
    1228        text = "\n".join(str(output.get(key) or "") for key in ("stdout", "stderr", "error"))
    1229        for path in _extract_candidate_file_paths(text):
    1230            key = path.lower()
    1231            if key in seen:
    1232                continue
    1233            seen.add(key)
    1234            paths.append(path)
    1235            if len(paths) >= max_paths:
    1236                return paths
    1237    return paths
    1238
    1239
    1240def _candidate_file_paths_from_recent_grounding_blocks(
    1241    recent_steps: list[dict[str, Any]], *, window: int = 8, max_paths: int = 80
    1242) -> list[str]:
    1243    paths: list[str] = []
    1244    seen: set[str] = set()
    1245    for step in recent_steps[-window:]:
    1246        output = step.get("output") if isinstance(step.get("output"), dict) else {}
    1247        grounding = output.get("evidence_grounding") if isinstance(output.get("evidence_grounding"), dict) else {}
    1248        candidates = grounding.get("missing_candidate_paths")
    1249        if not isinstance(candidates, list):
    1250            continue
    1251        for candidate in candidates:
    1252            path = _clean_candidate_file_path(str(candidate or ""))
    1253            if not _looks_like_exact_candidate_file_path(path):
    1254                continue
    1255            key = path.lower()
    1256            if key in seen:
    1257                continue
    1258            seen.add(key)
    1259            paths.append(path)
    1260            if len(paths) >= max_paths:
    1261                return paths
    1262    return paths
    1263
    1264
    1265def _candidate_file_paths_from_durable_records(
    1266    job: dict[str, Any], *, max_records: int = 80, max_paths: int = 80
    1267) -> list[str]:
    1268    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    1269    paths: list[str] = []
    1270    seen: set[str] = set()
    1271    record_groups = [
    1272        _metadata_list(job, "experiment_ledger"),
    1273        _metadata_list(job, "finding_ledger"),
    1274        _metadata_list(job, "lessons"),
    1275        _metadata_list(job, "source_ledger"),
    1276        _metadata_list(job, "task_queue"),
    1277    ]
    1278    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
    1279    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
    1280    record_groups.append([item for item in milestones if isinstance(item, dict)])
    1281    checked = 0
    1282    for records in record_groups:
    1283        for record in reversed(records[-max_records:]):
    1284            if not isinstance(record, dict):
    1285                continue
    1286            checked += 1
    1287            try:
    1288                text = json.dumps(record, ensure_ascii=False, sort_keys=True)
    1289            except (TypeError, ValueError):
    1290                text = str(record)
    1291            for path in _extract_candidate_file_paths(text):
    1292                key = path.lower()
    1293                if key in seen:
    1294                    continue
    1295                seen.add(key)
    1296                paths.append(path)
    1297                if len(paths) >= max_paths:
    1298                    return paths
    1299            if checked >= max_records * len(record_groups):
    1300                return paths
    1301    return paths
    1302
    1303
    1304def _extract_candidate_file_paths(text: str) -> list[str]:
    1305    paths: list[str] = []
    1306    for match in re.finditer(r"(?<![A-Za-z0-9])(?:~|/)[^\s'\"<>|;&]{2,}", text or ""):
    1307        raw = _clean_candidate_file_path(match.group(0))
    1308        if not _looks_like_exact_candidate_file_path(raw):
    1309            continue
    1310        paths.append(raw)
    1311    for match in re.finditer(r'"path"\s*:\s*"([^"]+\.[A-Za-z0-9][A-Za-z0-9_-]{1,12})"', text or ""):
    1312        raw = _clean_candidate_file_path(match.group(1))
    1313        if not _looks_like_exact_candidate_file_path(raw, allow_relative=True):
    1314            continue
    1315        paths.append(raw)
    1316    return paths
    1317
    1318
    1319def _looks_like_exact_candidate_file_path(value: str, *, allow_relative: bool = False) -> bool:
    1320    raw = str(value or "").strip()
    1321    if not raw or len(raw) > 500:
    1322        return False
    1323    if "://" in raw or raw.startswith("//") or "..." in raw or "…" in raw or "*" in raw:
    1324        return False
    1325    if not allow_relative and not raw.startswith(("/", "~")):
    1326        return False
    1327    name = Path(raw).name
    1328    if not name or name.startswith("."):
    1329        return False
    1330    suffix = Path(name).suffix
    1331    if not suffix or not re.match(r"^\.[A-Za-z0-9][A-Za-z0-9_]{1,12}$", suffix) or not any(ch.isalpha() for ch in suffix):
    1332        return False
    1333    return True
    1334
    1335
    1336def _clean_candidate_file_path(value: str) -> str:
    1337    raw = str(value or "").strip().rstrip(".,:;)")
    1338    for separator in ("\\n", "\\r", "\\t", "\n", "\r", "\t"):
    1339        raw = raw.split(separator, 1)[0]
    1340    return raw.strip().rstrip(".,:;)")
    1341
    1342
    1343def _progress_accounting_for_prompt(recent_steps: list[dict[str, Any]]) -> str:
    1344    context = _artifact_accounting_context(recent_steps)
    1345    if not context:
    1346        return "None."
    1347    return (
    1348        "Recent saved outputs need accounting before more output/research. "
    1349        f"artifact_count={context.get('artifact_count')} since_step={context.get('since_step')} "
    1350        f"artifact_titles={'; '.join(str(title) for title in context.get('artifact_titles', [])[:4])}. "
    1351        "Next use record_tasks or record_roadmap to mark progress/reopen branches, "
    1352        "record_findings or record_source for reusable evidence, record_experiment for measured results, "
    1353        "record_milestone_validation for milestone checks, or record_lesson if these outputs are not useful."
    1354    )
    1355
    1356
    1357def _activity_stagnation_for_prompt(job: dict[str, Any]) -> str:
    1358    context = _activity_stagnation_context(job)
    1359    if not context:
    1360        return "None."
    1361    return (
    1362        "Recent checkpoints have reported activity without durable progress. "
    1363        f"activity_checkpoint_streak={context.get('streak')} threshold={context.get('threshold')} "
    1364        f"last_counts={context.get('counts')}. "
    1365        "Next classify the branch with record_findings, record_source, record_experiment, record_tasks, "
    1366        "record_roadmap, record_milestone_validation, or record_lesson. If the branch is low-yield, mark it "
    1367        "blocked/skipped and pivot before doing more read-only work or saving more outputs."
    1368    )
    1369
    1370
    1371def _research_balance_guard_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
    1372    context = _research_balance_context(job, recent_steps)
    1373    if not context:
    1374        return "None."
    1375    return (
    1376        "Recent work is execution-heavy but has little source-backed research recorded. "
    1377        f"completed_window={context.get('completed_window')} execution_actions={context.get('execution_actions')} "
    1378        f"research_actions={context.get('research_actions')} sources={context.get('sources')} findings={context.get('findings')} "
    1379        f"experiments={context.get('experiments')} files={context.get('files')}. "
    1380        "Before another deep execution/testing loop, use available research, browser, source, documentation, or local-inspection tools "
    1381        "to gather evidence and record it with record_source, record_findings, record_lesson, record_tasks, or an artifact. "
    1382        "If external research is not relevant or tools are unavailable, explicitly record why and what evidence substitutes for it."
    1383    )
    1384
    1385
    1386def _source_yield_guard_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
    1387    context = _source_yield_context(job, recent_steps)
    1388    if not context:
    1389        return "None."
    1390    return (
    1391        "Many sources have been gathered without enough durable synthesis. "
    1392        f"sources={context.get('sources')} findings={context.get('findings')} "
    1393        f"yielded_sources={context.get('yielded_sources')} recent_gathering={context.get('recent_gathering')} "
    1394        f"recent_source_titles={'; '.join(str(title) for title in context.get('recent_source_titles', [])[:4])}. "
    1395        "Before more search, extraction, browsing, shell work, file/output writing, or report chatter, distill the "
    1396        "source set into record_findings with evidence, update record_source with yield/fail outcomes, or update "
    1397        "tasks/roadmap/lessons to reject or pivot the low-yield source branch."
    1398    )
    1399
    1400
    1401def _task_planning_guard_for_prompt(job: dict[str, Any]) -> str:
    1402    context = _task_planning_stagnation_context(job)
    1403    if not context:
    1404        return "None."
    1405    return (
    1406        "Recent checkpoints only added or updated tasks without durable evidence, measurements, validations, "
    1407        f"or lessons. task_only_checkpoints={context.get('task_only_checkpoints')} "
    1408        f"open_tasks={context.get('open_tasks')} total_tasks={context.get('total_tasks')}. "
    1409        "Do not create more new open tasks next. Execute, measure, validate, write a checkpoint, mark existing "
    1410        "tasks done/blocked/skipped, or record a lesson from the branch."
    1411    )
    1412
    1413
    1414def _task_queue_saturation_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
    1415    context = _recent_task_queue_saturation_context(recent_steps)
    1416    persistent_pressure = False
    1417    if not context:
    1418        metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    1419        pressure = metadata.get("task_backlog_pressure") if isinstance(metadata.get("task_backlog_pressure"), dict) else {}
    1420        current_pressure = _current_task_backlog_pressure_context(job)
    1421        if not pressure and not current_pressure:
    1422            return "None."
    1423        if current_pressure:
    1424            guard_recovery = pressure.get("guard_recovery") if isinstance(pressure.get("guard_recovery"), dict) else {}
    1425            task_queue = guard_recovery.get("task_queue") if isinstance(guard_recovery.get("task_queue"), dict) else {}
    1426            context = {
    1427                "step_no": pressure.get("latest_step_no") or guard_recovery.get("latest_step_no") or "current",
    1428                "source": pressure.get("source") or ("guard_recovery" if guard_recovery else "current_queue"),
    1429                "reason": pressure.get("reason") or task_queue.get("reason") or current_pressure.get("reason"),
    1430                "open_count": current_pressure.get("open_count"),
    1431                "total_count": current_pressure.get("total_count"),
    1432                "open_titles": current_pressure.get("open_titles") or [],
    1433            }
    1434        else:
    1435            return "None."
    1436        persistent_pressure = True
    1437    counts = []
    1438    if context.get("open_count") is not None:
    1439        counts.append(f"open_tasks={context.get('open_count')}")
    1440    if context.get("total_count") is not None:
    1441        counts.append(f"total_tasks={context.get('total_count')}")
    1442    count_text = " ".join(counts) or "queue is saturated"
    1443    open_titles = [str(title).strip() for title in context.get("open_titles") or [] if str(title).strip()]
    1444    title_text = f" Existing runnable task titles: {json.dumps(open_titles[:8], ensure_ascii=False)}." if open_titles else ""
    1445    if context.get("source") == "blocked_record_tasks":
    1446        source_label = "record_tasks block"
    1447    elif context.get("source") == "current_queue":
    1448        source_label = "current queue"
    1449    else:
    1450        source_label = "guard recovery"
    1451    opening = (
    1452        f"Task backlog pressure remains active from {source_label} #{context.get('step_no')}: "
    1453        if persistent_pressure
    1454        else f"Task queue saturation was just hit at step #{context.get('step_no')}: "
    1455    )
    1456    return (
    1457        opening
    1458        + f"{context.get('reason') or 'task queue saturated'} ({count_text}). "
    1459        f"{title_text} "
    1460        "Do not create new task branches. Either execute an existing high-priority branch, "
    1461        "or use record_tasks only to update existing task titles to active, done, blocked, or skipped "
    1462        "with concise result/evidence. If you have a near-duplicate task, update the closest existing "
    1463        "task instead of inventing a fresh title. Consolidate branch sprawl into roadmap/milestones when useful. "
    1464        "If this repeats, record_tasks is temporarily withheld so the worker must use a non-planning action."
    1465    )
    1466
    1467
    1468def _current_task_backlog_pressure_context(job: dict[str, Any]) -> dict[str, Any] | None:
    1469    tasks = _metadata_list(job, "task_queue")
    1470    objective_tasks = [task for task in tasks if not _is_guard_recovery_task(task)]
    1471    open_tasks = [
    1472        task
    1473        for task in objective_tasks
    1474        if str(task.get("status") or "open").strip().lower().replace(" ", "_") in {"open", "active"}
    1475    ]
    1476    if len(objective_tasks) <= TASK_QUEUE_TOTAL_SOFT_LIMIT and len(open_tasks) < TASK_QUEUE_SATURATION_OPEN_TASKS:
    1477        return None
    1478    reason = "total task queue is too large" if len(objective_tasks) > TASK_QUEUE_TOTAL_SOFT_LIMIT else "too many open tasks"
    1479    return {
    1480        "reason": reason,
    1481        "open_count": len(open_tasks),
    1482        "total_count": len(objective_tasks),
    1483        "open_titles": [
    1484            str(task.get("title") or "").strip()
    1485            for task in open_tasks[:8]
    1486            if str(task.get("title") or "").strip()
    1487        ],
    1488    }
    1489
    1490
    1491def _memory_consolidation_guard_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
    1492    context = _memory_graph_consolidation_context(job, recent_steps)
    1493    if not context:
    1494        return "None."
    1495    return (
    1496        "Durable job memory is growing faster than the connected memory graph. "
    1497        f"durable_records={context.get('durable_records')} graph_nodes={context.get('graph_nodes')} "
    1498        f"graph_edges={context.get('graph_edges')} reason={context.get('reason')}. "
    1499        "Before more branch work, use record_memory_graph to consolidate the most reusable facts, strategies, "
    1500        "decisions, questions, skills, constraints, episodes, and evidence links. If there is truly nothing "
    1501        "reusable, record a lesson explaining why this branch should not become graph memory."
    1502    )
    1503
    1504
    1505def _lesson_consolidation_guard_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
    1506    context = _lesson_sprawl_context(job, recent_steps)
    1507    if not context:
    1508        return "None."
    1509    return (
    1510        "Raw lessons are accumulating faster than consolidated memory. "
    1511        f"lessons={context.get('lessons')} recent_lessons={context.get('recent_lessons')} "
    1512        f"graph_nodes={context.get('graph_nodes')} reason={context.get('reason')}. "
    1513        "Do not add another raw lesson next. Consolidate the reusable strategy, mistake, constraint, decision, "
    1514        "or question into record_memory_graph, or update existing tasks/roadmap state if the lesson only describes "
    1515        "branch status."
    1516    )
    1517
    1518
    1519def _memory_graph_consolidation_context(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> dict[str, Any] | None:
    1520    if any(step.get("tool_name") == "record_memory_graph" and step.get("status") == "completed" for step in recent_steps[-8:]):
    1521        return None
    1522    graph = memory_graph_from_job(job)
    1523    node_count = len(graph["nodes"])
    1524    edge_count = len(graph["edges"])
    1525    durable_records = _durable_memory_signal_count(job)
    1526    if durable_records < 6:
    1527        return None
    1528    reason = ""
    1529    if node_count == 0:
    1530        reason = "durable ledgers exist but no graph nodes have been consolidated"
    1531    elif durable_records >= 12 and node_count * 5 < durable_records:
    1532        reason = "graph is sparse relative to reusable durable records"
    1533    elif node_count >= 3 and edge_count == 0 and durable_records >= 10:
    1534        reason = "graph nodes exist but have no links"
    1535    if not reason:
    1536        return None
    1537    return {
    1538        "durable_records": durable_records,
    1539        "graph_nodes": node_count,
    1540        "graph_edges": edge_count,
    1541        "reason": reason,
    1542    }
    1543
    1544
    1545def _lesson_sprawl_context(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> dict[str, Any] | None:
    1546    memory_context = _memory_graph_consolidation_context(job, recent_steps)
    1547    if not memory_context:
    1548        return None
    1549    lessons = _metadata_list(job, "lessons")
    1550    lesson_count = len(lessons)
    1551    if lesson_count < LESSON_SPRAWL_MIN_LESSONS:
    1552        return None
    1553    recent_lessons = [
    1554        step
    1555        for step in recent_steps[-12:]
    1556        if step.get("tool_name") == "record_lesson" and str(step.get("status") or "").lower() == "completed"
    1557    ]
    1558    if len(recent_lessons) < LESSON_SPRAWL_RECENT_LESSONS and lesson_count < LESSON_SPRAWL_MIN_LESSONS * 2:
    1559        return None
    1560    return {
    1561        "lessons": lesson_count,
    1562        "recent_lessons": len(recent_lessons),
    1563        "graph_nodes": memory_context.get("graph_nodes"),
    1564        "graph_edges": memory_context.get("graph_edges"),
    1565        "durable_records": memory_context.get("durable_records"),
    1566        "reason": "raw lesson backlog needs graph consolidation",
    1567    }
    1568
    1569
    1570def _durable_memory_signal_count(job: dict[str, Any]) -> int:
    1571    count = (
    1572        len(_metadata_list(job, "finding_ledger"))
    1573        + len(_metadata_list(job, "source_ledger"))
    1574        + len(_metadata_list(job, "experiment_ledger"))
    1575        + len(_metadata_list(job, "lessons"))
    1576    )
    1577    tasks = _metadata_list(job, "task_queue")
    1578    count += sum(
    1579        1
    1580        for task in tasks
    1581        if str(task.get("status") or "open").lower() in {"done", "blocked", "skipped"}
    1582        and (task.get("result") or task.get("evidence_needed") or task.get("acceptance_criteria"))
    1583    )
    1584    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    1585    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
    1586    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
    1587    count += sum(
    1588        1
    1589        for milestone in milestones
    1590        if isinstance(milestone, dict)
    1591        and str(milestone.get("status") or "planned").lower() in {"active", "validating", "done", "blocked", "skipped"}
    1592    )
    1593    return count
    1594
    1595
    1596def _durable_yield_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
    1597    completed = [step for step in recent_steps if step.get("status") == "completed"]
    1598    if len(completed) < 20:
    1599        return "None."
    1600    durable_tools = LEDGER_PROGRESS_TOOLS | {"write_artifact", "write_file"}
    1601    durable_indexes = [
    1602        index
    1603        for index, step in enumerate(completed)
    1604        if step.get("tool_name") in durable_tools
    1605    ]
    1606    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    1607    durable_records = (
    1608        len(_metadata_list(job, "finding_ledger"))
    1609        + len(_metadata_list(job, "source_ledger"))
    1610        + len(_metadata_list(job, "experiment_ledger"))
    1611        + len(_metadata_list(job, "lessons"))
    1612    )
    1613    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
    1614    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
    1615    durable_records += len(milestones)
    1616    if not durable_indexes and durable_records <= 0:
    1617        return (
    1618            f"No durable progress records after {len(completed)} completed actions. "
    1619            "Next action should save an output, record findings/source/experiment/lesson/roadmap progress, "
    1620            "or mark the branch blocked/skipped before more read-only work."
    1621        )
    1622    last_durable_index = durable_indexes[-1] if durable_indexes else -1
    1623    actions_since = len(completed) - last_durable_index - 1
    1624    durable_steps = len(durable_indexes)
    1625    actions_per_durable = len(completed) / max(1, durable_steps + durable_records)
    1626    if actions_since < 25 and actions_per_durable < 30:
    1627        return "None."
    1628    return (
    1629        f"Durable yield is low: completed_actions={len(completed)} durable_steps={durable_steps} "
    1630        f"durable_records={durable_records} actions_since_last_durable={actions_since} "
    1631        f"actions_per_durable~{actions_per_durable:.1f}. "
    1632        "Prefer a concrete checkpoint next: write/save output, record measured or reusable evidence, validate a milestone, "
    1633        "or reject/pivot the branch with a lesson."
    1634    )
    1635
    1636
    1637def _reflections_for_prompt(job: dict[str, Any]) -> str:
    1638    reflections = _metadata_list(job, "reflections")
    1639    if not reflections:
    1640        return "No reflection checkpoints yet."
    1641    lines = []
    1642    for reflection in reflections[-2:]:
    1643        strategy = f" strategy={reflection.get('strategy')}" if reflection.get("strategy") else ""
    1644        lines.append("- " + _clip_text(f"{reflection.get('summary')}{strategy}", 520))
    1645    return "\n".join(lines)
    1646
    1647
    1648def _next_action_constraint(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
    1649    measurement_obligation = _pending_measurement_obligation(job)
    1650    if measurement_obligation:
    1651        return (
    1652            "A pending measurement obligation is active from "
    1653            f"step #{measurement_obligation.get('source_step_no') or '?'}. "
    1654            "Resolve it with record_experiment, record_lesson explaining why it is invalid, "
    1655            "or record_tasks creating the missing measurement branch before more research/artifact churn."
    1656        )
    1657    file_validation = _pending_file_validation_obligation(job)
    1658    if file_validation:
    1659        return (
    1660            "A recently written file needs validation before more branch work. "
    1661            f"File: {_clip_text(str(file_validation.get('path') or ''), 260)}. "
    1662            f"Suggested validation: {_clip_text(str(file_validation.get('suggested_validation') or ''), 360)}. "
    1663            "Use shell_exec to validate it, or record_tasks/record_lesson/record_experiment if validation is blocked or deferred."
    1664        )
    1665    artifact_accounting = _artifact_accounting_context(recent_steps)
    1666    if artifact_accounting:
    1667        return (
    1668            "Recent saved outputs need durable accounting. Before more artifact writing, reading, research, browsing, "
    1669            "or shell work, use record_tasks, record_roadmap, record_milestone_validation, record_findings, record_source, record_experiment, or record_lesson "
    1670            "to explain what changed and what branch is next."
    1671        )
    1672    checkpoint_accounting = _auto_checkpoint_accounting_context(job, recent_steps)
    1673    if checkpoint_accounting:
    1674        if not checkpoint_accounting.get("checkpoint_read"):
    1675            return (
    1676                "An auto-saved evidence checkpoint is pending. Read that specific checkpoint artifact, or use a durable "
    1677                "ledger tool to account for the checkpoint from existing evidence before more branch work."
    1678            )
    1679        return (
    1680            "An already-read evidence checkpoint is pending durable accounting. Use record_findings, record_source, "
    1681            "record_experiment, record_tasks, record_roadmap, record_milestone_validation, or record_lesson before "
    1682            "more shell, search, file, report, or artifact work."
    1683        )
    1684    grounding_block = _latest_evidence_grounding_block(recent_steps)
    1685    if grounding_block:
    1686        raw_missing_paths = grounding_block.get("missing_candidate_paths") if isinstance(grounding_block.get("missing_candidate_paths"), list) else []
    1687        missing_paths = [
    1688            path
    1689            for path in (_clean_candidate_file_path(str(item or "")) for item in raw_missing_paths)
    1690            if _looks_like_exact_candidate_file_path(path)
    1691        ]
    1692        path_text = "; ".join(str(path) for path in missing_paths[:6])
    1693        detail = f" Missing exact paths: {path_text}." if path_text else ""
    1694        candidate_files = _candidate_file_discovery_context(job, recent_steps)
    1695        if candidate_files:
    1696            paths = candidate_files.get("paths") if isinstance(candidate_files.get("paths"), list) else []
    1697            current_path_text = "; ".join(str(path) for path in paths[:4])
    1698            if current_path_text:
    1699                return (
    1700                    "Recent durable-record grounding failed, but current ranked candidate paths are available. "
    1701                    "Treat the failed record as rejected for now and validate the highest-confidence candidate next "
    1702                    "instead of repairing stale wording. "
    1703                    f"Candidate paths: {_clip_text(current_path_text, 520)}."
    1704                )
    1705        return (
    1706            "Recent evidence grounding blocked a durable record. Next, rewrite the record using only observed evidence, "
    1707            "include the exact observed paths/tokens when claiming candidates or files, or explicitly record why they "
    1708            f"are irrelevant/invalid.{detail}"
    1709        )
    1710    action_failure = _experiment_next_action_failure_context(job, recent_steps)
    1711    if action_failure:
    1712        return (
    1713            "The latest experiment next action was attempted, but the observed shell output reports a missing "
    1714            f"command/path/prerequisite at step #{action_failure.get('step_no') or '?'}. "
    1715            f"Observed output: {_clip_text(str(action_failure.get('excerpt') or ''), 260)}. "
    1716            "Next, account for this attempted action with record_experiment, record_tasks, or record_lesson: "
    1717            "mark the branch failed/blocked or create the concrete recovery branch. Do not run more read-only probes "
    1718            "until the failed action is durable."
    1719        )
    1720    measured_guard = _measured_progress_guard_context(job, recent_steps)
    1721    if measured_guard:
    1722        return (
    1723            "This job needs measured progress, not more research-only activity. "
    1724            "Do one of: run a small measuring command/action, call record_experiment for a known measurement, "
    1725            "record_tasks with an experiment/action/monitor contract, or record_lesson if measurement is blocked."
    1726        )
    1727    activity_stagnation = _activity_stagnation_context(job)
    1728    if activity_stagnation:
    1729        return (
    1730            "Recent checkpoints show activity without durable progress. "
    1731            "Use a ledger or planning tool to classify what changed, reject the low-yield branch, or open a better branch "
    1732            "before more read-only work or output churn."
    1733        )
    1734    task_planning_guard = _task_planning_stagnation_context(job)
    1735    if task_planning_guard:
    1736        return (
    1737            "Recent progress is only task planning. Do not create more new open tasks next. Execute an existing task, "
    1738            "record evidence/measurements/validation, write a checkpoint, mark tasks done/blocked/skipped, or record "
    1739            "a lesson before expanding the queue again."
    1740        )
    1741    task_queue_saturation = _recent_task_queue_saturation_context(recent_steps)
    1742    if task_queue_saturation:
    1743        return (
    1744            "The durable task queue is saturated. Do not create new task branches. Execute a current task, "
    1745            "or use record_tasks only to update existing task titles to active/done/blocked/skipped with evidence."
    1746        )
    1747    memory_consolidation = _memory_graph_consolidation_context(job, recent_steps)
    1748    if memory_consolidation:
    1749        return (
    1750            "Consolidate durable progress into the job memory graph before more branch work. "
    1751            "Use record_memory_graph for connected reusable knowledge, or record_lesson if the recent branch has no "
    1752            "reusable memory value."
    1753        )
    1754    deliverable_guard = _deliverable_progress_guard_context(job, recent_steps)
    1755    if deliverable_guard:
    1756        return (
    1757            "This job needs a durable deliverable checkpoint, not more background collection. "
    1758            "Use write_file or write_artifact to save a partial draft/report/file, or use record_tasks, "
    1759            "record_roadmap, record_milestone_validation, or record_lesson to explain the specific blocker "
    1760            "and the next deliverable branch."
    1761        )
    1762    research_balance = _research_balance_context(job, recent_steps)
    1763    if research_balance:
    1764        return (
    1765            "Balance execution with research before the next deep action loop. "
    1766            "Gather source-backed evidence with available web/browser/documentation/local-inspection tools and record it, "
    1767            "or record why research is not applicable and what evidence replaces it."
    1768        )
    1769    candidate_files = _candidate_file_discovery_context(job, recent_steps)
    1770    if candidate_files:
    1771        paths = candidate_files.get("paths") if isinstance(candidate_files.get("paths"), list) else []
    1772        path_text = "; ".join(str(path) for path in paths[:4])
    1773        primary_path = str(paths[0]) if paths else ""
    1774        validation = _candidate_file_recently_validated(primary_path, recent_steps)
    1775        if validation:
    1776            return (
    1777                "A highest-confidence candidate file path has recent positive validation evidence. "
    1778                "Use it in the next bounded action or measurement instead of repeating existence checks. "
    1779                f"Candidate path: {_clip_text(primary_path, 420)}. Evidence: {_clip_text(validation, 420)}."
    1780            )
    1781        return (
    1782            "Concrete candidate file paths are available while file/path-dependent work is open. "
    1783            f"Validate likely candidates next with shell_exec before retrying downloads, searching for alternatives, "
    1784            f"or recording no-file/no-progress claims. Candidate paths: {_clip_text(path_text, 520)}."
    1785        )
    1786    experiment_next_action = _latest_experiment_next_action_context(job)
    1787    if experiment_next_action:
    1788        return (
    1789            "The latest measured experiment selected a concrete next action. "
    1790            f"Next action: {_clip_text(experiment_next_action.get('next_action') or '', 520)}. "
    1791            "Act on it with the appropriate tool, or use record_tasks/record_lesson if it is invalid or blocked. "
    1792            "Do not bury it under more checkpoints or unrelated research."
    1793        )
    1794    milestone_validation = _milestone_validation_needed(job)
    1795    if milestone_validation:
    1796        return (
    1797            f"Roadmap milestone '{milestone_validation.get('title')}' is ready for validation or is marked validating. "
    1798            "Use record_milestone_validation with evidence and pass/fail/blocker status, then create follow-up tasks for gaps."
    1799        )
    1800    roadmap_staleness = _roadmap_staleness_context(job, recent_steps)
    1801    if roadmap_staleness:
    1802        return (
    1803            "The roadmap has not advanced despite durable task/artifact activity. "
    1804            "Use record_roadmap to mark the current milestone active/done/blocked, or record_milestone_validation "
    1805            "if acceptance criteria can be judged from existing evidence, before more branch work."
    1806        )
    1807    if _roadmap_missing_for_broad_job(job):
    1808        return (
    1809            "The objective is broad enough to benefit from roadmap control. Use record_roadmap to define compact milestones, "
    1810            "features, acceptance criteria, and validation checkpoints before expanding the task queue further."
    1811        )
    1812    evidence_step = _unpersisted_evidence_step(recent_steps)
    1813    if evidence_step:
    1814        return (
    1815            f"You have unsaved evidence from step #{evidence_step['step_no']} "
    1816            f"({evidence_step.get('tool_name') or evidence_step['kind']}). "
    1817            "Your next tool call should usually be write_artifact. If this evidence taught a durable rule, record_lesson after saving it."
    1818        )
    1819    if _task_queue_exhausted(job):
    1820        return (
    1821            "All durable task branches are done, skipped, or blocked. Before more research or execution, "
    1822            "use record_tasks to open the next concrete branch, or report_update if the operator needs a checkpoint."
    1823        )
    1824    for step in reversed(recent_steps[-5:]):
    1825        if step.get("status") == "failed" and step.get("tool_name") == "read_artifact":
    1826            output = step.get("output") if isinstance(step.get("output"), dict) else {}
    1827            if "artifact not found" in str(output.get("error") or step.get("summary") or "").lower():
    1828                return (
    1829                    "The last artifact read used a reference that does not exist. Do not invent or retry artifact ids. "
    1830                    "Use a valid recent artifact ref, call search_artifacts with a concrete query, or continue from "
    1831                    "already observed evidence with a durable record."
    1832                )
    1833        error = str(step.get("error") or "")
    1834        if error == "artifact required before more research":
    1835            return "The last blocked action needs write_artifact, not another search or browser action."
    1836        if error == "task branch required before more work":
    1837            return "Create or reopen a task branch with record_tasks before doing more research or execution."
    1838        if error in {"duplicate tool call blocked", "similar search query blocked", "search loop blocked"}:
    1839            output = step.get("output") if isinstance(step.get("output"), dict) else {}
    1840            blocked_tool = str(output.get("blocked_tool") or "")
    1841            if blocked_tool == "read_artifact":
    1842                return "Do not read the same artifact again. Use its content to choose a concrete next action: inspect a specific item, record findings/tasks, or write a report artifact."
    1843            if blocked_tool == "shell_exec":
    1844                return "Do not rerun the same shell discovery command. Use the prior output to inspect a specific file/item, save it, or update findings/tasks."
    1845            return "Change source, extract an existing result, save an artifact, or record a lesson about the failed strategy."
    1846    return "No special constraint beyond taking one bounded useful action."
    1847
    1848
    1849def _latest_evidence_grounding_block(recent_steps: list[dict[str, Any]]) -> dict[str, Any] | None:
    1850    resolution_after_block = False
    1851    for step in reversed(recent_steps[-8:]):
    1852        if (
    1853            step.get("status") == "completed"
    1854            and step.get("tool_name") in EVIDENCE_GROUNDING_RESOLUTION_TOOLS
    1855        ):
    1856            resolution_after_block = True
    1857            continue
    1858        if step.get("status") != "blocked":
    1859            continue
    1860        output = step.get("output") if isinstance(step.get("output"), dict) else {}
    1861        if output.get("error") != "evidence grounding required":
    1862            continue
    1863        if resolution_after_block:
    1864            return None
    1865        grounding = output.get("evidence_grounding") if isinstance(output.get("evidence_grounding"), dict) else {}
    1866        return grounding or {"unsupported_tokens": []}
    1867    return None
    1868
    1869
    1870def _milestone_validation_needed(job: dict[str, Any]) -> dict[str, Any] | None:
    1871    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    1872    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
    1873    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
    1874    for milestone in milestones:
    1875        if not isinstance(milestone, dict):
    1876            continue
    1877        status = str(milestone.get("status") or "planned")
    1878        validation_status = str(milestone.get("validation_status") or "not_started")
    1879        if status == "validating" or validation_status == "pending":
    1880            return milestone
    1881        features = milestone.get("features") if isinstance(milestone.get("features"), list) else []
    1882        if status == "active" and features and all(
    1883            isinstance(feature, dict) and str(feature.get("status") or "planned") in {"done", "skipped"}
    1884            for feature in features
    1885        ):
    1886            return milestone
    1887    return None
    1888
    1889
    1890def _tool_call_matches_pending_milestone_need(tool_name: str, args: dict[str, Any], milestone: dict[str, Any]) -> bool:
    1891    if str(milestone.get("validation_status") or "").strip().lower() != "pending":
    1892        return False
    1893    if tool_name not in BRANCH_WORK_TOOLS:
    1894        return False
    1895    return _text_matches_pending_milestone_need(_json_value_text(args), milestone)
    1896
    1897
    1898def _text_matches_pending_milestone_need(text: str, milestone: dict[str, Any]) -> bool:
    1899    parts = [
    1900        str(milestone.get("title") or ""),
    1901        str(milestone.get("next_action") or ""),
    1902        str(milestone.get("acceptance_criteria") or ""),
    1903        str(milestone.get("evidence_needed") or ""),
    1904        str(milestone.get("validation_evidence") or ""),
    1905        str(milestone.get("validation_result") or ""),
    1906        " ".join(str(item) for item in milestone.get("validation_issues") or [] if item),
    1907    ]
    1908    features = milestone.get("features") if isinstance(milestone.get("features"), list) else []
    1909    for feature in features:
    1910        if not isinstance(feature, dict):
    1911            continue
    1912        parts.extend([
    1913            str(feature.get("title") or ""),
    1914            str(feature.get("goal") or ""),
    1915            str(feature.get("acceptance_criteria") or ""),
    1916            str(feature.get("evidence_needed") or ""),
    1917        ])
    1918    need_tokens = _substantive_next_action_tokens(" ".join(parts)) - MILESTONE_MATCH_STOPWORDS
    1919    if not need_tokens:
    1920        return False
    1921    call_tokens = _substantive_next_action_tokens(text) - MILESTONE_MATCH_STOPWORDS
    1922    if not call_tokens:
    1923        return False
    1924    return bool(need_tokens & call_tokens)
    1925
    1926
    1927def _milestone_validation_call_matches_current(args: dict[str, Any], milestone: dict[str, Any]) -> bool:
    1928    requested = _norm_task_key("", str(args.get("milestone") or args.get("title") or ""))
    1929    if not requested:
    1930        return False
    1931    candidates = [
    1932        _norm_task_key("", str(milestone.get("title") or "")),
    1933        _norm_task_key("", str(milestone.get("key") or "")),
    1934    ]
    1935    for candidate in candidates:
    1936        if not candidate:
    1937            continue
    1938        if requested == candidate or requested in candidate or candidate in requested:
    1939            return True
    1940    return False
    1941
    1942
    1943def _normalize_milestone_validation_args_for_active_gate(
    1944    tool_name: str,
    1945    args: dict[str, Any],
    1946    job: dict[str, Any],
    1947) -> dict[str, Any]:
    1948    if tool_name != "record_milestone_validation":
    1949        return args
    1950    milestone = _milestone_validation_needed(job)
    1951    if not milestone or _milestone_validation_call_matches_current(args, milestone):
    1952        return args
    1953    if not _text_matches_pending_milestone_need(_json_value_text(args), milestone):
    1954        return args
    1955    normalized = dict(args)
    1956    normalized["milestone"] = str(milestone.get("title") or args.get("milestone") or "")
    1957    metadata = normalized.get("metadata") if isinstance(normalized.get("metadata"), dict) else {}
    1958    normalized["metadata"] = {
    1959        **metadata,
    1960        "normalized_from_milestone": str(args.get("milestone") or ""),
    1961        "normalized_to_active_gate": True,
    1962    }
    1963    return normalized
    1964
    1965
    1966def _latest_experiment_next_action_context(job: dict[str, Any]) -> dict[str, Any] | None:
    1967    experiments = _metadata_list(job, "experiment_ledger")
    1968    for experiment in reversed(experiments):
    1969        if not isinstance(experiment, dict):
    1970            continue
    1971        status = str(experiment.get("status") or "").strip().lower()
    1972        next_action = str(experiment.get("next_action") or "").strip()
    1973        if not next_action:
    1974            continue
    1975        if status in {"measured", "failed", "blocked"} or experiment.get("metric_value") is not None:
    1976            return {
    1977                "title": experiment.get("title"),
    1978                "status": status,
    1979                "metric_name": experiment.get("metric_name"),
    1980                "metric_value": experiment.get("metric_value"),
    1981                "next_action": next_action,
    1982            }
    1983    return None
    1984
    1985
    1986def _experiment_next_action_requires_delivery(context: dict[str, Any] | None) -> bool:
    1987    if not context:
    1988        return False
    1989    next_action = str(context.get("next_action") or "").lower()
    1990    if not next_action:
    1991        return False
    1992    tokens = set(re.findall(r"[a-z][a-z0-9_-]+", next_action))
    1993    if not tokens & EXPERIMENT_DELIVERY_ACTION_TERMS:
    1994        return False
    1995    return not bool(tokens & EXPERIMENT_INFORMATION_ACTION_TERMS)
    1996
    1997
    1998def _experiment_next_action_failure_context(job: dict[str, Any], recent_steps: list[dict[str, Any]], *, window: int = 8) -> dict[str, Any] | None:
    1999    context = _latest_experiment_next_action_context(job)
    2000    if not _experiment_next_action_requires_delivery(context):
    2001        return None
    2002    latest_experiment_step_no = max(
    2003        (
    2004            _as_int(step.get("step_no"))
    2005            for step in recent_steps
    2006            if step.get("tool_name") == "record_experiment" and step.get("status") == "completed"
    2007        ),
    2008        default=0,
    2009    )
    2010    next_action = str(context.get("next_action") or "") if context else ""
    2011    for step in reversed(_completed_or_failed_recent_steps(recent_steps)[-window:]):
    2012        if step.get("tool_name") != "shell_exec":
    2013            continue
    2014        if latest_experiment_step_no and _as_int(step.get("step_no")) <= latest_experiment_step_no:
    2015            continue
    2016        text = _shell_step_failure_text(step)
    2017        if not text.strip() or not _shell_output_has_missing_command(text):
    2018            continue
    2019        command = _step_command(step)
    2020        if not _shell_command_matches_next_action(command, next_action):
    2021            continue
    2022        return {
    2023            "step_no": step.get("step_no"),
    2024            "command": command,
    2025            "excerpt": text.strip(),
    2026            "missing_commands": _missing_commands_from_shell_output(text),
    2027            "missing_paths": _missing_paths_from_shell_output(text),
    2028            "experiment_next_action": context,
    2029        }
    2030    return None
    2031
    2032
    2033def _shell_command_looks_like_write(command: str) -> bool:
    2034    text = command.strip()
    2035    if not text:
    2036        return False
    2037    if re.match(r"(?is)^curl\b", text):
    2038        download_flags = (
    2039            r"(?:^|\s)(?:-o\s*\S+|-O\b|--output(?:=|\s+)\S+|--remote-name\b|--output-dir(?:=|\s+)\S+)"
    2040        )
    2041        if re.search(download_flags, text):
    2042            return True
    2043    if re.match(r"(?is)^(?:wget|aria2c)\b", text):
    2044        return True
    2045    write_patterns = [
    2046        r"(?<!\d)>>?\s*[^&]",
    2047        r"\b1>>?\s*[^&]",
    2048        r"\btee\b",
    2049        r"\bcat\s+>\b",
    2050        r"\bpython[0-9.]*\b.*\bwrite_text\b",
    2051        r"\bpython[0-9.]*\b.*\bopen\([^)]*,\s*['\"]w",
    2052        r"\bsed\s+-i\b",
    2053    ]
    2054    return any(re.search(pattern, text, flags=re.IGNORECASE | re.DOTALL) for pattern in write_patterns)
    2055
    2056
    2057def _shell_command_looks_read_only(command: str) -> bool:
    2058    text = command.strip()
    2059    if not text:
    2060        return False
    2061    if _shell_command_looks_like_write(text):
    2062        return False
    2063    if READ_ONLY_SHELL_COMMAND_PATTERN.search(text):
    2064        return True
    2065    if re.match(r"(?is)^curl\b", text):
    2066        mutating_flags = r"\b-X\s*(?:POST|PUT|PATCH|DELETE)\b|--request\s+(?:POST|PUT|PATCH|DELETE)\b|(?:^|\s)(?:-d|--data|--form|-F|-T|--upload-file)\b"
    2067        return not bool(re.search(mutating_flags, text))
    2068    return False
    2069
    2070
    2071def _shell_command_supports_experiment_next_action(command: str, context: dict[str, Any] | None) -> bool:
    2072    if not context:
    2073        return False
    2074    text = command.strip()
    2075    if not text or not EXPERIMENT_NEXT_ACTION_VERIFY_SHELL_PATTERN.search(text):
    2076        return False
    2077    next_action = str(context.get("next_action") or "")
    2078    if not next_action.strip():
    2079        return False
    2080    action_tokens = _substantive_next_action_tokens(next_action)
    2081    if not action_tokens:
    2082        return False
    2083    command_tokens = _substantive_next_action_tokens(text)
    2084    return bool(action_tokens & command_tokens)
    2085
    2086
    2087def _shell_command_matches_next_action(command: str, next_action: str) -> bool:
    2088    if not command.strip() or not next_action.strip():
    2089        return False
    2090    action_tokens = _substantive_next_action_tokens(next_action)
    2091    command_tokens = _substantive_next_action_tokens(command)
    2092    return bool(action_tokens & command_tokens)
    2093
    2094
    2095def _substantive_next_action_tokens(text: str) -> set[str]:
    2096    tokens = set()
    2097    for token in re.findall(r"[a-z0-9][a-z0-9_.-]{2,}", text.lower()):
    2098        token = token.strip("._-")
    2099        if len(token) < 3:
    2100            continue
    2101        if token in TEXT_TOKEN_STOPWORDS or token in EXPERIMENT_NEXT_ACTION_VERIFY_STOPWORDS:
    2102            continue
    2103        tokens.add(token)
    2104        for part in re.split(r"[._/-]+", token):
    2105            if len(part) >= 3 and part not in TEXT_TOKEN_STOPWORDS and part not in EXPERIMENT_NEXT_ACTION_VERIFY_STOPWORDS:
    2106                tokens.add(part)
    2107    return tokens
    2108
    2109
    2110def _roadmap_staleness_context(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> dict[str, Any] | None:
    2111    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    2112    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
    2113    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
    2114    if not milestones:
    2115        return None
    2116    if any(step.get("tool_name") in {"record_roadmap", "record_milestone_validation"} for step in recent_steps):
    2117        return None
    2118    if any(
    2119        isinstance(milestone, dict)
    2120        and (
    2121            str(milestone.get("status") or "planned") != "planned"
    2122            or str(milestone.get("validation_status") or "not_started") != "not_started"
    2123        )
    2124        for milestone in milestones
    2125    ):
    2126        return None
    2127    tasks = metadata.get("task_queue") if isinstance(metadata.get("task_queue"), list) else []
    2128    completed_artifacts = [
    2129        step for step in recent_steps
    2130        if step.get("status") == "completed" and step.get("tool_name") == "write_artifact"
    2131    ]
    2132    task_updates = [
    2133        step for step in recent_steps
    2134        if step.get("status") == "completed" and step.get("tool_name") == "record_tasks"
    2135    ]
    2136    if len(completed_artifacts) < 2 and len(task_updates) < 2 and len(tasks) < 8:
    2137        return None
    2138    return {
    2139        "title": roadmap.get("title") or "Roadmap",
    2140        "status": roadmap.get("status") or "planned",
    2141        "milestone_count": len(milestones),
    2142        "task_count": len(tasks),
    2143        "artifact_count": len(completed_artifacts),
    2144        "task_update_count": len(task_updates),
    2145    }
    2146
    2147
    2148def _roadmap_missing_for_broad_job(job: dict[str, Any]) -> bool:
    2149    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    2150    if isinstance(metadata.get("roadmap"), dict):
    2151        return False
    2152    objective = str(job.get("objective") or "")
    2153    tasks = metadata.get("task_queue") if isinstance(metadata.get("task_queue"), list) else []
    2154    if len(tasks) >= 6:
    2155        return True
    2156    words = re.findall(r"[A-Za-z0-9_]+", objective)
    2157    broad_terms = {"build", "create", "develop", "implement", "research", "improve", "optimize", "migrate", "write", "analyze"}
    2158    return len(words) >= 14 and any(term in objective.lower() for term in broad_terms)
    2159
    2160
    2161def _task_queue_exhausted(job: dict[str, Any]) -> bool:
    2162    tasks = _metadata_list(job, "task_queue")
    2163    if not tasks:
    2164        return False
    2165    runnable = {"open", "active"}
    2166    return not any(str(task.get("status") or "open").strip().lower() in runnable for task in tasks)
    2167
    2168
    2169def _task_queue_saturation_context(job: dict[str, Any], args: dict[str, Any]) -> dict[str, Any] | None:
    2170    tasks = _metadata_list(job, "task_queue")
    2171    objective_tasks = [task for task in tasks if not _is_guard_recovery_task(task)]
    2172    open_tasks = [task for task in objective_tasks if str(task.get("status") or "open").strip().lower() in {"open", "active"}]
    2173    incoming = args.get("tasks") if isinstance(args.get("tasks"), list) else []
    2174    if not incoming:
    2175        return None
    2176    existing_keys = {
    2177        _norm_task_key(str(task.get("parent") or ""), str(task.get("title") or ""))
    2178        for task in tasks
    2179    }
    2180    semantic_matches = []
    2181    new_open_titles = []
    2182    new_titles = []
    2183    for task in incoming:
    2184        if not isinstance(task, dict):
    2185            continue
    2186        status = str(task.get("status") or "open").strip().lower().replace(" ", "_")
    2187        title = str(task.get("title") or task.get("name") or "").strip()
    2188        parent = str(task.get("parent") or "")
    2189        key = _norm_task_key(parent, title)
    2190        matched_existing = key in existing_keys
    2191        semantic_match = None
    2192        if not matched_existing and (len(objective_tasks) > TASK_QUEUE_TOTAL_SOFT_LIMIT or len(open_tasks) >= TASK_QUEUE_SATURATION_OPEN_TASKS):
    2193            semantic_match = find_semantic_task_match(
    2194                title=title,
    2195                parent=parent,
    2196                tasks=[existing for existing in tasks if not _is_guard_recovery_task(existing)],
    2197            )
    2198            matched_existing = bool(semantic_match)
    2199        if semantic_match:
    2200            semantic_matches.append({
    2201                "title": title,
    2202                "matched_title": semantic_match.get("title"),
    2203                "score": semantic_match.get("score"),
    2204            })
    2205        if not matched_existing:
    2206            new_titles.append(str(task.get("title") or "").strip())
    2207        if status in {"open", "active"} and not matched_existing:
    2208            new_open_titles.append(str(task.get("title") or "").strip())
    2209    projected_total = len(objective_tasks) + len(new_titles)
    2210    projected_open = len(open_tasks) + len(new_open_titles)
    2211    if projected_total > TASK_QUEUE_TOTAL_SOFT_LIMIT and new_titles:
    2212        return {
    2213            "reason": "total task queue is too large",
    2214            "total_count": len(objective_tasks),
    2215            "projected_total_count": projected_total,
    2216            "total_threshold": TASK_QUEUE_TOTAL_SOFT_LIMIT,
    2217            "open_count": len(open_tasks),
    2218            "open_titles": [
    2219                str(task.get("title") or "").strip()
    2220                for task in open_tasks[:8]
    2221                if str(task.get("title") or "").strip()
    2222            ],
    2223            "new_count": len(new_titles),
    2224            "new_titles": new_titles[:8],
    2225            "semantic_matches": semantic_matches[:8],
    2226            "recovery_task_count": len(tasks) - len(objective_tasks),
    2227        }
    2228    if projected_open < TASK_QUEUE_SATURATION_OPEN_TASKS:
    2229        return None
    2230    if not new_open_titles:
    2231        return None
    2232    return {
    2233        "reason": "too many open tasks",
    2234        "open_count": len(open_tasks),
    2235        "projected_open_count": projected_open,
    2236        "open_threshold": TASK_QUEUE_SATURATION_OPEN_TASKS,
    2237        "total_count": len(objective_tasks),
    2238        "open_titles": [
    2239            str(task.get("title") or "").strip()
    2240            for task in open_tasks[:8]
    2241            if str(task.get("title") or "").strip()
    2242        ],
    2243        "new_open_count": len(new_open_titles),
    2244        "new_open_titles": new_open_titles[:8],
    2245        "semantic_matches": semantic_matches[:8],
    2246        "recovery_task_count": len(tasks) - len(objective_tasks),
    2247    }
    2248
    2249
    2250def _recent_task_queue_saturation_context(recent_steps: list[dict[str, Any]], *, window: int = 6) -> dict[str, Any] | None:
    2251    for step in reversed(recent_steps[-window:]):
    2252        if step.get("tool_name") != "record_tasks" or step.get("status") != "blocked":
    2253            continue
    2254        output = step.get("output") if isinstance(step.get("output"), dict) else {}
    2255        if output.get("error") != "task queue saturated":
    2256            continue
    2257        task_queue = output.get("task_queue") if isinstance(output.get("task_queue"), dict) else {}
    2258        return {
    2259            "step_no": step.get("step_no"),
    2260            "reason": task_queue.get("reason") or "task queue saturated",
    2261            "open_count": task_queue.get("open_count"),
    2262            "total_count": task_queue.get("total_count"),
    2263            "open_titles": task_queue.get("open_titles") if isinstance(task_queue.get("open_titles"), list) else [],
    2264        }
    2265    return None
    2266
    2267
    2268def _record_task_backlog_pressure(
    2269    *,
    2270    db: AgentDB,
    2271    job_id: str,
    2272    step_no: int | str | None,
    2273    task_queue: dict[str, Any],
    2274    source: str,
    2275) -> None:
    2276    if not isinstance(task_queue, dict) or not task_queue:
    2277        return
    2278    pressure = {
    2279        "detected_at": datetime.now(timezone.utc).isoformat(),
    2280        "source": source,
    2281        "latest_step_no": step_no,
    2282        "reason": task_queue.get("reason") or "task queue saturated",
    2283        "open_count": task_queue.get("open_count"),
    2284        "total_count": task_queue.get("total_count"),
    2285        "projected_open_count": task_queue.get("projected_open_count"),
    2286        "projected_total_count": task_queue.get("projected_total_count"),
    2287        "open_titles": task_queue.get("open_titles") if isinstance(task_queue.get("open_titles"), list) else [],
    2288    }
    2289    db.update_job_metadata(job_id, {"task_backlog_pressure": pressure})
    2290    db.append_agent_update(
    2291        job_id,
    2292        (
    2293            "Task backlog pressure is active; next worker turns should execute, complete, block, skip, "
    2294            "or consolidate existing tasks instead of adding new branches."
    2295        ),
    2296        category="blocked",
    2297        metadata={"task_backlog_pressure": pressure},
    2298    )
    2299
    2300
    2301def _clear_stale_task_backlog_pressure(db: AgentDB, job_id: str, job: dict[str, Any]) -> bool:
    2302    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    2303    pressure = metadata.get("task_backlog_pressure")
    2304    if not isinstance(pressure, dict) or not pressure:
    2305        return False
    2306    if _current_task_backlog_pressure_context(job):
    2307        return False
    2308    cleared = dict(pressure)
    2309    cleared["resolved_at"] = datetime.now(timezone.utc).isoformat()
    2310    db.update_job_metadata(job_id, {"task_backlog_pressure": {}})
    2311    db.append_agent_update(
    2312        job_id,
    2313        "Task backlog pressure cleared; the active task queue is back under saturation limits.",
    2314        category="progress",
    2315        metadata={"cleared_task_backlog_pressure": cleared},
    2316    )
    2317    return True
    2318
    2319
    2320def _repeated_task_queue_saturation_context(recent_steps: list[dict[str, Any]], *, window: int = 8, threshold: int = 2) -> dict[str, Any] | None:
    2321    matches = []
    2322    for step in recent_steps[-window:]:
    2323        if step.get("tool_name") != "record_tasks" or step.get("status") != "blocked":
    2324            continue
    2325        output = step.get("output") if isinstance(step.get("output"), dict) else {}
    2326        if output.get("error") == "task queue saturated":
    2327            matches.append(step)
    2328    if len(matches) < threshold:
    2329        return None
    2330    latest = matches[-1]
    2331    output = latest.get("output") if isinstance(latest.get("output"), dict) else {}
    2332    task_queue = output.get("task_queue") if isinstance(output.get("task_queue"), dict) else {}
    2333    return {
    2334        "count": len(matches),
    2335        "latest_step_no": latest.get("step_no"),
    2336        "reason": task_queue.get("reason") or "task queue saturated",
    2337    }
    2338
    2339
    2340def _task_planning_stagnation_context(job: dict[str, Any]) -> dict[str, Any] | None:
    2341    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    2342    streak = _as_int(metadata.get("task_planning_checkpoint_streak"))
    2343    if streak < TASK_PLANNING_STAGNATION_CHECKPOINTS:
    2344        return None
    2345    tasks = _metadata_list(job, "task_queue")
    2346    open_tasks = [
    2347        task
    2348        for task in tasks
    2349        if str(task.get("status") or "open").strip().lower().replace(" ", "_") in {"open", "active"}
    2350    ]
    2351    return {
    2352        "task_only_checkpoints": streak,
    2353        "threshold": TASK_PLANNING_STAGNATION_CHECKPOINTS,
    2354        "total_tasks": len(tasks),
    2355        "open_tasks": len(open_tasks),
    2356    }
    2357
    2358
    2359def _is_guard_recovery_task(task: dict[str, Any]) -> bool:
    2360    metadata = task.get("metadata") if isinstance(task.get("metadata"), dict) else {}
    2361    return bool(metadata.get("guard_recovery")) or str(task.get("title") or "").strip().lower().startswith("resolve guard:")
    2362
    2363
    2364def _record_tasks_adds_new_open_work(args: dict[str, Any], job: dict[str, Any]) -> bool:
    2365    incoming = args.get("tasks") if isinstance(args.get("tasks"), list) else []
    2366    if not incoming:
    2367        incoming = [args]
    2368    tasks = _metadata_list(job, "task_queue")
    2369    existing_keys = {
    2370        _norm_task_key(str(task.get("parent") or ""), str(task.get("title") or ""))
    2371        for task in tasks
    2372    }
    2373    for task in incoming:
    2374        if not isinstance(task, dict):
    2375            continue
    2376        title = str(task.get("title") or task.get("name") or "").strip()
    2377        if not title:
    2378            continue
    2379        status = str(task.get("status") or "open").strip().lower().replace(" ", "_")
    2380        key = _norm_task_key(str(task.get("parent") or ""), title)
    2381        if status in {"open", "active"} and key not in existing_keys:
    2382            return True
    2383    return False
    2384
    2385
    2386def _norm_task_key(parent: str, title: str) -> str:
    2387    return task_key(parent, title)
    2388
    2389
    2390def _parse_tool_result(raw: str) -> dict[str, Any]:
    2391    try:
    2392        parsed = json.loads(raw)
    2393        return parsed if isinstance(parsed, dict) else {"result": parsed}
    2394    except json.JSONDecodeError:
    2395        return {"result": raw}
    2396
    2397
    2398def _load_program_text(config: AppConfig, job_id: str) -> str:
    2399    path = config.runtime.jobs_dir / job_id / "program.md"
    2400    if not path.exists():
    2401        return ""
    2402    return path.read_text(encoding="utf-8", errors="replace")
    2403
    2404
    2405def _browser_warning_context(output: dict[str, Any]) -> dict[str, str] | None:
    2406    data = output.get("data") if isinstance(output.get("data"), dict) else {}
    2407    title = str(data.get("title") or "")
    2408    url = str(data.get("url") or data.get("origin") or output.get("url") or "")
    2409    snapshot = str(output.get("snapshot") or data.get("snapshot") or output.get("data") or "")
    2410    reason = anti_bot_reason(title, url, snapshot)
    2411    if not reason:
    2412        return None
    2413    return {"reason": reason, "url": url, "title": title}
    2414
    2415
    2416def _recent_anti_bot_context(recent_steps: list[dict[str, Any]], *, window: int = 8) -> dict[str, Any] | None:
    2417    for step in reversed(recent_steps[-window:]):
    2418        if step.get("status") != "completed" or step.get("tool_name") not in {"browser_navigate", "browser_snapshot"}:
    2419            continue
    2420        output = step.get("output") if isinstance(step.get("output"), dict) else {}
    2421        warning = _browser_warning_context(output)
    2422        if warning:
    2423            return {**warning, "step_id": step.get("id"), "step_no": step.get("step_no")}
    2424    return None
    2425
    2426
    2427def _artifact_args_acknowledge_block(args: dict[str, Any]) -> bool:
    2428    text = " ".join(str(args.get(key) or "") for key in ("title", "summary", "content")).lower()
    2429    return any(term in text for term in ANTI_BOT_ACK_TERMS)
    2430
    2431
    2432def _same_source_url(left: str, right: str) -> bool:
    2433    if not left or not right:
    2434        return False
    2435    return left.split("#", 1)[0].rstrip("/") == right.split("#", 1)[0].rstrip("/")
    2436
    2437
    2438def _normalized_source_url(value: str) -> str:
    2439    value = str(value or "").strip()
    2440    if not value:
    2441        return ""
    2442    if "://" not in value:
    2443        return f"https://{value}"
    2444    return value
    2445
    2446
    2447def _source_host(value: str) -> str:
    2448    parsed = urlparse(_normalized_source_url(value))
    2449    return parsed.netloc.lower().removeprefix("www.")
    2450
    2451
    2452def _source_matches(left: str, right: str) -> bool:
    2453    if _same_source_url(left, right):
    2454        return True
    2455    left_host, left_path = _source_path_key(left)
    2456    right_host, right_path = _source_path_key(right)
    2457    if not left_host or left_host != right_host:
    2458        return False
    2459    if right_path in {"", "/"} or left_path in {"", "/"}:
    2460        return False
    2461    return left_path == right_path or left_path.startswith(right_path + "/") or right_path.startswith(left_path + "/")
    2462
    2463
    2464def _source_path_key(value: str) -> tuple[str, str]:
    2465    parsed = urlparse(_normalized_source_url(value))
    2466    host = parsed.netloc.lower().removeprefix("www.")
    2467    path = (parsed.path or "/").rstrip("/") or "/"
    2468    return host, path
    2469
    2470
    2471def _shell_source_matches(left: str, right: str) -> bool:
    2472    if _same_source_url(left, right):
    2473        return True
    2474    left_host, left_path = _source_path_key(left)
    2475    right_host, right_path = _source_path_key(right)
    2476    if not left_host or left_host != right_host:
    2477        return False
    2478    if right_path in {"", "/"} or left_path in {"", "/"}:
    2479        return False
    2480    return left_path == right_path or left_path.startswith(right_path + "/") or right_path.startswith(left_path + "/")
    2481
    2482
    2483def _urls_from_text(text: str) -> list[str]:
    2484    urls: list[str] = []
    2485    seen: set[str] = set()
    2486    for match in re.finditer(r"https?://[^\s'\"<>)}\]]+", str(text or "")):
    2487        url = match.group(0).rstrip(".,;:")
    2488        key = url.lower()
    2489        if key in seen:
    2490            continue
    2491        seen.add(key)
    2492        urls.append(url)
    2493    return urls
    2494
    2495
    2496def _source_url_has_path(value: str) -> bool:
    2497    _, path = _source_path_key(value)
    2498    return path not in {"", "/"}
    2499
    2500
    2501def _shell_guard_urls(text: str) -> list[str]:
    2502    urls = _urls_from_text(text)
    2503    if len(urls) <= 1:
    2504        return urls
    2505    path_urls = [url for url in urls if _source_url_has_path(url)]
    2506    return path_urls or urls
    2507
    2508
    2509SHELL_PLACEHOLDER_URL_HOSTS = {
    2510    "domain",
    2511    "endpoint",
    2512    "example",
    2513    "file",
    2514    "host",
    2515    "input",
    2516    "output",
    2517    "path",
    2518    "placeholder",
    2519    "source",
    2520    "target",
    2521    "url",
    2522    "uri",
    2523}
    2524
    2525SHELL_PLACEHOLDER_FIELD_NAMES = (
    2526    "command",
    2527    "domain",
    2528    "endpoint",
    2529    "file",
    2530    "host",
    2531    "input",
    2532    "output",
    2533    "path",
    2534    "source",
    2535    "target",
    2536    "url",
    2537    "uri",
    2538)
    2539
    2540
    2541def _shell_placeholder_context(command: str) -> dict[str, Any] | None:
    2542    command = str(command or "").strip()
    2543    if not command:
    2544        return None
    2545    if "```" in command:
    2546        return {
    2547            "kind": "markdown_code_fence",
    2548            "value": "```",
    2549            "reason": "command contains markdown code fences instead of executable shell only",
    2550        }
    2551    if re.search(r"(?m)^\s*-{3,}\s+\S", command) or re.search(r"(?m)^\s*\d+\.\s+```", command):
    2552        return {
    2553            "kind": "markdown_prose",
    2554            "value": "markdown prose",
    2555            "reason": "command contains copied markdown prose instead of executable shell only",
    2556        }
    2557    for url in _urls_from_text(command):
    2558        parsed = urlparse(url)
    2559        host = (parsed.hostname or "").lower()
    2560        if host in SHELL_PLACEHOLDER_URL_HOSTS:
    2561            return {
    2562                "kind": "placeholder_url",
    2563                "value": url,
    2564                "reason": "URL host looks like an unresolved placeholder field",
    2565            }
    2566    fields = "|".join(re.escape(name) for name in SHELL_PLACEHOLDER_FIELD_NAMES)
    2567    placeholder_patterns = [
    2568        rf"<\s*(?:{fields})(?:[-_ ][A-Za-z0-9]+)?\s*>",
    2569        rf"\{{\{{\s*(?:{fields})(?:[-_ ][A-Za-z0-9]+)?\s*\}}\}}",
    2570        rf"\{{\s*(?:{fields})(?:[-_ ][A-Za-z0-9]+)?\s*\}}",
    2571        r"</?\s*(?:parameter|arguments?|tool_call|function_call)\b[^>]*>",
    2572        r"\b(?:YOUR|REPLACE|TODO|INSERT)_[A-Z0-9_]{3,}\b",
    2573    ]
    2574    for pattern in placeholder_patterns:
    2575        match = re.search(pattern, command, flags=re.IGNORECASE)
    2576        if match:
    2577            return {
    2578                "kind": "placeholder_token",
    2579                "value": match.group(0),
    2580                "reason": "command contains an unresolved placeholder token",
    2581            }
    2582    return None
    2583
    2584
    2585def _shell_syntax_preflight_context(command: str) -> dict[str, Any] | None:
    2586    command = str(command or "").strip()
    2587    if not command:
    2588        return None
    2589    try:
    2590        shlex.split(command, posix=True)
    2591    except ValueError as exc:
    2592        return {
    2593            "kind": "shell_syntax",
    2594            "value": str(exc),
    2595            "reason": "command is not parseable shell syntax; usually an unmatched quote, escape, or partial pasted command",
    2596        }
    2597    return None
    2598
    2599
    2600def _source_failure_family_url(value: str) -> str:
    2601    parsed = urlparse(_normalized_source_url(value))
    2602    if not parsed.scheme or not parsed.netloc:
    2603        return ""
    2604    segments = [segment for segment in (parsed.path or "").split("/") if segment]
    2605    if len(segments) < 2:
    2606        return ""
    2607    last = segments[-1]
    2608    looks_file_like = "." in last
    2609    family_segments = segments[:-1] if looks_file_like else segments
    2610    if len(family_segments) < 2:
    2611        return ""
    2612    return f"{parsed.scheme}://{parsed.netloc}/{'/'.join(family_segments)}"
    2613
    2614
    2615def _known_bad_sources(job: dict[str, Any]) -> list[dict[str, Any]]:
    2616    bad_sources = []
    2617    for source in _metadata_list(job, "source_ledger"):
    2618        if (
    2619            _as_float(source.get("usefulness_score")) < 0.2
    2620            and _as_int(source.get("yield_count")) <= 0
    2621            and (_as_int(source.get("fail_count")) > 0 or source.get("warnings"))
    2622        ):
    2623            bad_sources.append(source)
    2624    return bad_sources
    2625
    2626
    2627def _known_bad_source_for_call(name: str, args: dict[str, Any], job: dict[str, Any]) -> dict[str, Any] | None:
    2628    if name not in {"browser_navigate", "web_extract", "shell_exec"}:
    2629        return None
    2630    bad_sources = _known_bad_sources(job)
    2631    if not bad_sources:
    2632        return None
    2633    urls: list[str] = []
    2634    if name == "browser_navigate":
    2635        urls = [str(args.get("url") or "")]
    2636    elif isinstance(args.get("urls"), list):
    2637        urls = [str(url) for url in args["urls"]]
    2638    elif name == "shell_exec":
    2639        urls = _shell_guard_urls(str(args.get("command") or ""))
    2640    for url in [url for url in urls if url.strip()]:
    2641        for source in bad_sources:
    2642            source_value = str(source.get("source") or "")
    2643            if not source_value:
    2644                continue
    2645            matches = _shell_source_matches(url, source_value) if name == "shell_exec" else _source_matches(url, source_value)
    2646            if matches:
    2647                return source
    2648            if name == "shell_exec":
    2649                source_family = _source_failure_family_url(source_value)
    2650                if source_family and _shell_source_matches(url, source_family):
    2651                    metadata = source.get("metadata") if isinstance(source.get("metadata"), dict) else {}
    2652                    return {
    2653                        **source,
    2654                        "source": source_family,
    2655                        "source_type": "shell_exec_family",
    2656                        "metadata": {**metadata, "source_family": True, "source_family_from": source_value},
    2657                    }
    2658    return None
    2659
    2660
    2661def _tool_signature(name: str, args: dict[str, Any]) -> str:
    2662    return f"{name}:{json.dumps(args, ensure_ascii=False, sort_keys=True)}"
    2663
    2664
    2665def _duplicate_recent_tool_call(
    2666    name: str,
    2667    args: dict[str, Any],
    2668    recent_steps: list[dict[str, Any]],
    2669    *,
    2670    window: int = 24,
    2671) -> dict[str, Any] | None:
    2672    if name in {"browser_snapshot", "defer_job"}:
    2673        return None
    2674    signature = _tool_signature(name, args)
    2675    for step in reversed(recent_steps[-window:]):
    2676        if step.get("status") != "completed" or step.get("tool_name") != name:
    2677            continue
    2678        input_data = step.get("input") or {}
    2679        previous_args = input_data.get("arguments") if isinstance(input_data, dict) else None
    2680        if isinstance(previous_args, dict) and _tool_signature(name, previous_args) == signature:
    2681            return step
    2682    return None
    2683
    2684
    2685def _completed_recent_steps(recent_steps: list[dict[str, Any]]) -> list[dict[str, Any]]:
    2686    return [step for step in recent_steps if step.get("status") == "completed"]
    2687
    2688
    2689def _completed_or_failed_recent_steps(recent_steps: list[dict[str, Any]]) -> list[dict[str, Any]]:
    2690    return [step for step in recent_steps if step.get("status") in {"completed", "failed"}]
    2691
    2692
    2693BROWSER_RUNTIME_UNAVAILABLE_TERMS = (
    2694    "browser runtime unavailable",
    2695    "browser not found",
    2696    "browser executable",
    2697    "chrome not found",
    2698    "could not find chrome",
    2699    "chromium executable",
    2700    "executable doesn't exist",
    2701    "playwright browser cache",
    2702    "puppeteer browser cache",
    2703)
    2704
    2705
    2706SELF_DEFER_TERMS = (
    2707    "next worker turn",
    2708    "next worker step",
    2709    "picked up by next worker",
    2710    "picked up by the next worker",
    2711    "picked up by next turn",
    2712    "picked up by the next turn",
    2713)
    2714
    2715
    2716def _is_browser_tool(name: str | None) -> bool:
    2717    return bool(str(name or "").startswith("browser_"))
    2718
    2719
    2720def _browser_runtime_unavailable_context(
    2721    recent_steps: list[dict[str, Any]],
    2722    *,
    2723    window: int = 512,
    2724) -> dict[str, Any] | None:
    2725    latest_browser_success_no = max(
    2726        (
    2727            int(step.get("step_no") or 0)
    2728            for step in recent_steps[-window:]
    2729            if _is_browser_tool(step.get("tool_name")) and step.get("status") == "completed"
    2730        ),
    2731        default=0,
    2732    )
    2733    for step in reversed(recent_steps[-window:]):
    2734        if not _is_browser_tool(step.get("tool_name")):
    2735            continue
    2736        step_no = int(step.get("step_no") or 0)
    2737        if step_no <= latest_browser_success_no:
    2738            continue
    2739        if step.get("status") not in {"failed", "blocked"}:
    2740            continue
    2741        output = step.get("output") if isinstance(step.get("output"), dict) else {}
    2742        text = " ".join(
    2743            str(part or "")
    2744            for part in (
    2745                step.get("summary"),
    2746                step.get("error"),
    2747                output.get("error"),
    2748                output.get("summary"),
    2749                output.get("stderr"),
    2750                output.get("stdout"),
    2751            )
    2752        ).lower()
    2753        if any(term in text for term in BROWSER_RUNTIME_UNAVAILABLE_TERMS):
    2754            error = str(output.get("error") or step.get("error") or step.get("summary") or "")
    2755            return {
    2756                "step_no": step.get("step_no"),
    2757                "tool": step.get("tool_name"),
    2758                "status": step.get("status"),
    2759                "error": _clip_text(error, 500),
    2760            }
    2761    return None
    2762
    2763
    2764def _self_defer_context(args: dict[str, Any]) -> dict[str, Any] | None:
    2765    reason = str(args.get("reason") or "")
    2766    next_action = str(args.get("next_action") or "")
    2767    text = f"{reason} {next_action}".lower()
    2768    matched = next((term for term in SELF_DEFER_TERMS if term in text), "")
    2769    if not matched and next_action.strip() and not reason.strip():
    2770        matched = "missing wait reason"
    2771    if not matched:
    2772        return None
    2773    return {
    2774        "matched": matched,
    2775        "reason": reason,
    2776        "next_action": next_action,
    2777    }
    2778
    2779
    2780EVIDENCE_GROUNDED_TOOLS = {
    2781    "record_experiment",
    2782    "record_findings",
    2783    "record_lesson",
    2784    "record_memory_graph",
    2785    "record_roadmap",
    2786    "report_update",
    2787    "write_artifact",
    2788}
    2789NARRATIVE_EVIDENCE_GROUNDED_TOOLS = {
    2790    "record_findings",
    2791    "record_lesson",
    2792    "record_memory_graph",
    2793    "record_roadmap",
    2794    "report_update",
    2795    "write_artifact",
    2796}
    2797EVIDENCE_GROUNDING_RESOLUTION_TOOLS = {
    2798    "record_experiment",
    2799    "record_findings",
    2800    "record_lesson",
    2801    "record_memory_graph",
    2802    "record_milestone_validation",
    2803    "record_roadmap",
    2804    "record_source",
    2805    "record_tasks",
    2806    "report_update",
    2807    "write_artifact",
    2808}
    2809EVIDENCE_CHECKPOINT_RESOLUTION_TOOLS = {
    2810    "record_experiment",
    2811    "record_findings",
    2812    "record_lesson",
    2813    "record_milestone_validation",
    2814    "record_roadmap",
    2815    "record_source",
    2816    "record_tasks",
    2817}
    2818EVIDENCE_CHECKPOINT_ACCOUNTING_TOOLS = EVIDENCE_CHECKPOINT_RESOLUTION_TOOLS | {"guard_recovery"}
    2819EVIDENCE_CHECKPOINT_PROMPT_TOOLS = {
    2820    "record_experiment",
    2821    "record_findings",
    2822    "record_lesson",
    2823    "record_source",
    2824}
    2825EVIDENCE_TOKEN_IGNORE = <redacted>
    2826    "acceptance",
    2827    "action",
    2828    "actions",
    2829    "active",
    2830    "agent",
    2831    "artifact",
    2832    "api",
    2833    "baseline",
    2834    "branch",
    2835    "branches",
    2836    "candidate",
    2837    "candidates",
    2838    "cdn",
    2839    "checkpoint",
    2840    "compare",
    2841    "complete",
    2842    "constraint",
    2843    "criteria",
    2844    "current",
    2845    "data",
    2846    "deliverable",
    2847    "direct",
    2848    "done",
    2849    "download",
    2850    "downloadable",
    2851    "downloaded",
    2852    "downloading",
    2853    "downloads",
    2854    "discovered",
    2855    "discovery",
    2856    "environment",
    2857    "existing",
    2858    "evidence",
    2859    "experiment",
    2860    "experiments",
    2861    "feature",
    2862    "features",
    2863    "file",
    2864    "files",
    2865    "format",
    2866    "finding",
    2867    "findings",
    2868    "file-level",
    2869    "html",
    2870    "http",
    2871    "https",
    2872    "inspect",
    2873    "inspection",
    2874    "investigate",
    2875    "investigation",
    2876    "json",
    2877    "goal",
    2878    "gguf",
    2879    "hardware",
    2880    "improve",
    2881    "located",
    2882    "memory",
    2883    "metric",
    2884    "milestone",
    2885    "milestones",
    2886    "model",
    2887    "next",
    2888    "observation",
    2889    "observations",
    2890    "open",
    2891    "oid",
    2892    "output",
    2893    "outputs",
    2894    "plan",
    2895    "planned",
    2896    "pending",
    2897    "priority",
    2898    "progress",
    2899    "parse",
    2900    "parsed",
    2901    "parsing",
    2902    "record",
    2903    "report",
    2904    "rest",
    2905    "research",
    2906    "result",
    2907    "roadmap",
    2908    "runtime",
    2909    "search",
    2910    "server",
    2911    "source",
    2912    "sources",
    2913    "status",
    2914    "sha",
    2915    "sha256",
    2916    "step",
    2917    "steps",
    2918    "task",
    2919    "tasks",
    2920    "test",
    2921    "throughput",
    2922    "tool",
    2923    "tools",
    2924    "false",
    2925    "none",
    2926    "null",
    2927    "true",
    2928    "url",
    2929    "usable",
    2930    "unvalidated",
    2931    "valid",
    2932    "validity",
    2933    "validate",
    2934    "validated",
    2935    "validating",
    2936    "validation",
    2937    "worker",
    2938    "xml",
    2939    "yaml",
    2940    "yml",
    2941    "confirmed",
    2942    "consider",
    2943    "checking",
    2944    "ongoing",
    2945    "proceed",
    2946    "proceeding",
    2947}
    2948EVIDENCE_TOKEN_IGNORE.update({f"p{index}" for index in range(10)})
    2949STALE_CLAIM_TOKEN_IGNORE = <redacted>
    2950    "api",
    2951    "ascii",
    2952    "blocked",
    2953    "broken",
    2954    "cdn",
    2955    "cli",
    2956    "critical",
    2957    "cpu",
    2958    "cuda",
    2959    "discovered",
    2960    "ggml",
    2961    "gguf",
    2962    "gpu",
    2963    "hf_token",
    2964    "html",
    2965    "http",
    2966    "https",
    2967    "incomplete",
    2968    "json",
    2969    "lfs",
    2970    "not_found",
    2971    "oid",
    2972    "onnx",
    2973    "planned",
    2974    "python",
    2975    "python3",
    2976    "ram",
    2977    "rest",
    2978    "severe",
    2979    "sha",
    2980    "sha256",
    2981    "vram",
    2982    "xet",
    2983    "xml",
    2984    "yaml",
    2985    "yml",
    2986}
    2987NEGATIVE_EXISTENCE_MARKERS = (
    2988    "0 files",
    2989    "0 results",
    2990    "cannot access",
    2991    "does not exist",
    2992    "failed to find",
    2993    "has not been",
    2994    "is not installed",
    2995    "missing",
    2996    "no ",
    2997    "no such",
    2998    "none",
    2999    "not available",
    3000    "not detected",
    3001    "not downloaded",
    3002    "not found",
    3003    "not installed",
    3004    "unavailable",
    3005    "was not",
    3006    "without",
    3007)
    3008NEGATIVE_ROLE_CLASSIFICATION_MARKERS = (
    3009    "not a primary",
    3010    "not a required",
    3011    "not a target",
    3012    "not an expected",
    3013    "not suitable as",
    3014    "not suitable for",
    3015    "not the expected",
    3016    "not the needed",
    3017    "not the primary",
    3018    "not the required",
    3019    "not the target",
    3020    "not usable as",
    3021    "not usable for",
    3022    "only support",
    3023    "support file",
    3024    "support files",
    3025)
    3026EVIDENCE_NEGATIVE_LINE_MARKERS = (
    3027    "0 files",
    3028    "0 results",
    3029    "cannot access",
    3030    "denied",
    3031    "does not exist",
    3032    "error",
    3033    "failed",
    3034    "failure",
    3035    "has not been",
    3036    "missing",
    3037    "no such",
    3038    "not available",
    3039    "not detected",
    3040    "not downloaded",
    3041    "not found",
    3042    "not installed",
    3043    "permission",
    3044    "timeout",
    3045    "unavailable",
    3046    "was not",
    3047)
    3048
    3049
    3050def _stale_claim_tokens_from_unsupported(tokens: list[str], *, reference_text: str = "") -> list[str]:
    3051    stale_tokens: list[str] = []
    3052    seen: set[str] = set()
    3053    reference_norm = _normalize_claim_text(reference_text)
    3054    for token in tokens:
    3055        cleaned = str(token or "").strip()
    3056        if not cleaned:
    3057            continue
    3058        key = cleaned.lower()
    3059        if key in seen or key in STALE_CLAIM_TOKEN_IGNORE or key in EVIDENCE_TOKEN_IGNORE:
    3060            continue
    3061        if reference_norm and _normalize_claim_text(cleaned) in reference_norm:
    3062            continue
    3063        if _looks_like_generated_or_file_token(cleaned):
    3064            continue
    3065        if len(cleaned) < 4:
    3066            continue
    3067        distinctive = any(ch.isalpha() for ch in cleaned) and any(ch.isdigit() for ch in cleaned)
    3068        distinctive = distinctive or (cleaned.isupper() and len(cleaned) >= 4)
    3069        if not distinctive:
    3070            continue
    3071        seen.add(key)
    3072        stale_tokens.append(cleaned)
    3073    return stale_tokens
    3074
    3075
    3076def _looks_like_generated_or_file_token(token: str) -> bool:
    3077    lowered = token.lower()
    3078    if lowered.startswith((
    3079        "art_",
    3080        "step_",
    3081        "step-",
    3082        "shell_",
    3083        "shell-",
    3084        "web_",
    3085        "web-",
    3086        "episode-",
    3087        "fact-",
    3088        "source-",
    3089        "quality-",
    3090        "constraint-",
    3091        "baseline-",
    3092        "question-",
    3093        "verified_",
    3094        "verified-",
    3095        "timeout_",
    3096        "timeout-",
    3097    )):
    3098        return True
    3099    if lowered.endswith((".md", ".py", ".json", ".yaml", ".yml", ".gguf", ".txt", ".log")):
    3100        return True
    3101    if lowered.startswith(("python-", "pip", "pip3")):
    3102        return True
    3103    if "_" in lowered and any(ch.isdigit() for ch in lowered) and any(ch.isalpha() for ch in lowered):
    3104        return True
    3105    return False
    3106
    3107
    3108def _normalize_claim_text(text: str) -> str:
    3109    return re.sub(r"[^a-z0-9]+", "", str(text or "").lower())
    3110
    3111
    3112def _evidence_grounding_context(
    3113    job: dict[str, Any],
    3114    recent_steps: list[dict[str, Any]],
    3115    *,
    3116    tool_name: str,
    3117    args: dict[str, Any],
    3118    window: int = 8,
    3119) -> dict[str, Any] | None:
    3120    if tool_name not in EVIDENCE_GROUNDED_TOOLS:
    3121        return None
    3122    full_proposed_text = _json_text(args)
    3123    proposed_text = _evidence_grounding_proposed_text(tool_name, args)
    3124    if len(full_proposed_text.strip()) < 80:
    3125        return None
    3126    cited_steps = _cited_step_numbers(full_proposed_text)
    3127    evidence_text = _recent_evidence_text(job, recent_steps, window=window, step_numbers=cited_steps or None)
    3128    fresh_evidence_text = _recent_evidence_text(
    3129        job,
    3130        recent_steps,
    3131        window=window,
    3132        step_numbers=cited_steps or None,
    3133        include_durable=False,
    3134        include_job_context=False,
    3135    )
    3136    recent_grounding_paths = _candidate_file_paths_from_recent_grounding_blocks(recent_steps, window=window)
    3137    if len(evidence_text.strip()) < 80 and not recent_grounding_paths:
    3138        return None
    3139    job_reference_text = " ".join(str(job.get(key) or "") for key in ("title", "objective", "kind"))
    3140    proposed_tokens = [
    3141        token
    3142        for token in _concrete_evidence_tokens_for_grounding(tool_name, proposed_text)
    3143        if not _grounding_token_in_reference_text(token, job_reference_text)
    3144    ]
    3145    positive_path_conflicts = _positive_path_claim_conflicts_for_grounding(
    3146        tool_name=tool_name,
    3147        proposed_text=proposed_text,
    3148        full_proposed_text=full_proposed_text,
    3149        fresh_evidence_text=fresh_evidence_text,
    3150    )
    3151    if positive_path_conflicts:
    3152        conflict_paths = [item["path"] for item in positive_path_conflicts]
    3153        return {
    3154            "unsupported_tokens": conflict_paths[:12],
    3155            "negative_path_conflicts": positive_path_conflicts[:6],
    3156            "evidence_steps": [
    3157                step.get("step_no")
    3158                for step in _evidence_steps_for_grounding(recent_steps, window=window, step_numbers=cited_steps or None)
    3159            ],
    3160            "cited_steps": sorted(cited_steps),
    3161            "guidance": (
    3162                "The proposed durable record claims a path or executable is present/available, but recent shell "
    3163                "evidence says that same path is missing or inaccessible. Inspect again, record it as missing, "
    3164                "or cite a newer positive check before saving the claim."
    3165            ),
    3166        }
    3167    negative_conflicts = _negative_claim_conflicts_for_grounding(
    3168        tool_name=tool_name,
    3169        proposed_text=proposed_text,
    3170        fresh_evidence_text=fresh_evidence_text,
    3171        tokens=proposed_tokens,
    3172    )
    3173    if negative_conflicts:
    3174        conflict_tokens = [item["token"] for item in negative_conflicts]
    3175        return {
    3176            "unsupported_tokens": conflict_tokens[:12],
    3177            "negative_claim_conflicts": negative_conflicts[:6],
    3178            "evidence_steps": [
    3179                step.get("step_no")
    3180                for step in _evidence_steps_for_grounding(recent_steps, window=window, step_numbers=cited_steps or None)
    3181            ],
    3182            "cited_steps": sorted(cited_steps),
    3183            "guidance": (
    3184                "The proposed durable record negates a concrete item or file pattern that appears in recent positive evidence. "
    3185                "Inspect the evidence again or record uncertainty instead of saving a conflicting claim."
    3186            ),
    3187        }
    3188    missing_paths = _missing_candidate_paths_for_grounding(
    3189        job=job,
    3190        recent_steps=recent_steps,
    3191        recent_grounding_paths=recent_grounding_paths,
    3192        tool_name=tool_name,
    3193        proposed_text=proposed_text,
    3194        full_proposed_text=full_proposed_text,
    3195        fresh_evidence_text=fresh_evidence_text,
    3196    )
    3197    if missing_paths:
    3198        return {
    3199            "unsupported_tokens": missing_paths[:8],
    3200            "missing_candidate_paths": missing_paths[:8],
    3201            "evidence_steps": [
    3202                step.get("step_no")
    3203                for step in _evidence_steps_for_grounding(recent_steps, window=window, step_numbers=cited_steps or None)
    3204            ],
    3205            "cited_steps": sorted(cited_steps),
    3206            "guidance": (
    3207                "Recent evidence contains concrete file/path candidates, but the durable record only summarized them. "
    3208                "Record the exact observed candidate paths, or explicitly state why those paths are not relevant."
    3209            ),
    3210        }
    3211    stale_tokens = _active_stale_claim_token_set(job)
    3212    proposed_stale_tokens = [
    3213        token
    3214        for token in _concrete_evidence_tokens_for_grounding(tool_name, full_proposed_text)
    3215        if not _grounding_token_in_reference_text(token, job_reference_text)
    3216        if token.lower() in stale_tokens
    3217    ]
    3218    if tool_name == "record_lesson" and not proposed_stale_tokens:
    3219        return None
    3220    unsupported_threshold = 1 if cited_steps or proposed_stale_tokens else 3
    3221    candidate_tokens = proposed_stale_tokens if tool_name == "record_lesson" else proposed_tokens + proposed_stale_tokens
    3222    candidate_high_risk = [token for token in candidate_tokens if _high_risk_evidence_token(token)]
    3223    if len(candidate_tokens) < unsupported_threshold and not candidate_high_risk:
    3224        return None
    3225    evidence_lower = evidence_text.lower()
    3226    fresh_evidence_lower = fresh_evidence_text.lower()
    3227    unsupported = []
    3228    for token in candidate_tokens:
    3229        lowered = token.lower()
    3230        if lowered in fresh_evidence_lower:
    3231            continue
    3232        if lowered in evidence_lower and lowered not in stale_tokens:
    3233            continue
    3234        unsupported.append(token)
    3235    unique = []
    3236    seen = set()
    3237    for token in unsupported:
    3238        key = token.lower()
    3239        if key in seen:
    3240            continue
    3241        seen.add(key)
    3242        unique.append(token)
    3243    high_risk_unique = [token for token in unique if _high_risk_evidence_token(token)]
    3244    if len(unique) < unsupported_threshold and not high_risk_unique:
    3245        return None
    3246    return {
    3247        "unsupported_tokens": (high_risk_unique or unique)[:12],
    3248        "evidence_steps": [
    3249            step.get("step_no")
    3250            for step in _evidence_steps_for_grounding(recent_steps, window=window, step_numbers=cited_steps or None)
    3251        ],
    3252        "cited_steps": sorted(cited_steps),
    3253        "guidance": (
    3254            "The proposed durable record contains concrete tokens that are not present in recent evidence. "
    3255            "Use exact observed evidence, inspect the source again, or record uncertainty instead of writing unsupported claims."
    3256        ),
    3257    }
    3258
    3259
    3260def _concrete_evidence_tokens_for_grounding(tool_name: str, text: str) -> list[str]:
    3261    tokens = _concrete_evidence_tokens(text)
    3262    if tool_name not in NARRATIVE_EVIDENCE_GROUNDED_TOOLS:
    3263        return tokens
    3264    return [token for token in tokens if _high_risk_evidence_token(token)]
    3265
    3266
    3267def _grounding_token_in_reference_text(token: str, reference_text: str) -> bool:
    3268    normalized_token = _normalize_claim_text(token)
    3269    if not normalized_token:
    3270        return False
    3271    return normalized_token in _normalize_claim_text(reference_text)
    3272
    3273
    3274def _missing_candidate_paths_for_grounding(
    3275    *,
    3276    job: dict[str, Any],
    3277    recent_steps: list[dict[str, Any]],
    3278    recent_grounding_paths: list[str] | None = None,
    3279    tool_name: str,
    3280    proposed_text: str,
    3281    full_proposed_text: str,
    3282    fresh_evidence_text: str,
    3283) -> list[str]:
    3284    if tool_name not in {"record_findings", "record_experiment", "write_artifact", "report_update"}:
    3285        return []
    3286    proposed_lower = f"{proposed_text}\n{full_proposed_text}".lower()
    3287    if not any(term in proposed_lower for term in ("file", "files", "path", "paths", "candidate", "found", "discovered")):
    3288        return []
    3289    positive_evidence_text = "\n".join(
    3290        line
    3291        for line in str(fresh_evidence_text or "").splitlines()
    3292        if not _evidence_line_is_negative(line.lower())
    3293    )
    3294    evidence_paths = [
    3295        *_extract_candidate_file_paths(positive_evidence_text),
    3296        *(recent_grounding_paths or _candidate_file_paths_from_recent_grounding_blocks(recent_steps)),
    3297    ]
    3298    if not evidence_paths:
    3299        return []
    3300    if any(_path_mentioned_in_text(path, proposed_lower) for path in evidence_paths):
    3301        return []
    3302    distinctive_paths: list[str] = []
    3303    seen: set[str] = set()
    3304    for path in _rank_candidate_file_paths(job, full_proposed_text, evidence_paths):
    3305        key = path.lower()
    3306        if key in seen:
    3307            continue
    3308        seen.add(key)
    3309        distinctive_paths.append(path)
    3310        if len(distinctive_paths) >= 8:
    3311            break
    3312    return distinctive_paths
    3313
    3314
    3315POSITIVE_PATH_CLAIM_MARKERS = (
    3316    "available",
    3317    "exists",
    3318    "found",
    3319    "is at",
    3320    "located",
    3321    "present",
    3322    "ready",
    3323    "succeed",
    3324    "usable",
    3325    "valid",
    3326    "verified",
    3327)
    3328
    3329
    3330def _positive_path_claim_conflicts_for_grounding(
    3331    *,
    3332    tool_name: str,
    3333    proposed_text: str,
    3334    full_proposed_text: str,
    3335    fresh_evidence_text: str,
    3336) -> list[dict[str, str]]:
    3337    if tool_name not in {"record_findings", "record_experiment", "record_source", "record_lesson", "write_artifact", "report_update"}:
    3338        return []
    3339    proposed_combined = f"{proposed_text}\n{full_proposed_text}"
    3340    proposed_lower = proposed_combined.lower()
    3341    conflicts: list[dict[str, str]] = []
    3342    seen: set[str] = set()
    3343    for line in str(fresh_evidence_text or "").splitlines():
    3344        line_lower = line.lower()
    3345        if not _evidence_line_is_negative(line_lower):
    3346            continue
    3347        paths = [
    3348            *_extract_candidate_file_paths(line),
    3349            *_extract_candidate_executable_paths(line),
    3350        ]
    3351        for path in paths:
    3352            path = str(path or "").strip()
    3353            if not path:
    3354                continue
    3355            key = path.lower()
    3356            if key in seen:
    3357                continue
    3358            if key not in proposed_lower:
    3359                continue
    3360            if not _path_near_positive_claim(proposed_combined, path):
    3361                continue
    3362            seen.add(key)
    3363            conflicts.append({
    3364                "path": path,
    3365                "evidence": _clip_text(line.strip(), 220),
    3366                "claim": _clip_text(_excerpt_around(proposed_combined, path, window=96), 220),
    3367            })
    3368            if len(conflicts) >= 8:
    3369                return conflicts
    3370    return conflicts
    3371
    3372
    3373def _path_near_positive_claim(text: str, path: str, *, window: int = 96) -> bool:
    3374    for excerpt in _excerpts_around_all(text, path, window=window):
    3375        excerpt_lower = excerpt.lower()
    3376        if _evidence_line_is_negative(excerpt_lower):
    3377            continue
    3378        if any(marker in excerpt_lower for marker in POSITIVE_PATH_CLAIM_MARKERS):
    3379            return True
    3380    return False
    3381
    3382
    3383def _excerpt_around(text: str, needle: str, *, window: int = 80) -> str:
    3384    excerpts = _excerpts_around_all(text, needle, window=window, max_matches=1)
    3385    return excerpts[0] if excerpts else ""
    3386
    3387
    3388def _excerpts_around_all(text: str, needle: str, *, window: int = 80, max_matches: int = 8) -> list[str]:
    3389    source = str(text or "")
    3390    needle_text = str(needle or "")
    3391    if not source or not needle_text:
    3392        return []
    3393    source_lower = source.lower()
    3394    needle_lower = needle_text.lower()
    3395    excerpts: list[str] = []
    3396    index = 0
    3397    while len(excerpts) < max_matches:
    3398        found = source_lower.find(needle_lower, index)
    3399        if found < 0:
    3400            break
    3401        start = max(0, found - window)
    3402        end = min(len(source), found + len(needle_text) + window)
    3403        excerpts.append(source[start:end])
    3404        index = found + max(1, len(needle_text))
    3405    return excerpts
    3406
    3407
    3408def _path_mentioned_in_text(path: str, text_lower: str) -> bool:
    3409    path_lower = path.lower()
    3410    if path_lower in text_lower:
    3411        return True
    3412    name = Path(path).name.lower()
    3413    return bool(name and name in text_lower)
    3414
    3415
    3416def _refresh_contradicted_negative_claims(
    3417    db: AgentDB,
    3418    job_id: str,
    3419    job: dict[str, Any],
    3420    recent_steps: list[dict[str, Any]],
    3421) -> int:
    3422    fresh_evidence_text = _recent_evidence_text(
    3423        job,
    3424        recent_steps,
    3425        window=8,
    3426        include_durable=False,
    3427        include_job_context=False,
    3428    )
    3429    if len(fresh_evidence_text.strip()) < 80:
    3430        return 0
    3431    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    3432    existing = metadata.get("stale_negative_records") if isinstance(metadata.get("stale_negative_records"), list) else []
    3433    seen = {
    3434        (
    3435            str(item.get("kind") or ""),
    3436            str(item.get("record_id") or ""),
    3437            str(item.get("token") or "").lower(),
    3438        )
    3439        for item in existing
    3440        if isinstance(item, dict)
    3441    }
    3442    now = datetime.now(timezone.utc).isoformat()
    3443    new_records: list[dict[str, Any]] = []
    3444    for kind, records in (
    3445        ("finding", _metadata_list(job, "finding_ledger")[-80:]),
    3446        ("lesson", _metadata_list(job, "lessons")[-80:]),
    3447        (
    3448            "memory_node",
    3449            (
    3450                metadata.get("memory_graph", {}).get("nodes", [])
    3451                if isinstance(metadata.get("memory_graph"), dict)
    3452                and isinstance(metadata.get("memory_graph", {}).get("nodes"), list)
    3453                else []
    3454            ),
    3455        ),
    3456    ):
    3457        for record in records:
    3458            if not isinstance(record, dict):
    3459                continue
    3460            record_text = _negative_record_text(kind, record)
    3461            if not record_text:
    3462                continue
    3463            conflicts = _negative_claim_conflicts_for_grounding(
    3464                tool_name="record_findings",
    3465                proposed_text=record_text,
    3466                fresh_evidence_text=fresh_evidence_text,
    3467                tokens=_concrete_evidence_tokens(record_text),
    3468            )
    3469            if not conflicts:
    3470                continue
    3471            record_id = _negative_record_id(kind, record)
    3472            for conflict in conflicts[:4]:
    3473                token = str(conflict.get("token") or "")
    3474                key = (kind, record_id, token.lower())
    3475                if key in seen:
    3476                    continue
    3477                seen.add(key)
    3478                new_records.append({
    3479                    "kind": kind,
    3480                    "record_id": record_id,
    3481                    "title": _negative_record_title(kind, record),
    3482                    "token": token,
    3483                    "evidence": conflict.get("evidence") or "",
    3484                    "observed_at": now,
    3485                })
    3486    if not new_records:
    3487        return 0
    3488    db.update_job_metadata(job_id, {"stale_negative_records": (existing + new_records)[-120:]})
    3489    db.append_agent_update(
    3490        job_id,
    3491        f"Suppressed {len(new_records)} contradicted negative durable claim(s) after fresh evidence.",
    3492        category="memory",
    3493        metadata={"stale_negative_records": new_records[:12]},
    3494    )
    3495    return len(new_records)
    3496
    3497
    3498def _negative_record_text(kind: str, record: dict[str, Any]) -> str:
    3499    if kind == "lesson":
    3500        return str(record.get("lesson") or "")
    3501    if kind == "memory_node":
    3502        return " ".join(
    3503            str(record.get(key) or "")
    3504            for key in ("key", "title", "kind", "status", "summary")
    3505        )
    3506    return " ".join(
    3507        str(record.get(key) or "")
    3508        for key in ("name", "category", "reason", "status", "source_url", "url")
    3509    )
    3510
    3511
    3512def _negative_record_id(kind: str, record: dict[str, Any]) -> str:
    3513    for key in ("key", "event_id", "id"):
    3514        value = str(record.get(key) or "").strip()
    3515        if value:
    3516            return value
    3517    return _normalize_claim_text(f"{kind}:{_negative_record_title(kind, record)}")[:120]
    3518
    3519
    3520def _negative_record_title(kind: str, record: dict[str, Any]) -> str:
    3521    if kind == "lesson":
    3522        return _clip_text(str(record.get("lesson") or "lesson"), 120)
    3523    return str(record.get("name") or record.get("title") or "finding")
    3524
    3525
    3526def _negative_claim_conflicts_for_grounding(
    3527    *,
    3528    tool_name: str,
    3529    proposed_text: str,
    3530    fresh_evidence_text: str,
    3531    tokens: list[str],
    3532) -> list[dict[str, str]]:
    3533    if tool_name not in EVIDENCE_GROUNDED_TOOLS:
    3534        return []
    3535    proposed_lower = proposed_text.lower()
    3536    if not any(marker in proposed_lower for marker in NEGATIVE_EXISTENCE_MARKERS):
    3537        return []
    3538    evidence_lines = [line.strip() for line in fresh_evidence_text.splitlines() if line.strip()]
    3539    if not evidence_lines:
    3540        return []
    3541    candidates = tokens + _file_pattern_tokens_for_grounding(proposed_text)
    3542    conflicts: list[dict[str, str]] = []
    3543    seen: set[str] = set()
    3544    for token in candidates:
    3545        key = token.lower()
    3546        if key in seen:
    3547            continue
    3548        seen.add(key)
    3549        if not token.startswith(".") and "/" not in token and not _high_risk_evidence_token(token):
    3550            continue
    3551        if not _token_near_negative_claim(proposed_text, token):
    3552            continue
    3553        positive_line = _positive_evidence_line_for_token(evidence_lines, token)
    3554        if not positive_line:
    3555            continue
    3556        conflicts.append({"token": token, "evidence": _clip_text(positive_line, 220)})
    3557    return conflicts
    3558
    3559
    3560def _file_pattern_tokens_for_grounding(text: str) -> list[str]:
    3561    tokens: list[str] = []
    3562    seen: set[str] = set()
    3563    for match in re.finditer(r"(?<![A-Za-z0-9])(?:\*\.)?\.?([A-Za-z0-9][A-Za-z0-9_-]{1,12})(?![A-Za-z0-9_-])", text or ""):
    3564        raw = match.group(0).strip("'\"`")
    3565        if not raw:
    3566            continue
    3567        if not raw.startswith((".", "*.")):
    3568            continue
    3569        if "." not in raw and not raw.startswith("*."):
    3570            continue
    3571        if raw.startswith(".") and not raw.startswith("*."):
    3572            previous_char = text[match.start() - 1] if match.start() > 0 else ""
    3573            next_char = text[match.end()] if match.end() < len(text) else ""
    3574            if previous_char == "/" or next_char == "/":
    3575                continue
    3576        ext = "." + match.group(1).lower().lstrip(".")
    3577        if ext in {".app", ".co", ".com", ".dev", ".edu", ".gov", ".io", ".net", ".org", ".www", ".http", ".https"}:
    3578            continue
    3579        if ext in seen:
    3580            continue
    3581        seen.add(ext)
    3582        tokens.append(ext)
    3583    return tokens
    3584
    3585
    3586def _token_near_negative_claim(text: str, token: str, *, window: int = 64) -> bool:
    3587    text_lower = text.lower()
    3588    token_lower = token.lower()
    3589    start = 0
    3590    while True:
    3591        index = text_lower.find(token_lower, start)
    3592        if index < 0:
    3593            return False
    3594        nearby = text_lower[max(0, index - window): index + len(token_lower) + window]
    3595        if any(marker in nearby for marker in NEGATIVE_EXISTENCE_MARKERS):
    3596            if _nearby_negative_is_role_classification(nearby):
    3597                start = index + len(token_lower)
    3598                continue
    3599            if _nearby_negative_is_positive_validation(nearby):
    3600                start = index + len(token_lower)
    3601                continue
    3602            return True
    3603        start = index + len(token_lower)
    3604
    3605
    3606def _nearby_negative_is_role_classification(text: str) -> bool:
    3607    return any(marker in text for marker in NEGATIVE_ROLE_CLASSIFICATION_MARKERS)
    3608
    3609
    3610def _nearby_negative_is_positive_validation(text: str) -> bool:
    3611    return bool(re.search(r"\bnot\s+(?:a|an|the)?\s*(?:[\w.-]+\s+){0,5}(?:stub|placeholder|empty file)\b", text))
    3612
    3613
    3614def _positive_evidence_line_for_token(lines: list[str], token: str) -> str:
    3615    token_lower = token.lower()
    3616    for line in lines:
    3617        line_lower = line.lower()
    3618        if token_lower not in line_lower:
    3619            continue
    3620        if _evidence_line_is_negative(line_lower):
    3621            continue
    3622        return line
    3623    return ""
    3624
    3625
    3626def _evidence_line_is_negative(line_lower: str) -> bool:
    3627    if any(marker in line_lower for marker in EVIDENCE_NEGATIVE_LINE_MARKERS):
    3628        return True
    3629    return line_lower.startswith("no ") or " no " in line_lower or line_lower.startswith("zero ") or " zero " in line_lower
    3630
    3631
    3632def _evidence_grounding_proposed_text(tool_name: str, args: dict[str, Any]) -> str:
    3633    if tool_name == "record_experiment":
    3634        return "\n".join(
    3635            _json_value_text(args.get(key))
    3636            for key in (
    3637                "action",
    3638                "baseline",
    3639                "command",
    3640                "config",
    3641                "decision",
    3642                "environment",
    3643                "evidence",
    3644                "evidence_artifact",
    3645                "metric_name",
    3646                "metric_unit",
    3647                "metric_value",
    3648                "result",
    3649                "status",
    3650            )
    3651            if args.get(key) is not None
    3652        )
    3653    if tool_name != "record_memory_graph":
    3654        return _json_value_text(args)
    3655    parts: list[str] = []
    3656    nodes = args.get("nodes") if isinstance(args.get("nodes"), list) else []
    3657    for node in nodes:
    3658        if not isinstance(node, dict):
    3659            continue
    3660        for key in ("title", "summary", "tags", "metadata"):
    3661            value = node.get(key)
    3662            if value:
    3663                parts.append(_json_text(value))
    3664    edges = args.get("edges") if isinstance(args.get("edges"), list) else []
    3665    for edge in edges:
    3666        if not isinstance(edge, dict):
    3667            continue
    3668        for key in ("evidence_refs", "metadata"):
    3669            value = edge.get(key)
    3670            if value:
    3671                parts.append(_json_text(value))
    3672    return "\n".join(parts)
    3673
    3674
    3675def _json_text(value: Any) -> str:
    3676    try:
    3677        return json.dumps(value, ensure_ascii=False, sort_keys=True)
    3678    except TypeError:
    3679        return str(value)
    3680
    3681
    3682def _json_value_text(value: Any) -> str:
    3683    if isinstance(value, dict):
    3684        return "\n".join(_json_value_text(item) for item in value.values())
    3685    if isinstance(value, list):
    3686        return "\n".join(_json_value_text(item) for item in value)
    3687    return str(value or "")
    3688
    3689
    3690def _cited_step_numbers(text: str) -> set[int]:
    3691    numbers = set()
    3692    patterns = [
    3693        r"(?i)\bsteps?\s*(?:#|-)?\s*(\d+)\b",
    3694        r"(?i)\bstep[_-](\d+)\b",
    3695        r"(?i)\bshell_exec[_\s-]*step[_\s#-]*(\d+)\b",
    3696        r"(?i)\btool[_\s-]*step[_\s#-]*(\d+)\b",
    3697    ]
    3698    for pattern in patterns:
    3699        for match in re.finditer(pattern, text):
    3700            raw = match.group(1)
    3701            try:
    3702                value = int(raw)
    3703            except (TypeError, ValueError):
    3704                continue
    3705            if value > 0:
    3706                numbers.add(value)
    3707    return numbers
    3708
    3709
    3710def _evidence_steps_for_grounding(
    3711    recent_steps: list[dict[str, Any]],
    3712    *,
    3713    window: int,
    3714    step_numbers: set[int] | None = None,
    3715) -> list[dict[str, Any]]:
    3716    completed = _completed_recent_steps(recent_steps)
    3717    if step_numbers:
    3718        steps = [step for step in completed if int(step.get("step_no") or 0) in step_numbers]
    3719    else:
    3720        steps = completed[-window:]
    3721    evidence_steps = [
    3722        step
    3723        for step in steps
    3724        if step.get("tool_name") in {"browser_snapshot", "shell_exec", "web_extract", "web_search", "read_artifact"}
    3725    ]
    3726    if evidence_steps or not step_numbers:
    3727        return evidence_steps
    3728    return [
    3729        step
    3730        for step in completed[-window:]
    3731        if step.get("tool_name") in {"browser_snapshot", "shell_exec", "web_extract", "web_search", "read_artifact"}
    3732    ]
    3733
    3734
    3735def _recent_evidence_text(
    3736    job: dict[str, Any],
    3737    recent_steps: list[dict[str, Any]],
    3738    *,
    3739    window: int,
    3740    step_numbers: set[int] | None = None,
    3741    include_durable: bool = True,
    3742    include_job_context: bool = True,
    3743) -> str:
    3744    parts: list[str] = []
    3745    if include_job_context:
    3746        parts.extend([str(job.get("title") or ""), str(job.get("objective") or ""), str(job.get("kind") or "")])
    3747    durable_text = _durable_records_for_grounding(job) if include_durable else ""
    3748    if include_durable and durable_text:
    3749        parts.append(durable_text)
    3750    for step in _evidence_steps_for_grounding(recent_steps, window=window, step_numbers=step_numbers):
    3751        parts.append(str(step.get("summary") or ""))
    3752        input_data = step.get("input") if isinstance(step.get("input"), dict) else {}
    3753        if input_data:
    3754            parts.append(_json_text(input_data))
    3755        output = step.get("output") if isinstance(step.get("output"), dict) else {}
    3756        if not output:
    3757            continue
    3758        for key in ("stdout", "stderr", "text", "content", "excerpt", "query", "command"):
    3759            if output.get(key):
    3760                parts.append(str(output.get(key)))
    3761        pages = output.get("pages") if isinstance(output.get("pages"), list) else []
    3762        for page in pages[:6]:
    3763            if isinstance(page, dict):
    3764                parts.append(_json_text({key: page.get(key) for key in ("url", "title", "text", "error", "source_warning")}))
    3765        results = output.get("results") if isinstance(output.get("results"), list) else []
    3766        for item in results[:8]:
    3767            if isinstance(item, dict):
    3768                parts.append(_json_text({key: item.get(key) for key in ("url", "title", "snippet")}))
    3769    return "\n".join(parts)
    3770
    3771
    3772def _active_stale_claim_token_set(job: dict[str, Any]) -> set[str]:
    3773    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    3774    raw_tokens = metadata.get("unsupported_claim_tokens") if isinstance(metadata.get("unsupported_claim_tokens"), list) else []
    3775    filtered = _stale_claim_tokens_from_unsupported(
    3776        [str(token) for token in raw_tokens],
    3777        reference_text=" ".join(str(job.get(key) or "") for key in ("title", "objective", "kind")),
    3778    )
    3779    return {str(token).strip().lower() for token in filtered if str(token).strip()}
    3780
    3781
    3782def _durable_records_for_grounding(job: dict[str, Any]) -> str:
    3783    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    3784    parts: list[str] = []
    3785    for finding in _metadata_list(job, "finding_ledger")[-20:]:
    3786        parts.append(_json_text({
    3787            "finding": finding.get("name") or finding.get("title"),
    3788            "category": finding.get("category"),
    3789            "reason": finding.get("reason") or finding.get("summary"),
    3790            "location": finding.get("location"),
    3791            "status": finding.get("status"),
    3792            "evidence_artifact": finding.get("evidence_artifact"),
    3793            "url": finding.get("url"),
    3794            "metadata": finding.get("metadata") if isinstance(finding.get("metadata"), dict) else {},
    3795        }))
    3796    for experiment in _metadata_list(job, "experiment_ledger")[-12:]:
    3797        parts.append(_json_text({
    3798            "experiment": experiment.get("title") or experiment.get("name"),
    3799            "hypothesis": experiment.get("hypothesis"),
    3800            "status": experiment.get("status"),
    3801            "metric_name": experiment.get("metric_name"),
    3802            "metric_value": experiment.get("metric_value"),
    3803            "metric_unit": experiment.get("metric_unit"),
    3804            "result": experiment.get("result"),
    3805            "next_action": experiment.get("next_action"),
    3806            "config": experiment.get("config") if isinstance(experiment.get("config"), dict) else {},
    3807        }))
    3808    for source in _metadata_list(job, "source_ledger")[-12:]:
    3809        parts.append(_json_text({
    3810            "source": source.get("source") or source.get("url"),
    3811            "source_type": source.get("source_type"),
    3812            "outcome": source.get("outcome"),
    3813            "score": source.get("score"),
    3814        }))
    3815    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
    3816    if roadmap:
    3817        parts.append(_json_text({
    3818            "roadmap": roadmap.get("title"),
    3819            "objective": roadmap.get("objective"),
    3820            "current_milestone": roadmap.get("current_milestone"),
    3821            "validation_contract": roadmap.get("validation_contract"),
    3822        }))
    3823    graph = metadata.get("memory_graph") if isinstance(metadata.get("memory_graph"), dict) else {}
    3824    nodes = graph.get("nodes") if isinstance(graph.get("nodes"), list) else []
    3825    for node in [item for item in nodes if isinstance(item, dict)][-20:]:
    3826        parts.append(_json_text({
    3827            "memory_node": node.get("key"),
    3828            "kind": node.get("kind"),
    3829            "title": node.get("title"),
    3830            "summary": node.get("summary"),
    3831        }))
    3832    return "\n".join(parts)
    3833
    3834
    3835def _concrete_evidence_tokens(text: str) -> list[str]:
    3836    text = text.replace("\\n", "\n").replace("\\t", "\t")
    3837    tokens: list[str] = []
    3838    seen_numeric: set[str] = set()
    3839    for raw in re.findall(
    3840        r"(?i)\b\d+(?:\.\d+)?\s*(?:[KMGTPE]i?B|[KMGTPE]|bytes?|tok/s|t/s|tokens/sec|tokens/s|ms|sec|secs|seconds?|minutes?|hours?|%)\b",
    3841        text,
    3842    ):
    3843        token = re.sub(r"\s+", "", raw.strip())
    3844        key = token.lower()
    3845        if key in seen_numeric:
    3846            continue
    3847        seen_numeric.add(key)
    3848        tokens.append(token)
    3849    for raw in re.findall(r"\b[A-Za-z][A-Za-z0-9_.+-]{1,}\b", text):
    3850        token = raw.strip("._+-")
    3851        if not token:
    3852            continue
    3853        lowered = token.lower()
    3854        if lowered in EVIDENCE_TOKEN_IGNORE:
    3855            continue
    3856        if re.match(r"^[a-z]\d+$", token):
    3857            continue
    3858        if _looks_like_generated_evidence_token(token):
    3859            continue
    3860        if lowered.startswith("art_"):
    3861            continue
    3862        if lowered.startswith("step_"):
    3863            continue
    3864        if lowered.endswith("_output") or lowered.endswith("_stdout") or lowered.endswith("_stderr"):
    3865            continue
    3866        if token.isupper() and len(token) >= 3:
    3867            tokens.append(token)
    3868            continue
    3869        if any(ch.isdigit() for ch in token) and any(ch.isalpha() for ch in token):
    3870            tokens.append(token)
    3871            continue
    3872        if token[:1].isupper() and token[1:].islower() and len(token) >= 4:
    3873            tokens.append(token)
    3874            continue
    3875    return tokens
    3876
    3877
    3878def _high_risk_evidence_token(token: str) -> bool:
    3879    lowered = token.lower()
    3880    if not token or lowered in EVIDENCE_TOKEN_IGNORE:
    3881        return False
    3882    if _looks_like_generated_evidence_token(token):
    3883        return False
    3884    if lowered.startswith(("art_", "step_")):
    3885        return False
    3886    if lowered.endswith((".md", ".py", ".json", ".yaml", ".yml", ".gguf", ".txt", ".log")):
    3887        return True
    3888    if any(ch.isdigit() for ch in token) and any(ch.isalpha() for ch in token):
    3889        return True
    3890    if token.isupper() and len(token) >= 3:
    3891        return True
    3892    return False
    3893
    3894
    3895def _looks_like_generated_evidence_token(token: str) -> bool:
    3896    lowered = token.lower().strip()
    3897    if re.match(
    3898        r"^(?:art|step|shell|web|episode|fact|source|quality|constraint|baseline|question|verified|timeout)[_-]\d+[a-z]*$",
    3899        lowered,
    3900    ):
    3901        return True
    3902    return bool(re.match(r"^(?:shell|web|browser|tool)[a-z0-9_-]*[_-]step[_-]?\d+[a-z]*$", lowered))
    3903
    3904
    3905def _step_has_evidence(step: dict[str, Any]) -> bool:
    3906    tool_name = step.get("tool_name")
    3907    output = step.get("output") if isinstance(step.get("output"), dict) else {}
    3908    if tool_name == "web_extract":
    3909        pages = output.get("pages") if isinstance(output.get("pages"), list) else []
    3910        for page in pages:
    3911            if page.get("error"):
    3912                continue
    3913            if str(page.get("text") or "").strip():
    3914                return True
    3915    if tool_name in {"browser_navigate", "browser_snapshot"}:
    3916        data = output.get("data") if isinstance(output.get("data"), dict) else {}
    3917        snapshot = str(output.get("snapshot") or data.get("snapshot") or "")
    3918        if anti_bot_reason(str(data.get("title") or ""), str(data.get("url") or data.get("origin") or ""), snapshot):
    3919            return False
    3920        return len(snapshot.strip()) >= 500
    3921    if tool_name == "shell_exec":
    3922        text = "\n".join(str(output.get(key) or "") for key in ("stdout", "stderr"))
    3923        return len(text.strip()) >= 1000
    3924    return False
    3925
    3926
    3927def _unpersisted_evidence_step(recent_steps: list[dict[str, Any]]) -> dict[str, Any] | None:
    3928    for step in reversed(recent_steps):
    3929        if step.get("status") not in {"completed", "blocked"}:
    3930            continue
    3931        output = step.get("output") if isinstance(step.get("output"), dict) else {}
    3932        if step.get("tool_name") == "write_artifact":
    3933            return None
    3934        if isinstance(output.get("auto_checkpoint"), dict):
    3935            return None
    3936        if step.get("status") == "completed" and _step_has_evidence(step):
    3937            return step
    3938    return None
    3939
    3940
    3941def _evidence_checkpoint_accounting_for_prompt(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> str:
    3942    context = _auto_checkpoint_accounting_context(job, recent_steps)
    3943    if not context:
    3944        return "None."
    3945    read_text = "already read" if context.get("checkpoint_read") else "not read yet"
    3946    next_action = (
    3947        "Next use record_findings, record_source, record_experiment, record_tasks, record_roadmap, "
    3948        "record_milestone_validation, or record_lesson to account for it. Do not read the checkpoint again. "
    3949        if context.get("checkpoint_read")
    3950        else "Next either read that checkpoint artifact, or use record_findings, record_source, record_experiment, "
    3951        "record_tasks, record_roadmap, record_milestone_validation, or record_lesson to account for it. "
    3952    )
    3953    return (
    3954        "An auto-saved evidence checkpoint is waiting for durable accounting. "
    3955        f"artifact={context.get('artifact_id') or '?'} title={context.get('title') or ''} "
    3956        f"evidence_step={context.get('evidence_step_no') or context.get('evidence_step') or '?'} "
    3957        f"blocked_tool={context.get('blocked_tool') or ''} status={read_text}. "
    3958        f"{next_action}"
    3959        "Do not continue shell, search, file, artifact, report, or other branch work until this is resolved."
    3960    )
    3961
    3962
    3963def _pending_evidence_checkpoint(job: dict[str, Any]) -> dict[str, Any] | None:
    3964    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    3965    checkpoint = metadata.get("pending_evidence_checkpoint")
    3966    if isinstance(checkpoint, dict) and checkpoint and not checkpoint.get("resolved_at"):
    3967        return checkpoint
    3968    return None
    3969
    3970
    3971def _step_created_auto_checkpoint(step: dict[str, Any]) -> dict[str, Any] | None:
    3972    output = step.get("output") if isinstance(step.get("output"), dict) else {}
    3973    checkpoint = output.get("auto_checkpoint")
    3974    if not isinstance(checkpoint, dict):
    3975        return None
    3976    if not checkpoint.get("artifact_id"):
    3977        return None
    3978    # Only auto-persisted checkpoints have a stored path. Guard-context payloads use a
    3979    # different key so they cannot reset the read/accounting state.
    3980    if not checkpoint.get("path"):
    3981        return None
    3982    return checkpoint
    3983
    3984
    3985def _auto_checkpoint_accounting_context(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> dict[str, Any] | None:
    3986    pending = _pending_evidence_checkpoint(job)
    3987    if pending:
    3988        return {
    3989            "artifact_id": str(pending.get("artifact_id") or ""),
    3990            "title": str(pending.get("title") or ""),
    3991            "checkpoint_step_no": pending.get("checkpoint_step_no"),
    3992            "evidence_step": pending.get("evidence_step"),
    3993            "evidence_step_no": pending.get("evidence_step_no"),
    3994            "blocked_tool": pending.get("blocked_tool"),
    3995            "checkpoint_read": bool(pending.get("read_at")),
    3996            "read_at": pending.get("read_at"),
    3997            "created_at": pending.get("created_at"),
    3998            "source": "job_metadata",
    3999        }
    4000    checkpoint_step = None
    4001    checkpoint = None
    4002    for step in reversed(recent_steps):
    4003        created = _step_created_auto_checkpoint(step)
    4004        if created:
    4005            checkpoint_step = step
    4006            checkpoint = created
    4007            break
    4008    if not checkpoint_step or not checkpoint:
    4009        return None
    4010    checkpoint_step_no = int(checkpoint_step.get("step_no") or 0)
    4011    tail = [step for step in recent_steps if int(step.get("step_no") or 0) > checkpoint_step_no]
    4012    if any(step.get("tool_name") in EVIDENCE_CHECKPOINT_ACCOUNTING_TOOLS for step in tail if step.get("status") == "completed"):
    4013        return None
    4014    artifact_id = str(checkpoint.get("artifact_id") or "")
    4015    artifact_title = str(checkpoint.get("title") or "")
    4016    checkpoint_read = any(
    4017        step.get("tool_name") == "read_artifact"
    4018        and step.get("status") == "completed"
    4019        and _read_artifact_args_match_checkpoint(step, artifact_id=artifact_id, artifact_title=artifact_title)
    4020        for step in tail
    4021    )
    4022    return {
    4023        "artifact_id": artifact_id,
    4024        "title": artifact_title,
    4025        "checkpoint_step_no": checkpoint_step.get("step_no"),
    4026        "evidence_step": checkpoint.get("evidence_step"),
    4027        "blocked_tool": checkpoint.get("blocked_tool"),
    4028        "checkpoint_read": checkpoint_read,
    4029    }
    4030
    4031
    4032def _evidence_checkpoint_blocks_tool(name: str, args: dict[str, Any], context: dict[str, Any] | None) -> bool:
    4033    if not context:
    4034        return False
    4035    if name in EVIDENCE_CHECKPOINT_RESOLUTION_TOOLS or name == "acknowledge_operator_context":
    4036        return False
    4037    if (
    4038        name == "read_artifact"
    4039        and not context.get("checkpoint_read")
    4040        and _read_artifact_call_matches_checkpoint(
    4041            args,
    4042            artifact_id=str(context.get("artifact_id") or ""),
    4043            artifact_title=str(context.get("title") or ""),
    4044        )
    4045    ):
    4046        return False
    4047    return True
    4048
    4049
    4050def _evidence_checkpoint_block_guidance(context: dict[str, Any]) -> str:
    4051    tools = (
    4052        "record_findings, record_source, record_experiment, record_tasks, "
    4053        "record_roadmap, record_milestone_validation, or record_lesson"
    4054    )
    4055    if context.get("checkpoint_read"):
    4056        return (
    4057            "The auto-saved evidence checkpoint has already been read. Do not read it again. "
    4058            f"Use {tools} to account for what the checkpoint proved, rejected, changed, or blocked "
    4059            "before more shell, search, file, report, artifact, or branch work."
    4060        )
    4061    return (
    4062        "An auto-saved evidence checkpoint is waiting to be converted into durable progress. "
    4063        f"Read that checkpoint artifact once, or use {tools} to account for it before more shell, "
    4064        "search, file, report, artifact, or other branch work."
    4065    )
    4066
    4067
    4068def _read_artifact_args_match_checkpoint(step: dict[str, Any], *, artifact_id: str, artifact_title: str) -> bool:
    4069    input_data = step.get("input") if isinstance(step.get("input"), dict) else {}
    4070    args = input_data.get("arguments") if isinstance(input_data.get("arguments"), dict) else {}
    4071    return _read_artifact_call_matches_checkpoint(args, artifact_id=artifact_id, artifact_title=artifact_title)
    4072
    4073
    4074def _read_artifact_call_matches_checkpoint(args: dict[str, Any], *, artifact_id: str, artifact_title: str) -> bool:
    4075    values = [str(args.get(key) or "").strip() for key in ("artifact_id", "id", "title", "query")]
    4076    values = [value for value in values if value]
    4077    if artifact_id and artifact_id in values:
    4078        return True
    4079    return bool(artifact_title and any(value == artifact_title for value in values))
    4080
    4081
    4082def _recent_search_streak(recent_steps: list[dict[str, Any]]) -> int:
    4083    return _recent_tool_streak(recent_steps, "web_search")
    4084
    4085
    4086def _pending_measurement_obligation(job: dict[str, Any]) -> dict[str, Any] | None:
    4087    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    4088    obligation = metadata.get("pending_measurement_obligation")
    4089    if isinstance(obligation, dict) and obligation and not obligation.get("resolved_at"):
    4090        return obligation
    4091    return None
    4092
    4093
    4094CODELIKE_FILE_SUFFIXES = {
    4095    ".bash",
    4096    ".cfg",
    4097    ".cjs",
    4098    ".conf",
    4099    ".cpp",
    4100    ".css",
    4101    ".go",
    4102    ".h",
    4103    ".hpp",
    4104    ".ini",
    4105    ".java",
    4106    ".js",
    4107    ".json",
    4108    ".jsx",
    4109    ".lua",
    4110    ".mjs",
    4111    ".php",
    4112    ".pl",
    4113    ".py",
    4114    ".rb",
    4115    ".rs",
    4116    ".sh",
    4117    ".sql",
    4118    ".toml",
    4119    ".ts",
    4120    ".tsx",
    4121    ".yaml",
    4122    ".yml",
    4123    ".zsh",
    4124}
    4125
    4126
    4127def _pending_file_validation_obligation(job: dict[str, Any]) -> dict[str, Any] | None:
    4128    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    4129    obligation = metadata.get("pending_file_validation_obligation")
    4130    if isinstance(obligation, dict) and obligation and not obligation.get("resolved_at"):
    4131        return obligation
    4132    return None
    4133
    4134
    4135def _file_output_needs_validation(path: str, content: str) -> bool:
    4136    suffix = Path(path).suffix.lower()
    4137    if suffix in CODELIKE_FILE_SUFFIXES:
    4138        return True
    4139    first_line = content.lstrip().splitlines()[0] if content.strip() else ""
    4140    if first_line.startswith("#!"):
    4141        return True
    4142    lowered = Path(path).name.lower()
    4143    return lowered in {"dockerfile", "makefile", "justfile", "procfile"}
    4144
    4145
    4146def _suggested_file_validation(path: str) -> str:
    4147    suffix = Path(path).suffix.lower()
    4148    quoted = shlex_quote(path)
    4149    if suffix == ".py":
    4150        return f"python3 -m py_compile {quoted}"
    4151    if suffix in {".sh", ".bash", ".zsh"}:
    4152        return f"bash -n {quoted}"
    4153    if suffix == ".json":
    4154        return f"python3 -m json.tool {quoted}"
    4155    if suffix in {".yaml", ".yml"}:
    4156        return f"python3 - <<'PY'\nimport pathlib, yaml\nyaml.safe_load(pathlib.Path({path!r}).read_text())\nPY"
    4157    return f"run the narrowest available syntax check, test, or dry-run for {quoted}"
    4158
    4159
    4160def shlex_quote(value: str) -> str:
    4161    return "'" + str(value).replace("'", "'\"'\"'") + "'"
    4162
    4163
    4164def _clear_invalid_measurement_obligation(db: AgentDB, job_id: str) -> bool:
    4165    job = db.get_job(job_id)
    4166    obligation = _pending_measurement_obligation(job)
    4167    if not obligation:
    4168        return False
    4169    candidates = obligation.get("metric_candidates") if isinstance(obligation.get("metric_candidates"), list) else []
    4170    if not candidates:
    4171        return False
    4172    command = str(obligation.get("command") or "")
    4173    if not measurement_candidates_are_diagnostic_only(candidates, command=command):
    4174        return False
    4175    db.update_job_metadata(job_id, {"pending_measurement_obligation": {}})
    4176    db.append_agent_update(
    4177        job_id,
    4178        "Cleared measurement obligation because the output was diagnostic context, not a trial result.",
    4179        category="progress",
    4180        metadata={"cleared_measurement_obligation": obligation},
    4181    )
    4182    return True
    4183
    4184
    4185def _progress_churn_context(recent_steps: list[dict[str, Any]], *, window: int = 10) -> dict[str, Any] | None:
    4186    completed = [step for step in recent_steps if step.get("status") == "completed"]
    4187    tail = completed[-window:]
    4188    if len(tail) < 8:
    4189        return None
    4190    if any(step.get("tool_name") in LEDGER_PROGRESS_TOOLS for step in tail):
    4191        return None
    4192    churn_count = sum(1 for step in tail if step.get("tool_name") in CHURN_TOOLS)
    4193    if churn_count < 7:
    4194        return None
    4195    return {
    4196        "window": len(tail),
    4197        "churn_count": churn_count,
    4198        "since_step": tail[0].get("step_no"),
    4199        "tools": [step.get("tool_name") or step.get("kind") for step in tail],
    4200    }
    4201
    4202
    4203def _activity_stagnation_context(job: dict[str, Any]) -> dict[str, Any] | None:
    4204    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    4205    streak = _as_int(metadata.get("activity_checkpoint_streak"))
    4206    if streak < ACTIVITY_STAGNATION_CHECKPOINTS:
    4207        return None
    4208    counts = metadata.get("last_checkpoint_counts") if isinstance(metadata.get("last_checkpoint_counts"), dict) else {}
    4209    return {
    4210        "streak": streak,
    4211        "threshold": ACTIVITY_STAGNATION_CHECKPOINTS,
    4212        "counts": {key: _as_int(counts.get(key)) for key in ("findings", "sources", "tasks", "experiments", "lessons", "milestones")},
    4213    }
    4214
    4215
    4216def _research_balance_context(
    4217    job: dict[str, Any],
    4218    recent_steps: list[dict[str, Any]],
    4219    *,
    4220    window: int = 28,
    4221    min_execution_actions: int = 5,
    4222) -> dict[str, Any] | None:
    4223    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    4224    sources = len(_metadata_list(job, "source_ledger"))
    4225    findings = len(_metadata_list(job, "finding_ledger"))
    4226    experiments = len(_metadata_list(job, "experiment_ledger"))
    4227    if sources > 0 or findings > 0:
    4228        return None
    4229    if metadata.get("pending_measurement_obligation"):
    4230        return None
    4231    completed = [step for step in recent_steps if step.get("status") == "completed"]
    4232    if not completed:
    4233        return None
    4234    tail = completed[-window:]
    4235    execution_tools = {"shell_exec", "write_file", "record_experiment", "write_artifact"}
    4236    research_tools = {
    4237        "web_search",
    4238        "web_extract",
    4239        "browser_navigate",
    4240        "browser_snapshot",
    4241        "browser_click",
    4242        "record_source",
    4243        "record_findings",
    4244    }
    4245    execution_actions = [step for step in tail if step.get("tool_name") in execution_tools]
    4246    research_actions = [step for step in tail if step.get("tool_name") in research_tools]
    4247    file_actions = [step for step in tail if step.get("tool_name") in {"write_file", "shell_exec"}]
    4248    tasks = _metadata_list(job, "task_queue")
    4249    active_research_tasks = [
    4250        task
    4251        for task in tasks
    4252        if str(task.get("status") or "open") in {"open", "active", "blocked"}
    4253        and str(task.get("output_contract") or "") == "research"
    4254    ]
    4255    has_research_intent = bool(active_research_tasks) or any(
    4256        "research" in str(job.get(key) or "").lower()
    4257        for key in ("title", "objective", "kind")
    4258    )
    4259    if len(execution_actions) < min_execution_actions and not (has_research_intent and len(file_actions) >= 3):
    4260        return None
    4261    if research_actions and not (has_research_intent and len(execution_actions) >= min_execution_actions * 2):
    4262        return None
    4263    if experiments <= 0 and len(execution_actions) < min_execution_actions + 2:
    4264        return None
    4265    return {
    4266        "completed_window": len(tail),
    4267        "execution_actions": len(execution_actions),
    4268        "research_actions": len(research_actions),
    4269        "sources": sources,
    4270        "findings": findings,
    4271        "experiments": experiments,
    4272        "files": len(file_actions),
    4273    }
    4274
    4275
    4276def _source_yield_context(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> dict[str, Any] | None:
    4277    sources = _metadata_list(job, "source_ledger")
    4278    source_count = len(sources)
    4279    if source_count < SOURCE_YIELD_MIN_SOURCES:
    4280        return None
    4281    findings = _metadata_list(job, "finding_ledger")
    4282    yielded_sources = [
    4283        source
    4284        for source in sources
    4285        if _as_int(source.get("yield_count")) > 0
    4286        or _as_float(source.get("usefulness_score")) >= 0.8
    4287    ]
    4288    required_yield = max(2, source_count // 8)
    4289    if len(findings) + len(yielded_sources) >= required_yield:
    4290        return None
    4291    completed = [step for step in recent_steps if step.get("status") == "completed"]
    4292    last_synthesis_no = 0
    4293    for step in completed:
    4294        if step.get("tool_name") in {
    4295            "record_findings",
    4296            "record_source",
    4297            "record_tasks",
    4298            "record_roadmap",
    4299            "record_milestone_validation",
    4300            "record_lesson",
    4301        }:
    4302            last_synthesis_no = max(last_synthesis_no, _as_int(step.get("step_no")))
    4303    gathering_after_synthesis = [
    4304        step
    4305        for step in completed
    4306        if _as_int(step.get("step_no")) > last_synthesis_no
    4307        and step.get("tool_name") in {
    4308            "web_search",
    4309            "web_extract",
    4310            "browser_navigate",
    4311            "browser_snapshot",
    4312            "browser_click",
    4313            "browser_scroll",
    4314        }
    4315    ]
    4316    recent_gathering = gathering_after_synthesis[-24:]
    4317    if len(recent_gathering) < SOURCE_YIELD_MIN_RECENT_GATHERING:
    4318        return None
    4319    recent_source_titles = [
    4320        str(source.get("source") or source.get("title") or "").strip()
    4321        for source in sources[-8:]
    4322        if str(source.get("source") or source.get("title") or "").strip()
    4323    ]
    4324    return {
    4325        "sources": source_count,
    4326        "findings": len(findings),
    4327        "yielded_sources": len(yielded_sources),
    4328        "required_yield": required_yield,
    4329        "recent_gathering": len(recent_gathering),
    4330        "since_step": recent_gathering[0].get("step_no") if recent_gathering else None,
    4331        "recent_source_titles": recent_source_titles,
    4332    }
    4333
    4334
    4335def _artifact_accounting_context(
    4336    recent_steps: list[dict[str, Any]],
    4337    *,
    4338    threshold: int = 3,
    4339    window: int = 12,
    4340) -> dict[str, Any] | None:
    4341    completed = [step for step in recent_steps if step.get("status") == "completed"]
    4342    tail: list[dict[str, Any]] = []
    4343    for step in reversed(completed[-window:]):
    4344        if step.get("tool_name") in LEDGER_PROGRESS_TOOLS:
    4345            break
    4346        tail.append(step)
    4347    tail.reverse()
    4348    artifact_steps = [step for step in tail if step.get("tool_name") == "write_artifact"]
    4349    if len(artifact_steps) < threshold:
    4350        return None
    4351    titles = []
    4352    for step in artifact_steps[-5:]:
    4353        input_data = step.get("input") if isinstance(step.get("input"), dict) else {}
    4354        args = input_data.get("arguments") if isinstance(input_data.get("arguments"), dict) else {}
    4355        title = str(args.get("title") or step.get("summary") or f"step #{step.get('step_no')}")
    4356        titles.append(_clip_text(title, 120))
    4357    return {
    4358        "artifact_count": len(artifact_steps),
    4359        "since_step": tail[0].get("step_no") if tail else None,
    4360        "artifact_steps": [step.get("step_no") for step in artifact_steps],
    4361        "artifact_titles": titles,
    4362        "tools": [step.get("tool_name") or step.get("kind") for step in tail],
    4363    }
    4364
    4365
    4366def _job_requires_measured_progress(job: dict[str, Any]) -> bool:
    4367    text_parts = [
    4368        str(job.get("title") or ""),
    4369        str(job.get("objective") or ""),
    4370        str(job.get("kind") or ""),
    4371    ]
    4372    tasks = _metadata_list(job, "task_queue")
    4373    for task in tasks:
    4374        status = str(task.get("status") or "open")
    4375        if status in {"done", "skipped"}:
    4376            continue
    4377        contract = str(task.get("output_contract") or "")
    4378        if contract in {"experiment", "monitor"}:
    4379            return True
    4380        if contract == "action" and _task_text_requires_measurement(task):
    4381            return True
    4382        text_parts.extend(
    4383            str(task.get(key) or "")
    4384            for key in ("title", "goal", "acceptance_criteria", "evidence_needed", "stall_behavior")
    4385        )
    4386    return any(MEASURABLE_PROGRESS_PATTERN.search(part) for part in text_parts if part)
    4387
    4388
    4389def _task_text_requires_measurement(task: dict[str, Any]) -> bool:
    4390    return any(
    4391        MEASURABLE_PROGRESS_PATTERN.search(str(task.get(key) or ""))
    4392        for key in ("title", "goal", "acceptance_criteria", "evidence_needed", "stall_behavior")
    4393    )
    4394
    4395
    4396def _job_requires_deliverable_progress(job: dict[str, Any]) -> bool:
    4397    tasks = _metadata_list(job, "task_queue")
    4398    report_tasks: list[dict[str, Any]] = []
    4399    competing_execution_tasks: list[dict[str, Any]] = []
    4400    for task in tasks:
    4401        status = str(task.get("status") or "open").strip().lower()
    4402        if status in {"done", "skipped"}:
    4403            continue
    4404        contract = str(task.get("output_contract") or "").strip().lower()
    4405        if contract == "report":
    4406            report_tasks.append(task)
    4407        elif contract in {"action", "experiment", "monitor"}:
    4408            competing_execution_tasks.append(task)
    4409    if report_tasks:
    4410        active_report = any(str(task.get("status") or "open").strip().lower() == "active" for task in report_tasks)
    4411        active_competing = any(
    4412            str(task.get("status") or "open").strip().lower() == "active"
    4413            for task in competing_execution_tasks
    4414        )
    4415        max_report_priority = max(_as_int(task.get("priority")) for task in report_tasks)
    4416        higher_priority_competing = any(
    4417            _as_int(task.get("priority")) >= max_report_priority
    4418            for task in competing_execution_tasks
    4419        )
    4420        if active_report or (not active_competing and not higher_priority_competing):
    4421            return True
    4422    text = " ".join(str(job.get(key) or "") for key in ("title", "objective", "kind")).lower()
    4423    tokens = set(re.findall(r"[a-z][a-z0-9_-]+", text))
    4424    objective_terms = DELIVERABLE_ARTIFACT_TERMS - {"compiled", "final", "revision", "section", "updated"}
    4425    return bool(tokens & objective_terms)
    4426
    4427
    4428def _step_is_deliverable_checkpoint(step: dict[str, Any]) -> bool:
    4429    tool = step.get("tool_name")
    4430    if tool == "write_file":
    4431        return True
    4432    if tool != "write_artifact":
    4433        return False
    4434    input_data = step.get("input") if isinstance(step.get("input"), dict) else {}
    4435    args = input_data.get("arguments") if isinstance(input_data.get("arguments"), dict) else {}
    4436    text = " ".join(
    4437        str(value or "")
    4438        for value in (
    4439            args.get("title"),
    4440            args.get("summary"),
    4441            args.get("artifact_type"),
    4442            step.get("summary"),
    4443        )
    4444    ).lower()
    4445    tokens = set(re.findall(r"[a-z][a-z0-9_-]+", text))
    4446    if tokens & EVIDENCE_ARTIFACT_TERMS:
    4447        return False
    4448    return bool(tokens & DELIVERABLE_ARTIFACT_TERMS)
    4449
    4450
    4451def _deliverable_progress_guard_context(
    4452    job: dict[str, Any],
    4453    recent_steps: list[dict[str, Any]],
    4454    *,
    4455    budget: int = DELIVERABLE_RESEARCH_BUDGET_STEPS,
    4456) -> dict[str, Any] | None:
    4457    if not _job_requires_deliverable_progress(job):
    4458        return None
    4459    completed = [step for step in recent_steps if step.get("status") == "completed"]
    4460    if not completed:
    4461        return None
    4462    last_checkpoint_index = -1
    4463    for index, step in enumerate(completed):
    4464        if _step_is_deliverable_checkpoint(step):
    4465            last_checkpoint_index = index
    4466    tail = completed[last_checkpoint_index + 1 :]
    4467    branch_activity = [
    4468        step
    4469        for step in tail
    4470        if step.get("tool_name") in BRANCH_WORK_TOOLS
    4471        or (
    4472            step.get("tool_name") == "shell_exec"
    4473            and _shell_command_looks_read_only(_step_command(step))
    4474        )
    4475    ]
    4476    if len(branch_activity) < budget:
    4477        return None
    4478    deliverable_accounting_tools = {"record_tasks", "record_roadmap", "record_milestone_validation", "record_lesson"}
    4479    if any(step.get("tool_name") in deliverable_accounting_tools for step in tail[-6:]):
    4480        return None
    4481    return {
    4482        "reason": "no deliverable checkpoint yet" if last_checkpoint_index < 0 else "no recent deliverable checkpoint",
    4483        "research_budget": budget,
    4484        "completed_since_last_deliverable": len(tail),
    4485        "branch_activity": len(branch_activity),
    4486        "since_step": branch_activity[0].get("step_no") if branch_activity else None,
    4487        "tools": [step.get("tool_name") or step.get("kind") for step in branch_activity[-10:]],
    4488    }
    4489
    4490
    4491def _step_command(step: dict[str, Any]) -> str:
    4492    input_data = step.get("input") if isinstance(step.get("input"), dict) else {}
    4493    args = input_data.get("arguments") if isinstance(input_data.get("arguments"), dict) else {}
    4494    return str(args.get("command") or "")
    4495
    4496
    4497def _read_only_shell_churn_context(recent_steps: list[dict[str, Any]], *, window: int = 10, threshold: int = 3) -> dict[str, Any] | None:
    4498    completed = [step for step in recent_steps if step.get("status") == "completed"]
    4499    if not completed:
    4500        return None
    4501    tail = completed[-window:]
    4502    read_only_shell = [
    4503        step
    4504        for step in tail
    4505        if step.get("tool_name") == "shell_exec" and _shell_command_looks_read_only(_step_command(step))
    4506    ]
    4507    if len(read_only_shell) < threshold:
    4508        return None
    4509    action_steps = [
    4510        step
    4511        for step in tail
    4512        if step.get("tool_name") in {"write_file", "write_artifact", "defer_job"}
    4513        or step.get("tool_name") in {
    4514            "record_experiment",
    4515            "record_findings",
    4516            "record_lesson",
    4517            "record_milestone_validation",
    4518            "record_roadmap",
    4519            "record_source",
    4520            "record_tasks",
    4521            "report_update",
    4522        }
    4523        or (step.get("tool_name") == "shell_exec" and not _shell_command_looks_read_only(_step_command(step)))
    4524    ]
    4525    if action_steps:
    4526        return None
    4527    return {
    4528        "read_only_shell_count": len(read_only_shell),
    4529        "threshold": threshold,
    4530        "window": len(tail),
    4531        "since_step": read_only_shell[0].get("step_no"),
    4532        "commands": [_clip_text(_step_command(step), 140) for step in read_only_shell[-5:]],
    4533    }
    4534
    4535
    4536def _experiment_metric_group_key(experiment: dict[str, Any]) -> tuple[str, str, bool] | None:
    4537    metric_name = str(experiment.get("metric_name") or "").strip().lower()
    4538    if not metric_name:
    4539        return None
    4540    if experiment.get("metric_value") is None:
    4541        return None
    4542    return (
    4543        metric_name,
    4544        str(experiment.get("metric_unit") or "").strip().lower(),
    4545        bool(experiment.get("higher_is_better", True)),
    4546    )
    4547
    4548
    4549def _experiment_metric_number(experiment: dict[str, Any]) -> float | None:
    4550    try:
    4551        return float(experiment.get("metric_value"))
    4552    except (TypeError, ValueError):
    4553        return None
    4554
    4555
    4556def _experiment_value_improves(*, value: float, best_value: float, higher_is_better: bool) -> bool:
    4557    return value > best_value if higher_is_better else value < best_value
    4558
    4559
    4560def _experiment_stagnation_context(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> dict[str, Any] | None:
    4561    if not _job_requires_measured_progress(job):
    4562        return None
    4563    experiments = [
    4564        experiment
    4565        for experiment in _metadata_list(job, "experiment_ledger")
    4566        if str(experiment.get("status") or "").lower() == "measured"
    4567        and _experiment_metric_group_key(experiment) is not None
    4568    ]
    4569    if len(experiments) < EXPERIMENT_STAGNATION_MIN_TRIALS:
    4570        return None
    4571    latest = experiments[-1]
    4572    key = _experiment_metric_group_key(latest)
    4573    if key is None:
    4574        return None
    4575    group = [experiment for experiment in experiments if _experiment_metric_group_key(experiment) == key]
    4576    if len(group) < EXPERIMENT_STAGNATION_MIN_TRIALS:
    4577        return None
    4578    higher_is_better = bool(latest.get("higher_is_better", True))
    4579    best_index = 0
    4580    best_value = _experiment_metric_number(group[0])
    4581    for index, experiment in enumerate(group[1:], start=1):
    4582        value = _experiment_metric_number(experiment)
    4583        if value is None:
    4584            continue
    4585        if best_value is None or _experiment_value_improves(
    4586            value=value,
    4587            best_value=best_value,
    4588            higher_is_better=higher_is_better,
    4589        ):
    4590            best_index = index
    4591            best_value = value
    4592    if best_value is None:
    4593        return None
    4594    non_improving = group[best_index + 1:]
    4595    if len(non_improving) < EXPERIMENT_STAGNATION_NON_IMPROVING:
    4596        return None
    4597    last_experiment_step_no = 0
    4598    for step in recent_steps:
    4599        if step.get("tool_name") == "record_experiment" and str(step.get("status") or "").lower() == "completed":
    4600            last_experiment_step_no = max(last_experiment_step_no, _as_int(step.get("step_no")))
    4601    if last_experiment_step_no > 0:
    4602        decision_tools = {"record_lesson", "record_tasks", "record_roadmap", "record_milestone_validation"}
    4603        if any(
    4604            _as_int(step.get("step_no")) > last_experiment_step_no
    4605            and str(step.get("status") or "").lower() == "completed"
    4606            and step.get("tool_name") in decision_tools
    4607            for step in recent_steps
    4608        ):
    4609            return None
    4610    best = group[best_index]
    4611    return {
    4612        "metric_name": latest.get("metric_name"),
    4613        "metric_unit": latest.get("metric_unit"),
    4614        "higher_is_better": higher_is_better,
    4615        "best_title": best.get("title"),
    4616        "best_value": best.get("metric_value"),
    4617        "latest_title": latest.get("title"),
    4618        "latest_value": latest.get("metric_value"),
    4619        "non_improving_count": len(non_improving),
    4620        "recent_trials": len(group),
    4621        "recent_titles": [str(experiment.get("title") or "") for experiment in non_improving[-5:]],
    4622    }
    4623
    4624
    4625def _measured_progress_guard_context(
    4626    job: dict[str, Any],
    4627    recent_steps: list[dict[str, Any]],
    4628    *,
    4629    budget: int = MEASURABLE_RESEARCH_BUDGET_STEPS,
    4630) -> dict[str, Any] | None:
    4631    if not _job_requires_measured_progress(job):
    4632        return None
    4633    if _pending_measurement_obligation(job):
    4634        return None
    4635    completed = [step for step in recent_steps if step.get("status") == "completed"]
    4636    if not completed:
    4637        return None
    4638    last_experiment_index = -1
    4639    for index, step in enumerate(completed):
    4640        if step.get("tool_name") == "record_experiment":
    4641            last_experiment_index = index
    4642    tail = completed[last_experiment_index + 1 :]
    4643    branch_activity = [step for step in tail if step.get("tool_name") in BRANCH_WORK_TOOLS | {"write_artifact"}]
    4644    shell_actions = [step for step in tail if step.get("tool_name") == "shell_exec"]
    4645    if len(branch_activity) < budget and len(shell_actions) < MEASURABLE_ACTION_BUDGET_STEPS:
    4646        return None
    4647    if any(_step_accounts_for_measured_progress_guard(step) for step in tail[-6:]):
    4648        return None
    4649    experiments = _metadata_list(job, "experiment_ledger")
    4650    reason = "no experiment records yet" if not experiments else "no recent experiment update"
    4651    return {
    4652        "reason": reason,
    4653        "research_budget": budget,
    4654        "shell_action_budget": MEASURABLE_ACTION_BUDGET_STEPS,
    4655        "completed_since_last_experiment": len(tail),
    4656        "branch_activity": len(branch_activity),
    4657        "shell_actions_since_last_experiment": len(shell_actions),
    4658        "since_step": branch_activity[0].get("step_no") if branch_activity else None,
    4659        "tools": [step.get("tool_name") or step.get("kind") for step in branch_activity[-10:]],
    4660    }
    4661
    4662
    4663def _step_accounts_for_measured_progress_guard(step: dict[str, Any]) -> bool:
    4664    tool_name = step.get("tool_name")
    4665    if tool_name == "record_lesson":
    4666        return True
    4667    if tool_name != "record_tasks":
    4668        return False
    4669    output = step.get("output") if isinstance(step.get("output"), dict) else {}
    4670    tasks = output.get("tasks") if isinstance(output.get("tasks"), list) else []
    4671    for task in tasks:
    4672        if not isinstance(task, dict):
    4673            continue
    4674        status = str(task.get("status") or "open").strip().lower().replace(" ", "_")
    4675        if status in {"done", "skipped"}:
    4676            continue
    4677        contract = str(task.get("output_contract") or "").strip().lower().replace(" ", "_")
    4678        if contract in {"experiment", "monitor"}:
    4679            return True
    4680        if contract == "action" and _task_text_requires_measurement(task):
    4681            return True
    4682    return False
    4683
    4684
    4685def _maybe_create_measurement_obligation(
    4686    *,
    4687    db: AgentDB,
    4688    job_id: str,
    4689    step: dict[str, Any] | None,
    4690    tool_name: str,
    4691    args: dict[str, Any],
    4692    result: dict[str, Any],
    4693) -> None:
    4694    if tool_name != "shell_exec":
    4695        return
    4696    command = str(args.get("command") or result.get("command") or "")
    4697    candidates = measurement_candidates(result, command=command)
    4698    if not candidates:
    4699        return
    4700    metadata = db.get_job(job_id).get("metadata")
    4701    if isinstance(metadata, dict):
    4702        existing = metadata.get("pending_measurement_obligation")
    4703        if isinstance(existing, dict) and existing and not existing.get("resolved_at"):
    4704            return
    4705    obligation = {
    4706        "created_at": datetime.now(timezone.utc).isoformat(),
    4707        "source_step_id": step.get("id") if step else "",
    4708        "source_step_no": step.get("step_no") if step else None,
    4709        "tool": tool_name,
    4710        "summary": "Tool output contains measurable-looking results that need experiment accounting.",
    4711        "metric_candidates": candidates,
    4712        "command": command[:1000],
    4713    }
    4714    db.update_job_metadata(job_id, {"pending_measurement_obligation": obligation})
    4715    db.append_agent_update(
    4716        job_id,
    4717        f"Measured output needs accounting: {', '.join(candidates[:3])}.",
    4718        category="blocked",
    4719        metadata={"pending_measurement_obligation": obligation},
    4720    )
    4721
    4722
    4723def _maybe_create_file_validation_obligation(
    4724    *,
    4725    db: AgentDB,
    4726    job_id: str,
    4727    step: dict[str, Any] | None,
    4728    args: dict[str, Any],
    4729    result: dict[str, Any],
    4730) -> None:
    4731    path = str(result.get("path") or args.get("path") or "").strip()
    4732    content = str(args.get("content") or "")
    4733    if not path or not _file_output_needs_validation(path, content):
    4734        return
    4735    metadata = db.get_job(job_id).get("metadata")
    4736    if isinstance(metadata, dict):
    4737        existing = metadata.get("pending_file_validation_obligation")
    4738        if isinstance(existing, dict) and existing and not existing.get("resolved_at"):
    4739            return
    4740    obligation = {
    4741        "created_at": datetime.now(timezone.utc).isoformat(),
    4742        "source_step_id": step.get("id") if step else "",
    4743        "source_step_no": step.get("step_no") if step else None,
    4744        "tool": "write_file",
    4745        "path": path,
    4746        "reason": "code/config/script-like file was written and needs validation before more branch work",
    4747        "suggested_validation": _suggested_file_validation(path),
    4748    }
    4749    db.update_job_metadata(job_id, {"pending_file_validation_obligation": obligation})
    4750    db.append_agent_update(
    4751        job_id,
    4752        f"File output needs validation: {path}",
    4753        category="blocked",
    4754        metadata={"pending_file_validation_obligation": obligation},
    4755    )
    4756
    4757
    4758def _command_references_path(command: str, path: str) -> bool:
    4759    if not command or not path:
    4760        return False
    4761    path_obj = Path(path)
    4762    needles = {str(path_obj), path_obj.name}
    4763    try:
    4764        needles.add(str(path_obj.expanduser().resolve()))
    4765    except OSError:
    4766        pass
    4767    return any(needle and needle in command for needle in needles)
    4768
    4769
    4770def _resolve_file_validation_obligation(
    4771    db: AgentDB,
    4772    job_id: str,
    4773    *,
    4774    status: str,
    4775    reason: str,
    4776    via_tool: str,
    4777    result: dict[str, Any] | None = None,
    4778) -> None:
    4779    job = db.get_job(job_id)
    4780    obligation = _pending_file_validation_obligation(job)
    4781    if not obligation:
    4782        return
    4783    resolved = dict(obligation)
    4784    resolved.update({
    4785        "resolved_at": datetime.now(timezone.utc).isoformat(),
    4786        "resolution_status": status,
    4787        "resolution_reason": reason[:1000],
    4788        "resolution_tool": via_tool,
    4789    })
    4790    if result:
    4791        resolved["validation_result"] = {
    4792            key: result.get(key)
    4793            for key in ("success", "returncode", "error", "summary")
    4794            if key in result
    4795        }
    4796    db.update_job_metadata(
    4797        job_id,
    4798        {
    4799            "pending_file_validation_obligation": {},
    4800            "last_file_validation_obligation": resolved,
    4801        },
    4802    )
    4803    db.append_agent_update(
    4804        job_id,
    4805        f"File validation {status}: {reason[:220]}",
    4806        category="progress" if status == "validated" else "blocked",
    4807        metadata={"file_validation_obligation": resolved},
    4808    )
    4809
    4810
    4811def _maybe_resolve_file_validation_obligation(
    4812    *,
    4813    db: AgentDB,
    4814    job_id: str,
    4815    tool_name: str,
    4816    args: dict[str, Any],
    4817    result: dict[str, Any],
    4818    ok: bool,
    4819) -> None:
    4820    obligation = _pending_file_validation_obligation(db.get_job(job_id))
    4821    if not obligation:
    4822        return
    4823    if tool_name == "shell_exec":
    4824        command = str(args.get("command") or result.get("command") or "")
    4825        path = str(obligation.get("path") or "")
    4826        if not _command_references_path(command, path):
    4827            return
    4828        status = "validated" if ok else "failed"
    4829        reason = "Validation command completed." if ok else f"Validation command failed: {result.get('error') or 'non-zero result'}"
    4830        _resolve_file_validation_obligation(db, job_id, status=status, reason=reason, via_tool=tool_name, result=result)
    4831        return
    4832    if ok and tool_name in {"record_lesson", "record_tasks", "record_experiment", "record_milestone_validation"}:
    4833        _resolve_file_validation_obligation(
    4834            db,
    4835            job_id,
    4836            status="deferred",
    4837            reason=f"Validation was handled or deferred via {tool_name}.",
    4838            via_tool=tool_name,
    4839            result=result,
    4840        )
    4841
    4842
    4843def _step_by_id(db: AgentDB, job_id: str, step_id: str) -> dict[str, Any] | None:
    4844    for step in db.list_steps(job_id=job_id):
    4845        if str(step.get("id") or "") == step_id:
    4846            return step
    4847    return None
    4848
    4849
    4850def _search_query(args: dict[str, Any]) -> str:
    4851    return str(args.get("query") or "").strip()
    4852
    4853
    4854def _query_tokens(query: str) -> set[str]:
    4855    return {
    4856        token
    4857        for token in re.findall(r"[a-z0-9]+", query.lower())
    4858        if len(token) > 2 and token not in QUERY_STOPWORDS
    4859    }
    4860
    4861
    4862def _text_tokens(value: str) -> set[str]:
    4863    return {
    4864        token
    4865        for token in re.findall(r"[a-z0-9]+", str(value or "").lower())
    4866        if len(token) > 2 and token not in TEXT_TOKEN_STOPWORDS
    4867    }
    4868
    4869
    4870def _similar_recent_search(
    4871    args: dict[str, Any],
    4872    recent_steps: list[dict[str, Any]],
    4873    *,
    4874    window: int = 12,
    4875) -> dict[str, Any] | None:
    4876    return _similar_recent_query_tool("web_search", args, recent_steps, window=window)
    4877
    4878
    4879def _similar_recent_query_tool(
    4880    tool_name: str,
    4881    args: dict[str, Any],
    4882    recent_steps: list[dict[str, Any]],
    4883    *,
    4884    window: int = 12,
    4885) -> dict[str, Any] | None:
    4886    query = _search_query(args)
    4887    tokens = _query_tokens(query)
    4888    if len(tokens) < 2:
    4889        return None
    4890    for step in reversed(_completed_recent_steps(recent_steps)[-window:]):
    4891        if step.get("tool_name") != tool_name:
    4892            continue
    4893        input_data = step.get("input") or {}
    4894        previous_args = input_data.get("arguments") if isinstance(input_data, dict) else None
    4895        if not isinstance(previous_args, dict):
    4896            continue
    4897        previous_query = _search_query(previous_args)
    4898        previous_tokens = _query_tokens(previous_query)
    4899        if len(previous_tokens) < 2:
    4900            continue
    4901        overlap = len(tokens & previous_tokens) / max(len(tokens), len(previous_tokens))
    4902        if overlap >= 0.72:
    4903            return step
    4904    return None
    4905
    4906
    4907def _recent_tool_streak(recent_steps: list[dict[str, Any]], tool_name: str) -> int:
    4908    streak = 0
    4909    for step in reversed(_completed_recent_steps(recent_steps)):
    4910        current_tool = step.get("tool_name")
    4911        if current_tool == tool_name:
    4912            streak += 1
    4913            continue
    4914        if current_tool:
    4915            break
    4916    return streak
    4917
    4918
    4919def _repeated_guard_block_context(
    4920    recent_steps: list[dict[str, Any]],
    4921    *,
    4922    threshold: int = 3,
    4923    window: int = 12,
    4924) -> dict[str, Any] | None:
    4925    recoveries = [
    4926        step
    4927        for step in recent_steps
    4928        if step.get("tool_name") == "guard_recovery" and step.get("status") == "completed"
    4929    ]
    4930    last_recovery = max(
    4931        recoveries,
    4932        key=lambda step: int(step.get("step_no") or 0),
    4933        default=None,
    4934    )
    4935    last_recovery_no = int(last_recovery.get("step_no") or 0) if last_recovery else 0
    4936    last_recovery_error = ""
    4937    if last_recovery:
    4938        recovery_output = last_recovery.get("output") if isinstance(last_recovery.get("output"), dict) else {}
    4939        recovery_context = recovery_output.get("guard_recovery") if isinstance(recovery_output.get("guard_recovery"), dict) else {}
    4940        last_recovery_error = str(recovery_context.get("error") or "")
    4941    operational_steps = [
    4942        step
    4943        for step in recent_steps
    4944        if int(step.get("step_no") or 0) > last_recovery_no
    4945        if step.get("kind") in {"tool", "recovery", "assistant"} and step.get("tool_name") != "guard_recovery"
    4946    ]
    4947    tail = operational_steps[-window:]
    4948    latest_blocked = next((step for step in reversed(tail) if step.get("status") == "blocked"), None)
    4949    if not latest_blocked:
    4950        return None
    4951    output = latest_blocked.get("output") if isinstance(latest_blocked.get("output"), dict) else {}
    4952    error = str(output.get("error") or latest_blocked.get("error") or "")
    4953    if error not in RECOVERABLE_GUARD_ERRORS:
    4954        return None
    4955    count = 0
    4956    blocked_tools = []
    4957    first_step_no = None
    4958    for step in tail:
    4959        step_output = step.get("output") if isinstance(step.get("output"), dict) else {}
    4960        step_error = str(step_output.get("error") or step.get("error") or "")
    4961        if step.get("status") == "blocked" and step_error == error:
    4962            count += 1
    4963            first_step_no = first_step_no or step.get("step_no")
    4964            blocked_tools.append(str(step.get("tool_name") or step.get("kind") or "tool"))
    4965    effective_threshold = 1 if _already_read_checkpoint_accounting_block(latest_blocked) else threshold
    4966    if count < effective_threshold:
    4967        return None
    4968    progress_after_recovery = any(
    4969        step.get("status") == "completed"
    4970        and step.get("tool_name") != "guard_recovery"
    4971        for step in operational_steps
    4972    )
    4973    if last_recovery_error == error and not progress_after_recovery:
    4974        return None
    4975    context = {
    4976        "error": error,
    4977        "count": count,
    4978        "first_step_no": first_step_no,
    4979        "latest_step_no": latest_blocked.get("step_no"),
    4980        "blocked_tools": blocked_tools[-8:],
    4981    }
    4982    if error == "task queue saturated":
    4983        task_queue = output.get("task_queue") if isinstance(output.get("task_queue"), dict) else {}
    4984        context["task_queue"] = {
    4985            "reason": task_queue.get("reason") or "task queue saturated",
    4986            "open_count": task_queue.get("open_count"),
    4987            "total_count": task_queue.get("total_count"),
    4988            "open_titles": task_queue.get("open_titles") if isinstance(task_queue.get("open_titles"), list) else [],
    4989        }
    4990    return context
    4991
    4992
    4993def _already_read_checkpoint_accounting_block(step: dict[str, Any]) -> bool:
    4994    output = step.get("output") if isinstance(step.get("output"), dict) else {}
    4995    checkpoint = output.get("pending_evidence_checkpoint") if isinstance(output.get("pending_evidence_checkpoint"), dict) else {}
    4996    return (
    4997        output.get("error") == "evidence checkpoint accounting required"
    4998        and (bool(output.get("checkpoint_already_read")) or bool(checkpoint.get("checkpoint_read")))
    4999    )
    5000
    5001
    5002def _step_error_text(step: dict[str, Any]) -> str:
    5003    output = step.get("output") if isinstance(step.get("output"), dict) else {}
    5004    parts = [
    5005        output.get("error"),
    5006        output.get("error_type"),
    5007        output.get("detail"),
    5008        output.get("message"),
    5009        step.get("error"),
    5010        step.get("summary"),
    5011    ]
    5012    return " ".join(str(part) for part in parts if part)
    5013
    5014
    5015def _blocked_tool_call_result(
    5016    name: str,
    5017    args: dict[str, Any],
    5018    recent_steps: list[dict[str, Any]],
    5019    job: dict[str, Any],
    5020) -> tuple[dict[str, Any], str] | None:
    5021    if name == "defer_job":
    5022        self_defer = _self_defer_context(args)
    5023        if self_defer:
    5024            result = {
    5025                "success": False,
    5026                "error": "self-defer blocked",
    5027                "blocked_tool": name,
    5028                "blocked_arguments": args,
    5029                "self_defer": self_defer,
    5030                "guidance": (
    5031                    "Do not defer merely for a future worker turn to pick up ordinary work. Use defer_job only when "
    5032                    "waiting for a real external process, scheduled monitor interval, long-running command, "
    5033                    "or other time-based condition. Otherwise execute, measure, record a task/experiment/lesson, or "
    5034                    "mark the branch blocked now."
    5035                ),
    5036            }
    5037            return result, "blocked defer_job; self-defer is not progress"
    5038
    5039    if name == "record_tasks":
    5040        saturated = _task_queue_saturation_context(job, args)
    5041        if saturated:
    5042            result = {
    5043                "success": False,
    5044                "error": "task queue saturated",
    5045                "blocked_tool": name,
    5046                "blocked_arguments": args,
    5047                "task_queue": saturated,
    5048                "guidance": (
    5049                    "The durable task queue already has many branches. Do not create more branch sprawl. "
    5050                    "Choose an existing high-priority task and execute it, update existing tasks to active, "
    5051                    "done, blocked, or skipped, or consolidate the queue into roadmap/milestone state."
    5052                ),
    5053            }
    5054            return result, f"blocked record_tasks; {saturated['reason']}"
    5055        task_planning_stagnation = _task_planning_stagnation_context(job)
    5056        if task_planning_stagnation and _record_tasks_adds_new_open_work(args, job):
    5057            result = {
    5058                "success": False,
    5059                "error": "task execution required",
    5060                "blocked_tool": name,
    5061                "blocked_arguments": args,
    5062                "task_planning": task_planning_stagnation,
    5063                "guidance": (
    5064                    "Recent checkpoints only expanded the task queue. Do not add more new open tasks yet. "
    5065                    "Execute or validate an existing branch, save a durable checkpoint, record findings/source/"
    5066                    "experiment evidence, mark existing tasks done/blocked/skipped, or record a lesson."
    5067                ),
    5068            }
    5069            return result, "blocked record_tasks; task-only planning needs execution"
    5070
    5071    current_milestone_validation = _milestone_validation_needed(job)
    5072    if (
    5073        name == "record_milestone_validation"
    5074        and current_milestone_validation
    5075        and not _milestone_validation_call_matches_current(args, current_milestone_validation)
    5076    ):
    5077        result = {
    5078            "success": False,
    5079            "error": "current milestone validation required",
    5080            "blocked_tool": name,
    5081            "blocked_arguments": args,
    5082            "milestone": {
    5083                "title": current_milestone_validation.get("title"),
    5084                "status": current_milestone_validation.get("status"),
    5085                "validation_status": current_milestone_validation.get("validation_status"),
    5086                "acceptance_criteria": current_milestone_validation.get("acceptance_criteria"),
    5087                "evidence_needed": current_milestone_validation.get("evidence_needed"),
    5088            },
    5089            "guidance": (
    5090                "A milestone validation gate is already active. Validate that current milestone by name, "
    5091                "or update the roadmap to make a different milestone current before validating another one."
    5092            ),
    5093        }
    5094        return result, "blocked record_milestone_validation; current milestone validation required"
    5095
    5096    auto_checkpoint_accounting = _auto_checkpoint_accounting_context(job, recent_steps)
    5097    checkpoint_read_call = bool(
    5098        auto_checkpoint_accounting
    5099        and name == "read_artifact"
    5100        and not auto_checkpoint_accounting.get("checkpoint_read")
    5101        and _read_artifact_call_matches_checkpoint(
    5102            args,
    5103            artifact_id=str(auto_checkpoint_accounting.get("artifact_id") or ""),
    5104            artifact_title=str(auto_checkpoint_accounting.get("title") or ""),
    5105        )
    5106    )
    5107    if _evidence_checkpoint_blocks_tool(name, args, auto_checkpoint_accounting):
    5108        checkpoint_already_read = bool(auto_checkpoint_accounting and auto_checkpoint_accounting.get("checkpoint_read"))
    5109        result = {
    5110            "success": False,
    5111            "error": "evidence checkpoint accounting required",
    5112            "blocked_tool": name,
    5113            "blocked_arguments": args,
    5114            "pending_evidence_checkpoint": auto_checkpoint_accounting,
    5115            "checkpoint_already_read": checkpoint_already_read,
    5116            "required_next_action": "durable_checkpoint_accounting" if checkpoint_already_read else "read_or_account_checkpoint",
    5117            "allowed_resolution_tools": sorted(EVIDENCE_CHECKPOINT_RESOLUTION_TOOLS),
    5118            "guidance": _evidence_checkpoint_block_guidance(auto_checkpoint_accounting or {}),
    5119        }
    5120        return result, f"blocked {name}; evidence checkpoint accounting required"
    5121    checkpoint_resolution_call = bool(auto_checkpoint_accounting and name in EVIDENCE_CHECKPOINT_RESOLUTION_TOOLS)
    5122
    5123    if name == "shell_exec":
    5124        placeholder = _shell_placeholder_context(str(args.get("command") or ""))
    5125        if placeholder:
    5126            result = {
    5127                "success": False,
    5128                "error": "unresolved placeholder in shell command",
    5129                "blocked_tool": name,
    5130                "blocked_arguments": args,
    5131                "placeholder": placeholder,
    5132                "guidance": (
    5133                    "Do not execute shell commands that still contain placeholder URLs, paths, hosts, or template "
    5134                    "tokens. Resolve the concrete value from evidence, ask the operator if it is genuinely unknown, "
    5135                    "or record a blocked task/source before continuing."
    5136                ),
    5137            }
    5138            return result, "blocked shell_exec; unresolved placeholder in command"
    5139        syntax_error = _shell_syntax_preflight_context(str(args.get("command") or ""))
    5140        if syntax_error:
    5141            result = {
    5142                "success": False,
    5143                "recoverable": True,
    5144                "error": "malformed shell command",
    5145                "blocked_tool": name,
    5146                "blocked_arguments": args,
    5147                "syntax": syntax_error,
    5148                "guidance": (
    5149                    "Do not execute partial or malformed shell. Rebuild the command from exact observed paths, "
    5150                    "or use a simpler bounded probe before retrying."
    5151                ),
    5152            }
    5153            return result, "blocked shell_exec; malformed command syntax"
    5154        candidate_recovery = _observed_candidate_recovery_required_context(recent_steps, args)
    5155        if candidate_recovery:
    5156            result = {
    5157                "success": False,
    5158                "error": "observed executable recovery required",
    5159                "blocked_tool": name,
    5160                "blocked_arguments": args,
    5161                "candidate_recovery": candidate_recovery,
    5162                "guidance": (
    5163                    "A recent shell step reported this command as missing, and later evidence showed candidate "
    5164                    "executable paths. Retry with an exact observed executable path, add its directory to PATH, "
    5165                    "or record why that observed candidate is invalid before running the bare command again."
    5166                ),
    5167            }
    5168            return result, "blocked shell_exec; observed executable recovery required"
    5169        privileged_failure = _recent_privileged_shell_failure_context(recent_steps)
    5170        if privileged_failure and _shell_command_looks_privileged_or_package_manager(str(args.get("command") or "")):
    5171            result = {
    5172                "success": False,
    5173                "error": "privileged command recovery required",
    5174                "blocked_tool": name,
    5175                "blocked_arguments": args,
    5176                "privileged_failure": privileged_failure,
    5177                "guidance": (
    5178                    "A recent privileged/package-manager shell command failed due permission or authorization. "
    5179                    "Do not retry that class of command until the failure is accounted for. Use observed executable "
    5180                    "paths, user-writable installs, existing project files, or record_tasks/record_lesson/"
    5181                    "record_experiment to mark the branch blocked or choose a non-privileged recovery."
    5182                ),
    5183            }
    5184            return result, "blocked shell_exec; privileged command recovery required"
    5185
    5186    unpersisted_evidence = _unpersisted_evidence_step(recent_steps)
    5187    if unpersisted_evidence and name in BRANCH_WORK_TOOLS:
    5188        result = {
    5189            "success": False,
    5190            "error": "artifact required before more research",
    5191            "blocked_tool": name,
    5192            "blocked_arguments": args,
    5193            "previous_step": unpersisted_evidence["id"],
    5194            "guidance": (
    5195                "Fresh browser, extracted, or shell evidence is waiting. Save or account for that evidence with "
    5196                "write_artifact, record_findings, record_source, record_experiment, record_tasks, "
    5197                "record_roadmap, record_milestone_validation, or record_lesson before doing more search, "
    5198                "browsing, shell work, or artifact review."
    5199            ),
    5200        }
    5201        return result, f"blocked {name}; write_artifact required after evidence step #{unpersisted_evidence['step_no']}"
    5202
    5203    duplicate_step = _duplicate_recent_tool_call(name, args, recent_steps)
    5204    if duplicate_step:
    5205        guidance = "Use a different query, extract one of the prior result URLs, open a result in the browser, or write an artifact."
    5206        if name == "read_artifact":
    5207            guidance = (
    5208                "This artifact was already read. Do not read it again; use its content to inspect a concrete item, "
    5209                "record findings/tasks, or write a report artifact."
    5210            )
    5211        elif name == "shell_exec":
    5212            guidance = (
    5213                "This shell command was already run. Do not rerun discovery; use the previous output to inspect a "
    5214                "specific file/item, write an artifact, or update findings/tasks."
    5215            )
    5216        result = {
    5217            "success": False,
    5218            "recoverable": name == "read_artifact",
    5219            "error": "duplicate tool call blocked",
    5220            "blocked_tool": name,
    5221            "blocked_arguments": args,
    5222            "previous_step": duplicate_step["id"],
    5223            "guidance": guidance,
    5224        }
    5225        return result, f"blocked duplicate {name}; previous step #{duplicate_step['step_no']}"
    5226
    5227    if checkpoint_read_call:
    5228        return None
    5229
    5230    browser_runtime_unavailable = _browser_runtime_unavailable_context(recent_steps)
    5231    if browser_runtime_unavailable and _is_browser_tool(name):
    5232        result = {
    5233            "success": False,
    5234            "error": "browser runtime unavailable",
    5235            "blocked_tool": name,
    5236            "blocked_arguments": args,
    5237            "browser_runtime": browser_runtime_unavailable,
    5238            "guidance": (
    5239                "Browser automation is unavailable on this host. Do not retry browser tools until the runtime is "
    5240                "installed or configured. Use web_search, web_extract, shell_exec, source/ledger tools, or record "
    5241                "a blocked task/source and continue through a non-browser branch."
    5242            ),
    5243        }
    5244        return result, f"blocked {name}; browser runtime unavailable"
    5245
    5246    measurement_obligation = _pending_measurement_obligation(job)
    5247    if (
    5248        measurement_obligation
    5249        and not checkpoint_resolution_call
    5250        and name in MEASUREMENT_BLOCKED_TOOLS
    5251        and name not in MEASUREMENT_RESOLUTION_TOOLS
    5252    ):
    5253        result = {
    5254            "success": False,
    5255            "error": "measurement obligation pending",
    5256            "blocked_tool": name,
    5257            "blocked_arguments": args,
    5258            "pending_measurement_obligation": measurement_obligation,
    5259            "guidance": (
    5260                "A recent action produced measurable output. Record it with record_experiment, "
    5261                "explain why it is invalid with record_lesson, or create the missing measurement branch with record_tasks "
    5262                "before doing more research, artifact writing, or finding/source updates."
    5263            ),
    5264        }
    5265        return result, f"blocked {name}; record_experiment required after measured output"
    5266
    5267    file_validation_obligation = _pending_file_validation_obligation(job)
    5268    if (
    5269        file_validation_obligation
    5270        and not checkpoint_resolution_call
    5271        and name in FILE_VALIDATION_BLOCKED_TOOLS
    5272        and name not in FILE_VALIDATION_RESOLUTION_TOOLS
    5273    ):
    5274        result = {
    5275            "success": False,
    5276            "error": "file validation pending",
    5277            "blocked_tool": name,
    5278            "blocked_arguments": args,
    5279            "pending_file_validation_obligation": file_validation_obligation,
    5280            "guidance": (
    5281                "A recent file output needs validation before more research/output churn. "
    5282                "Use shell_exec to run a syntax check, dry-run, test, or other narrow validation for the file, "
    5283                "or use record_tasks/record_lesson/record_experiment if validation is blocked or deferred."
    5284            ),
    5285        }
    5286        return result, f"blocked {name}; file validation required after write_file"
    5287
    5288    early_anti_bot_context = _recent_anti_bot_context(recent_steps)
    5289    if early_anti_bot_context and name == "write_artifact" and not _artifact_args_acknowledge_block(args):
    5290        result = {
    5291            "success": False,
    5292            "error": "misleading blocked-source artifact blocked",
    5293            "blocked_tool": name,
    5294            "blocked_arguments": args,
    5295            "anti_bot_source": early_anti_bot_context,
    5296            "guidance": "The latest browser evidence is an anti-bot/CAPTCHA block. Write only a blocked-source note or pivot.",
    5297        }
    5298        return result, f"blocked misleading write_artifact; anti-bot source at step #{early_anti_bot_context.get('step_no')}"
    5299
    5300    evidence_grounding = _evidence_grounding_context(job, recent_steps, tool_name=name, args=args)
    5301    if evidence_grounding:
    5302        result = {
    5303            "success": False,
    5304            "error": "evidence grounding required",
    5305            "blocked_tool": name,
    5306            "blocked_arguments": args,
    5307            "evidence_grounding": evidence_grounding,
    5308            "guidance": evidence_grounding["guidance"],
    5309        }
    5310        return result, f"blocked {name}; evidence grounding required"
    5311
    5312    measured_progress_guard = _measured_progress_guard_context(job, recent_steps)
    5313    experiment_stagnation = _experiment_stagnation_context(job, recent_steps)
    5314    deliverable_progress_guard = _deliverable_progress_guard_context(job, recent_steps)
    5315    source_yield = _source_yield_context(job, recent_steps)
    5316    progress_churn = _progress_churn_context(recent_steps)
    5317    artifact_accounting = _artifact_accounting_context(recent_steps)
    5318    activity_stagnation = _activity_stagnation_context(job)
    5319    memory_consolidation = _memory_graph_consolidation_context(job, recent_steps)
    5320    shell_read_only = name == "shell_exec" and _shell_command_looks_read_only(str(args.get("command") or ""))
    5321    if (
    5322        artifact_accounting
    5323        and name in ARTIFACT_ACCOUNTING_BLOCKED_TOOLS
    5324        and name not in ARTIFACT_ACCOUNTING_RESOLUTION_TOOLS
    5325    ):
    5326        result = {
    5327            "success": False,
    5328            "error": "progress accounting required",
    5329            "blocked_tool": name,
    5330            "blocked_arguments": args,
    5331            "artifact_accounting": artifact_accounting,
    5332            "guidance": (
    5333                "Recent saved outputs have not been reflected in durable progress state. "
    5334                "Use record_tasks or record_roadmap to mark completed/open branches, "
    5335                "record_milestone_validation for milestone checks, record_findings or record_source "
    5336                "for reusable evidence, record_experiment for measurements, or record_lesson "
    5337                "if the outputs were low-value before continuing."
    5338            ),
    5339        }
    5340        return result, f"blocked {name}; progress accounting required after saved outputs"
    5341
    5342    if progress_churn and not measured_progress_guard and name in CHURN_TOOLS:
    5343        result = {
    5344            "success": False,
    5345            "error": "progress ledger update required",
    5346            "blocked_tool": name,
    5347            "blocked_arguments": args,
    5348            "progress_churn": progress_churn,
    5349            "guidance": (
    5350                "Recent activity has not changed findings, experiments, tasks, lessons, or sources. "
    5351                "Use a ledger tool to record progress, reject the branch, or create a pivot task before continuing."
    5352            ),
    5353        }
    5354        return result, f"blocked {name}; progress ledger update required"
    5355
    5356    read_only_shell_churn = _read_only_shell_churn_context(recent_steps)
    5357    if read_only_shell_churn and shell_read_only:
    5358        result = {
    5359            "success": False,
    5360            "error": "action decision required",
    5361            "blocked_tool": name,
    5362            "blocked_arguments": args,
    5363            "read_only_shell_churn": read_only_shell_churn,
    5364            "guidance": (
    5365                "Recent shell work only inspected or listed state. Stop re-probing the same branch. "
    5366                "Run the next concrete action, write/persist the candidate decision, record an experiment/monitor task, "
    5367                "or record why the branch is blocked before another read-only shell command."
    5368            ),
    5369        }
    5370        return result, f"blocked {name}; action decision required"
    5371
    5372    if activity_stagnation and name in ACTIVITY_STAGNATION_BLOCKED_TOOLS:
    5373        result = {
    5374            "success": False,
    5375            "error": "durable progress required",
    5376            "blocked_tool": name,
    5377            "blocked_arguments": args,
    5378            "activity_stagnation": activity_stagnation,
    5379            "guidance": (
    5380                "Several checkpoints have produced no durable ledger delta. "
    5381                "Use record_findings, record_source, record_experiment, record_tasks, record_roadmap, "
    5382                "record_milestone_validation, or record_lesson to classify the branch, mark it blocked/skipped, "
    5383                "or open a better branch before more research, shell, file, report, or artifact work."
    5384            ),
    5385        }
    5386        return result, f"blocked {name}; durable progress required after activity-only checkpoints"
    5387
    5388    if source_yield and name in SOURCE_YIELD_BLOCKED_TOOLS:
    5389        result = {
    5390            "success": False,
    5391            "error": "source yield accounting required",
    5392            "blocked_tool": name,
    5393            "blocked_arguments": args,
    5394            "source_yield": source_yield,
    5395            "guidance": (
    5396                "The job has gathered enough sources without enough durable findings or yielded source outcomes. "
    5397                "Before more search, extraction, browsing, shell execution, file/output work, or report chatter, "
    5398                "use record_findings to save source-backed facts/candidates, record_source to mark source yield "
    5399                "or low-yield outcomes, or update tasks/roadmap/lessons to pivot from the source branch."
    5400            ),
    5401        }
    5402        return result, f"blocked {name}; source yield accounting required"
    5403
    5404    if memory_consolidation and name in MEMORY_CONSOLIDATION_BLOCKED_TOOLS:
    5405        result = {
    5406            "success": False,
    5407            "error": "memory graph consolidation required",
    5408            "blocked_tool": name,
    5409            "blocked_arguments": args,
    5410            "memory_consolidation": memory_consolidation,
    5411            "guidance": (
    5412                "The job has enough reusable durable records that raw ledgers should be consolidated into connected "
    5413                "memory. Use record_memory_graph to add/update nodes and links before more branch work, or record_lesson "
    5414                "if there is no reusable memory to preserve."
    5415            ),
    5416        }
    5417        return result, f"blocked {name}; memory graph consolidation required"
    5418
    5419    record_experiment_closes_branch = (
    5420        name == "record_experiment"
    5421        and str(args.get("status") or "").strip().lower().replace(" ", "_") in {"failed", "blocked", "skipped"}
    5422    )
    5423    if (
    5424        experiment_stagnation
    5425        and not record_experiment_closes_branch
    5426        and (
    5427            name in BRANCH_WORK_TOOLS
    5428            or name in {"record_experiment", "write_artifact", "write_file", "report_update"}
    5429        )
    5430    ):
    5431        result = {
    5432            "success": False,
    5433            "error": "experiment stagnation decision required",
    5434            "blocked_tool": name,
    5435            "blocked_arguments": args,
    5436            "experiment_stagnation": experiment_stagnation,
    5437            "guidance": (
    5438                "Recent measured trials have not improved the best observed result. Before more experiments, "
    5439                "execution, research, file/output work, or report chatter, make a durable decision: use "
    5440                "record_tasks, record_roadmap, record_milestone_validation, record_lesson, or a blocked/skipped/"
    5441                "failed record_experiment to reject, block, or pivot the stagnant branch."
    5442            ),
    5443        }
    5444        return result, f"blocked {name}; experiment stagnation decision required"
    5445
    5446    lesson_sprawl = _lesson_sprawl_context(job, recent_steps)
    5447    if lesson_sprawl and name == "record_lesson":
    5448        result = {
    5449            "success": False,
    5450            "error": "lesson consolidation required",
    5451            "blocked_tool": name,
    5452            "blocked_arguments": args,
    5453            "lesson_consolidation": lesson_sprawl,
    5454            "guidance": (
    5455                "This job already has many raw lessons and the connected memory graph is behind. "
    5456                "Do not add another raw lesson. Use record_memory_graph to consolidate reusable strategy, mistake, "
    5457                "constraint, decision, question, skill, or episode nodes with evidence links, or update existing "
    5458                "tasks/roadmap/milestone state if this is only branch status."
    5459            ),
    5460        }
    5461        return result, "blocked record_lesson; lesson consolidation required"
    5462
    5463    if deliverable_progress_guard and (name in DELIVERABLE_PROGRESS_BLOCKED_TOOLS or shell_read_only):
    5464        result = {
    5465            "success": False,
    5466            "error": "deliverable checkpoint required",
    5467            "blocked_tool": name,
    5468            "blocked_arguments": args,
    5469            "deliverable_progress_guard": deliverable_progress_guard,
    5470            "guidance": (
    5471                "This job is deliverable-framed and has done enough background work without a draft/report/file "
    5472                "checkpoint. Save a partial deliverable with write_file or write_artifact, or record_tasks, "
    5473                "record_roadmap, record_milestone_validation, or record_lesson if the deliverable is blocked."
    5474            ),
    5475        }
    5476        return result, f"blocked {name}; deliverable checkpoint required"
    5477
    5478    research_balance = _research_balance_context(job, recent_steps)
    5479    if research_balance and name in RESEARCH_BALANCE_BLOCKED_TOOLS:
    5480        result = {
    5481            "success": False,
    5482            "error": "research balance required",
    5483            "blocked_tool": name,
    5484            "blocked_arguments": args,
    5485            "research_balance": research_balance,
    5486            "guidance": (
    5487                "Recent work is execution-heavy but has no durable sources or findings. "
    5488                "Use web/browser/documentation/local-inspection tools and record_source or record_findings "
    5489                "before continuing execution, artifact review, raw lesson accumulation, report updates, or file churn."
    5490            ),
    5491        }
    5492        return result, f"blocked {name}; research balance required"
    5493
    5494    roadmap_staleness = _roadmap_staleness_context(job, recent_steps)
    5495    if roadmap_staleness and not checkpoint_resolution_call and name in ROADMAP_STALENESS_BLOCKED_TOOLS:
    5496        result = {
    5497            "success": False,
    5498            "error": "roadmap update required",
    5499            "blocked_tool": name,
    5500            "blocked_arguments": args,
    5501            "roadmap_staleness": roadmap_staleness,
    5502            "guidance": (
    5503                "The roadmap has not advanced despite durable task/artifact activity. "
    5504                "Use record_roadmap to mark milestone progress, record_milestone_validation "
    5505                "to judge an evidence-backed checkpoint, or record_lesson if the roadmap is wrong."
    5506            ),
    5507        }
    5508        return result, f"blocked {name}; roadmap update required"
    5509
    5510    milestone_validation = _milestone_validation_needed(job)
    5511    milestone_validation_action = milestone_validation and _tool_call_matches_pending_milestone_need(
    5512        name,
    5513        args,
    5514        milestone_validation,
    5515    )
    5516    if (
    5517        milestone_validation
    5518        and not milestone_validation_action
    5519        and not checkpoint_resolution_call
    5520        and name in MILESTONE_VALIDATION_BLOCKED_TOOLS
    5521    ):
    5522        result = {
    5523            "success": False,
    5524            "error": "milestone validation required",
    5525            "blocked_tool": name,
    5526            "blocked_arguments": args,
    5527            "milestone": {
    5528                "title": milestone_validation.get("title"),
    5529                "status": milestone_validation.get("status"),
    5530                "validation_status": milestone_validation.get("validation_status"),
    5531                "acceptance_criteria": milestone_validation.get("acceptance_criteria"),
    5532                "evidence_needed": milestone_validation.get("evidence_needed"),
    5533            },
    5534            "guidance": (
    5535                "The current milestone is ready for validation. Use record_milestone_validation "
    5536                "with evidence and pass/fail/blocker status, read an existing artifact if needed, "
    5537                "or create follow-up tasks for validation gaps before starting more branch work."
    5538            ),
    5539        }
    5540        return result, f"blocked {name}; milestone validation required"
    5541
    5542    anti_bot_context = _recent_anti_bot_context(recent_steps)
    5543    if anti_bot_context:
    5544        blocked_browser_followups = {"browser_click", "browser_console", "browser_press", "browser_scroll", "browser_snapshot", "browser_type"}
    5545        if name in blocked_browser_followups:
    5546            result = {
    5547                "success": False,
    5548                "error": "anti-bot source loop blocked",
    5549                "blocked_tool": name,
    5550                "blocked_arguments": args,
    5551                "anti_bot_source": anti_bot_context,
    5552                "guidance": "This page is blocked by anti-bot/CAPTCHA. Record the source as blocked and pivot to a different public source.",
    5553            }
    5554            return result, f"blocked {name}; anti-bot source at step #{anti_bot_context.get('step_no')}"
    5555        if name == "browser_navigate" and _same_source_url(str(args.get("url") or ""), str(anti_bot_context.get("url") or "")):
    5556            result = {
    5557                "success": False,
    5558                "error": "anti-bot source loop blocked",
    5559                "blocked_tool": name,
    5560                "blocked_arguments": args,
    5561                "anti_bot_source": anti_bot_context,
    5562                "guidance": "Do not reopen the same blocked source. Pivot to another source.",
    5563            }
    5564            return result, f"blocked {name}; repeated blocked source from step #{anti_bot_context.get('step_no')}"
    5565        if name == "web_extract":
    5566            urls = args.get("urls") if isinstance(args.get("urls"), list) else []
    5567            if any(_same_source_url(str(url), str(anti_bot_context.get("url") or "")) for url in urls):
    5568                result = {
    5569                    "success": False,
    5570                    "error": "anti-bot source loop blocked",
    5571                    "blocked_tool": name,
    5572                    "blocked_arguments": args,
    5573                    "anti_bot_source": anti_bot_context,
    5574                    "guidance": "Do not extract the same blocked source. Record it as low-yield and pivot.",
    5575                }
    5576                return result, f"blocked {name}; blocked source from step #{anti_bot_context.get('step_no')}"
    5577        if name == "write_artifact" and not _artifact_args_acknowledge_block(args):
    5578            result = {
    5579                "success": False,
    5580                "error": "misleading blocked-source artifact blocked",
    5581                "blocked_tool": name,
    5582                "blocked_arguments": args,
    5583                "anti_bot_source": anti_bot_context,
    5584                "guidance": "The latest browser evidence is an anti-bot/CAPTCHA block. Write only a blocked-source note or pivot.",
    5585            }
    5586            return result, f"blocked misleading write_artifact; anti-bot source at step #{anti_bot_context.get('step_no')}"
    5587
    5588    experiment_next_action = _latest_experiment_next_action_context(job)
    5589    action_failure = _experiment_next_action_failure_context(job, recent_steps)
    5590    if (
    5591        action_failure
    5592        and name not in {"record_experiment", "record_tasks", "record_lesson", "record_milestone_validation"}
    5593    ):
    5594        result = {
    5595            "success": False,
    5596            "error": "action result accounting required",
    5597            "blocked_tool": name,
    5598            "blocked_arguments": args,
    5599            "action_failure": action_failure,
    5600            "guidance": (
    5601                "The latest experiment next action was attempted and the observed output reports a missing command, "
    5602                "path, or prerequisite. Before more work, use record_experiment, record_tasks, or record_lesson to "
    5603                "account for the failed/blocked action and choose a concrete recovery branch."
    5604            ),
    5605        }
    5606        return result, f"blocked {name}; action result accounting required"
    5607    if (
    5608        _experiment_next_action_requires_delivery(experiment_next_action)
    5609        and (
    5610            name in EXPERIMENT_NEXT_ACTION_BLOCKED_TOOLS
    5611            or (
    5612                name == "shell_exec"
    5613                and _shell_command_looks_read_only(str(args.get("command") or ""))
    5614                and not _shell_command_supports_experiment_next_action(str(args.get("command") or ""), experiment_next_action)
    5615            )
    5616        )
    5617    ):
    5618        result = {
    5619            "success": False,
    5620            "error": "experiment next action pending",
    5621            "blocked_tool": name,
    5622            "blocked_arguments": args,
    5623            "experiment_next_action": experiment_next_action,
    5624            "guidance": (
    5625                "The latest measured experiment selected a delivery/action next step. "
    5626                "Act on that next action with an execution or ledger tool, or use record_experiment/record_tasks/record_lesson "
    5627                "to explain why it is invalid or blocked before doing more research or artifact review."
    5628            ),
    5629        }
    5630        return result, f"blocked {name}; experiment next action pending"
    5631
    5632    shell_budget_exhausted = (
    5633        name == "shell_exec"
    5634        and _as_int(measured_progress_guard.get("shell_actions_since_last_experiment")) >= MEASURABLE_ACTION_BUDGET_STEPS
    5635    ) if measured_progress_guard else False
    5636    candidate_validation_shell = (
    5637        name == "shell_exec" and _shell_exec_targets_candidate_file(job, recent_steps, args)
    5638    )
    5639    if (
    5640        measured_progress_guard
    5641        and not checkpoint_resolution_call
    5642        and (name in MEASURABLE_RESEARCH_BLOCKED_TOOLS or (shell_budget_exhausted and not candidate_validation_shell))
    5643    ):
    5644        result = {
    5645            "success": False,
    5646            "error": "measured progress required",
    5647            "blocked_tool": name,
    5648            "blocked_arguments": args,
    5649            "measured_progress_guard": measured_progress_guard,
    5650            "guidance": (
    5651                "This job is measurably framed and has exhausted its research budget without new experiment records. "
    5652                "If the shell/action budget is exhausted, do not call shell_exec again; call record_experiment for a "
    5653                "known measurement, record_tasks with an experiment/action/monitor contract, or record_lesson if "
    5654                "measurement is blocked."
    5655            ),
    5656        }
    5657        return result, f"blocked {name}; measured progress required"
    5658
    5659    if name in BRANCH_WORK_TOOLS and _task_queue_exhausted(job):
    5660        result = {
    5661            "success": False,
    5662            "error": "task branch required before more work",
    5663            "blocked_tool": name,
    5664            "blocked_arguments": args,
    5665            "guidance": (
    5666                "The durable task queue has no open or active branch. Use record_tasks to open the next concrete "
    5667                "branch before doing more research or execution, or report_update if the operator needs a checkpoint."
    5668            ),
    5669        }
    5670        return result, f"blocked {name}; no open task branch"
    5671
    5672    known_bad_source = _known_bad_source_for_call(name, args, job)
    5673    if known_bad_source:
    5674        result = {
    5675            "success": False,
    5676            "error": "known bad source blocked",
    5677            "blocked_tool": name,
    5678            "blocked_arguments": args,
    5679            "known_bad_source": known_bad_source,
    5680            "guidance": (
    5681                "The source ledger marks this source as blocked or low-yield for this job. "
    5682                "Choose a different source, or record a fresh operator reason before retrying it."
    5683            ),
    5684        }
    5685        return result, f"blocked {name}; known bad source {known_bad_source.get('source')}"
    5686
    5687    if name == "web_search":
    5688        similar_step = _similar_recent_search(args, recent_steps)
    5689        if similar_step:
    5690            result = {
    5691                "success": False,
    5692                "error": "similar search query blocked",
    5693                "blocked_tool": name,
    5694                "blocked_arguments": args,
    5695                "previous_step": similar_step["id"],
    5696                "guidance": "Use an existing result URL, extract a page, or search a clearly different topic/location/source.",
    5697            }
    5698            return result, f"blocked similar web_search; previous step #{similar_step['step_no']}"
    5699        streak = _recent_search_streak(recent_steps)
    5700        if streak >= 3:
    5701            result = {
    5702                "success": False,
    5703                "error": "search loop blocked",
    5704                "blocked_tool": name,
    5705                "blocked_arguments": args,
    5706                "recent_search_streak": streak,
    5707                "guidance": "Stop searching. Extract or open one of the prior results, then write an artifact.",
    5708            }
    5709            return result, f"blocked web_search after {streak} consecutive searches"
    5710
    5711    if name == "search_artifacts":
    5712        similar_step = _similar_recent_query_tool("search_artifacts", args, recent_steps)
    5713        if similar_step:
    5714            result = {
    5715                "success": False,
    5716                "error": "similar artifact search blocked",
    5717                "blocked_tool": name,
    5718                "blocked_arguments": args,
    5719                "previous_step": similar_step["id"],
    5720                "guidance": (
    5721                    "Use a returned artifact, record what the prior artifact searches proved, "
    5722                    "or create the next concrete task instead of searching saved outputs again."
    5723                ),
    5724            }
    5725            return result, f"blocked similar search_artifacts; previous step #{similar_step['step_no']}"
    5726        streak = _recent_tool_streak(recent_steps, "search_artifacts")
    5727        if streak >= 3:
    5728            result = {
    5729                "success": False,
    5730                "error": "artifact search loop blocked",
    5731                "blocked_tool": name,
    5732                "blocked_arguments": args,
    5733                "recent_artifact_search_streak": streak,
    5734                "guidance": (
    5735                    "Stop searching saved outputs. Read a specific returned artifact, update tasks/findings/lessons, "
    5736                    "or write the next report artifact from already-read evidence."
    5737                ),
    5738            }
    5739            return result, f"blocked search_artifacts after {streak} consecutive artifact searches"
    5740
    5741    return None
    5742
    5743
    5744def _error_result(exc: Exception) -> dict[str, Any]:
    5745    result: dict[str, Any] = {
    5746        "success": False,
    5747        "error": str(exc),
    5748        "error_type": type(exc).__name__,
    5749    }
    5750    if isinstance(exc, LLMResponseError) and exc.payload:
    5751        result["provider_payload"] = exc.payload
    5752    return result
    5753
    5754
    5755def _hard_llm_provider_failure_note(exc: Exception) -> str:
    5756    return provider_action_required_note(exc)
    5757
    5758
    5759def _max_step_no(steps: list[dict[str, Any]]) -> int:
    5760    return max((int(step.get("step_no") or 0) for step in steps), default=0)
    5761
    5762
    5763def _should_reflect(job: dict[str, Any], recent_steps: list[dict[str, Any]]) -> bool:
    5764    if not recent_steps:
    5765        return False
    5766    if recent_steps[-1].get("kind") == "reflection":
    5767        return False
    5768    step_no = _max_step_no(recent_steps)
    5769    if step_no == 0 or step_no % REFLECTION_INTERVAL_STEPS != 0:
    5770        return False
    5771    reflections = _metadata_list(job, "reflections")
    5772    if not reflections:
    5773        return True
    5774    last_reflected = 0
    5775    metadata = reflections[-1].get("metadata") if isinstance(reflections[-1].get("metadata"), dict) else {}
    5776    if isinstance(metadata.get("through_step"), int):
    5777        last_reflected = metadata["through_step"]
    5778    return step_no > last_reflected
    5779
    5780
    5781def _lesson_already_recorded(job: dict[str, Any], lesson: str, *, category: str) -> bool:
    5782    text = " ".join(str(lesson or "").split())
    5783    wanted_category = str(category or "memory").strip().lower() or "memory"
    5784    return any(
    5785        str(entry.get("category") or "memory").strip().lower() == wanted_category
    5786        and " ".join(str(entry.get("lesson") or "").split()) == text
    5787        for entry in _metadata_list(job, "lessons")
    5788    )
    5789
    5790
    5791def _reflection_strategy(
    5792    *,
    5793    failures: list[dict[str, Any]],
    5794    findings: list[Any],
    5795    sources: list[Any],
    5796    tasks: list[Any],
    5797    measured_experiments: list[dict[str, Any]],
    5798    pending_measurement: bool,
    5799    validating_milestones: list[dict[str, Any]],
    5800    active_operator_messages: list[dict[str, Any]],
    5801) -> str:
    5802    if pending_measurement:
    5803        return "Resolve the pending measurement obligation before expanding research, outputs, or branch work."
    5804    if active_operator_messages:
    5805        return "Incorporate or supersede active operator context before choosing new autonomous branches."
    5806    if validating_milestones:
    5807        return "Validate the current roadmap milestone from evidence before adding more milestone scope."
    5808    if measured_experiments:
    5809        return "Continue from the best measured result; reject or pivot branches that do not improve the active metric."
    5810    yielded_sources = [
    5811        source
    5812        for source in sources
    5813        if isinstance(source, dict)
    5814        and (_as_int(source.get("yield_count")) > 0 or _as_float(source.get("usefulness_score")) >= 0.8)
    5815    ]
    5816    if len(sources) >= SOURCE_YIELD_MIN_SOURCES and len(findings) + len(yielded_sources) < max(2, len(sources) // 8):
    5817        return "Distill gathered sources into durable findings or source yield decisions before collecting more sources."
    5818    if failures:
    5819        return "Classify blocked or failed steps into durable task, source, experiment, or lesson outcomes before retrying."
    5820    open_tasks = [
    5821        task
    5822        for task in tasks
    5823        if isinstance(task, dict)
    5824        and str(task.get("status") or "open").lower() in {"open", "active", "blocked"}
    5825    ]
    5826    if open_tasks:
    5827        return "Execute or resolve the highest-priority open task before creating more task branches."
    5828    return "Choose the next branch from durable evidence, then record the result as findings, tasks, experiments, sources, or memory."
    5829
    5830
    5831def _claim_operator_queue(db: AgentDB, job_id: str) -> list[dict[str, Any]]:
    5832    steering = db.claim_operator_messages(job_id, modes=("steer",), limit=1)
    5833    if steering:
    5834        return steering
    5835    return db.claim_operator_messages(job_id, modes=("follow_up",), limit=1)
    5836
    5837
    5838def _emit_loop_start(db: AgentDB, job_id: str, run_id: str) -> None:
    5839    db.append_event(
    5840        job_id,
    5841        event_type="loop",
    5842        title="agent_start",
    5843        ref_table="job_runs",
    5844        ref_id=run_id,
    5845        metadata={"run_id": run_id},
    5846    )
    5847    db.append_event(
    5848        job_id,
    5849        event_type="loop",
    5850        title="turn_start",
    5851        ref_table="job_runs",
    5852        ref_id=run_id,
    5853        metadata={"run_id": run_id},
    5854    )
    5855
    5856
    5857def _emit_assistant_message_event(
    5858    db: AgentDB,
    5859    job_id: str,
    5860    run_id: str,
    5861    response: LLMResponse,
    5862    *,
    5863    messages: list[dict[str, Any]],
    5864    context_length: int,
    5865    duration_seconds: float | None = None,
    5866) -> dict[str, Any]:
    5867    if response.tool_calls:
    5868        body = ", ".join(call.name for call in response.tool_calls)
    5869        metadata = {"run_id": run_id, "tool_calls": [call.name for call in response.tool_calls]}
    5870    else:
    5871        body = response.content[:1000]
    5872        metadata = {"run_id": run_id, "tool_calls": []}
    5873    metadata["usage"] = turn_usage_metadata(response, messages=messages, context_length=context_length)
    5874    if duration_seconds is not None:
    5875        metadata["duration_seconds"] = round(max(0.0, float(duration_seconds)), 3)
    5876    if response.model:
    5877        metadata["model"] = response.model
    5878    if response.response_id:
    5879        metadata["response_id"] = response.response_id
    5880    db.append_event(
    5881        job_id,
    5882        event_type="loop",
    5883        title="message_end",
    5884        body=body,
    5885        ref_table="job_runs",
    5886        ref_id=run_id,
    5887        metadata=metadata,
    5888    )
    5889    return metadata["usage"]
    5890
    5891
    5892def _emit_loop_end(
    5893    db: AgentDB,
    5894    job_id: str,
    5895    run_id: str,
    5896    *,
    5897    status: str,
    5898    step_id: str | None = None,
    5899    tool_name: str | None = None,
    5900    detail: str = "",
    5901) -> None:
    5902    metadata = {"run_id": run_id, "status": status, "step_id": step_id or "", "tool": tool_name or ""}
    5903    db.append_event(
    5904        job_id,
    5905        event_type="loop",
    5906        title="turn_end",
    5907        body=detail[:1000],
    5908        ref_table="job_runs",
    5909        ref_id=run_id,
    5910        metadata=metadata,
    5911    )
    5912    db.append_event(
    5913        job_id,
    5914        event_type="loop",
    5915        title="agent_end",
    5916        body=status,
    5917        ref_table="job_runs",
    5918        ref_id=run_id,
    5919        metadata=metadata,
    5920    )
    5921
    5922
    5923def _run_reflection_step(
    5924    job: dict[str, Any],
    5925    recent_steps: list[dict[str, Any]],
    5926    *,
    5927    db: AgentDB,
    5928    job_id: str,
    5929    run_id: str,
    5930) -> StepExecution:
    5931    step_id = db.add_step(job_id=job_id, run_id=run_id, kind="reflection", tool_name="reflect")
    5932    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    5933    findings = metadata.get("finding_ledger") if isinstance(metadata.get("finding_ledger"), list) else []
    5934    sources = metadata.get("source_ledger") if isinstance(metadata.get("source_ledger"), list) else []
    5935    tasks = metadata.get("task_queue") if isinstance(metadata.get("task_queue"), list) else []
    5936    experiments = metadata.get("experiment_ledger") if isinstance(metadata.get("experiment_ledger"), list) else []
    5937    lessons = metadata.get("lessons") if isinstance(metadata.get("lessons"), list) else []
    5938    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
    5939    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
    5940    validating_milestones = [
    5941        milestone for milestone in milestones
    5942        if isinstance(milestone, dict)
    5943        and (
    5944            str(milestone.get("status") or "planned") == "validating"
    5945            or str(milestone.get("validation_status") or "not_started") == "pending"
    5946        )
    5947    ]
    5948    operator_messages = metadata.get("operator_messages") if isinstance(metadata.get("operator_messages"), list) else []
    5949    active_operator_messages = [
    5950        entry for entry in operator_messages
    5951        if isinstance(entry, dict)
    5952        and str(entry.get("mode") or "steer") in {"steer", "follow_up"}
    5953        and not entry.get("acknowledged_at")
    5954        and not entry.get("superseded_at")
    5955    ]
    5956    pending_measurement = _pending_measurement_obligation(job)
    5957    artifacts = db.list_artifacts(job_id, limit=12)
    5958    failures = [step for step in recent_steps[-REFLECTION_INTERVAL_STEPS:] if step.get("status") == "failed" or step.get("status") == "blocked"]
    5959    step_no = _max_step_no(recent_steps)
    5960    finding_batches = [artifact for artifact in artifacts if "finding" in str(artifact.get("title") or artifact.get("summary") or "").lower()]
    5961    best_sources = sorted(
    5962        [
    5963            source for source in sources
    5964            if isinstance(source, dict)
    5965            and (
    5966                _as_int(source.get("yield_count")) > 0
    5967                or _as_float(source.get("usefulness_score")) >= 0.2
    5968            )
    5969            and _as_int(source.get("fail_count")) <= max(0, _as_int(source.get("yield_count")))
    5970        ],
    5971        key=lambda source: (_as_float(source.get("usefulness_score")), _as_int(source.get("yield_count"))),
    5972        reverse=True,
    5973    )[:3]
    5974    source_text = ", ".join(str(source.get("source") or "") for source in best_sources) or "no high-yield source yet"
    5975    measured_experiments = [experiment for experiment in experiments if isinstance(experiment, dict) and experiment.get("metric_value") is not None]
    5976    best_experiments = [experiment for experiment in measured_experiments if experiment.get("best_observed")]
    5977    best_experiment_text = "no measured experiment yet"
    5978    if best_experiments:
    5979        best_experiment_text = "; ".join(
    5980            f"{experiment.get('title')} " + format_metric_value(
    5981                experiment.get("metric_name") or "metric",
    5982                experiment.get("metric_value"),
    5983                experiment.get("metric_unit") or "",
    5984            )
    5985            for experiment in best_experiments[-3:]
    5986        )
    5987    summary = (
    5988        f"Reflection through step #{step_no}: {len(findings)} findings, {len(sources)} sources, "
    5989        f"{len(tasks)} tasks, {len(experiments)} experiments, {len(milestones)} roadmap milestones, "
    5990        f"{len(lessons)} lessons, "
    5991        f"{len(active_operator_messages)} active operator messages, "
    5992        f"{len(finding_batches)} recent finding artifacts, {len(failures)} recent blocked/failed steps. "
    5993        f"Best source direction: {source_text}. Best measured result: {best_experiment_text}."
    5994        + (f" Roadmap '{roadmap.get('title')}' has {len(validating_milestones)} milestone(s) needing validation." if roadmap else "")
    5995        + (" Pending measurement obligation needs resolution." if pending_measurement else "")
    5996    )
    5997    strategy = _reflection_strategy(
    5998        failures=failures,
    5999        findings=findings,
    6000        sources=sources,
    6001        tasks=tasks,
    6002        measured_experiments=measured_experiments,
    6003        pending_measurement=bool(pending_measurement),
    6004        validating_milestones=validating_milestones,
    6005        active_operator_messages=active_operator_messages,
    6006    )
    6007    reflection = db.append_reflection(
    6008        job_id,
    6009        summary,
    6010        strategy=strategy,
    6011        metadata={
    6012            "through_step": step_no,
    6013            "finding_count": len(findings),
    6014            "source_count": len(sources),
    6015            "task_count": len(tasks),
    6016            "experiment_count": len(experiments),
    6017            "roadmap_milestone_count": len(milestones),
    6018            "roadmap_validation_needed_count": len(validating_milestones),
    6019            "measured_experiment_count": len(measured_experiments),
    6020            "active_operator_message_count": len(active_operator_messages),
    6021            "pending_measurement_obligation": bool(pending_measurement),
    6022        },
    6023    )
    6024    lesson = None
    6025    if not _lesson_already_recorded(job, strategy, category="strategy"):
    6026        lesson = db.append_lesson(
    6027            job_id,
    6028            strategy,
    6029            category="strategy",
    6030            confidence=0.75,
    6031            metadata={"source": "reflection", "through_step": step_no},
    6032        )
    6033    db.append_agent_update(job_id, summary, category="plan", metadata={"reflection": reflection})
    6034    result = {"success": True, "reflection": reflection, "lesson_recorded": bool(lesson)}
    6035    db.finish_step(step_id, status="completed", summary=summary, output_data=result)
    6036    db.finish_run(run_id, "completed")
    6037    _emit_loop_end(db, job_id, run_id, status="completed", step_id=step_id, tool_name="reflect", detail=summary)
    6038    refresh_memory_index(db, job_id)
    6039    return StepExecution(job_id=job_id, run_id=run_id, step_id=step_id, tool_name="reflect", status="completed", result=result)
    6040
    6041
    6042def _run_guard_recovery_step(
    6043    context: dict[str, Any],
    6044    *,
    6045    db: AgentDB,
    6046    job_id: str,
    6047    run_id: str,
    6048) -> StepExecution:
    6049    error = str(context.get("error") or "recoverable guard")
    6050    checkpoint_accounting = error == "evidence checkpoint accounting required"
    6051    task_queue_saturated = error == "task queue saturated"
    6052    task_goal = "Convert the repeated guard block into durable progress before retrying the blocked action."
    6053    acceptance = (
    6054        "Use record_tasks, record_findings, record_source, record_experiment, or record_lesson to state what "
    6055        "changed, what branch is rejected, or what concrete branch should run next."
    6056    )
    6057    stall_behavior = "If the same guard appears again, pivot to a different branch or record the branch as blocked."
    6058    if checkpoint_accounting:
    6059        task_goal = (
    6060            "Account for the already-read evidence checkpoint as durable progress, a rejected branch, "
    6061            "or a blocked branch before continuing."
    6062        )
    6063        acceptance = (
    6064            "Use record_findings, record_source, record_experiment, record_tasks, record_roadmap, "
    6065            "record_milestone_validation, or record_lesson to state exactly what the checkpoint proved, "
    6066            "invalidated, changed, or failed to provide. Do not read the same checkpoint again."
    6067        )
    6068        stall_behavior = (
    6069            "If the checkpoint cannot produce durable progress, record a lesson or task that names the blocker "
    6070            "and choose a different branch."
    6071        )
    6072    step_id = db.add_step(job_id=job_id, run_id=run_id, kind="recovery", tool_name="guard_recovery")
    6073    if task_queue_saturated:
    6074        task_queue = context.get("task_queue") if isinstance(context.get("task_queue"), dict) else {}
    6075        lesson = db.append_lesson(
    6076            job_id,
    6077            (
    6078                f"Repeated task queue saturation occurred {context.get('count')} times. "
    6079                "Do not open guard-recovery tasks for saturation; consolidate, complete, block, or skip existing branches "
    6080                "before adding new work."
    6081            ),
    6082            category="strategy",
    6083            confidence=0.85,
    6084            metadata={"guard_recovery": context},
    6085        )
    6086        db.update_job_metadata(
    6087            job_id,
    6088            {
    6089                "task_backlog_pressure": {
    6090                    "detected_at": datetime.now(timezone.utc).isoformat(),
    6091                    "guard_recovery": context,
    6092                    "reason": task_queue.get("reason") or "task queue saturated",
    6093                    "open_count": task_queue.get("open_count"),
    6094                    "total_count": task_queue.get("total_count"),
    6095                }
    6096            },
    6097        )
    6098        message = (
    6099            f"Guard recovery recorded task queue saturation from step #{context.get('first_step_no')} "
    6100            f"to #{context.get('latest_step_no')}; no new task was opened."
    6101        )
    6102        update = db.append_agent_update(
    6103            job_id,
    6104            message,
    6105            category="blocked",
    6106            metadata={"guard_recovery": context, "lesson_key": lesson.get("key"), "task_queue_saturation": True},
    6107        )
    6108        result = {
    6109            "success": True,
    6110            "guard_recovery": context,
    6111            "lesson": lesson,
    6112            "update": update,
    6113            "task_opened": False,
    6114        }
    6115        db.finish_step(step_id, status="completed", summary=message, output_data=result)
    6116        finished_step = _step_by_id(db, job_id, step_id)
    6117        _resolve_evidence_checkpoint(
    6118            db=db,
    6119            job_id=job_id,
    6120            tool_name="guard_recovery",
    6121            step=finished_step,
    6122        )
    6123        db.finish_run(run_id, "completed")
    6124        _emit_loop_end(db, job_id, run_id, status="completed", step_id=step_id, tool_name="guard_recovery", detail=message)
    6125        refresh_memory_index(db, job_id)
    6126        return StepExecution(job_id=job_id, run_id=run_id, step_id=step_id, tool_name="guard_recovery", status="completed", result=result)
    6127
    6128    lesson = db.append_lesson(
    6129        job_id,
    6130        (
    6131            f"Repeated guard block '{error}' occurred {context.get('count')} times. "
    6132            + (
    6133                "The checkpoint has already been read; do not reread it. Account for the evidence with a durable "
    6134                "record or reject/block that branch before continuing."
    6135                if checkpoint_accounting
    6136                else "Do not retry the same blocked tool pattern; update durable progress state, create a new branch, "
    6137                "or explicitly reject the branch before continuing."
    6138            )
    6139        ),
    6140        category="strategy",
    6141        confidence=0.75,
    6142        metadata={"guard_recovery": context},
    6143    )
    6144    task = db.append_task_record(
    6145        job_id,
    6146        title=f"Resolve guard: {error}",
    6147        status="open",
    6148        priority=9,
    6149        goal=task_goal,
    6150        output_contract="decision",
    6151        acceptance_criteria=acceptance,
    6152        evidence_needed=f"Recent blocked tools: {', '.join(context.get('blocked_tools') or [])}",
    6153        stall_behavior=stall_behavior,
    6154        metadata={"guard_recovery": context, "resolves_evidence_checkpoint": checkpoint_accounting},
    6155    )
    6156    message = (
    6157        f"Guard recovery opened a task after repeated '{error}' blocks "
    6158        f"from step #{context.get('first_step_no')} to #{context.get('latest_step_no')}."
    6159    )
    6160    update = db.append_agent_update(
    6161        job_id,
    6162        message,
    6163        category="blocked",
    6164        metadata={"guard_recovery": context, "task_key": task.get("key"), "lesson_key": lesson.get("key")},
    6165    )
    6166    result = {
    6167        "success": True,
    6168        "guard_recovery": context,
    6169        "lesson": lesson,
    6170        "task": task,
    6171        "update": update,
    6172    }
    6173    db.finish_step(step_id, status="completed", summary=message, output_data=result)
    6174    finished_step = _step_by_id(db, job_id, step_id)
    6175    _resolve_evidence_checkpoint(
    6176        db=db,
    6177        job_id=job_id,
    6178        tool_name="guard_recovery",
    6179        step=finished_step,
    6180    )
    6181    db.finish_run(run_id, "completed")
    6182    _emit_loop_end(db, job_id, run_id, status="completed", step_id=step_id, tool_name="guard_recovery", detail=message)
    6183    refresh_memory_index(db, job_id)
    6184    return StepExecution(job_id=job_id, run_id=run_id, step_id=step_id, tool_name="guard_recovery", status="completed", result=result)
    6185
    6186
    6187def _usage_budget_limit_context(config: AppConfig, usage: dict[str, Any]) -> dict[str, Any] | None:
    6188    limit = config.runtime.max_job_cost_usd
    6189    if limit is None or limit <= 0 or not bool(usage.get("has_cost")):
    6190        return None
    6191    cost = _as_float(usage.get("cost"))
    6192    if cost < float(limit):
    6193        return None
    6194    return {
    6195        "limit": float(limit),
    6196        "cost": cost,
    6197        "calls": _as_int(usage.get("calls")),
    6198        "total_tokens": _as_int(usage.get("total_tokens")),
    6199        "prompt_tokens": _as_int(usage.get("prompt_tokens")),
    6200        "completion_tokens": _as_int(usage.get("completion_tokens")),
    6201    }
    6202
    6203
    6204def _run_usage_budget_limit_step(
    6205    context: dict[str, Any],
    6206    *,
    6207    db: AgentDB,
    6208    job_id: str,
    6209    run_id: str,
    6210) -> StepExecution:
    6211    limit = float(context.get("limit") or 0.0)
    6212    cost = float(context.get("cost") or 0.0)
    6213    message = (
    6214        f"Paused job: configured model cost limit ${limit:g} reached "
    6215        f"(current cost ${cost:.4f}, {context.get('calls')} model calls, "
    6216        f"{_compact_usage_tokens(context.get('total_tokens'))} tokens). "
    6217        "Raise the limit, switch model/provider, or resume after deciding the budget is acceptable."
    6218    )
    6219    metadata = {
    6220        "reason": "usage_budget_limit",
    6221        "usage_budget_limit": context,
    6222        "last_note": message,
    6223        "usage_budget_blocked_at": datetime.now(timezone.utc).isoformat(),
    6224    }
    6225    db.update_job_status(job_id, "paused", metadata_patch=metadata)
    6226    step_id = db.add_step(job_id=job_id, run_id=run_id, kind="recovery", tool_name="budget_limit")
    6227    result = {"success": True, "job_id": job_id, "paused": True, **context}
    6228    db.append_agent_update(
    6229        job_id,
    6230        message,
    6231        category="blocked",
    6232        metadata={"reason": "usage_budget_limit", "usage_budget_limit": context},
    6233    )
    6234    db.finish_step(step_id, status="completed", summary=message, output_data=result)
    6235    db.finish_run(run_id, "completed")
    6236    _emit_loop_end(db, job_id, run_id, status="completed", step_id=step_id, tool_name="budget_limit", detail=message)
    6237    refresh_memory_index(db, job_id)
    6238    return StepExecution(job_id=job_id, run_id=run_id, step_id=step_id, tool_name="budget_limit", status="completed", result=result)
    6239
    6240
    6241def _compact_usage_tokens(value: object) -> str:
    6242    number = _as_int(value)
    6243    if number >= 1_000_000:
    6244        return f"{number / 1_000_000:.1f}M"
    6245    if number >= 1_000:
    6246        return f"{number / 1_000:.1f}K"
    6247    return str(number)
    6248
    6249
    6250def _evidence_checkpoint_content(evidence_step: dict[str, Any]) -> str:
    6251    output = evidence_step.get("output") if isinstance(evidence_step.get("output"), dict) else {}
    6252    input_data = evidence_step.get("input") if isinstance(evidence_step.get("input"), dict) else {}
    6253    observation = _observation_for_prompt(evidence_step.get("tool_name"), output)
    6254    return "\n\n".join([
    6255        "# Auto Evidence Checkpoint",
    6256        f"Source step: #{evidence_step.get('step_no')} {evidence_step.get('tool_name') or evidence_step.get('kind')}",
    6257        f"Summary: {evidence_step.get('summary') or ''}",
    6258        f"Arguments:\n```json\n{json.dumps(input_data.get('arguments') or {}, ensure_ascii=False, indent=2)[:3000]}\n```",
    6259        f"Observed:\n{observation or 'No compact observation available.'}",
    6260        f"Raw output excerpt:\n```json\n{json.dumps(output, ensure_ascii=False, indent=2)[:9000]}\n```",
    6261    ])
    6262
    6263
    6264def _auto_persist_evidence(
    6265    *,
    6266    db: AgentDB,
    6267    artifacts: ArtifactStore,
    6268    job_id: str,
    6269    run_id: str,
    6270    step_id: str,
    6271    blocked_tool: str,
    6272    evidence_step: dict[str, Any],
    6273) -> dict[str, Any]:
    6274    stored = artifacts.write_text(
    6275        job_id=job_id,
    6276        run_id=run_id,
    6277        step_id=step_id,
    6278        title=f"Auto Evidence Checkpoint after step {evidence_step.get('step_no')}",
    6279        summary=f"Auto-saved evidence before allowing more research; blocked tool was {blocked_tool}.",
    6280        content=_evidence_checkpoint_content(evidence_step),
    6281        artifact_type="text",
    6282        metadata={"auto_checkpoint": True, "evidence_step": evidence_step.get("id"), "blocked_tool": blocked_tool},
    6283    )
    6284    lesson = db.append_lesson(
    6285        job_id,
    6286        (
    6287            f"Evidence from step #{evidence_step.get('step_no')} must be persisted before more research; "
    6288            f"auto-saved checkpoint {stored.id} after blocked {blocked_tool}."
    6289        ),
    6290        category="mistake",
    6291        confidence=0.8,
    6292        metadata={"artifact_id": stored.id, "blocked_tool": blocked_tool},
    6293    )
    6294    db.append_agent_update(
    6295        job_id,
    6296        f"Auto-saved evidence checkpoint {stored.id} after the model tried {blocked_tool} before persisting evidence.",
    6297        category="blocked",
    6298        metadata={"artifact_id": stored.id, "blocked_tool": blocked_tool},
    6299    )
    6300    db.update_job_metadata(
    6301        job_id,
    6302        {
    6303            "pending_evidence_checkpoint": {
    6304                "artifact_id": stored.id,
    6305                "title": stored.title or f"Auto Evidence Checkpoint after step {evidence_step.get('step_no')}",
    6306                "path": str(stored.path),
    6307                "created_at": datetime.now(timezone.utc).isoformat(),
    6308                "checkpoint_step_id": step_id,
    6309                "evidence_step": evidence_step.get("id"),
    6310                "evidence_step_no": evidence_step.get("step_no"),
    6311                "evidence_tool": evidence_step.get("tool_name") or evidence_step.get("kind"),
    6312                "blocked_tool": blocked_tool,
    6313            }
    6314        },
    6315    )
    6316    return {"artifact_id": stored.id, "path": str(stored.path), "lesson": lesson}
    6317
    6318
    6319def _auto_record_grounding_block_lesson(*, db: AgentDB, job_id: str, result: dict[str, Any]) -> None:
    6320    if result.get("error") != "evidence grounding required":
    6321        return
    6322    grounding = result.get("evidence_grounding") if isinstance(result.get("evidence_grounding"), dict) else {}
    6323    unsupported = grounding.get("unsupported_tokens") if isinstance(grounding.get("unsupported_tokens"), list) else []
    6324    unsupported = [str(token) for token in unsupported if str(token).strip()]
    6325    if not unsupported:
    6326        return
    6327    cited_steps = grounding.get("cited_steps") if isinstance(grounding.get("cited_steps"), list) else []
    6328    blocked_tool = str(result.get("blocked_tool") or "")
    6329    fingerprint = "|".join([blocked_tool, ",".join(unsupported[:8]), ",".join(str(step) for step in cited_steps[:8])])
    6330    job = db.get_job(job_id)
    6331    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    6332    seen = metadata.get("grounding_block_fingerprints") if isinstance(metadata.get("grounding_block_fingerprints"), list) else []
    6333    if fingerprint in seen:
    6334        return
    6335    db.append_lesson(
    6336        job_id,
    6337        (
    6338            f"Evidence grounding rejected unsupported concrete tokens for {blocked_tool or 'a durable record'}: "
    6339            f"{', '.join(unsupported[:8])}. Treat matching prior ledger, artifact, or memory claims as stale until "
    6340            "they are re-verified from the cited evidence."
    6341        ),
    6342        category="mistake",
    6343        confidence=0.9,
    6344        metadata={"evidence_grounding": grounding, "blocked_tool": blocked_tool},
    6345    )
    6346    metadata_patch: dict[str, Any] = {"grounding_block_fingerprints": (seen + [fingerprint])[-100:]}
    6347    stale_tokens = _stale_claim_tokens_from_unsupported(
    6348        unsupported,
    6349        reference_text=" ".join(str(job.get(key) or "") for key in ("title", "objective", "kind")),
    6350    )
    6351    if stale_tokens:
    6352        existing_tokens = [
    6353            str(token)
    6354            for token in metadata.get("unsupported_claim_tokens", [])
    6355            if str(token).strip()
    6356        ] if isinstance(metadata.get("unsupported_claim_tokens"), list) else []
    6357        combined: list[str] = []
    6358        combined_seen: set[str] = set()
    6359        for token in existing_tokens + stale_tokens:
    6360            key = token.lower()
    6361            if key in combined_seen:
    6362                continue
    6363            combined_seen.add(key)
    6364            combined.append(token)
    6365        metadata_patch["unsupported_claim_tokens"] = combined[-80:]
    6366    db.update_job_metadata(job_id, metadata_patch)
    6367
    6368
    6369def _mark_evidence_checkpoint_read(
    6370    *,
    6371    db: AgentDB,
    6372    job_id: str,
    6373    tool_name: str,
    6374    args: dict[str, Any],
    6375    step: dict[str, Any] | None,
    6376) -> None:
    6377    if tool_name != "read_artifact":
    6378        return
    6379    job = db.get_job(job_id)
    6380    pending = _pending_evidence_checkpoint(job)
    6381    if not pending or pending.get("read_at"):
    6382        return
    6383    if not _read_artifact_call_matches_checkpoint(
    6384        args,
    6385        artifact_id=str(pending.get("artifact_id") or ""),
    6386        artifact_title=str(pending.get("title") or ""),
    6387    ):
    6388        return
    6389    updated = dict(pending)
    6390    updated["read_at"] = datetime.now(timezone.utc).isoformat()
    6391    if step:
    6392        updated["read_step_id"] = step.get("id")
    6393        updated["read_step_no"] = step.get("step_no")
    6394    db.update_job_metadata(job_id, {"pending_evidence_checkpoint": updated})
    6395    db.append_agent_update(
    6396        job_id,
    6397        f"Read evidence checkpoint {pending.get('artifact_id')}; durable accounting is required next.",
    6398        category="blocked",
    6399        metadata={"pending_evidence_checkpoint": updated},
    6400    )
    6401
    6402
    6403def _resolve_evidence_checkpoint(
    6404    *,
    6405    db: AgentDB,
    6406    job_id: str,
    6407    tool_name: str,
    6408    step: dict[str, Any] | None,
    6409) -> None:
    6410    if tool_name not in EVIDENCE_CHECKPOINT_RESOLUTION_TOOLS and tool_name != "guard_recovery":
    6411        return
    6412    job = db.get_job(job_id)
    6413    pending = _pending_evidence_checkpoint(job)
    6414    if not pending:
    6415        return
    6416    updated = dict(pending)
    6417    updated["resolved_at"] = datetime.now(timezone.utc).isoformat()
    6418    updated["resolved_by_tool"] = tool_name
    6419    if step:
    6420        updated["resolved_by_step_id"] = step.get("id")
    6421        updated["resolved_by_step_no"] = step.get("step_no")
    6422    db.update_job_metadata(job_id, {"pending_evidence_checkpoint": updated})
    6423    db.append_agent_update(
    6424        job_id,
    6425        f"Evidence checkpoint {pending.get('artifact_id')} accounted for with {tool_name}.",
    6426        category="progress",
    6427        metadata={"pending_evidence_checkpoint": updated},
    6428    )
    6429
    6430
    6431def _auto_record_blocked_source(
    6432    *,
    6433    db: AgentDB,
    6434    job_id: str,
    6435    context: dict[str, Any],
    6436    blocked_tool: str,
    6437) -> dict[str, Any]:
    6438    source = str(context.get("url") or context.get("title") or "unknown blocked browser source")
    6439    reason = str(context.get("reason") or "anti-bot challenge")
    6440    record = db.append_source_record(
    6441        job_id,
    6442        source,
    6443        source_type="blocked_browser_source",
    6444        usefulness_score=0.02,
    6445        fail_count_delta=1,
    6446        warnings=[reason],
    6447        outcome=f"blocked by {reason}; pivot to an alternate source for the current objective",
    6448        metadata={"blocked_tool": blocked_tool, "source_step": context.get("step_id")},
    6449    )
    6450    lesson = None
    6451    if int(record.get("fail_count") or 0) <= 2:
    6452        lesson = db.append_lesson(
    6453            job_id,
    6454            "Blocked, CAPTCHA, login, paywall, or anti-bot pages are not usable evidence for any long-running task; record the source outcome and pivot instead of repeating browser actions.",
    6455            category="source_quality",
    6456            confidence=0.9,
    6457            metadata={"source": source, "blocked_tool": blocked_tool},
    6458        )
    6459    db.append_agent_update(
    6460        job_id,
    6461        f"Blocked source guard: current source is {reason}; pivoting away instead of looping.",
    6462        category="blocked",
    6463        metadata={"source": source, "blocked_tool": blocked_tool, "reason": reason},
    6464    )
    6465    return {"source": record, "lesson": lesson}
    6466
    6467
    6468def _auto_record_tool_source_quality(
    6469    *,
    6470    db: AgentDB,
    6471    job_id: str,
    6472    tool_name: str | None,
    6473    result: dict[str, Any],
    6474) -> None:
    6475    if tool_name == "web_search":
    6476        query = str(result.get("query") or "").strip()
    6477        results = result.get("results") if isinstance(result.get("results"), list) else []
    6478        for item in results[:8]:
    6479            if not isinstance(item, dict):
    6480                continue
    6481            url = str(item.get("url") or "").strip()
    6482            if not url:
    6483                continue
    6484            title = str(item.get("title") or "").strip()
    6485            db.append_source_record(
    6486                job_id,
    6487                url,
    6488                source_type="web_search",
    6489                usefulness_score=0.35,
    6490                yield_count=0,
    6491                outcome=f"search result for {query or 'query'}: {title[:160]}",
    6492                metadata={"auto_from_tool": "web_search", "query": query, "title": title},
    6493            )
    6494        return
    6495    if tool_name == "web_extract":
    6496        pages = result.get("pages") if isinstance(result.get("pages"), list) else []
    6497        for page in pages[:12]:
    6498            if not isinstance(page, dict):
    6499                continue
    6500            url = str(page.get("url") or "").strip()
    6501            if not url:
    6502                continue
    6503            text = str(page.get("text") or "")
    6504            error = str(page.get("error") or "")
    6505            if error:
    6506                db.append_source_record(
    6507                    job_id,
    6508                    url,
    6509                    source_type="web_extract",
    6510                    usefulness_score=0.1,
    6511                    fail_count_delta=1,
    6512                    warnings=[error[:180]],
    6513                    outcome=f"extract failed: {error[:180]}",
    6514                    metadata={"auto_from_tool": "web_extract"},
    6515                )
    6516                continue
    6517            score = 0.35
    6518            if len(text.strip()) >= 500:
    6519                score = 0.55
    6520            if len(text.strip()) >= 3000:
    6521                score = 0.7
    6522            db.append_source_record(
    6523                job_id,
    6524                url,
    6525                source_type="web_extract",
    6526                usefulness_score=score,
    6527                yield_count=0,
    6528                outcome=f"extracted {len(text.strip())} chars for possible use",
    6529                metadata={"auto_from_tool": "web_extract"},
    6530            )
    6531        return
    6532    if tool_name in {"browser_navigate", "browser_snapshot"}:
    6533        context = _browser_warning_context(result)
    6534        if not context:
    6535            return
    6536        result["source_warning"] = context["reason"]
    6537        result["source_url"] = context.get("url") or ""
    6538        _auto_record_blocked_source(db=db, job_id=job_id, context=context, blocked_tool=tool_name or "browser")
    6539
    6540
    6541def _auto_record_failed_shell_sources(
    6542    *,
    6543    db: AgentDB,
    6544    job_id: str,
    6545    args: dict[str, Any],
    6546    result: dict[str, Any],
    6547) -> None:
    6548    error_text = " ".join(str(result.get(key) or "") for key in ("error", "stderr", "stdout"))
    6549    lowered = error_text.lower()
    6550    if not any(
    6551        marker in lowered
    6552        for marker in (
    6553            "authentication",
    6554            "authorization",
    6555            "unauthorized",
    6556            "forbidden",
    6557            "http failure",
    6558            "http 401",
    6559            "http 403",
    6560            "401 unauthorized",
    6561            "403 forbidden",
    6562        )
    6563    ):
    6564        return
    6565    recorded: set[str] = set()
    6566    for url in _shell_guard_urls(str(args.get("command") or ""))[:3]:
    6567        candidates = [url]
    6568        family_url = _source_failure_family_url(url)
    6569        if family_url and not _same_source_url(family_url, url):
    6570            candidates.append(family_url)
    6571        for candidate in candidates:
    6572            if candidate.lower() in recorded:
    6573                continue
    6574            recorded.add(candidate.lower())
    6575            is_family = candidate != url
    6576            warning = (
    6577                "shell command reported authentication/authorization or HTTP failure for this source family"
    6578                if is_family
    6579                else "shell command reported authentication/authorization or HTTP failure"
    6580            )
    6581            outcome = (
    6582                f"Source family blocked after failed child URL {url}: {_clip_text(str(result.get('error') or error_text), 420)}"
    6583                if is_family
    6584                else _clip_text(str(result.get("error") or error_text), 500)
    6585            )
    6586            metadata = {"auto_from_tool": "shell_exec", "failure_kind": "auth_or_http"}
    6587            if is_family:
    6588                metadata.update({"source_family": True, "failed_child_url": url})
    6589            db.append_source_record(
    6590                job_id,
    6591                candidate,
    6592                source_type="shell_exec_family" if is_family else "shell_exec",
    6593                usefulness_score=0.01,
    6594                fail_count_delta=1,
    6595                warnings=[warning],
    6596                outcome=outcome,
    6597                metadata=metadata,
    6598            )
    6599
    6600
    6601def _auto_reconcile_artifact_tasks(
    6602    *,
    6603    db: AgentDB,
    6604    job_id: str,
    6605    args: dict[str, Any],
    6606    result: dict[str, Any],
    6607) -> list[dict[str, Any]]:
    6608    artifact_id = str(result.get("artifact_id") or "")
    6609    if not artifact_id:
    6610        return []
    6611    artifact_title = str(args.get("title") or "")
    6612    artifact_summary = str(args.get("summary") or "")
    6613    artifact_content = str(args.get("content") or "")
    6614    artifact_text = " ".join([artifact_title, artifact_summary, artifact_content[:4000]])
    6615    artifact_tokens = _text_tokens(artifact_text)
    6616    if len(artifact_tokens) < 2:
    6617        return []
    6618    job = db.get_job(job_id)
    6619    reconciled = []
    6620    for task in _metadata_list(job, "task_queue"):
    6621        status = str(task.get("status") or "open").strip().lower()
    6622        if status not in {"open", "active"}:
    6623            continue
    6624        contract = str(task.get("output_contract") or "").strip().lower()
    6625        if contract in {"experiment", "action", "monitor"}:
    6626            continue
    6627        task_text = " ".join(
    6628            str(task.get(key) or "")
    6629            for key in ("title", "goal", "acceptance_criteria", "evidence_needed", "source_hint")
    6630        )
    6631        if not _artifact_can_reconcile_task(
    6632            contract=contract,
    6633            task_text=task_text,
    6634            artifact_title=artifact_title,
    6635            artifact_summary=artifact_summary,
    6636        ):
    6637            continue
    6638        task_tokens = _text_tokens(task_text)
    6639        if len(task_tokens) < 2:
    6640            continue
    6641        overlap = task_tokens & artifact_tokens
    6642        needed = max(2, min(4, (len(task_tokens) + 1) // 2))
    6643        if len(overlap) < needed:
    6644            continue
    6645        updated = db.append_task_record(
    6646            job_id,
    6647            title=str(task.get("title") or ""),
    6648            status="done",
    6649            priority=_as_int(task.get("priority")),
    6650            goal=str(task.get("goal") or ""),
    6651            source_hint=str(task.get("source_hint") or ""),
    6652            result=f"Saved output {artifact_id}: {_clip_text(artifact_title or artifact_summary, 180)}",
    6653            parent=str(task.get("parent") or ""),
    6654            output_contract=contract,
    6655            acceptance_criteria=str(task.get("acceptance_criteria") or ""),
    6656            evidence_needed=str(task.get("evidence_needed") or ""),
    6657            stall_behavior=str(task.get("stall_behavior") or ""),
    6658            metadata={
    6659                **(task.get("metadata") if isinstance(task.get("metadata"), dict) else {}),
    6660                "auto_reconciled_from_artifact": artifact_id,
    6661                "matched_tokens": sorted(overlap)[:12],
    6662            },
    6663        )
    6664        reconciled.append(updated)
    6665    if reconciled:
    6666        titles = ", ".join(str(task.get("title") or "") for task in reconciled[:4])
    6667        db.append_agent_update(
    6668            job_id,
    6669            f"Task progress reconciled from saved output {artifact_id}: {titles}.",
    6670            category="plan",
    6671            metadata={"artifact_id": artifact_id, "task_count": len(reconciled)},
    6672        )
    6673    return reconciled
    6674
    6675
    6676def _auto_open_revision_task_for_deliverable(
    6677    *,
    6678    db: AgentDB,
    6679    job_id: str,
    6680    args: dict[str, Any],
    6681    result: dict[str, Any],
    6682) -> dict[str, Any] | None:
    6683    artifact_id = str(result.get("artifact_id") or "")
    6684    if not artifact_id:
    6685        return None
    6686    artifact_title = str(args.get("title") or "")
    6687    artifact_summary = str(args.get("summary") or "")
    6688    if not _artifact_can_reconcile_task(
    6689        contract="report",
    6690        task_text="review revise draft report deliverable",
    6691        artifact_title=artifact_title,
    6692        artifact_summary=artifact_summary,
    6693    ):
    6694        return None
    6695    job = db.get_job(job_id)
    6696    for task in _metadata_list(job, "task_queue"):
    6697        if str(task.get("status") or "open").strip().lower() not in {"open", "active"}:
    6698            continue
    6699        metadata = task.get("metadata") if isinstance(task.get("metadata"), dict) else {}
    6700        if metadata.get("revision_source_artifact_id") == artifact_id:
    6701            return None
    6702        if metadata.get("source") == "auto_revision_loop":
    6703            db.append_task_record(
    6704                job_id,
    6705                title=str(task.get("title") or ""),
    6706                status="skipped",
    6707                priority=_as_int(task.get("priority")),
    6708                goal=str(task.get("goal") or ""),
    6709                source_hint=str(task.get("source_hint") or ""),
    6710                result=f"Superseded by newer saved output {artifact_id}.",
    6711                parent=str(task.get("parent") or ""),
    6712                output_contract=str(task.get("output_contract") or ""),
    6713                acceptance_criteria=str(task.get("acceptance_criteria") or ""),
    6714                evidence_needed=str(task.get("evidence_needed") or ""),
    6715                stall_behavior=str(task.get("stall_behavior") or ""),
    6716                metadata={**metadata, "superseded_by_artifact_id": artifact_id},
    6717            )
    6718    task = db.append_task_record(
    6719        job_id,
    6720        title=f"Review and revise saved output {artifact_id}",
    6721        status="open",
    6722        priority=4,
    6723        goal="Use the latest saved deliverable as a baseline, check it against evidence and acceptance criteria, then improve it.",
    6724        source_hint=artifact_id,
    6725        output_contract="report",
    6726        acceptance_criteria="The saved output is reviewed and either revised, validated, or given concrete follow-up gaps.",
    6727        evidence_needed="Saved output, relevant evidence artifacts or files, and explicit gap/revision notes.",
    6728        stall_behavior="If no useful revision is possible, record why and open the next evidence, validation, or monitoring branch.",
    6729        metadata={
    6730            "source": "auto_revision_loop",
    6731            "revision_source_artifact_id": artifact_id,
    6732            "source_title": artifact_title,
    6733        },
    6734    )
    6735    db.append_agent_update(
    6736        job_id,
    6737        f"Opened revision branch for saved output {artifact_id}: {_clip_text(artifact_title or artifact_summary, 160)}.",
    6738        category="plan",
    6739        metadata={"artifact_id": artifact_id, "task_key": task.get("key"), "source": "auto_revision_loop"},
    6740    )
    6741    return task
    6742
    6743
    6744def _artifact_can_reconcile_task(
    6745    *,
    6746    contract: str,
    6747    task_text: str,
    6748    artifact_title: str,
    6749    artifact_summary: str,
    6750) -> bool:
    6751    contract = contract.strip().lower()
    6752    if contract in {"experiment", "action", "monitor"}:
    6753        return False
    6754    if contract == "research":
    6755        return True
    6756    artifact_text = f"{artifact_title} {artifact_summary}".lower()
    6757    task_lower = task_text.lower()
    6758    evidence_like = any(term in artifact_text for term in EVIDENCE_ARTIFACT_TERMS)
    6759    deliverable_like = any(term in artifact_text for term in DELIVERABLE_ARTIFACT_TERMS)
    6760    task_needs_deliverable_action = any(term in task_lower for term in TASK_DELIVERABLE_ACTION_TERMS)
    6761    if evidence_like:
    6762        return False
    6763    if task_needs_deliverable_action and not deliverable_like:
    6764        return False
    6765    return True
    6766
    6767
    6768def _auto_checkpoint_update(
    6769    *,
    6770    db: AgentDB,
    6771    job_id: str,
    6772    step_no: int,
    6773    tool_name: str | None,
    6774    args: dict[str, Any],
    6775    result: dict[str, Any],
    6776) -> None:
    6777    title_text = " ".join(str(args.get(key) or "") for key in ("title", "summary", "type")).lower()
    6778    is_finding_batch = tool_name == "write_artifact" and "finding" in title_text
    6779    if not is_finding_batch and step_no % 10 != 0:
    6780        return
    6781    job = db.get_job(job_id)
    6782    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    6783    previous = metadata.get("last_checkpoint_counts") if isinstance(metadata.get("last_checkpoint_counts"), dict) else {}
    6784    checkpoint = build_progress_checkpoint(
    6785        metadata,
    6786        previous_counts=previous,
    6787        step_no=step_no,
    6788        tool_name=tool_name,
    6789        artifact_id=str(result.get("artifact_id") or ""),
    6790        is_finding_output=is_finding_batch,
    6791    )
    6792    db.append_agent_update(
    6793        job_id,
    6794        checkpoint.message,
    6795        category=checkpoint.category,
    6796        metadata={
    6797            "step_no": step_no,
    6798            "tool": tool_name,
    6799            "deltas": checkpoint.deltas,
    6800            "updates": checkpoint.updates,
    6801            "resolutions": checkpoint.resolutions,
    6802        },
    6803    )
    6804    streak = _as_int(metadata.get("activity_checkpoint_streak"))
    6805    streak = streak + 1 if checkpoint.category == "activity" else 0
    6806    task_durable_change = checkpoint.deltas.get("tasks", 0) + checkpoint.updates.get("tasks", 0)
    6807    non_task_durable_change = any(
    6808        checkpoint.deltas.get(key, 0) > 0
    6809        or checkpoint.updates.get(key, 0) > 0
    6810        or checkpoint.resolutions.get(key, 0) > 0
    6811        for key in ("findings", "sources", "experiments", "lessons", "milestones")
    6812    )
    6813    task_resolution = checkpoint.resolutions.get("tasks", 0) > 0
    6814    task_only_progress = task_durable_change > 0 and not non_task_durable_change and not task_resolution
    6815    task_planning_streak = _as_int(metadata.get("task_planning_checkpoint_streak"))
    6816    task_planning_streak = task_planning_streak + 1 if task_only_progress else 0
    6817    db.update_job_metadata(
    6818        job_id,
    6819        {
    6820            "last_checkpoint_counts": checkpoint.counts,
    6821            "last_checkpoint_at": datetime.now(timezone.utc).isoformat(),
    6822            "activity_checkpoint_streak": streak,
    6823            "task_planning_checkpoint_streak": task_planning_streak,
    6824        },
    6825    )
    6826
    6827
    6828def _execute_tool_call(
    6829    call: Any,
    6830    *,
    6831    job: dict[str, Any],
    6832    recent_steps: list[dict[str, Any]],
    6833    config: AppConfig,
    6834    db: AgentDB,
    6835    artifacts: ArtifactStore,
    6836    registry: ToolRegistry,
    6837    job_id: str,
    6838    run_id: str,
    6839) -> tuple[StepExecution, bool, str, str | None]:
    6840    args = _normalize_milestone_validation_args_for_active_gate(call.name, call.arguments, job)
    6841    input_data = {"tool_call_id": call.id, "arguments": args}
    6842    if args != call.arguments:
    6843        input_data["original_arguments"] = call.arguments
    6844    step_id = db.add_step(
    6845        job_id=job_id,
    6846        run_id=run_id,
    6847        kind="tool",
    6848        tool_name=call.name,
    6849        input_data=input_data,
    6850    )
    6851    validate_arguments = getattr(registry, "validate_arguments", None)
    6852    argument_block = validate_arguments(call.name, args, config) if callable(validate_arguments) else None
    6853    if argument_block:
    6854        concrete_fields = [*(argument_block.get("missing_arguments") or []), *(argument_block.get("placeholder_arguments") or [])]
    6855        reason = "missing required arguments" if argument_block.get("missing_arguments") else str(argument_block.get("error") or "invalid tool arguments")
    6856        summary = f"blocked {call.name}; {reason}: {', '.join(concrete_fields)}"
    6857        db.finish_step(
    6858            step_id,
    6859            status="blocked",
    6860            summary=summary,
    6861            output_data=argument_block,
    6862            error=None,
    6863        )
    6864        db.append_agent_update(
    6865            job_id,
    6866            summary,
    6867            category="blocked",
    6868            metadata={
    6869                "reason": "tool_arguments_missing",
    6870                "tool": call.name,
    6871                "missing_arguments": argument_block.get("missing_arguments") or [],
    6872                "placeholder_arguments": argument_block.get("placeholder_arguments") or [],
    6873            },
    6874        )
    6875        return (
    6876            StepExecution(
    6877                job_id=job_id,
    6878                run_id=run_id,
    6879                step_id=step_id,
    6880                tool_name=call.name,
    6881                status="blocked",
    6882                result=argument_block,
    6883            ),
    6884            True,
    6885            summary,
    6886            None,
    6887        )
    6888    blocked = _blocked_tool_call_result(call.name, args, recent_steps, job)
    6889    if blocked:
    6890        result, summary = blocked
    6891        result = {**result, "success": True, "recoverable": True}
    6892        evidence_checkpoint = None
    6893        if result.get("error") == "artifact required before more research":
    6894            evidence_step = next(
    6895                (step for step in recent_steps if step.get("id") == result.get("previous_step")),
    6896                None,
    6897            )
    6898            if evidence_step:
    6899                evidence_checkpoint = _auto_persist_evidence(
    6900                    db=db,
    6901                    artifacts=artifacts,
    6902                    job_id=job_id,
    6903                    run_id=run_id,
    6904                    step_id=step_id,
    6905                    blocked_tool=call.name,
    6906                    evidence_step=evidence_step,
    6907                )
    6908                result["auto_checkpoint"] = evidence_checkpoint
    6909                summary = f"blocked {call.name}; auto-saved evidence checkpoint {evidence_checkpoint['artifact_id']}"
    6910        anti_bot_source = result.get("anti_bot_source") if isinstance(result.get("anti_bot_source"), dict) else None
    6911        if anti_bot_source:
    6912            result["auto_source_record"] = _auto_record_blocked_source(
    6913                db=db,
    6914                job_id=job_id,
    6915                context=anti_bot_source,
    6916                blocked_tool=call.name,
    6917            )
    6918        known_bad_source = result.get("known_bad_source") if isinstance(result.get("known_bad_source"), dict) else None
    6919        if known_bad_source:
    6920            db.append_agent_update(
    6921                job_id,
    6922                f"Source ledger blocked retry of {known_bad_source.get('source')}; choosing a different route next.",
    6923                category="blocked",
    6924                metadata={"source": known_bad_source, "blocked_tool": call.name},
    6925            )
    6926        if result.get("error") == "task queue saturated":
    6927            step = _step_by_id(db, job_id, step_id)
    6928            task_queue = result.get("task_queue") if isinstance(result.get("task_queue"), dict) else {}
    6929            _record_task_backlog_pressure(
    6930                db=db,
    6931                job_id=job_id,
    6932                step_no=(step or {}).get("step_no"),
    6933                task_queue=task_queue,
    6934                source="blocked_record_tasks",
    6935            )
    6936        _auto_record_grounding_block_lesson(db=db, job_id=job_id, result=result)
    6937        db.finish_step(
    6938            step_id,
    6939            status="blocked",
    6940            summary=summary,
    6941            output_data=result,
    6942            error=None,
    6943        )
    6944        return (
    6945            StepExecution(job_id=job_id, run_id=run_id, step_id=step_id, tool_name=call.name, status="blocked", result=result),
    6946            True,
    6947            summary,
    6948            None,
    6949        )
    6950
    6951    ctx = ToolContext(
    6952        config=config,
    6953        db=db,
    6954        artifacts=artifacts,
    6955        job_id=job_id,
    6956        run_id=run_id,
    6957        step_id=step_id,
    6958        task_id=job_id,
    6959    )
    6960    try:
    6961        raw_result = registry.handle(call.name, args, ctx)
    6962        result = _parse_tool_result(raw_result)
    6963        ok = bool(result.get("success", True)) and not result.get("error")
    6964        status = "completed" if ok else "blocked" if result.get("recoverable") is True else "failed"
    6965        if ok:
    6966            _auto_record_tool_source_quality(db=db, job_id=job_id, tool_name=call.name, result=result)
    6967        elif call.name == "shell_exec":
    6968            _auto_record_failed_shell_sources(db=db, job_id=job_id, args=args, result=result)
    6969        summary = _summarize_tool_result(call.name, args, result, ok=ok)
    6970        db.finish_step(step_id, status=status, summary=summary, output_data=result, error=result.get("error"))
    6971        if call.name == "shell_exec":
    6972            _maybe_resolve_file_validation_obligation(
    6973                db=db,
    6974                job_id=job_id,
    6975                tool_name=call.name,
    6976                args=args,
    6977                result=result,
    6978                ok=ok,
    6979            )
    6980            finished_step = _step_by_id(db, job_id, step_id)
    6981            _maybe_create_measurement_obligation(
    6982                db=db,
    6983                job_id=job_id,
    6984                step=finished_step,
    6985                tool_name=call.name,
    6986                args=args,
    6987                result=result,
    6988            )
    6989        if ok:
    6990            finished_step = _step_by_id(db, job_id, step_id)
    6991            _mark_evidence_checkpoint_read(
    6992                db=db,
    6993                job_id=job_id,
    6994                tool_name=call.name,
    6995                args=args,
    6996                step=finished_step,
    6997            )
    6998            _resolve_evidence_checkpoint(
    6999                db=db,
    7000                job_id=job_id,
    7001                tool_name=call.name,
    7002                step=finished_step,
    7003            )
    7004            if call.name == "write_file":
    7005                _maybe_create_file_validation_obligation(
    7006                    db=db,
    7007                    job_id=job_id,
    7008                    step=finished_step,
    7009                    args=args,
    7010                    result=result,
    7011                )
    7012            elif call.name in {"record_lesson", "record_tasks", "record_experiment", "record_milestone_validation"}:
    7013                _maybe_resolve_file_validation_obligation(
    7014                    db=db,
    7015                    job_id=job_id,
    7016                    tool_name=call.name,
    7017                    args=args,
    7018                    result=result,
    7019                    ok=ok,
    7020                )
    7021            _auto_checkpoint_update(
    7022                db=db,
    7023                job_id=job_id,
    7024                step_no=(finished_step or db.list_steps(job_id=job_id)[-1])["step_no"],
    7025                tool_name=call.name,
    7026                args=args,
    7027                result=result,
    7028            )
    7029            if call.name == "write_artifact":
    7030                reconciled_tasks = _auto_reconcile_artifact_tasks(
    7031                    db=db,
    7032                    job_id=job_id,
    7033                    args=args,
    7034                    result=result,
    7035                )
    7036                if reconciled_tasks:
    7037                    result["auto_reconciled_tasks"] = [
    7038                        {"title": task.get("title"), "status": task.get("status")}
    7039                        for task in reconciled_tasks[:8]
    7040                    ]
    7041                revision_task = _auto_open_revision_task_for_deliverable(
    7042                    db=db,
    7043                    job_id=job_id,
    7044                    args=args,
    7045                    result=result,
    7046                )
    7047                if revision_task:
    7048                    result["auto_revision_task"] = {
    7049                        "title": revision_task.get("title"),
    7050                        "status": revision_task.get("status"),
    7051                        "key": revision_task.get("key"),
    7052                    }
    7053        return (
    7054            StepExecution(job_id=job_id, run_id=run_id, step_id=step_id, tool_name=call.name, status=status, result=result),
    7055            status != "completed",
    7056            summary,
    7057            result.get("error") if status == "failed" else None,
    7058        )
    7059    except Exception as exc:
    7060        result = _error_result(exc)
    7061        db.finish_step(step_id, status="failed", summary=f"{call.name} raised", output_data=result, error=str(exc))
    7062        return (
    7063            StepExecution(job_id=job_id, run_id=run_id, step_id=step_id, tool_name=call.name, status="failed", result=result),
    7064            True,
    7065            str(exc),
    7066            str(exc),
    7067        )
    7068
    7069
    7070def _is_continuable_recoverable_input_block(execution: StepExecution) -> bool:
    7071    result = execution.result if isinstance(execution.result, dict) else {}
    7072    error = str(result.get("error") or "")
    7073    if execution.status != "blocked" or result.get("recoverable") is not True:
    7074        return False
    7075    if error in {"missing required tool arguments", "placeholder tool arguments"}:
    7076        return bool(result.get("missing_arguments") or result.get("placeholder_arguments"))
    7077    if error == "malformed shell command" and execution.tool_name == "shell_exec":
    7078        return True
    7079    if error == "duplicate tool call blocked" and execution.tool_name == "read_artifact":
    7080        return True
    7081    return error.startswith("artifact not found:") or error == "no active operator context to acknowledge"
    7082
    7083
    7084def _ordered_tool_calls_for_execution(
    7085    tool_calls: list[ToolCall],
    7086    *,
    7087    job: dict[str, Any],
    7088    recent_steps: list[dict[str, Any]],
    7089) -> list[ToolCall]:
    7090    """Run guard-unblocking calls before branch work when a model batches both."""
    7091
    7092    if len(tool_calls) < 2:
    7093        return tool_calls
    7094    if _browser_runtime_unavailable_context(recent_steps) and any(not _is_browser_tool(call.name) for call in tool_calls):
    7095        tool_calls = [call for call in tool_calls if not _is_browser_tool(call.name)]
    7096        if len(tool_calls) < 2:
    7097            return tool_calls
    7098    checkpoint = _auto_checkpoint_accounting_context(job, recent_steps)
    7099    saturated_record_tasks = any(
    7100        call.name == "record_tasks" and _task_queue_saturation_context(job, call.arguments)
    7101        for call in tool_calls
    7102    )
    7103    if not checkpoint and not saturated_record_tasks:
    7104        return tool_calls
    7105
    7106    artifact_id = str(checkpoint.get("artifact_id") or "") if checkpoint else ""
    7107    artifact_title = str(checkpoint.get("title") or "") if checkpoint else ""
    7108    checkpoint_read = bool(checkpoint and checkpoint.get("checkpoint_read"))
    7109    accounting_tools = {
    7110        "record_experiment",
    7111        "record_findings",
    7112        "record_lesson",
    7113        "record_memory_graph",
    7114        "record_milestone_validation",
    7115        "record_roadmap",
    7116        "record_source",
    7117        "report_update",
    7118        "write_artifact",
    7119    }
    7120
    7121    def priority(call: ToolCall) -> int:
    7122        if checkpoint:
    7123            if call.name in EVIDENCE_CHECKPOINT_RESOLUTION_TOOLS:
    7124                return 0
    7125            if (
    7126                not checkpoint_read
    7127                and call.name == "read_artifact"
    7128                and _read_artifact_call_matches_checkpoint(
    7129                    call.arguments,
    7130                    artifact_id=artifact_id,
    7131                    artifact_title=artifact_title,
    7132                )
    7133            ):
    7134                return 0
    7135        if saturated_record_tasks:
    7136            if call.name == "record_tasks" and _task_queue_saturation_context(job, call.arguments):
    7137                return 2
    7138            if call.name in accounting_tools:
    7139                return 0
    7140        return 1
    7141
    7142    ordered = sorted(enumerate(tool_calls), key=lambda item: (priority(item[1]), item[0]))
    7143    return [call for _, call in ordered]
    7144
    7145
    7146def _registry_tools(registry: ToolRegistry, config: AppConfig) -> list[dict[str, Any]]:
    7147    try:
    7148        return registry.openai_tools(config=config)
    7149    except TypeError:
    7150        return registry.openai_tools()
    7151
    7152
    7153def _registry_tools_for_step(
    7154    registry: ToolRegistry,
    7155    config: AppConfig,
    7156    recent_steps: list[dict[str, Any]],
    7157    *,
    7158    job: dict[str, Any] | None = None,
    7159) -> list[dict[str, Any]]:
    7160    tools = _registry_tools(registry, config)
    7161    resolution_tools = _active_obligation_tool_names(job, recent_steps) if job else None
    7162    if resolution_tools:
    7163        tools = [tool for tool in tools if _openai_tool_name(tool) in resolution_tools]
    7164    suppressed_tools = _suppressed_tool_names(job, recent_steps)
    7165    if resolution_tools:
    7166        suppressed_tools -= resolution_tools
    7167    if suppressed_tools:
    7168        tools = [tool for tool in tools if _openai_tool_name(tool) not in suppressed_tools]
    7169    if not _browser_runtime_unavailable_context(recent_steps):
    7170        return tools
    7171    return [tool for tool in tools if not _is_browser_tool(_openai_tool_name(tool))]
    7172
    7173
    7174def _active_obligation_tool_names(job: dict[str, Any] | None, recent_steps: list[dict[str, Any]]) -> set[str] | None:
    7175    if not job:
    7176        return None
    7177    allowed: set[str] = set()
    7178    checkpoint = _auto_checkpoint_accounting_context(job, recent_steps)
    7179    if checkpoint:
    7180        if not checkpoint.get("checkpoint_read"):
    7181            allowed.add("read_artifact")
    7182        allowed.update(EVIDENCE_CHECKPOINT_PROMPT_TOOLS)
    7183    if _pending_measurement_obligation(job):
    7184        allowed.update(MEASUREMENT_RESOLUTION_TOOLS)
    7185    if _experiment_next_action_failure_context(job, recent_steps):
    7186        allowed.update(MEASUREMENT_RESOLUTION_TOOLS)
    7187    measured_progress = _measured_progress_guard_context(job, recent_steps)
    7188    if measured_progress:
    7189        allowed.update(MEASUREMENT_RESOLUTION_TOOLS)
    7190        if _as_int(measured_progress.get("shell_actions_since_last_experiment")) < MEASURABLE_ACTION_BUDGET_STEPS:
    7191            allowed.add("shell_exec")
    7192    if _pending_file_validation_obligation(job):
    7193        allowed.update(FILE_VALIDATION_RESOLUTION_TOOLS)
    7194    return allowed or None
    7195
    7196
    7197def _suppressed_tool_names(job: dict[str, Any] | None, recent_steps: list[dict[str, Any]]) -> set[str]:
    7198    if not job:
    7199        return set()
    7200    suppressed: set[str] = set()
    7201    if _repeated_task_queue_saturation_context(recent_steps):
    7202        suppressed.add("record_tasks")
    7203    elif (
    7204        (backlog := _task_backlog_pressure_context(job))
    7205        and _as_int(backlog.get("total")) > TASK_QUEUE_TOTAL_SOFT_LIMIT
    7206        and not _pending_measurement_obligation(job)
    7207        and not _pending_file_validation_obligation(job)
    7208        and not _auto_checkpoint_accounting_context(job, recent_steps)
    7209        and not _task_queue_exhausted(job)
    7210    ):
    7211        suppressed.add("record_tasks")
    7212    if not _has_acknowledgeable_operator_context(job):
    7213        suppressed.add("acknowledge_operator_context")
    7214    return suppressed
    7215
    7216
    7217def _has_acknowledgeable_operator_context(job: dict[str, Any]) -> bool:
    7218    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
    7219    messages = metadata.get("operator_messages") if isinstance(metadata.get("operator_messages"), list) else []
    7220    for entry in messages:
    7221        if not isinstance(entry, dict):
    7222            continue
    7223        mode = str(entry.get("mode") or "steer").strip().lower().replace("-", "_")
    7224        if mode not in {"steer", "follow_up"}:
    7225            continue
    7226        if not entry.get("claimed_at"):
    7227            continue
    7228        if entry.get("acknowledged_at") or entry.get("superseded_at"):
    7229            continue
    7230        return True
    7231    return False
    7232
    7233
    7234def _openai_tool_name(tool: dict[str, Any]) -> str:
    7235    function = tool.get("function") if isinstance(tool, dict) else None
    7236    if isinstance(function, dict):
    7237        return str(function.get("name") or "")
    7238    return str(tool.get("name") or "") if isinstance(tool, dict) else ""
    7239
    7240
    7241def _call_next_action_with_timeout(
    7242    llm: StepLLM,
    7243    *,
    7244    messages: list[dict[str, Any]],
    7245    tools: list[dict[str, Any]],
    7246    timeout_seconds: float,
    7247) -> LLMResponse:
    7248    timeout = max(0.0, float(timeout_seconds or 0.0))
    7249    if timeout <= 0 or threading.current_thread() is not threading.main_thread():
    7250        return llm.next_action(messages=messages, tools=tools)
    7251
    7252    previous_handler = signal.getsignal(signal.SIGALRM)
    7253    previous_timer = signal.getitimer(signal.ITIMER_REAL)
    7254    started = time.monotonic()
    7255
    7256    def _raise_timeout(_signum: int, _frame: Any) -> None:
    7257        raise TimeoutError(f"model call timed out after {timeout:g}s")
    7258
    7259    signal.signal(signal.SIGALRM, _raise_timeout)
    7260    signal.setitimer(signal.ITIMER_REAL, timeout)
    7261    try:
    7262        return llm.next_action(messages=messages, tools=tools)
    7263    finally:
    7264        signal.setitimer(signal.ITIMER_REAL, 0)
    7265        signal.signal(signal.SIGALRM, previous_handler)
    7266        if previous_timer[0] > 0:
    7267            elapsed = max(0.0, time.monotonic() - started)
    7268            remaining = max(0.001, previous_timer[0] - elapsed)
    7269            signal.setitimer(signal.ITIMER_REAL, remaining, previous_timer[1])
    7270
    7271
    7272def _tool_repair_messages(messages: list[dict[str, Any]], response: LLMResponse) -> list[dict[str, Any]]:
    7273    content = str(response.content or "").strip()
    7274    if len(content) > 2000:
    7275        content = content[:2000] + " ..."
    7276    repair_prompt = (
    7277        "Your previous worker response did not call a tool. This worker must advance by calling exactly "
    7278        "one available tool now. Do not answer in prose. Choose one bounded action that fits the current "
    7279        "state, such as executing existing work, recording a measurement, updating an existing task, "
    7280        "saving an evidence-backed output, recording a lesson/finding/source, or deferring only for a real wait."
    7281    )
    7282    repaired = list(messages)
    7283    if content:
    7284        repaired.append({"role": "assistant", "content": content})
    7285    repaired.append({"role": "user", "content": repair_prompt})
    7286    return repaired
    7287
    7288
    7289def run_one_step(
    7290    job_id: str,
    7291    *,
    7292    config: AppConfig | None = None,
    7293    db: AgentDB | None = None,
    7294    llm: StepLLM | None = None,
    7295    registry: ToolRegistry = DEFAULT_REGISTRY,
    7296) -> StepExecution:
    7297    config = config or load_config()
    7298    config.ensure_dirs()
    7299    owns_db = db is None
    7300    db = db or AgentDB(config.runtime.state_db_path)
    7301    try:
    7302        artifacts = ArtifactStore(config.runtime.home, db=db)
    7303        job = db.get_job(job_id)
    7304        if _acknowledge_non_prompt_operator_context(db, job_id):
    7305            job = db.get_job(job_id)
    7306        if _clear_invalid_measurement_obligation(db, job_id):
    7307            job = db.get_job(job_id)
    7308        if _clear_stale_task_backlog_pressure(db, job_id, job):
    7309            job = db.get_job(job_id)
    7310        run_id = db.start_run(job_id, model=config.model.model)
    7311        _emit_loop_start(db, job_id, run_id)
    7312        recent_steps = db.list_steps(job_id=job_id)
    7313        if _refresh_contradicted_negative_claims(db, job_id, job, recent_steps):
    7314            job = db.get_job(job_id)
    7315        model_config = config.model
    7316        if _should_reflect(job, recent_steps):
    7317            return _run_reflection_step(job, recent_steps, db=db, job_id=job_id, run_id=run_id)
    7318        guard_recovery = _repeated_guard_block_context(recent_steps)
    7319        if guard_recovery:
    7320            return _run_guard_recovery_step(guard_recovery, db=db, job_id=job_id, run_id=run_id)
    7321        active_operator_messages = _claim_operator_queue(db, job_id)
    7322        if active_operator_messages:
    7323            job = db.get_job(job_id)
    7324        usage = db.job_token_usage(job_id)
    7325        usage_budget_limit = _usage_budget_limit_context(config, usage)
    7326        if usage_budget_limit:
    7327            return _run_usage_budget_limit_step(
    7328                usage_budget_limit,
    7329                db=db,
    7330                job_id=job_id,
    7331                run_id=run_id,
    7332            )
    7333        messages = build_messages(
    7334            job,
    7335            recent_steps,
    7336            memory_entries=db.list_memory(job_id),
    7337            program_text=_load_program_text(config, job_id),
    7338            timeline_events=db.list_timeline_events(job_id, limit=30),
    7339            active_operator_messages=active_operator_messages,
    7340            include_unclaimed_operator_messages=True,
    7341            token_usage=usage,
    7342        )
    7343        llm = llm or OpenAIChatLLM(model_config)
    7344        llm_started = time.monotonic()
    7345        try:
    7346            response: LLMResponse = _call_next_action_with_timeout(
    7347                llm,
    7348                messages=messages,
    7349                tools=_registry_tools_for_step(registry, config, recent_steps, job=job),
    7350                timeout_seconds=model_config.request_timeout_seconds,
    7351            )
    7352        except Exception as exc:
    7353            llm_duration_seconds = round(max(0.0, time.monotonic() - llm_started), 3)
    7354            step_id = db.add_step(
    7355                job_id=job_id,
    7356                run_id=run_id,
    7357                kind="llm",
    7358                status="failed",
    7359                summary=f"model call failed: {type(exc).__name__}",
    7360                input_data={
    7361                    "model": config.model.model,
    7362                    "duration_seconds": llm_duration_seconds,
    7363                    "request_timeout_seconds": model_config.request_timeout_seconds,
    7364                },
    7365            )
    7366            result = _error_result(exc)
    7367            result["duration_seconds"] = llm_duration_seconds
    7368            hard_failure_note = _hard_llm_provider_failure_note(exc)
    7369            if hard_failure_note:
    7370                result["provider_action_required"] = True
    7371                result["pause_reason"] = "llm_provider_blocked"
    7372                db.update_job_status(
    7373                    job_id,
    7374                    "paused",
    7375                    metadata_patch={
    7376                        "last_note": hard_failure_note,
    7377                        "provider_blocked_at": datetime.now(timezone.utc).isoformat(),
    7378                    },
    7379                )
    7380                db.append_agent_update(
    7381                    job_id,
    7382                    hard_failure_note,
    7383                    category="error",
    7384                    metadata={"reason": "llm_provider_blocked", "error_type": type(exc).__name__},
    7385                )
    7386            db.finish_step(step_id, status="failed", output_data=result, error=str(exc))
    7387            db.finish_run(run_id, "failed", error=str(exc))
    7388            _emit_loop_end(db, job_id, run_id, status="failed", step_id=step_id, detail=str(exc))
    7389            refresh_memory_index(db, job_id)
    7390            return StepExecution(job_id=job_id, run_id=run_id, step_id=step_id, tool_name=None, status="failed", result=result)
    7391
    7392        llm_duration_seconds = round(max(0.0, time.monotonic() - llm_started), 3)
    7393        job = db.get_job(job_id)
    7394        usage = _emit_assistant_message_event(
    7395            db,
    7396            job_id,
    7397            run_id,
    7398            response,
    7399            messages=messages,
    7400            context_length=config.model.context_length,
    7401            duration_seconds=llm_duration_seconds,
    7402        )
    7403        emit_context_pressure_update(db, job_id, usage)
    7404        emit_usage_pressure_update(db, job_id, db.job_token_usage(job_id))
    7405
    7406        tool_repair_attempted = False
    7407        tool_repair_error: dict[str, Any] | None = None
    7408        original_content = response.content
    7409        if not response.tool_calls and getattr(llm, "tool_repair", False):
    7410            tool_repair_attempted = True
    7411            repair_messages = _tool_repair_messages(messages, response)
    7412            repair_started = time.monotonic()
    7413            try:
    7414                repair_response = _call_next_action_with_timeout(
    7415                    llm,
    7416                    messages=repair_messages,
    7417                    tools=_registry_tools_for_step(registry, config, recent_steps, job=job),
    7418                    timeout_seconds=model_config.request_timeout_seconds,
    7419                )
    7420            except Exception as exc:
    7421                tool_repair_error = _error_result(exc)
    7422                tool_repair_error["duration_seconds"] = round(max(0.0, time.monotonic() - repair_started), 3)
    7423            else:
    7424                repair_duration_seconds = round(max(0.0, time.monotonic() - repair_started), 3)
    7425                repair_usage = _emit_assistant_message_event(
    7426                    db,
    7427                    job_id,
    7428                    run_id,
    7429                    repair_response,
    7430                    messages=repair_messages,
    7431                    context_length=config.model.context_length,
    7432                    duration_seconds=repair_duration_seconds,
    7433                )
    7434                emit_context_pressure_update(db, job_id, repair_usage)
    7435                emit_usage_pressure_update(db, job_id, db.job_token_usage(job_id))
    7436                if repair_response.tool_calls:
    7437                    response = repair_response
    7438
    7439        if response.tool_calls:
    7440            executions: list[StepExecution] = []
    7441            details: list[str] = []
    7442            run_error: str | None = None
    7443            ordered_tool_calls = _ordered_tool_calls_for_execution(
    7444                response.tool_calls,
    7445                job=db.get_job(job_id),
    7446                recent_steps=db.list_steps(job_id=job_id),
    7447            )
    7448            for index, call in enumerate(ordered_tool_calls):
    7449                current_job = db.get_job(job_id)
    7450                current_recent_steps = db.list_steps(job_id=job_id)
    7451                execution, stop_batch, detail, error = _execute_tool_call(
    7452                    call,
    7453                    job=current_job,
    7454                    recent_steps=current_recent_steps,
    7455                    config=config,
    7456                    db=db,
    7457                    artifacts=artifacts,
    7458                    registry=registry,
    7459                    job_id=job_id,
    7460                    run_id=run_id,
    7461                )
    7462                executions.append(execution)
    7463                details.append(detail)
    7464                if error:
    7465                    run_error = error
    7466                if stop_batch:
    7467                    if index < len(ordered_tool_calls) - 1 and _is_continuable_recoverable_input_block(execution):
    7468                        details.append(f"continued after recoverable {call.name} input block")
    7469                        continue
    7470                    break
    7471
    7472            final_execution = executions[-1]
    7473            run_status = "failed" if any(item.status == "failed" for item in executions) else "completed"
    7474            db.finish_run(run_id, run_status, error=run_error)
    7475            detail = f"executed {len(executions)}/{len(response.tool_calls)} tool calls"
    7476            if details:
    7477                detail = f"{detail}; last: {details[-1]}"
    7478            _emit_loop_end(
    7479                db,
    7480                job_id,
    7481                run_id,
    7482                status=final_execution.status,
    7483                step_id=final_execution.step_id,
    7484                tool_name=final_execution.tool_name,
    7485                detail=detail,
    7486            )
    7487            refresh_memory_index(db, job_id)
    7488            return final_execution
    7489
    7490        step_id = db.add_step(
    7491            job_id=job_id,
    7492            run_id=run_id,
    7493            kind="assistant",
    7494            status="blocked",
    7495            summary="worker returned content without a tool call",
    7496            input_data={},
    7497        )
    7498        result = {
    7499            "success": False,
    7500            "recoverable": True,
    7501            "error": "worker tool call required",
    7502            "content": response.content,
    7503            "original_content": original_content,
    7504            "tool_repair_attempted": tool_repair_attempted,
    7505            "tool_repair_error": tool_repair_error,
    7506            "next": (
    7507                "Worker turns must use a tool call. Continue by choosing one bounded action such as "
    7508                "record_tasks, report_update, write_artifact, write_file, shell_exec, record_findings, "
    7509                "record_source, record_experiment, record_lesson, or defer_job."
    7510            ),
    7511        }
    7512        db.append_agent_update(
    7513            job_id,
    7514            "Worker returned a message without a tool call; continuing with a tool-action recovery constraint.",
    7515            category="blocked",
    7516            metadata={"reason": "worker_tool_call_required", "step_id": step_id},
    7517        )
    7518        db.finish_step(
    7519            step_id,
    7520            status="blocked",
    7521            summary="blocked assistant-only worker turn; tool call required",
    7522            output_data=result,
    7523            error="worker tool call required",
    7524        )
    7525        db.finish_run(run_id, "blocked", error="worker tool call required")
    7526        _emit_loop_end(
    7527            db,
    7528            job_id,
    7529            run_id,
    7530            status="blocked",
    7531            step_id=step_id,
    7532            detail="worker tool call required",
    7533        )
    7534        refresh_memory_index(db, job_id)
    7535        return StepExecution(job_id=job_id, run_id=run_id, step_id=step_id, tool_name=None, status="blocked", result=result)
    7536    finally:
    7537        if owns_db:
    7538            db.close()

nipux_cli/worker_policy.py 489 lines

       1"""Static worker prompt and loop policy constants."""
       2
       3from __future__ import annotations
       4
       5import re
       6
       7
       8REFLECTION_INTERVAL_STEPS = 12
       9WORKER_PROTOCOL_VERSION = "2026-05-01-contract-first-v1"
      10
      11SYSTEM_PROMPT = """You are a long-running local work agent.
      12
      13Operate as a bounded worker, not a chat assistant. Choose one useful next step,
      14call one of the available tools, and persist important evidence as artifacts.
      15Do not claim the whole job is complete. A strong result is only a checkpoint:
      16save it, report it, add the next tasks, and continue improving or broadening.
      17
      18Use a contract-first durable cycle. Read the objective, operator context,
      19roadmap, active task, and recent evidence; choose the next action that satisfies
      20the active output contract; produce or measure concrete evidence; update the
      21right ledger; report the checkpoint; then open or continue the next branch.
      22Research is only one possible contract. For action, experiment, monitor, report,
      23or file-deliverable work, prefer execution, measurement, validation, or writing
      24over more background collection. Keep moving forever until the operator pauses
      25or cancels the job.
      26The worker must not mark jobs completed or failed; use record_tasks,
      27record_lesson, report_update, and artifacts to describe checkpoints, blockers,
      28and next branches while the job stays runnable.
      29
      30Avoid loops. Do not repeat the same search query or the same exact tool call.
      31If search results already exist, move forward by extracting source pages,
      32opening a useful site in the browser, or saving a finding/evidence artifact.
      33If a page has already been extracted and contains useful evidence, save that
      34evidence with write_artifact before doing more searching or browsing.
      35Only click or type browser refs from the most recent successful browser snapshot
      36or navigation result. If a click/type fails with an unknown ref, use the fresh
      37recovery snapshot or call browser_snapshot before retrying.
      38If a source shows Cloudflare, login, paywall, or anti-bot verification, keep it
      39visible in the trace. Do not bypass protections. Continue with normal visible
      40browser actions when possible, persist what you have, or use alternate public
      41sources if stuck.
      42If a tool returns a list of actionable candidates such as files, packages,
      43configurations, commands, sources, venues, records, branches, or options, do not
      44keep re-listing the same candidate set with small formatting changes. Persist the
      45candidate list once, choose the best candidate for the active contract, and move
      46to execution, measurement, validation, or an explicit blocked decision.
      47If a probe discovers a local/runtime candidate that might satisfy the active
      48contract, promote that candidate immediately: record the fact, validate it with
      49the smallest relevant action, and measure it before continuing external
      50acquisition or research retries. Do not let an available local candidate fall
      51out of context while pursuing lower-confidence external sources.
      52If repeated external acquisition attempts fail with authentication, permission,
      53quota, missing credentials, or unavailable resources, mark that branch blocked
      54or low-yield and pivot to another source, local candidate, monitor/defer branch,
      55or operator-visible credential requirement instead of retrying small variants.
      56If a browser page says blocked, CAPTCHA, bot check, login required, paywall, or
      57anti-bot, treat that page as a failed/low-yield source for the current job. Do
      58not write an artifact that claims usable evidence exists unless the evidence is
      59actually visible. Record the source outcome or pivot to another public source.
      60Use report_update for short operator-readable progress notes when you need to
      61say what you found or why you are blocked. Do not use report_update instead of
      62write_artifact when you have durable evidence, findings, or report content to save.
      63Use write_file when the objective requires a concrete file deliverable, source
      64file, document, config, dataset, or other workspace output. If a measured
      65experiment says the next action is to write, merge, update, compile, or insert
      66content, prefer write_file or an execution command that actually changes the
      67target over more read-only inspection.
      68Use defer_job when the next useful step is to wait for an external process,
      69scheduled check, long-running command, or monitor interval. Do not
      70simulate waiting with repeated searches, reports, or shell probes.
      71Use record_lesson when you learn something that should change future behavior:
      72bad source patterns, task-specific success criteria, repeated mistakes, operator
      73preferences, or a better strategy. Keep lessons short and reusable.
      74Use record_memory_graph when work produces reusable connected knowledge: an
      75episode worth remembering, a stable fact, a strategy, a reusable skill, an open
      76question, a decision, or a constraint. Link nodes to their evidence and to each
      77other. Treat this as the job's durable brain: recent events are fast episodic
      78memory, while stable graph nodes are consolidated knowledge that should guide
      79future branches without replaying raw history.
      80Durable memory is not automatically true forever. If newer evidence contradicts
      81an older memory-graph fact, constraint, strategy, or finding, update the older
      82record as deprecated/resolved/stale and link the newer evidence before acting.
      83Prefer fresh measured or directly observed evidence over stale summaries.
      84Use record_source when a source is high-yield, low-yield, blocked, repetitive,
      85or otherwise useful to score for future behavior.
      86Use record_findings after finding durable candidates, facts, opportunities,
      87experiments, files, bugs, sources, or other reusable outputs. Dedupe against the
      88finding ledger and artifacts before saving.
      89Use record_tasks to maintain a durable queue of objective-neutral branches:
      90open work, active branch, blocked branch, completed branch, and skipped branch.
      91Each task should include an output_contract (research, artifact, experiment,
      92action, monitor, decision, or report), acceptance criteria, evidence needed,
      93and stall behavior so progress is judged by evidence, not activity volume.
      94Before marking a task or milestone done, audit the claim against the objective:
      95list the requirement, the concrete artifact/file/finding/measurement/validation
      96that proves it, and any remaining gap. If the audit is incomplete, keep the
      97branch active or blocked and create the next smallest follow-up task.
      98When the job is broad or starts looping, split it into tasks and move to the
      99highest-priority open task rather than staying on one source or tactic forever.
     100Use record_roadmap for broad, multi-phase, or ambiguous objectives that need a
     101higher-level orchestration plan. A roadmap is generic: milestones group related
     102features or work units; each milestone has acceptance criteria, evidence needed,
     103and a validation contract. Use record_milestone_validation at milestone checkpoints
     104to pass, fail, block, or create follow-up tasks from validation gaps. Keep the
     105roadmap compact and update it from durable evidence, not from activity count.
     106Use record_experiment for measurable trials, benchmarks, comparisons,
     107optimization attempts, or hypothesis tests. A saved note, source, or artifact is
     108not enough progress for a measurable objective: record the exact configuration,
     109metric, result, whether higher or lower is better, and the next experiment. Keep
     110improving against the best observed result instead of declaring victory after a
     111single measurement.
     112Use shell_exec for command-line work, repository inspection, diagnostics,
     113benchmarks, repeatable experiments, and other command execution that the
     114objective requires. Prefer small read-only probes before changing anything, use
     115explicit timeouts, and save important command output with write_artifact before
     116continuing. Do not run destructive or high-risk cyber commands.
     117For long downloads, builds, training runs, crawls, benchmarks, or other slow
     118actions, treat the action as a monitored branch: choose a timeout that can make
     119meaningful progress, use resumable commands when available, record partial
     120progress as an experiment/task/checkpoint, and use defer_job when the next useful
     121step is to wait and check again. Do not repeatedly restart the same long action
     122with short timeouts without recording what changed and how the next attempt will
     123resume or differ.
     124If a probe shows a partial output, incomplete file, running process, cache entry,
     125checkpoint, or other unfinished artifact from an action branch, stop re-listing
     126the same state. Either resume/continue the action with a resumable command,
     127record a monitor/defer step for the still-running work, or record the branch as
     128blocked with the concrete missing condition and next action.
     129read_artifact only reads saved Nipux artifacts. Use shell_exec for repository,
     130workspace, project, or filesystem files that are not saved artifacts.
     131write_file writes workspace/local files directly; write_artifact writes Nipux's
     132separate saved-output store. Use the right one for the operator-facing result.
     133Operator messages are durable context from the human operator. Messages marked
     134steer are active constraints until acknowledged or superseded. Messages marked
     135follow_up are lower-priority queued work; keep them in the task queue and act on
     136them after the current active branch has a durable checkpoint. Messages marked
     137note are durable preferences. Use acknowledge_operator_context only after you
     138have incorporated or intentionally superseded a steer/follow_up message.
     139"""
     140
     141INFORMATION_GATHERING_TOOLS = {
     142    "browser_back",
     143    "browser_click",
     144    "browser_console",
     145    "browser_navigate",
     146    "browser_press",
     147    "browser_scroll",
     148    "browser_snapshot",
     149    "browser_type",
     150    "web_extract",
     151    "web_search",
     152}
     153
     154ARTIFACT_REVIEW_TOOLS = {"read_artifact", "search_artifacts"}
     155MEMORY_REVIEW_TOOLS = {"search_memory_graph"}
     156BRANCH_WORK_TOOLS = INFORMATION_GATHERING_TOOLS | ARTIFACT_REVIEW_TOOLS | MEMORY_REVIEW_TOOLS | {"shell_exec"}
     157LEDGER_PROGRESS_TOOLS = {
     158    "guard_recovery",
     159    "record_findings",
     160    "record_memory_graph",
     161    "record_source",
     162    "record_tasks",
     163    "record_roadmap",
     164    "record_milestone_validation",
     165    "record_experiment",
     166    "record_lesson",
     167}
     168MEASUREMENT_RESOLUTION_TOOLS = {"record_experiment", "record_lesson", "record_tasks", "record_milestone_validation"}
     169FILE_VALIDATION_RESOLUTION_TOOLS = {
     170    "shell_exec",
     171    "record_experiment",
     172    "record_lesson",
     173    "record_tasks",
     174    "record_milestone_validation",
     175    "record_memory_graph",
     176    "acknowledge_operator_context",
     177}
     178ARTIFACT_ACCOUNTING_RESOLUTION_TOOLS = LEDGER_PROGRESS_TOOLS | {"acknowledge_operator_context"}
     179ARTIFACT_ACCOUNTING_BLOCKED_TOOLS = INFORMATION_GATHERING_TOOLS | {
     180    "shell_exec",
     181    "write_file",
     182    "write_artifact",
     183    "read_artifact",
     184    "search_artifacts",
     185    "report_update",
     186}
     187MEASUREMENT_BLOCKED_TOOLS = INFORMATION_GATHERING_TOOLS | {
     188    "shell_exec",
     189    "write_file",
     190    "write_artifact",
     191    "record_findings",
     192    "record_memory_graph",
     193    "record_source",
     194    "acknowledge_operator_context",
     195    "report_update",
     196}
     197FILE_VALIDATION_BLOCKED_TOOLS = INFORMATION_GATHERING_TOOLS | ARTIFACT_REVIEW_TOOLS | MEMORY_REVIEW_TOOLS | {
     198    "write_file",
     199    "write_artifact",
     200    "record_findings",
     201    "record_source",
     202    "report_update",
     203}
     204MILESTONE_VALIDATION_BLOCKED_TOOLS = INFORMATION_GATHERING_TOOLS | {
     205    "shell_exec",
     206    "write_file",
     207    "write_artifact",
     208    "record_findings",
     209    "record_source",
     210    "record_experiment",
     211    "report_update",
     212}
     213ROADMAP_STALENESS_BLOCKED_TOOLS = INFORMATION_GATHERING_TOOLS | {
     214    "shell_exec",
     215    "write_file",
     216    "write_artifact",
     217    "record_findings",
     218    "record_source",
     219    "record_tasks",
     220    "record_experiment",
     221    "report_update",
     222}
     223CHURN_TOOLS = INFORMATION_GATHERING_TOOLS | ARTIFACT_REVIEW_TOOLS | MEMORY_REVIEW_TOOLS | {"shell_exec"}
     224MEMORY_CONSOLIDATION_BLOCKED_TOOLS = CHURN_TOOLS | {"write_artifact", "write_file", "report_update"}
     225ACTIVITY_STAGNATION_BLOCKED_TOOLS = CHURN_TOOLS | {"write_artifact", "write_file", "report_update"}
     226DELIVERABLE_PROGRESS_BLOCKED_TOOLS = INFORMATION_GATHERING_TOOLS | ARTIFACT_REVIEW_TOOLS | {"report_update"}
     227RESEARCH_BALANCE_BLOCKED_TOOLS = ARTIFACT_REVIEW_TOOLS | MEMORY_REVIEW_TOOLS | {
     228    "shell_exec",
     229    "write_file",
     230    "write_artifact",
     231    "record_lesson",
     232    "report_update",
     233}
     234SOURCE_YIELD_BLOCKED_TOOLS = INFORMATION_GATHERING_TOOLS | ARTIFACT_REVIEW_TOOLS | MEMORY_REVIEW_TOOLS | {
     235    "shell_exec",
     236    "write_file",
     237    "write_artifact",
     238    "report_update",
     239}
     240MEASURABLE_RESEARCH_BLOCKED_TOOLS = INFORMATION_GATHERING_TOOLS | {
     241    "write_artifact",
     242    "record_findings",
     243    "record_source",
     244    "report_update",
     245}
     246MEASURABLE_PROGRESS_PATTERN = re.compile(
     247    r"(?i)\b("
     248    r"benchmark|baseline|compare|comparison|experiment|improv(?:e|ing|ement)|increase|latency|"
     249    r"measure|metric|minimi[sz]e|maximi[sz]e|optim(?:ize|ise|ization|isation)|performance|"
     250    r"rate|reduce|score|speed|throughput|tune|tuning"
     251    r")\b"
     252)
     253RECOVERABLE_GUARD_ERRORS = {
     254    "artifact search loop blocked",
     255    "browser runtime unavailable",
     256    "deliverable checkpoint required",
     257    "durable progress required",
     258    "evidence checkpoint accounting required",
     259    "evidence grounding required",
     260    "duplicate tool call blocked",
     261    "experiment stagnation decision required",
     262    "experiment next action pending",
     263    "known bad source blocked",
     264    "lesson consolidation required",
     265    "memory graph consolidation required",
     266    "measurement obligation pending",
     267    "measured progress required",
     268    "progress accounting required",
     269    "progress ledger update required",
     270    "action decision required",
     271    "similar artifact search blocked",
     272    "similar search query blocked",
     273    "source yield accounting required",
     274    "task execution required",
     275    "task branch required before more work",
     276    "task queue saturated",
     277    "usage pressure recovery required",
     278    "worker tool call required",
     279}
     280MEASURABLE_RESEARCH_BUDGET_STEPS = 18
     281MEASURABLE_ACTION_BUDGET_STEPS = 4
     282DELIVERABLE_RESEARCH_BUDGET_STEPS = 18
     283ACTIVITY_STAGNATION_CHECKPOINTS = 3
     284TASK_QUEUE_SATURATION_OPEN_TASKS = 40
     285TASK_QUEUE_TOTAL_SOFT_LIMIT = 80
     286TASK_PLANNING_STAGNATION_CHECKPOINTS = 2
     287PROGRAM_PROMPT_CHARS = 2000
     288MEMORY_ENTRY_PROMPT_CHARS = 700
     289MEMORY_PROMPT_CHARS = 1800
     290RECENT_STATE_STEPS = 5
     291RECENT_STATE_PROMPT_CHARS = 3000
     292TIMELINE_PROMPT_EVENTS = 8
     293SECTION_ITEM_CHARS = 420
     294MAX_WORKER_PROMPT_CHARS = 18_000
     295TIMELINE_PROMPT_EVENT_TYPES = {
     296    "agent_message",
     297    "artifact",
     298    "error",
     299    "experiment",
     300    "finding",
     301    "lesson",
     302    "memory_node",
     303    "milestone_validation",
     304    "reflection",
     305    "roadmap",
     306    "source",
     307    "task",
     308}
     309TIMELINE_PROMPT_AGENT_TITLES = {"blocked", "error", "plan", "progress", "report", "update"}
     310TIMELINE_PROMPT_TOOL_STATUSES = {"blocked", "failed"}
     311PROMPT_SECTION_BUDGETS = {
     312    "Workspace": 520,
     313    "Operator context": 2_200,
     314    "Current execution focus": 1_600,
     315    "Pending measurement obligation": 1_100,
     316    "Candidate file discovery": 2_000,
     317    "Measured progress guard": 1_000,
     318    "Experiment stagnation guard": 1_000,
     319    "Source yield guard": 1_000,
     320    "Deliverable progress guard": 1_000,
     321    "Progress accounting guard": 900,
     322    "Activity stagnation": 900,
     323    "Task planning guard": 900,
     324    "Memory consolidation guard": 900,
     325    "Lesson consolidation guard": 900,
     326    "Durable progress yield": 900,
     327    "Program": 1_400,
     328    "Usage pressure": 900,
     329    "Lessons learned": 1_100,
     330    "Memory graph": 1_800,
     331    "Roadmap": 2_000,
     332    "Task queue": 2_400,
     333    "Durable outcomes": 1_200,
     334    "Ledgers": 2_400,
     335    "Experiment ledger": 2_200,
     336    "Reflections": 900,
     337    "Compact memory": 1_100,
     338    "Recent visible timeline": 1_000,
     339    "Recent state": 1_800,
     340    "Next-action constraint": 1_100,
     341}
     342
     343QUERY_STOPWORDS = {
     344    "and",
     345    "are",
     346    "does",
     347    "for",
     348    "from",
     349    "how",
     350    "offer",
     351    "product",
     352    "service",
     353    "services",
     354    "the",
     355    "they",
     356    "what",
     357    "with",
     358}
     359TEXT_TOKEN_STOPWORDS = <redacted>
     360    "and",
     361    "are",
     362    "for",
     363    "from",
     364    "into",
     365    "that",
     366    "the",
     367    "this",
     368    "with",
     369}
     370
     371EVIDENCE_ARTIFACT_TERMS = {
     372    "checkpoint",
     373    "evidence",
     374    "extract",
     375    "extracted",
     376    "notes",
     377    "source",
     378    "sources",
     379}
     380DELIVERABLE_ARTIFACT_TERMS = {
     381    "article",
     382    "checklist",
     383    "compiled",
     384    "deck",
     385    "deliverable",
     386    "doc",
     387    "document",
     388    "draft",
     389    "final",
     390    "guide",
     391    "manual",
     392    "memo",
     393    "outline",
     394    "paper",
     395    "presentation",
     396    "report",
     397    "revision",
     398    "section",
     399    "spec",
     400    "template",
     401    "updated",
     402    "writeup",
     403}
     404TASK_DELIVERABLE_ACTION_TERMS = {
     405    "add",
     406    "append",
     407    "compile",
     408    "create",
     409    "edit",
     410    "insert",
     411    "polish",
     412    "rewrite",
     413    "update",
     414    "write",
     415}
     416EXPERIMENT_DELIVERY_ACTION_TERMS = {
     417    "append",
     418    "apply",
     419    "build",
     420    "compile",
     421    "create",
     422    "edit",
     423    "finish",
     424    "fix",
     425    "generate",
     426    "implement",
     427    "insert",
     428    "merge",
     429    "patch",
     430    "produce",
     431    "publish",
     432    "replace",
     433    "rewrite",
     434    "save",
     435    "update",
     436    "write",
     437}
     438EXPERIMENT_INFORMATION_ACTION_TERMS = {
     439    "audit",
     440    "collect",
     441    "extract",
     442    "find",
     443    "gather",
     444    "inspect",
     445    "read",
     446    "research",
     447    "review",
     448    "search",
     449    "source",
     450    "survey",
     451}
     452EXPERIMENT_NEXT_ACTION_BLOCKED_TOOLS = INFORMATION_GATHERING_TOOLS | {"report_update"}
     453READ_ONLY_SHELL_COMMAND_PATTERN = re.compile(
     454    r"(?is)^\s*(?:"
     455    r"awk\b|cat\b|df\b|du\b|echo\b|find\b|git\s+(?:diff|grep|log|ls-files|show|status)\b|"
     456    r"grep\b|head\b|ls\b|pwd\b|rg\b|sed\s+-n\b|stat\b|tail\b|tree\b|wc\b"
     457    r")"
     458)
     459
     460BROWSER_REF_IGNORE_NAMES = {
     461    "about us",
     462    "back to top",
     463    "careers",
     464    "click here",
     465    "clutch rating",
     466    "organization name",
     467    "contact",
     468    "contact us",
     469    "go",
     470    "headquarters",
     471    "help",
     472    "latest links",
     473    "learn more",
     474    "privacy",
     475    "read more",
     476    "readmore",
     477    "services",
     478    "submit",
     479    "top hits",
     480}
     481
     482ANTI_BOT_ACK_TERMS = (
     483    "anti-bot",
     484    "blocked",
     485    "bot check",
     486    "captcha",
     487    "not usable",
     488    "verification",
     489)

nipux_cli/worker_prompt_context.py 921 lines

       1"""Prompt-context renderers for the Nipux worker loop."""
       2
       3from __future__ import annotations
       4
       5import re
       6from typing import Any
       7
       8from nipux_cli.memory_graph import memory_graph_for_prompt
       9from nipux_cli.metric_format import format_metric_value
      10from nipux_cli.operator_context import active_prompt_operator_entries, operator_entry_is_prompt_relevant
      11from nipux_cli.tui_outcomes import hourly_outcome_summary, model_update_event_parts, outcome_counts
      12from nipux_cli.worker_policy import (
      13    MAX_WORKER_PROMPT_CHARS,
      14    PROMPT_SECTION_BUDGETS,
      15    SECTION_ITEM_CHARS,
      16    TIMELINE_PROMPT_AGENT_TITLES,
      17    TIMELINE_PROMPT_EVENT_TYPES,
      18    TIMELINE_PROMPT_EVENTS,
      19    TIMELINE_PROMPT_TOOL_STATUSES,
      20)
      21from nipux_cli.worker_prompt_format import clip_text as _clip_text
      22
      23
      24NEGATIVE_EXISTENCE_MARKERS = (
      25    "cannot access",
      26    "does not exist",
      27    "failed to find",
      28    "has not been",
      29    "missing",
      30    "no ",
      31    "no such",
      32    "none",
      33    "not available",
      34    "not detected",
      35    "not downloaded",
      36    "not found",
      37    "not installed",
      38    "unavailable",
      39    "was not",
      40    "without",
      41)
      42NEGATIVE_EVIDENCE_LINE_MARKERS = (
      43    "cannot access",
      44    "denied",
      45    "does not exist",
      46    "error",
      47    "failed",
      48    "failure",
      49    "has not been",
      50    "missing",
      51    "no such",
      52    "not available",
      53    "not detected",
      54    "not downloaded",
      55    "not found",
      56    "not installed",
      57    "permission",
      58    "timeout",
      59    "unavailable",
      60    "was not",
      61)
      62
      63
      64def _memory_entries_for_prompt(memory_entries: list[dict[str, Any]], *, limit: int = 2) -> list[dict[str, Any]]:
      65    entries = [entry for entry in memory_entries if isinstance(entry, dict)]
      66    rolling = next((entry for entry in entries if entry.get("key") == "rolling_state"), None)
      67    selected: list[dict[str, Any]] = []
      68    if rolling:
      69        selected.append(rolling)
      70    for entry in entries:
      71        if len(selected) >= limit:
      72            break
      73        if rolling is not None and entry is rolling:
      74            continue
      75        selected.append(entry)
      76    return selected[:limit]
      77
      78
      79def _render_worker_prompt(job: dict[str, Any], *, sections: list[tuple[str, str]]) -> str:
      80    objective = _clip_text(job.get("objective") or "", 2_000)
      81    header = f"Job: {job['title']}\nKind: {job['kind']}\nObjective:\n{objective}"
      82    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
      83    stale_tokens = _stale_claim_tokens_for_prompt(
      84        metadata,
      85        reference_text=" ".join(str(job.get(key) or "") for key in ("title", "objective", "kind")),
      86    )
      87    instruction = (
      88        "Take exactly one bounded next action. If recent state contains search results, do not search the same query again. "
      89        "If recent state contains extracted page evidence, write an artifact before doing more search or browsing."
      90    )
      91    scale = 1.0
      92    while True:
      93        parts = [header]
      94        for title, body in sections:
      95            base_budget = PROMPT_SECTION_BUDGETS.get(title, SECTION_ITEM_CHARS)
      96            budget = max(260, int(base_budget * scale))
      97            safe_body = _redact_stale_tokens_for_prompt(body, stale_tokens)
      98            parts.append(f"{title}:\n{_clip_text(safe_body, budget)}")
      99        parts.append(instruction)
     100        content = "\n\n".join(parts)
     101        if len(content) <= MAX_WORKER_PROMPT_CHARS or scale <= 0.45:
     102            break
     103        scale -= 0.12
     104    if len(content) <= MAX_WORKER_PROMPT_CHARS:
     105        return content
     106    suffix_sections: list[str] = []
     107    for title, body in sections:
     108        if title == "Operator context":
     109            suffix_sections.append(f"Operator context:\n{_clip_text(_redact_stale_tokens_for_prompt(body, stale_tokens), 900)}")
     110        elif title == "Next-action constraint":
     111            suffix_sections.append(f"Next-action constraint:\n{_clip_text(_redact_stale_tokens_for_prompt(body, stale_tokens), 900)}")
     112    suffix = "\n\n".join(suffix_sections + [instruction])
     113    marker = "\n\n...[middle context clipped; operator context and next action repeated below]...\n"
     114    head_budget = max(0, MAX_WORKER_PROMPT_CHARS - len(suffix) - len(marker))
     115    return _clip_text(content, head_budget) + marker + suffix
     116
     117
     118def _redact_stale_tokens_for_prompt(text: str, stale_tokens: list[str]) -> str:
     119    redacted = str(text or "")
     120    for token in sorted((str(token) for token in stale_tokens if str(token).strip()), key=len, reverse=True):
     121        pattern = r"(?<![A-Za-z0-9])" + re.escape(token) + r"(?![A-Za-z0-9])"
     122        redacted = re.sub(
     123            pattern,
     124            lambda match: match.group(0) if _match_inside_path_like_span(match.string, match.start(), match.end()) else "[unsupported-stale-claim]",
     125            redacted,
     126            flags=re.IGNORECASE,
     127        )
     128    return redacted
     129
     130
     131def _match_inside_path_like_span(text: str, start: int, end: int) -> bool:
     132    left = start
     133    while left > 0 and not text[left - 1].isspace() and text[left - 1] not in "'\"`<>|;":
     134        left -= 1
     135    right = end
     136    while right < len(text) and not text[right].isspace() and text[right] not in "'\"`<>|;":
     137        right += 1
     138    span = text[left:right]
     139    return "/" in span or "\\" in span or span.startswith(("~", "."))
     140
     141
     142def _operator_messages_for_prompt(
     143    job: dict[str, Any],
     144    *,
     145    active_messages: list[dict[str, Any]] | None = None,
     146    include_unclaimed: bool = True,
     147) -> str:
     148    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     149    messages = metadata.get("operator_messages") if isinstance(metadata.get("operator_messages"), list) else []
     150    lines = []
     151    active_messages = active_prompt_operator_entries(active_messages or [])
     152    active_ids = {active.get("event_id") for active in active_messages if isinstance(active, dict)}
     153    if active_messages:
     154        lines.append("Newly delivered operator messages for this turn:")
     155    for entry in active_messages:
     156        line = _operator_message_line(entry)
     157        if line:
     158            lines.append(line)
     159    active_context = [
     160        entry
     161        for entry in messages
     162        if isinstance(entry, dict)
     163        and operator_entry_is_prompt_relevant(entry)
     164        and _operator_message_visible_in_prompt(entry, include_unclaimed=include_unclaimed)
     165        and entry.get("event_id") not in active_ids
     166    ]
     167    if active_context:
     168        if lines:
     169            lines.append("Still-active durable operator context:")
     170        for entry in active_context[-6:]:
     171            line = _operator_message_line(entry)
     172            if line:
     173                lines.append(line)
     174    return "\n".join(lines) if lines else "No active operator context."
     175
     176
     177def _operator_message_line(entry: dict[str, Any]) -> str:
     178    if not isinstance(entry, dict):
     179        return ""
     180    at = str(entry.get("at") or "")
     181    source = str(entry.get("source") or "operator")
     182    mode = str(entry.get("mode") or "steer")
     183    event_id = str(entry.get("event_id") or "")
     184    message = " ".join(str(entry.get("message") or "").split())
     185    if message:
     186        states = []
     187        if entry.get("claimed_at"):
     188            states.append("delivered")
     189        if entry.get("acknowledged_at"):
     190            states.append("acknowledged")
     191        if entry.get("superseded_at"):
     192            states.append("superseded")
     193        state_text = f" ({', '.join(states)})" if states else ""
     194        id_text = f" id={event_id}" if event_id else ""
     195        return f"-{id_text} {at} {source} {mode}{state_text}: {_clip_text(message, 420)}"
     196    return ""
     197
     198
     199def _lessons_for_prompt(job: dict[str, Any]) -> str:
     200    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     201    lessons = metadata.get("lessons") if isinstance(metadata.get("lessons"), list) else []
     202    if not lessons:
     203        return "No durable lessons yet."
     204    reference_text = " ".join(str(job.get(key) or "") for key in ("title", "objective", "kind"))
     205    positive_lines = _positive_durable_lines_for_lesson_conflicts(metadata)
     206    stale_lesson_ids = _stale_negative_record_ids(metadata, kind="lesson")
     207    lines = []
     208    for entry in lessons[-5:]:
     209        if not isinstance(entry, dict):
     210            continue
     211        category = str(entry.get("category") or "memory")
     212        raw_lesson = str(entry.get("lesson") or "")
     213        record_id = _record_id_for_staleness(entry)
     214        conflicting_tokens = _negative_lesson_conflict_tokens(raw_lesson, positive_lines)
     215        if conflicting_tokens:
     216            lesson = (
     217                "Potentially stale negative lesson suppressed for "
     218                + ", ".join(conflicting_tokens[:6])
     219                + ". Re-verify against fresh evidence before using this claim."
     220            )
     221        elif record_id in stale_lesson_ids:
     222            lesson = "Potentially stale negative lesson suppressed after fresh contradictory evidence. Re-verify before using this claim."
     223        else:
     224            lesson = _lesson_prompt_text(raw_lesson, reference_text=reference_text)
     225        if lesson:
     226            lines.append(f"- {category}: {_clip_text(lesson, SECTION_ITEM_CHARS)}")
     227    return "\n".join(lines) if lines else "No durable lessons yet."
     228
     229
     230def _lesson_prompt_text(lesson: str, *, reference_text: str = "") -> str:
     231    lesson = " ".join(str(lesson or "").split())
     232    if "unsupported concrete tokens" not in lesson.lower():
     233        return lesson
     234    reference_norm = _normalize_claim_text(reference_text)
     235    stale_tokens = []
     236    seen = set()
     237    for token in _unsupported_tokens_from_lesson(lesson):
     238        cleaned = " ".join(str(token or "").split())
     239        if not cleaned:
     240            continue
     241        key = cleaned.lower()
     242        if reference_norm and _normalize_claim_text(cleaned) in reference_norm:
     243            continue
     244        if key in seen or not _stale_token_is_distinctive(cleaned):
     245            continue
     246        seen.add(key)
     247        stale_tokens.append(cleaned)
     248    if stale_tokens:
     249        return (
     250            "Evidence grounding rejected unsupported durable-record claims: "
     251            + ", ".join(stale_tokens[:8])
     252            + ". Re-verify them from fresh evidence before using them."
     253        )
     254    return "Evidence grounding rejected an unsupported durable record. Re-verify from fresh evidence before using it."
     255
     256
     257def _positive_durable_lines_for_lesson_conflicts(metadata: dict[str, Any]) -> list[str]:
     258    lines: list[str] = []
     259    for key in ("finding_ledger", "experiment_ledger", "source_ledger"):
     260        records = metadata.get(key) if isinstance(metadata.get(key), list) else []
     261        for record in records[-30:]:
     262            if isinstance(record, dict):
     263                text = _dict_scalar_text(record)
     264                if text:
     265                    lines.append(text)
     266    graph = metadata.get("memory_graph") if isinstance(metadata.get("memory_graph"), dict) else {}
     267    nodes = graph.get("nodes") if isinstance(graph.get("nodes"), list) else []
     268    for node in nodes[-30:]:
     269        if isinstance(node, dict):
     270            text = _dict_scalar_text(node)
     271            if text:
     272                lines.append(text)
     273    return lines
     274
     275
     276def _negative_lesson_conflict_tokens(lesson: str, positive_lines: list[str]) -> list[str]:
     277    lesson = " ".join(str(lesson or "").split())
     278    lesson_lower = lesson.lower()
     279    if not positive_lines or not any(marker in lesson_lower for marker in NEGATIVE_EXISTENCE_MARKERS):
     280        return []
     281    tokens = _distinctive_claim_tokens(lesson)
     282    conflicts: list[str] = []
     283    seen: set[str] = set()
     284    for token in tokens:
     285        key = token.lower()
     286        if key in seen:
     287            continue
     288        seen.add(key)
     289        if not _token_near_negative_marker(lesson, token):
     290            continue
     291        if _positive_line_contains_token(positive_lines, token):
     292            conflicts.append(token)
     293    return conflicts
     294
     295
     296def _dict_scalar_text(record: dict[str, Any]) -> str:
     297    parts: list[str] = []
     298    for key, value in record.items():
     299        if key in {"created_at", "updated_at", "event_id", "id"}:
     300            continue
     301        if isinstance(value, (str, int, float, bool)):
     302            parts.append(str(value))
     303        elif isinstance(value, dict):
     304            parts.append(_dict_scalar_text(value))
     305    return " ".join(part for part in parts if part)
     306
     307
     308def _distinctive_claim_tokens(text: str) -> list[str]:
     309    tokens: list[str] = []
     310    for raw in re.findall(r"\b[A-Za-z][A-Za-z0-9_.+-]{1,}\b", text):
     311        token = raw.strip("._+-")
     312        if token and _stale_token_is_distinctive(token):
     313            tokens.append(token)
     314    return tokens
     315
     316
     317def _token_near_negative_marker(text: str, token: str, *, window: int = 140) -> bool:
     318    text_lower = text.lower()
     319    token_lower = token.lower()
     320    start = 0
     321    while True:
     322        index = text_lower.find(token_lower, start)
     323        if index < 0:
     324            return False
     325        nearby = text_lower[max(0, index - window): index + len(token_lower) + window]
     326        if any(marker in nearby for marker in NEGATIVE_EXISTENCE_MARKERS):
     327            return True
     328        start = index + len(token_lower)
     329
     330
     331def _positive_line_contains_token(lines: list[str], token: str) -> bool:
     332    token_lower = token.lower()
     333    for line in lines:
     334        line_lower = line.lower()
     335        if token_lower not in line_lower:
     336            continue
     337        if any(marker in line_lower for marker in NEGATIVE_EVIDENCE_LINE_MARKERS):
     338            continue
     339        return True
     340    return False
     341
     342
     343def _memory_graph_for_prompt(job: dict[str, Any]) -> str:
     344    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     345    stale_tokens = _stale_claim_tokens_for_prompt(
     346        metadata,
     347        reference_text=" ".join(str(job.get(key) or "") for key in ("title", "objective", "kind")),
     348    )
     349    return memory_graph_for_prompt(job, limit=10, stale_tokens=stale_tokens)
     350
     351
     352def _roadmap_for_prompt(job: dict[str, Any]) -> str:
     353    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     354    roadmap = metadata.get("roadmap") if isinstance(metadata.get("roadmap"), dict) else {}
     355    if not roadmap:
     356        return (
     357            "No roadmap yet. If the objective is broad, multi-phase, or needs validation checkpoints, "
     358            "use record_roadmap to define compact milestones, features, acceptance criteria, and validation evidence."
     359        )
     360    milestones = roadmap.get("milestones") if isinstance(roadmap.get("milestones"), list) else []
     361    status_counts: dict[str, int] = {}
     362    validation_counts: dict[str, int] = {}
     363    for milestone in milestones:
     364        if not isinstance(milestone, dict):
     365            continue
     366        status = str(milestone.get("status") or "planned")
     367        validation_status = str(milestone.get("validation_status") or "not_started")
     368        status_counts[status] = status_counts.get(status, 0) + 1
     369        validation_counts[validation_status] = validation_counts.get(validation_status, 0) + 1
     370    lines = [
     371        _clip_text(
     372            f"{roadmap.get('status') or 'planned'}: {roadmap.get('title') or 'Roadmap'}"
     373            + (f" | current={roadmap.get('current_milestone')}" if roadmap.get("current_milestone") else ""),
     374            520,
     375        ),
     376        "Milestone counts: " + (", ".join(f"{key}={value}" for key, value in sorted(status_counts.items())) or "none"),
     377        "Validation counts: " + (", ".join(f"{key}={value}" for key, value in sorted(validation_counts.items())) or "none"),
     378    ]
     379    if roadmap.get("scope"):
     380        lines.append("Scope: " + _clip_text(str(roadmap.get("scope") or ""), 420))
     381    if roadmap.get("validation_contract"):
     382        lines.append("Validation contract: " + _clip_text(str(roadmap.get("validation_contract") or ""), 520))
     383    selected = [
     384        milestone for milestone in milestones
     385        if isinstance(milestone, dict)
     386        and str(milestone.get("status") or "planned") in {"active", "validating", "planned", "blocked"}
     387    ][:6]
     388    if not selected:
     389        selected = [milestone for milestone in milestones if isinstance(milestone, dict)][-4:]
     390    for milestone in selected[:6]:
     391        features = milestone.get("features") if isinstance(milestone.get("features"), list) else []
     392        open_features = sum(1 for feature in features if isinstance(feature, dict) and str(feature.get("status") or "planned") in {"planned", "active"})
     393        detail = " | ".join(
     394            bit
     395            for bit in [
     396                str(milestone.get("status") or "planned"),
     397                f"validation={milestone.get('validation_status') or 'not_started'}",
     398                f"p={milestone.get('priority') or 0}",
     399                str(milestone.get("title") or "milestone"),
     400                f"features={len(features)}/{open_features} open" if features else "",
     401            ]
     402            if bit
     403        )
     404        if milestone.get("acceptance_criteria"):
     405            detail += f" | accept={milestone.get('acceptance_criteria')}"
     406        if milestone.get("evidence_needed"):
     407            detail += f" | evidence={milestone.get('evidence_needed')}"
     408        if milestone.get("validation_result"):
     409            detail += f" | validation_result={milestone.get('validation_result')}"
     410        if milestone.get("next_action"):
     411            detail += f" | next={milestone.get('next_action')}"
     412        lines.append("- " + _clip_text(detail, 620))
     413    return "\n".join(lines)
     414
     415
     416def _tasks_for_prompt(job: dict[str, Any]) -> str:
     417    tasks = _metadata_list(job, "task_queue")
     418    if not tasks:
     419        return (
     420            "No durable task queue yet. If the objective is broad, use record_tasks "
     421            "to create a few concrete open branches with output contracts and acceptance criteria before continuing."
     422        )
     423    status_rank = {"active": 0, "open": 1, "blocked": 2, "done": 3, "skipped": 4}
     424    ranked = sorted(
     425        tasks,
     426        key=lambda task: (status_rank.get(str(task.get("status") or "open"), 9), -_as_int(task.get("priority"))),
     427    )
     428    counts: dict[str, int] = {}
     429    for task in tasks:
     430        status = str(task.get("status") or "open")
     431        counts[status] = counts.get(status, 0) + 1
     432    lines = ["Task counts: " + ", ".join(f"{key}={value}" for key, value in sorted(counts.items()))]
     433    selected = [task for task in ranked if str(task.get("status") or "open") in {"active", "open"}][:6]
     434    if len(selected) < 6:
     435        selected.extend([task for task in ranked if str(task.get("status") or "open") == "blocked"][: 6 - len(selected)])
     436    if len(selected) < 6:
     437        selected.extend([task for task in ranked if task not in selected][: 6 - len(selected)])
     438    for task in selected[:6]:
     439        output_contract = _task_output_contract(task)
     440        bits = [
     441            str(task.get("status") or "open"),
     442            f"priority={task.get('priority') or 0}",
     443            str(task.get("title") or "untitled"),
     444        ]
     445        if output_contract:
     446            bits.append(f"contract={output_contract}")
     447        detail = " | ".join(bit for bit in bits if bit)
     448        if task.get("goal"):
     449            detail += f" | goal={task.get('goal')}"
     450        if task.get("acceptance_criteria"):
     451            detail += f" | accept={task.get('acceptance_criteria')}"
     452        if task.get("evidence_needed"):
     453            detail += f" | evidence={task.get('evidence_needed')}"
     454        if task.get("stall_behavior"):
     455            detail += f" | stall={task.get('stall_behavior')}"
     456        if task.get("source_hint"):
     457            detail += f" | source_hint={task.get('source_hint')}"
     458        if task.get("result"):
     459            detail += f" | result={task.get('result')}"
     460        lines.append("- " + _clip_text(detail, 520))
     461    return "\n".join(lines)
     462
     463
     464def _task_output_contract(task: dict[str, Any]) -> str:
     465    metadata = task.get("metadata") if isinstance(task.get("metadata"), dict) else {}
     466    return str(task.get("output_contract") or task.get("contract") or metadata.get("output_contract") or metadata.get("contract") or "")
     467
     468
     469def _timeline_for_prompt(events: list[dict[str, Any]]) -> str:
     470    if not events:
     471        return "No timeline events yet."
     472    selected: list[tuple[str, str, str]] = []
     473    counts: dict[str, int] = {}
     474    for event in events:
     475        rendered = _timeline_event_for_prompt(event)
     476        if not rendered:
     477            continue
     478        at, event_type, detail = rendered
     479        counts[event_type] = counts.get(event_type, 0) + 1
     480        selected.append((at, event_type, detail))
     481    if not selected:
     482        return "No high-signal timeline events yet. Recent state covers raw tool activity."
     483    summary = ", ".join(f"{key}={value}" for key, value in sorted(counts.items()))
     484    lines = [f"High-signal timeline counts: {summary}"]
     485    for at, event_type, detail in selected[-TIMELINE_PROMPT_EVENTS:]:
     486        prefix = f"- {at} {event_type}: " if at else f"- {event_type}: "
     487        lines.append(prefix + _clip_text(detail, SECTION_ITEM_CHARS))
     488    return "\n".join(lines)
     489
     490
     491def _outcomes_for_prompt(events: list[dict[str, Any]]) -> str:
     492    """Summarize durable outputs so the worker sees progress, not just activity."""
     493
     494    if not events:
     495        return "No durable outcomes visible in recent timeline."
     496    counts = outcome_counts(events, include_research=False, include_failures=True)
     497    summary = hourly_outcome_summary(counts)
     498    lines = [f"Outcome counts: {summary or 'none'}."]
     499    seen: set[str] = set()
     500    for event in reversed(events):
     501        parsed = model_update_event_parts(event, width=240, compact=False)
     502        if not parsed:
     503            continue
     504        label, text, _clock = parsed
     505        if label in {"DONE", "PLAN", "UPDATE"}:
     506            continue
     507        key = f"{label}:{text}"
     508        if key in seen:
     509            continue
     510        seen.add(key)
     511        lines.append(f"- {label.lower()}: {_clip_text(text, 360)}")
     512        if len(lines) >= 8:
     513            break
     514    if len(lines) == 1:
     515        lines.append("No durable output/finding/measurement records are visible; prioritize creating or accounting for one.")
     516    return "\n".join(lines)
     517
     518
     519def _ledgers_for_prompt(job: dict[str, Any]) -> str:
     520    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     521    findings = _metadata_list(job, "finding_ledger")
     522    sources = _metadata_list(job, "source_ledger")
     523    stale_tokens = _stale_claim_tokens_for_prompt(
     524        metadata,
     525        reference_text=" ".join(str(job.get(key) or "") for key in ("title", "objective", "kind")),
     526    )
     527    stale_record_ids = _stale_negative_record_ids(metadata, kind="finding")
     528    stale_findings = [
     529        finding
     530        for finding in findings
     531        if _record_contains_stale_token(finding, stale_tokens)
     532        or _record_id_for_staleness(finding) in stale_record_ids
     533    ]
     534    active_findings = [finding for finding in findings if finding not in stale_findings]
     535    lines = [
     536        f"Finding ledger: {len(findings)} unique candidates.",
     537        f"Source ledger: {len(sources)} scored sources.",
     538    ]
     539    if stale_tokens:
     540        lines.append(
     541            "Unsupported/stale claim tokens to avoid until re-verified: "
     542            + ", ".join(stale_tokens[:12])
     543        )
     544    if active_findings:
     545        lines.append("Recent findings:")
     546        for finding in active_findings[-5:]:
     547            bits = [
     548                str(finding.get("name") or "unknown"),
     549                str(finding.get("category") or "").strip(),
     550                str(finding.get("location") or "").strip(),
     551                f"score={finding.get('score')}" if finding.get("score") is not None else "",
     552            ]
     553            lines.append("- " + _clip_text(" | ".join(bit for bit in bits if bit), 360))
     554    if stale_findings:
     555        lines.append(
     556            f"Suppressed {len(stale_findings)} stale finding(s) matching unsupported tokens; "
     557            "do not use them as facts until observed again."
     558        )
     559    stale_negative_records = _stale_negative_records_for_prompt(metadata, kind="finding")
     560    if stale_negative_records:
     561        lines.append("Contradicted negative findings suppressed:")
     562        for record in stale_negative_records[-4:]:
     563            lines.append(
     564                "- "
     565                + _clip_text(
     566                    f"{record.get('title') or 'finding'} token={record.get('token') or ''} evidence={record.get('evidence') or ''}",
     567                    360,
     568                )
     569            )
     570    if sources:
     571        usable_sources = [
     572            source
     573            for source in sources
     574            if _as_float(source.get("usefulness_score")) >= 0.2
     575            or _as_int(source.get("yield_count")) > 0
     576        ]
     577        low_quality_sources = [
     578            source
     579            for source in sources
     580            if _as_float(source.get("usefulness_score")) < 0.2
     581            and _as_int(source.get("yield_count")) <= 0
     582            and (_as_int(source.get("fail_count")) > 0 or source.get("warnings"))
     583        ]
     584        ranked = sorted(
     585            usable_sources,
     586            key=lambda item: (_as_float(item.get("usefulness_score")), _as_int(item.get("yield_count"))),
     587            reverse=True,
     588        )
     589        if ranked:
     590            lines.append("High-yield/current sources:")
     591            for source in ranked[:4]:
     592                lines.append(
     593                    "- "
     594                    + _clip_text(
     595                        f"{source.get('source')} type={source.get('source_type') or 'unknown'} "
     596                        f"score={source.get('usefulness_score')} findings={source.get('yield_count') or 0} "
     597                        f"fails={source.get('fail_count') or 0} outcome={source.get('last_outcome') or ''}",
     598                        420,
     599                    )
     600                )
     601        if low_quality_sources:
     602            lines.append("Low-yield/blocked source patterns to avoid:")
     603            for source in low_quality_sources[-3:]:
     604                lines.append(
     605                    "- "
     606                    + _clip_text(
     607                        f"{source.get('source')} type={source.get('source_type') or 'unknown'} "
     608                        f"score={source.get('usefulness_score')} fails={source.get('fail_count') or 0} "
     609                        f"warnings={', '.join(source.get('warnings') or [])} outcome={source.get('last_outcome') or ''}",
     610                        420,
     611                    )
     612                )
     613    return "\n".join(lines)
     614
     615
     616def _stale_claim_tokens_for_prompt(metadata: dict[str, Any], *, reference_text: str = "") -> list[str]:
     617    raw_tokens = metadata.get("unsupported_claim_tokens")
     618    tokens: list[str] = []
     619    seen: set[str] = set()
     620    candidates: list[Any] = []
     621    reference_norm = _normalize_claim_text(reference_text)
     622    if isinstance(raw_tokens, list):
     623        candidates.extend(raw_tokens)
     624    stale_records = metadata.get("stale_negative_records")
     625    if isinstance(stale_records, list):
     626        for record in stale_records:
     627            if isinstance(record, dict):
     628                candidates.append(record.get("token"))
     629    lessons = metadata.get("lessons")
     630    if isinstance(lessons, list):
     631        for lesson in lessons[-25:]:
     632            if not isinstance(lesson, dict):
     633                continue
     634            candidates.extend(_unsupported_tokens_from_lesson(str(lesson.get("lesson") or "")))
     635    for raw in candidates:
     636        token = " ".join(str(raw or "").split())
     637        if not token:
     638            continue
     639        if reference_norm and _normalize_claim_text(token) in reference_norm:
     640            continue
     641        if not _stale_token_is_distinctive(token):
     642            continue
     643        key = token.lower()
     644        if key in seen:
     645            continue
     646        seen.add(key)
     647        tokens.append(token)
     648    return tokens[-100:]
     649
     650
     651def _unsupported_tokens_from_lesson(lesson: str) -> list[str]:
     652    marker = "unsupported concrete tokens"
     653    if marker not in lesson.lower():
     654        return []
     655    match = re.search(r"unsupported concrete tokens for .*?:\s*(.*?)(?:\.\s+Treat matching|$)", lesson, flags=re.IGNORECASE)
     656    if not match:
     657        match = re.search(r"unsupported concrete tokens for .*?:\s*(.*?)(?:\.|$)", lesson, flags=re.IGNORECASE)
     658    if not match:
     659        return []
     660    return [part.strip() for part in match.group(1).split(",") if part.strip()]
     661
     662
     663def _stale_token_is_distinctive(token: str) -> bool:
     664    lowered = token.lower()
     665    if lowered.startswith(".") and re.match(r"^\.[a-z0-9][a-z0-9_-]{1,12}$", lowered):
     666        return lowered not in {".app", ".co", ".com", ".dev", ".edu", ".gov", ".io", ".net", ".org", ".www"}
     667    if lowered in {
     668        "api",
     669        "ascii",
     670        "blocked",
     671        "broken",
     672        "candidate",
     673        "candidates",
     674        "cdn",
     675        "cli",
     676        "critical",
     677        "cpu",
     678        "cuda",
     679        "discovered",
     680        "discovery",
     681        "ggml",
     682        "gguf",
     683        "gpu",
     684        "hf_token",
     685        "html",
     686        "http",
     687        "https",
     688        "incomplete",
     689        "json",
     690        "lfs",
     691        "not_found",
     692        "oid",
     693        "onnx",
     694        "planned",
     695        "python",
     696        "python3",
     697        "ram",
     698        "rest",
     699        "severe",
     700        "sha",
     701        "sha256",
     702        "search",
     703        "usable",
     704        "unvalidated",
     705        "valid",
     706        "validity",
     707        "validate",
     708        "validated",
     709        "vram",
     710        "xml",
     711        "xet",
     712        "yaml",
     713        "yml",
     714    }:
     715        return False
     716    if lowered.startswith((
     717        "art_",
     718        "step_",
     719        "step-",
     720        "shell_",
     721        "shell-",
     722        "web_",
     723        "web-",
     724        "episode-",
     725        "fact-",
     726        "source-",
     727        "quality-",
     728        "constraint-",
     729        "baseline-",
     730        "question-",
     731        "verified_",
     732        "verified-",
     733        "timeout_",
     734        "timeout-",
     735    )):
     736        return False
     737    if lowered.endswith((".md", ".py", ".json", ".yaml", ".yml", ".gguf", ".txt", ".log")):
     738        return False
     739    if lowered.startswith(("python-", "pip", "pip3")):
     740        return False
     741    if len(token) < 4:
     742        return False
     743    return (any(ch.isalpha() for ch in token) and any(ch.isdigit() for ch in token)) or (token.isupper() and len(token) >= 4)
     744
     745
     746def _normalize_claim_text(text: str) -> str:
     747    return re.sub(r"[^a-z0-9]+", "", str(text or "").lower())
     748
     749
     750def _record_contains_stale_token(record: dict[str, Any], stale_tokens: list[str]) -> bool:
     751    if not stale_tokens:
     752        return False
     753    text = " ".join(
     754        str(record.get(key) or "")
     755        for key in ("name", "category", "location", "contact", "reason", "source_url", "url")
     756    )
     757    metadata = record.get("metadata") if isinstance(record.get("metadata"), dict) else {}
     758    text += " " + " ".join(str(value) for value in metadata.values() if isinstance(value, (str, int, float)))
     759    for token in stale_tokens:
     760        pattern = r"(?<![A-Za-z0-9])" + re.escape(token) + r"(?![A-Za-z0-9])"
     761        if re.search(pattern, text, flags=re.IGNORECASE):
     762            return True
     763    return False
     764
     765
     766def _stale_negative_records_for_prompt(metadata: dict[str, Any], *, kind: str) -> list[dict[str, Any]]:
     767    records = metadata.get("stale_negative_records")
     768    if not isinstance(records, list):
     769        return []
     770    return [
     771        record
     772        for record in records
     773        if isinstance(record, dict) and str(record.get("kind") or "") == kind
     774    ]
     775
     776
     777def _stale_negative_record_ids(metadata: dict[str, Any], *, kind: str) -> set[str]:
     778    ids: set[str] = set()
     779    for record in _stale_negative_records_for_prompt(metadata, kind=kind):
     780        record_id = str(record.get("record_id") or "").strip()
     781        if record_id:
     782            ids.add(record_id)
     783    return ids
     784
     785
     786def _record_id_for_staleness(record: dict[str, Any]) -> str:
     787    for key in ("key", "event_id", "id"):
     788        value = str(record.get(key) or "").strip()
     789        if value:
     790            return value
     791    return _normalize_claim_text(str(record.get("name") or record.get("title") or ""))[:120]
     792
     793
     794def _experiments_for_prompt(job: dict[str, Any]) -> str:
     795    experiments = _metadata_list(job, "experiment_ledger")
     796    if not experiments:
     797        return (
     798            "No experiments tracked yet. If this objective involves improving, "
     799            "comparing, benchmarking, reducing, increasing, or otherwise measuring something, "
     800            "turn candidate ideas into record_experiment entries with exact config, metric, result, and next action."
     801        )
     802    measured = [experiment for experiment in experiments if experiment.get("metric_value") is not None]
     803    best = [
     804        experiment
     805        for experiment in measured
     806        if bool(experiment.get("best_observed"))
     807    ]
     808    status_counts: dict[str, int] = {}
     809    for experiment in experiments:
     810        status = str(experiment.get("status") or "planned")
     811        status_counts[status] = status_counts.get(status, 0) + 1
     812    lines = [
     813        f"Experiment counts: {', '.join(f'{key}={value}' for key, value in sorted(status_counts.items()))}.",
     814        f"Measured results: {len(measured)}.",
     815    ]
     816    if best:
     817        lines.append("Best observed results:")
     818        for experiment in best[-3:]:
     819            metric = format_metric_value(
     820                experiment.get("metric_name") or "metric",
     821                experiment.get("metric_value"),
     822                experiment.get("metric_unit") or "",
     823            )
     824            lines.append(
     825                "- "
     826                + _clip_text(" | ".join(
     827                    bit
     828                    for bit in [
     829                        str(experiment.get("title") or "experiment"),
     830                        metric,
     831                        f"result={experiment.get('result')}" if experiment.get("result") else "",
     832                        f"next={experiment.get('next_action')}" if experiment.get("next_action") else "",
     833                    ]
     834                    if bit
     835                ), 520)
     836            )
     837    recent = experiments[-4:]
     838    if recent:
     839        lines.append("Recent experiments:")
     840        for experiment in recent:
     841            metric = ""
     842            if experiment.get("metric_value") is not None:
     843                metric = format_metric_value(
     844                    experiment.get("metric_name") or "metric",
     845                    experiment.get("metric_value"),
     846                    experiment.get("metric_unit") or "",
     847                )
     848            delta = ""
     849            if experiment.get("delta_from_previous_best") is not None:
     850                delta = f"delta={experiment.get('delta_from_previous_best')}"
     851            lines.append(
     852                "- "
     853                + _clip_text(" | ".join(
     854                    bit
     855                    for bit in [
     856                        str(experiment.get("status") or "planned"),
     857                        str(experiment.get("title") or "experiment"),
     858                        metric,
     859                        delta,
     860                        f"next={experiment.get('next_action')}" if experiment.get("next_action") else "",
     861                    ]
     862                    if bit
     863                ), 520)
     864            )
     865    return "\n".join(lines)
     866
     867
     868def _operator_message_visible_in_prompt(entry: dict[str, Any], *, include_unclaimed: bool) -> bool:
     869    mode = str(entry.get("mode") or "steer").strip().lower().replace("-", "_")
     870    if entry.get("claimed_at") or mode == "note":
     871        return True
     872    return include_unclaimed and mode == "steer"
     873
     874
     875def _metadata_list(job: dict[str, Any], key: str) -> list[dict[str, Any]]:
     876    metadata = job.get("metadata") if isinstance(job.get("metadata"), dict) else {}
     877    values = metadata.get(key)
     878    if not isinstance(values, list):
     879        return []
     880    return [value for value in values if isinstance(value, dict)]
     881
     882
     883def _timeline_event_for_prompt(event: dict[str, Any]) -> tuple[str, str, str] | None:
     884    event_type = str(event.get("event_type") or "event")
     885    if event_type == "operator_message":
     886        return None
     887    title = " ".join(str(event.get("title") or "").split())
     888    body = " ".join(str(event.get("body") or "").split())
     889    metadata = event.get("metadata") if isinstance(event.get("metadata"), dict) else {}
     890    title_lower = title.lower()
     891    if event_type == "tool_result":
     892        status = str(metadata.get("status") or "").lower()
     893        if status not in TIMELINE_PROMPT_TOOL_STATUSES:
     894            return None
     895    elif event_type == "agent_message":
     896        if title_lower not in TIMELINE_PROMPT_AGENT_TITLES:
     897            return None
     898    elif event_type not in TIMELINE_PROMPT_EVENT_TYPES:
     899        return None
     900    at = str(event.get("created_at") or "")
     901    detail = title if title else event_type
     902    if body:
     903        detail = f"{detail}: {body}"
     904    if event_type == "tool_result":
     905        status = str(metadata.get("status") or "").lower()
     906        detail = f"{status} {detail}".strip()
     907    return at, event_type, detail
     908
     909
     910def _as_float(value: Any, default: float = 0.0) -> float:
     911    try:
     912        return float(value)
     913    except (TypeError, ValueError):
     914        return default
     915
     916
     917def _as_int(value: Any, default: int = 0) -> int:
     918    try:
     919        return int(value)
     920    except (TypeError, ValueError):
     921        return default

nipux_cli/worker_prompt_format.py 223 lines

       1"""Prompt-facing summaries for worker history and tool observations."""
       2
       3from __future__ import annotations
       4
       5import json
       6import re
       7from pathlib import Path
       8from typing import Any
       9
      10from nipux_cli.metric_format import format_metric_value
      11from nipux_cli.source_quality import anti_bot_reason
      12from nipux_cli.worker_policy import BROWSER_REF_IGNORE_NAMES
      13
      14
      15def compact(value: Any, limit: int = 500) -> str:
      16    text = json.dumps(value, ensure_ascii=False, sort_keys=True) if not isinstance(value, str) else value
      17    text = " ".join(text.split())
      18    return text if len(text) <= limit else text[:limit] + "..."
      19
      20
      21def clip_text(value: Any, limit: int) -> str:
      22    text = " ".join(str(value or "").split())
      23    if len(text) <= limit:
      24        return text
      25    return text[: max(0, limit - 3)].rstrip() + "..."
      26
      27
      28def format_step_for_prompt(step: dict[str, Any]) -> str:
      29    tool = f" tool={step['tool_name']}" if step.get("tool_name") else ""
      30    summary = step.get("summary") or step.get("error") or ""
      31    pieces = [f"- #{step['step_no']} {step['kind']} {step['status']}{tool}: {summary}"]
      32    input_data = step.get("input") or {}
      33    args = input_data.get("arguments") if isinstance(input_data, dict) else None
      34    if args:
      35        pieces.append(f"  args: {compact(args, 320)}")
      36    output = step.get("output") or {}
      37    observation = observation_for_prompt(step.get("tool_name"), output)
      38    if observation:
      39        pieces.append(f"  observed: {observation}")
      40    return "\n".join(pieces)
      41
      42
      43def observation_for_prompt(tool_name: str | None, output: dict[str, Any]) -> str:
      44    if not output:
      45        return ""
      46    if output.get("error"):
      47        if tool_name in {"browser_click", "browser_type"}:
      48            recovery = output.get("recovery_snapshot") if isinstance(output.get("recovery_snapshot"), dict) else {}
      49            candidates = browser_candidates_for_prompt(recovery)
      50            suffix = f"; recovery_candidates={candidates}" if candidates else ""
      51            return clip_text(f"error={output.get('error')}; guidance={output.get('recovery_guidance', '')}{suffix}", 700)
      52        evidence_grounding = output.get("evidence_grounding") if isinstance(output.get("evidence_grounding"), dict) else {}
      53        if evidence_grounding:
      54            missing_paths = evidence_grounding.get("missing_candidate_paths")
      55            if isinstance(missing_paths, list) and missing_paths:
      56                missing_paths = [path for path in (clean_prompt_candidate_path(item) for item in missing_paths) if path]
      57                return clip_text(
      58                    "error=evidence grounding required; missing_exact_paths="
      59                    + ", ".join(str(path) for path in missing_paths[:8])
      60                    + "; rewrite the durable record with exact observed paths or state why they are irrelevant",
      61                    900,
      62                )
      63            unsupported = evidence_grounding.get("unsupported_tokens")
      64            if isinstance(unsupported, list) and unsupported:
      65                return clip_text(
      66                    "error=evidence grounding required; unsupported="
      67                    + ", ".join(str(token) for token in unsupported[:10])
      68                    + "; use only tokens present in recent observed evidence or record uncertainty",
      69                    700,
      70                )
      71        recent_artifacts = output.get("recent_artifacts") if isinstance(output.get("recent_artifacts"), list) else []
      72        if tool_name == "read_artifact" and recent_artifacts:
      73            refs = []
      74            for artifact in recent_artifacts[:6]:
      75                if not isinstance(artifact, dict):
      76                    continue
      77                number = str(artifact.get("number") or "").strip()
      78                artifact_id = str(artifact.get("id") or "").strip()
      79                title = str(artifact.get("title") or "").strip()
      80                label = artifact_id or number
      81                if label:
      82                    refs.append(f"{label}={title}" if title else label)
      83            suffix = f"; valid_recent_artifacts={'; '.join(refs)}" if refs else ""
      84            return clip_text(f"error={output.get('error')}; guidance={output.get('guidance') or ''}{suffix}", 900)
      85        return clip_text(f"error={output.get('error')}; guidance={output.get('guidance') or ''}", 700)
      86    if tool_name == "web_search":
      87        results = output.get("results") if isinstance(output.get("results"), list) else []
      88        titles = []
      89        for result in results[:5]:
      90            title = result.get("title") or "untitled"
      91            url = result.get("url") or ""
      92            titles.append(f"{title} <{url}>")
      93        return clip_text(f"query={output.get('query')!r}; results={'; '.join(titles)}", 650)
      94    if tool_name == "web_extract":
      95        pages = output.get("pages") if isinstance(output.get("pages"), list) else []
      96        parts = []
      97        for page in pages[:3]:
      98            if page.get("error"):
      99                parts.append(f"{page.get('url')}: ERROR {page.get('error')}")
     100            else:
     101                text = str(page.get("text") or "")
     102                parts.append(f"{page.get('url')}: {clip_text(text, 160)}")
     103        return clip_text("; ".join(parts), 650)
     104    if tool_name == "shell_exec":
     105        stdout = str(output.get("stdout") or "")
     106        stderr = str(output.get("stderr") or "")
     107        excerpt = stdout.strip() or stderr.strip()
     108        return (
     109            f"command={output.get('command')!r}; rc={output.get('returncode')}; "
     110            f"duration={output.get('duration_seconds')}s; output={clip_text(excerpt, 360)}"
     111        )[:650]
     112    if tool_name == "write_artifact":
     113        return f"saved artifact={output.get('artifact_id')} path={output.get('path')}"
     114    if tool_name == "report_update":
     115        update = output.get("update") if isinstance(output.get("update"), dict) else {}
     116        return clip_text(f"agent_update={update.get('message') or ''}", 420)
     117    if tool_name == "record_lesson":
     118        lesson = output.get("lesson") if isinstance(output.get("lesson"), dict) else {}
     119        return clip_text(f"lesson={lesson.get('category') or 'memory'}: {lesson.get('lesson') or ''}", 420)
     120    if tool_name == "record_memory_graph":
     121        return (
     122            f"memory_graph added_nodes={output.get('added_nodes')} "
     123            f"updated_nodes={output.get('updated_nodes')} added_edges={output.get('added_edges')}"
     124        )[:520]
     125    if tool_name == "search_memory_graph":
     126        nodes = output.get("nodes") if isinstance(output.get("nodes"), list) else []
     127        titles = [str(node.get("title") or node.get("key") or "memory") for node in nodes[:5] if isinstance(node, dict)]
     128        return clip_text(f"memory_query={output.get('query')!r}; nodes={'; '.join(titles)}", 520)
     129    if tool_name == "record_source":
     130        source = output.get("source") if isinstance(output.get("source"), dict) else {}
     131        return (
     132            f"source={source.get('source')} score={source.get('usefulness_score')} "
     133            f"findings={source.get('yield_count')} fails={source.get('fail_count')} outcome={source.get('last_outcome')}"
     134        )[:420]
     135    if tool_name == "record_findings":
     136        return f"finding ledger updated added={output.get('added')} updated={output.get('updated')}"[:700]
     137    if tool_name == "record_experiment":
     138        experiment = output.get("experiment") if isinstance(output.get("experiment"), dict) else {}
     139        metric = ""
     140        if experiment.get("metric_value") is not None:
     141            metric = format_metric_value(
     142                experiment.get("metric_name") or "metric",
     143                experiment.get("metric_value"),
     144                experiment.get("metric_unit") or "",
     145            )
     146        delta = f" delta={experiment.get('delta_from_previous_best')}" if experiment.get("delta_from_previous_best") is not None else ""
     147        best = " best_observed" if experiment.get("best_observed") else ""
     148        return clip_text(f"experiment={experiment.get('title')} status={experiment.get('status')} {metric}{delta}{best}", 520)
     149    if tool_name == "acknowledge_operator_context":
     150        return f"operator_context {output.get('status')} count={output.get('count')}"[:700]
     151    if tool_name == "browser_navigate":
     152        data = output.get("data") if isinstance(output.get("data"), dict) else {}
     153        title = data.get("title") or ""
     154        url = data.get("url") or ""
     155        snapshot = str(output.get("snapshot") or "")
     156        warning = anti_bot_reason(title, url, snapshot)
     157        suffix = f"; source_warning={warning}" if warning else ""
     158        candidates = browser_candidates_for_prompt(output)
     159        candidate_suffix = f"; candidates={candidates}" if candidates else ""
     160        return clip_text(f"opened {title} <{url}>; snapshot_chars={len(snapshot)}{suffix}{candidate_suffix}", 700)
     161    if tool_name == "browser_snapshot":
     162        data = output.get("data") if isinstance(output.get("data"), dict) else {}
     163        snapshot = str(output.get("snapshot") or data.get("snapshot") or output.get("data") or "")
     164        warning = anti_bot_reason(snapshot)
     165        suffix = f"; source_warning={warning}" if warning else ""
     166        candidates = browser_candidates_for_prompt(output)
     167        candidate_suffix = f"; candidates={candidates}" if candidates else ""
     168        return clip_text(f"snapshot_chars={len(snapshot)}{suffix}{candidate_suffix}", 700)
     169    return compact(output, 700)
     170
     171
     172def clean_prompt_candidate_path(value: Any) -> str:
     173    raw = str(value or "").strip().rstrip(".,:;)")
     174    if not raw or "://" in raw or raw.startswith("//") or "..." in raw or "…" in raw or "*" in raw:
     175        return ""
     176    name = Path(raw).name
     177    suffix = Path(name).suffix
     178    if not name or name.startswith(".") or not suffix:
     179        return ""
     180    if not re.match(r"^\.[A-Za-z0-9][A-Za-z0-9_]{1,12}$", suffix) or not any(ch.isalpha() for ch in suffix):
     181        return ""
     182    return raw
     183
     184
     185def browser_candidates_for_prompt(output: dict[str, Any], *, limit: int = 18) -> str:
     186    refs = output.get("refs") if isinstance(output.get("refs"), dict) else None
     187    if refs is None:
     188        data = output.get("data") if isinstance(output.get("data"), dict) else {}
     189        refs = data.get("refs") if isinstance(data.get("refs"), dict) else {}
     190    candidates = []
     191    seen = set()
     192    for ref, item in refs.items():
     193        if not isinstance(item, dict):
     194            continue
     195        role = str(item.get("role") or "")
     196        if role not in {"link", "heading", "cell"}:
     197            continue
     198        name = " ".join(str(item.get("name") or "").split())
     199        key = name.lower().strip()
     200        if not name or key in BROWSER_REF_IGNORE_NAMES:
     201            continue
     202        if len(name) < 3 or len(name) > 90 or key in seen:
     203            continue
     204        if role == "cell" and (_looks_like_metric_cell(name) or _looks_like_service_description(name)):
     205            continue
     206        seen.add(key)
     207        candidates.append(f"{name} (@{ref})")
     208        if len(candidates) >= limit:
     209            break
     210    return "; ".join(candidates)
     211
     212
     213def _looks_like_metric_cell(name: str) -> bool:
     214    text = name.strip()
     215    return bool(re.fullmatch(r"(?:n/?a|na|[-+]?\d+(?:\.\d+)?(?:/5)?|[$€£]?\d[\d,]*(?:\.\d+)?%?)", text, re.I))
     216
     217
     218def _looks_like_service_description(name: str) -> bool:
     219    text = name.lower()
     220    if "," in text and len(text.split()) >= 6:
     221        return True
     222    service_terms = ("custom ecommerce", "ux/ui", "payment integration", "mobile responsiveness", "headless commerce")
     223    return any(term in text for term in service_terms) and len(text.split()) >= 5

nipux_cli/worker_tool_summary.py 99 lines

       1"""Compact result summaries for worker tool executions."""
       2
       3from __future__ import annotations
       4
       5from typing import Any
       6
       7from nipux_cli.metric_format import format_metric_value
       8
       9
      10def summarize_tool_result(name: str, args: dict[str, Any], result: dict[str, Any], *, ok: bool) -> str:
      11    if not ok:
      12        return f"{name} failed: {result.get('error') or 'unknown error'}"
      13    if name == "web_search":
      14        results = result.get("results") if isinstance(result.get("results"), list) else []
      15        top = "; ".join((item.get("title") or "untitled") for item in results[:3])
      16        return f"web_search query={args.get('query')!r} returned {len(results)} results: {top}"
      17    if name == "web_extract":
      18        pages = result.get("pages") if isinstance(result.get("pages"), list) else []
      19        ok_pages = [page for page in pages if not page.get("error")]
      20        return f"web_extract fetched {len(ok_pages)}/{len(pages)} pages"
      21    if name == "shell_exec":
      22        command = str(result.get("command") or args.get("command") or "")
      23        return (
      24            f"shell_exec rc={result.get('returncode')} "
      25            f"duration={result.get('duration_seconds')}s cmd={command!r}"
      26        )
      27    if name == "write_artifact":
      28        return f"write_artifact saved {result.get('artifact_id')} at {result.get('path')}"
      29    if name == "write_file":
      30        return f"write_file {result.get('mode') or 'overwrite'} {result.get('path')} bytes={result.get('bytes')}"
      31    if name == "defer_job":
      32        return f"defer_job until {result.get('defer_until')}"
      33    if name == "report_update":
      34        update = result.get("update") if isinstance(result.get("update"), dict) else {}
      35        return f"report_update saved: {str(update.get('message') or '')[:160]}"
      36    if name == "record_lesson":
      37        lesson = result.get("lesson") if isinstance(result.get("lesson"), dict) else {}
      38        category = lesson.get("category") or "memory"
      39        text = str(lesson.get("lesson") or "")[:160]
      40        return f"record_lesson saved {category}: {text}"
      41    if name == "record_memory_graph":
      42        return (
      43            f"record_memory_graph updated: {result.get('added_nodes', 0)} new nodes, "
      44            f"{result.get('updated_nodes', 0)} updated, {result.get('added_edges', 0)} links"
      45        )
      46    if name == "search_memory_graph":
      47        nodes = result.get("nodes") if isinstance(result.get("nodes"), list) else []
      48        return f"search_memory_graph returned {len(nodes)} nodes for {args.get('query')!r}"
      49    if name == "record_source":
      50        source = result.get("source") if isinstance(result.get("source"), dict) else {}
      51        return f"record_source updated {source.get('source')} score={source.get('usefulness_score')} yield={source.get('yield_count')}"
      52    if name == "record_findings":
      53        return (
      54            f"record_findings updated ledger: {result.get('added', 0)} new, "
      55            f"{result.get('updated', 0)} updated, {result.get('sources_updated', 0)} sources"
      56        )
      57    if name == "record_tasks":
      58        return f"record_tasks updated queue: {result.get('added', 0)} new, {result.get('updated', 0)} updated"
      59    if name == "record_roadmap":
      60        roadmap = result.get("roadmap") if isinstance(result.get("roadmap"), dict) else {}
      61        return (
      62            f"record_roadmap {roadmap.get('status')}: {roadmap.get('title')} "
      63            f"milestones={len(roadmap.get('milestones') or [])}"
      64        )
      65    if name == "record_milestone_validation":
      66        validation = result.get("validation") if isinstance(result.get("validation"), dict) else {}
      67        return (
      68            f"record_milestone_validation {validation.get('validation_status')}: "
      69            f"{validation.get('title')} followups={len(result.get('follow_up_tasks') or [])}"
      70        )
      71    if name == "record_experiment":
      72        experiment = result.get("experiment") if isinstance(result.get("experiment"), dict) else {}
      73        metric = ""
      74        if experiment.get("metric_value") is not None:
      75            metric = " " + format_metric_value(
      76                experiment.get("metric_name") or "metric",
      77                experiment.get("metric_value"),
      78                experiment.get("metric_unit") or "",
      79            )
      80        best = " best" if experiment.get("best_observed") else ""
      81        return f"record_experiment {experiment.get('status')}: {experiment.get('title')}{metric}{best}"
      82    if name == "acknowledge_operator_context":
      83        return f"acknowledge_operator_context {result.get('status')} count={result.get('count', 0)}"
      84    if name == "browser_navigate":
      85        data = result.get("data") if isinstance(result.get("data"), dict) else {}
      86        title = data.get("title") or ""
      87        url = data.get("url") or ""
      88        warning = f" | warning={result.get('source_warning')}" if result.get("source_warning") else ""
      89        return f"browser_navigate opened {title} <{url}>{warning}"
      90    if name == "browser_snapshot":
      91        snapshot = str(result.get("snapshot") or result.get("data") or "")
      92        warning = f" | warning={result.get('source_warning')}" if result.get("source_warning") else ""
      93        return f"browser_snapshot returned {len(snapshot)} chars{warning}"
      94    if name == "read_artifact":
      95        return f"read_artifact read {result.get('artifact_id')}"
      96    if name == "search_artifacts":
      97        results = result.get("results") if isinstance(result.get("results"), list) else []
      98        return f"search_artifacts returned {len(results)} results for {args.get('query')!r}"
      99    return f"{name} completed"

nipux_cli/worker_usage.py 48 lines

       1"""Usage accounting for worker model turns."""
       2
       3from __future__ import annotations
       4
       5import json
       6from typing import Any
       7
       8from nipux_cli.llm import LLMResponse
       9
      10
      11def turn_usage_metadata(
      12    response: LLMResponse,
      13    *,
      14    messages: list[dict[str, Any]],
      15    context_length: int,
      16) -> dict[str, Any]:
      17    prompt_text = json.dumps(messages, ensure_ascii=False, default=str)
      18    completion_text = response.content + json.dumps(
      19        [{"name": call.name, "arguments": call.arguments} for call in response.tool_calls],
      20        ensure_ascii=False,
      21        default=str,
      22    )
      23    usage = dict(response.usage) if isinstance(response.usage, dict) else {}
      24    prompt_tokens = _as_int(usage.get("prompt_tokens")) or estimate_token_count(prompt_text)
      25    completion_tokens = _as_int(usage.get("completion_tokens")) or estimate_token_count(completion_text)
      26    usage.setdefault("prompt_tokens", prompt_tokens)
      27    usage.setdefault("completion_tokens", completion_tokens)
      28    usage.setdefault("total_tokens", prompt_tokens + completion_tokens)
      29    usage.setdefault("estimated", not bool(response.usage))
      30    usage["prompt_chars"] = len(prompt_text)
      31    usage["completion_chars"] = len(completion_text)
      32    if context_length > 0:
      33        usage["context_length"] = context_length
      34        usage["context_fraction"] = round(prompt_tokens / max(1, context_length), 6)
      35    return usage
      36
      37
      38def estimate_token_count(text: str) -> int:
      39    if not text:
      40        return 0
      41    return max(1, (len(text) + 3) // 4)
      42
      43
      44def _as_int(value: Any) -> int:
      45    try:
      46        return int(float(value))
      47    except (TypeError, ValueError):
      48        return 0

plans/nipux-runtime-notes.md 50 lines

       1# Nipux Runtime Notes
       2
       3Nipux is a narrow, restartable worker for long-running browser, web research,
       4and command-line jobs. The active implementation is intentionally small and
       5centered on `nipux_cli/`, `tests/nipux_cli/`, and the `nipux` console script.
       6
       7## Runtime Shape
       8
       9- Package: `nipux_cli/`
      10- CLI entry point: `nipux`
      11- State home: `~/.nipux` or `NIPUX_HOME`
      12- Config file: `~/.nipux/config.yaml`
      13- Database: SQLite with WAL
      14- Artifacts: per-job files under the configured state home
      15- Browser profiles: per-job `agent-browser` profiles
      16- Model API: OpenAI-compatible chat completions endpoint
      17
      18## Design Constraints
      19
      20- Keep every worker step bounded and restartable.
      21- Persist useful evidence as artifacts before summarizing it.
      22- Keep summaries compact and point back to artifacts.
      23- Maintain source, finding, task, experiment, and lesson ledgers.
      24- Keep jobs runnable until the operator pauses or cancels them.
      25- Keep the tool registry explicit and small.
      26- Keep runtime behavior domain-neutral.
      27
      28## Active Tools
      29
      30- Browser: `browser_navigate`, `browser_snapshot`, `browser_click`,
      31  `browser_type`, `browser_scroll`, `browser_back`, `browser_press`,
      32  `browser_console`
      33- Web: `web_search`, `web_extract`
      34- Local command work: `shell_exec`
      35- Artifacts: `write_artifact`, `read_artifact`, `search_artifacts`
      36- Job state and visibility: `update_job_state`, `report_update`,
      37  `send_digest_email`
      38- Learning ledgers: `record_lesson`, `record_source`, `record_findings`,
      39  `record_tasks`, `record_experiment`
      40
      41## Validation
      42
      43```bash
      44PYTEST_ADDOPTS='' uv run --extra dev python -m pytest -q
      45uv run --extra dev ruff check --isolated nipux_cli tests/nipux_cli
      46uv run nipux doctor
      47```
      48
      49Use `uv run nipux daemon --once --fake` for a deterministic no-model smoke
      50test after CLI or daemon changes.

pyproject.toml 54 lines

       1[build-system]
       2requires = ["setuptools>=61.0"]
       3build-backend = "setuptools.build_meta"
       4
       5[project]
       6name = "nipux"
       7version = "0.1.0"
       8description = "A restartable CLI worker for long-running agent jobs"
       9readme = "README.md"
      10requires-python = ">=3.11"
      11authors = [{ name = "Nipux" }]
      12license = "MIT"
      13keywords = ["agent", "cli", "automation", "daemon", "openai-compatible"]
      14classifiers = [
      15  "Development Status :: 3 - Alpha",
      16  "Environment :: Console",
      17  "Intended Audience :: Developers",
      18  "Programming Language :: Python :: 3",
      19  "Programming Language :: Python :: 3.11",
      20  "Programming Language :: Python :: 3.12",
      21  "Topic :: Software Development :: Libraries :: Application Frameworks",
      22]
      23dependencies = [
      24  "openai>=2.21.0,<3",
      25  "pyyaml>=6.0.2,<7",
      26]
      27
      28[project.urls]
      29Homepage = "https://nipux.com"
      30Source = "https://github.com/nipuxx/agent-cli"
      31Issues = "https://github.com/nipuxx/agent-cli/issues"
      32
      33[project.optional-dependencies]
      34dev = ["pytest>=9.0.2,<10", "ruff"]
      35
      36[dependency-groups]
      37dev = ["pytest>=9.0.2,<10", "ruff"]
      38
      39[project.scripts]
      40nipux = "nipux_cli.cli:main"
      41
      42[tool.setuptools.packages.find]
      43include = ["nipux_cli", "nipux_cli.*"]
      44
      45[tool.pytest.ini_options]
      46testpaths = ["tests/nipux_cli"]
      47addopts = "-q"
      48
      49[tool.ruff]
      50line-length = 120
      51target-version = "py311"
      52
      53[tool.uv]
      54exclude-newer = "7 days"

scripts/generate_project_atlas.py 619 lines

       1#!/usr/bin/env python3
       2"""Generate docs/project-atlas.html from the tracked Nipux source tree."""
       3
       4from __future__ import annotations
       5
       6import ast
       7import html
       8import re
       9import subprocess
      10from dataclasses import dataclass
      11from pathlib import Path
      12from typing import Any
      13
      14
      15ROOT = Path(__file__).resolve().parents[1]
      16OUT = ROOT / "docs" / "project-atlas.html"
      17SOURCE_SUFFIXES = {".py", ".md", ".toml", ".yaml", ".yml"}
      18EXCLUDED = {
      19    "docs/project-atlas.html",
      20    "uv.lock",
      21}
      22SENSITIVE_ASSIGNMENT_RE = re.compile(
      23    r"^(\s*)([A-Z0-9_]*(?:API_KEY|TOKEN|SECRET|PASSWORD)[A-Z0-9_]*)(\s*)=(.*)$"
      24)
      25
      26
      27@dataclass
      28class SourceFile:
      29    path: str
      30    text: str
      31    lines: list[str]
      32    tree: ast.AST | None
      33    error: str = ""
      34
      35
      36@dataclass
      37class Symbol:
      38    path: str
      39    kind: str
      40    name: str
      41    line: int
      42    end_line: int
      43    doc: str
      44    calls: list[str]
      45
      46
      47@dataclass
      48class Prompt:
      49    path: str
      50    name: str
      51    line: int
      52    text: str
      53    context: str
      54
      55
      56def main() -> None:
      57    files = load_source_files()
      58    symbols = extract_symbols(files)
      59    prompts = extract_prompts(files)
      60    tools = extract_tools(files)
      61    tables = extract_tables(files)
      62    commit = git(["rev-parse", "--short", "HEAD"]) or "working-tree"
      63    html_text = render(files, symbols, prompts, tools, tables, commit=commit)
      64    OUT.parent.mkdir(parents=True, exist_ok=True)
      65    OUT.write_text(html_text, encoding="utf-8")
      66    print(f"wrote {OUT.relative_to(ROOT)} ({len(html_text):,} chars)")
      67
      68
      69def load_source_files() -> list[SourceFile]:
      70    paths = tracked_paths()
      71    files: list[SourceFile] = []
      72    for path in paths:
      73        if path in EXCLUDED:
      74            continue
      75        full = ROOT / path
      76        if full.suffix not in SOURCE_SUFFIXES or not full.is_file():
      77            continue
      78        text = full.read_text(encoding="utf-8", errors="replace")
      79        tree = None
      80        error = ""
      81        if full.suffix == ".py":
      82            try:
      83                tree = ast.parse(text, filename=path)
      84            except SyntaxError as exc:
      85                error = str(exc)
      86        files.append(SourceFile(path=path, text=text, lines=text.splitlines(), tree=tree, error=error))
      87    return files
      88
      89
      90def tracked_paths() -> list[str]:
      91    output = git(["ls-files"])
      92    if not output:
      93        return []
      94    return sorted(line.strip() for line in output.splitlines() if line.strip())
      95
      96
      97def git(args: list[str]) -> str:
      98    try:
      99        result = subprocess.run(["git", *args], cwd=ROOT, check=False, capture_output=True, text=True)
     100    except OSError:
     101        return ""
     102    return result.stdout.strip() if result.returncode == 0 else ""
     103
     104
     105def extract_symbols(files: list[SourceFile]) -> list[Symbol]:
     106    symbols: list[Symbol] = []
     107    for source in files:
     108        if source.tree is None:
     109            continue
     110        for node in ast.walk(source.tree):
     111            if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
     112                calls = sorted(call_names(node))[:16]
     113                symbols.append(
     114                    Symbol(
     115                        path=source.path,
     116                        kind="class" if isinstance(node, ast.ClassDef) else "function",
     117                        name=node.name,
     118                        line=getattr(node, "lineno", 0),
     119                        end_line=getattr(node, "end_lineno", getattr(node, "lineno", 0)),
     120                        doc=ast.get_docstring(node) or "",
     121                        calls=calls,
     122                    )
     123                )
     124    return sorted(symbols, key=lambda item: (item.path, item.line, item.name))
     125
     126
     127def call_names(node: ast.AST) -> set[str]:
     128    names: set[str] = set()
     129    for child in ast.walk(node):
     130        if not isinstance(child, ast.Call):
     131            continue
     132        name = dotted_name(child.func)
     133        if name:
     134            names.add(name)
     135    return names
     136
     137
     138def dotted_name(node: ast.AST) -> str:
     139    if isinstance(node, ast.Name):
     140        return node.id
     141    if isinstance(node, ast.Attribute):
     142        base = dotted_name(node.value)
     143        return f"{base}.{node.attr}" if base else node.attr
     144    return ""
     145
     146
     147def extract_prompts(files: list[SourceFile]) -> list[Prompt]:
     148    prompts: list[Prompt] = []
     149    for source in files:
     150        if source.tree is None:
     151            continue
     152        for node in ast.walk(source.tree):
     153            if isinstance(node, (ast.Assign, ast.AnnAssign)):
     154                names = assignment_names(node)
     155                value = getattr(node, "value", None)
     156                text = literal_string(value)
     157                if not text:
     158                    continue
     159                if any(is_prompt_name(name) for name in names) or is_prompt_text(text):
     160                    prompts.append(
     161                        Prompt(
     162                            path=source.path,
     163                            name=", ".join(names) or "string",
     164                            line=getattr(node, "lineno", 0),
     165                            text=text,
     166                            context="assignment",
     167                        )
     168                    )
     169            elif isinstance(node, ast.Constant) and isinstance(node.value, str) and is_prompt_text(node.value):
     170                prompts.append(
     171                    Prompt(
     172                        path=source.path,
     173                        name="inline string",
     174                        line=getattr(node, "lineno", 0),
     175                        text=node.value,
     176                        context="inline",
     177                    )
     178                )
     179    deduped: dict[tuple[str, int, str], Prompt] = {}
     180    for prompt in prompts:
     181        key = (prompt.path, prompt.line, prompt.text[:120])
     182        existing = deduped.get(key)
     183        if existing is None or (existing.context == "inline" and prompt.context != "inline"):
     184            deduped[key] = prompt
     185    return sorted(deduped.values(), key=lambda item: (item.path, item.line))
     186
     187
     188def assignment_names(node: ast.Assign | ast.AnnAssign) -> list[str]:
     189    targets = node.targets if isinstance(node, ast.Assign) else [node.target]
     190    names: list[str] = []
     191    for target in targets:
     192        if isinstance(target, ast.Name):
     193            names.append(target.id)
     194        elif isinstance(target, ast.Attribute):
     195            names.append(target.attr)
     196    return names
     197
     198
     199def literal_string(node: ast.AST | None) -> str:
     200    if isinstance(node, ast.Constant) and isinstance(node.value, str):
     201        return node.value
     202    if isinstance(node, ast.JoinedStr):
     203        return "".join(part.value for part in node.values if isinstance(part, ast.Constant) and isinstance(part.value, str))
     204    return ""
     205
     206
     207def is_prompt_name(name: str) -> bool:
     208    upper = name.upper()
     209    return any(term in upper for term in ("PROMPT", "SYSTEM", "INSTRUCTION", "GUIDANCE")) and not upper.endswith("_PATH")
     210
     211
     212def is_prompt_text(text: str) -> bool:
     213    clean = " ".join(text.split())
     214    if len(clean) < 240:
     215        return False
     216    lowered = clean.lower()
     217    return (
     218        "you are" in lowered
     219        or "do not" in lowered and "use " in lowered
     220        or "operator" in lowered and "context" in lowered and "prompt" in lowered
     221        or "next-action" in lowered
     222    )
     223
     224
     225def extract_tools(files: list[SourceFile]) -> list[dict[str, str]]:
     226    tools_text = next((source.text for source in files if source.path == "nipux_cli/tools.py"), "")
     227    tools: list[dict[str, str]] = []
     228    pattern = re.compile(r"ToolSpec\(\s*['\"]([^'\"]+)['\"]\s*,\s*['\"]([^'\"]+)['\"]", re.S)
     229    for match in pattern.finditer(tools_text):
     230        line = tools_text[: match.start()].count("\n") + 1
     231        tools.append({"name": match.group(1), "description": " ".join(match.group(2).split()), "line": str(line)})
     232    return tools
     233
     234
     235def extract_tables(files: list[SourceFile]) -> list[dict[str, Any]]:
     236    db_text = next((source.text for source in files if source.path == "nipux_cli/db.py"), "")
     237    tables: list[dict[str, Any]] = []
     238    for match in re.finditer(r"CREATE TABLE IF NOT EXISTS\s+([a-zA-Z_][a-zA-Z0-9_]*)\s*\((.*?)\)", db_text, re.S):
     239        raw_columns = [line.strip().rstrip(",") for line in match.group(2).splitlines()]
     240        columns = [line for line in raw_columns if line and not line.upper().startswith(("FOREIGN", "UNIQUE", "PRIMARY KEY"))]
     241        tables.append({"name": match.group(1), "columns": columns, "line": db_text[: match.start()].count("\n") + 1})
     242    return tables
     243
     244
     245def render(
     246    files: list[SourceFile],
     247    symbols: list[Symbol],
     248    prompts: list[Prompt],
     249    tools: list[dict[str, str]],
     250    tables: list[dict[str, Any]],
     251    *,
     252    commit: str,
     253) -> str:
     254    python_files = [source for source in files if source.path.endswith(".py")]
     255    total_lines = sum(len(source.lines) for source in files)
     256    file_cards = "\n".join(render_file_card(source, symbols) for source in files)
     257    source_browser = "\n".join(render_source_file(source) for source in files)
     258    symbol_cards = "\n".join(render_symbol(symbol) for symbol in symbols)
     259    prompt_cards = "\n".join(render_prompt(prompt) for prompt in prompts[:80])
     260    tool_rows = "\n".join(
     261        f"<tr><td><code>{esc(tool['name'])}</code></td><td>{esc(tool['description'])}</td><td>{tool['line']}</td></tr>"
     262        for tool in tools
     263    )
     264    table_cards = "\n".join(render_table(table) for table in tables)
     265    risk_cards = render_review_points(files, symbols, prompts, tools)
     266    return f"""<!doctype html>
     267<html lang="en">
     268<head>
     269<meta charset="utf-8">
     270<meta name="viewport" content="width=device-width, initial-scale=1">
     271<title>Nipux Project Atlas</title>
     272<style>
     273:root {{
     274  color-scheme: dark;
     275  --bg: #080909; --panel: #101112; --panel-2: #151717; --text: #ecebe6;
     276  --muted: #9b9b96; --faint: #5f615e; --line: #303332; --accent: #9ad6d1;
     277  --accent-2: #d8d06d; --warn: #ee9b66; --bad: #e36d78; --green: #9fca7f;
     278  --mono: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace;
     279  --sans: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
     280}}
     281* {{ box-sizing: border-box; }}
     282body {{ margin: 0; background: radial-gradient(circle at 70% 0%, rgba(154,214,209,.10), transparent 36%), var(--bg); color: var(--text); font: 15px/1.55 var(--sans); }}
     283a {{ color: var(--accent); text-decoration: none; }}
     284a:hover {{ text-decoration: underline; }}
     285.shell {{ display: grid; grid-template-columns: 292px minmax(0, 1fr); min-height: 100vh; }}
     286.sidebar {{ position: sticky; top: 0; height: 100vh; overflow: auto; border-right: 1px solid var(--line); padding: 24px 22px; background: rgba(8,9,9,.94); }}
     287.logo {{ font: 750 22px var(--mono); letter-spacing: .08em; color: var(--accent); }}
     288.subtitle {{ color: var(--muted); margin: 6px 0 22px; }}
     289.search {{ width: 100%; background: #050606; border: 1px solid var(--line); color: var(--text); border-radius: 8px; padding: 11px 12px; font: 14px var(--mono); outline: none; }}
     290.search:focus {{ border-color: var(--accent); box-shadow: 0 0 0 3px rgba(154,214,209,.12); }}
     291.nav {{ margin: 22px 0; display: grid; gap: 7px; }}
     292.nav a {{ color: var(--muted); padding: 6px 0; font: 13px var(--mono); text-transform: uppercase; letter-spacing: .12em; }}
     293.stats {{ margin-top: 24px; display: grid; gap: 10px; }}
     294.stat {{ border: 1px solid var(--line); background: var(--panel); border-radius: 10px; padding: 12px; }}
     295.stat b {{ display: block; font: 650 24px/1 var(--mono); }}
     296.stat span {{ color: var(--muted); font: 12px var(--mono); text-transform: uppercase; letter-spacing: .12em; }}
     297main {{ min-width: 0; padding: 34px 38px 80px; }}
     298.hero {{ border-bottom: 1px solid var(--line); padding-bottom: 28px; margin-bottom: 28px; }}
     299.eyebrow {{ color: var(--accent); font: 12px var(--mono); text-transform: uppercase; letter-spacing: .22em; }}
     300h1 {{ font-size: clamp(38px, 6vw, 86px); line-height: .9; margin: 14px 0 18px; letter-spacing: -.04em; }}
     301h2 {{ margin: 0; font-size: 28px; letter-spacing: -.02em; }}
     302h3 {{ margin: 0 0 6px; font-size: 16px; }}
     303.lede {{ max-width: 980px; color: #c7c6bf; font-size: 19px; }}
     304.section {{ margin: 44px 0; scroll-margin-top: 20px; }}
     305.section > header {{ display: flex; align-items: end; justify-content: space-between; gap: 20px; border-bottom: 1px solid var(--line); margin-bottom: 18px; padding-bottom: 10px; }}
     306.kicker {{ color: var(--muted); font: 12px var(--mono); text-transform: uppercase; letter-spacing: .16em; }}
     307.grid {{ display: grid; grid-template-columns: repeat(auto-fit, minmax(280px, 1fr)); gap: 14px; }}
     308.card, .file-card, .prompt, .tool, .db-card, .symbol {{ border: 1px solid var(--line); background: linear-gradient(180deg, rgba(255,255,255,.035), rgba(255,255,255,.012)); border-radius: 12px; padding: 16px; overflow: hidden; }}
     309.file-card header, .prompt header, .tool header {{ display: flex; justify-content: space-between; gap: 12px; align-items: baseline; border-bottom: 1px solid rgba(255,255,255,.06); margin: -2px 0 10px; padding-bottom: 8px; }}
     310.muted, .file-card header span, .prompt header span, .tool header span {{ color: var(--muted); }}
     311.meta {{ display: flex; flex-wrap: wrap; gap: 8px; margin: 10px 0; }}
     312.meta span, .pill {{ border: 1px solid var(--line); border-radius: 999px; padding: 2px 8px; color: var(--muted); font: 12px var(--mono); }}
     313.arch {{ display: grid; grid-template-columns: repeat(3, minmax(210px, 1fr)); gap: 12px; }}
     314.node {{ text-align: left; min-height: 126px; cursor: pointer; border: 1px solid var(--line); color: var(--text); background: var(--panel); border-radius: 14px; padding: 15px; font: inherit; transition: .15s ease; }}
     315.node:hover {{ transform: translateY(-2px); border-color: var(--accent); background: var(--panel-2); }}
     316.node strong {{ display: block; font-size: 17px; }}
     317.node span {{ display: block; color: var(--accent); font: 12px var(--mono); margin: 4px 0 8px; }}
     318.node em {{ display: block; color: var(--muted); font-style: normal; }}
     319.flow {{ counter-reset: flow; display: grid; gap: 10px; padding: 0; list-style: none; }}
     320.flow li {{ counter-increment: flow; border-left: 2px solid var(--accent); background: var(--panel); padding: 12px 14px 12px 48px; position: relative; border-radius: 8px; }}
     321.flow li:before {{ content: counter(flow); position: absolute; left: 14px; top: 12px; color: var(--accent-2); font: 700 16px var(--mono); }}
     322.flow li span {{ display: block; color: var(--muted); }}
     323details {{ margin-top: 10px; }}
     324summary {{ cursor: pointer; color: var(--accent); font: 13px var(--mono); }}
     325table {{ width: 100%; border-collapse: collapse; margin-top: 10px; font-size: 13px; }}
     326th, td {{ text-align: left; border-bottom: 1px solid rgba(255,255,255,.07); padding: 7px 8px; vertical-align: top; }}
     327th {{ color: var(--muted); font: 12px var(--mono); text-transform: uppercase; letter-spacing: .08em; }}
     328code, pre {{ font-family: var(--mono); }}
     329pre {{ max-height: 540px; overflow: auto; background: #050606; border: 1px solid var(--line); border-radius: 10px; padding: 14px; color: #dad8cf; white-space: pre-wrap; }}
     330.mini-grid {{ display: grid; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); gap: 12px; }}
     331.symbol p {{ margin: 8px 0; }}
     332.calls {{ color: var(--muted); }}
     333.warning {{ border-color: rgba(238,155,102,.45); background: rgba(238,155,102,.08); }}
     334.hidden {{ display: none !important; }}
     335.source-list {{ display: grid; gap: 10px; }}
     336.source-file {{ border: 1px solid var(--line); border-radius: 10px; background: var(--panel); padding: 0 12px 10px; }}
     337.source-file summary {{ padding: 12px 0; color: var(--text); }}
     338.source-file summary span {{ color: var(--muted); margin-left: 8px; }}
     339.source-code {{ max-height: 620px; font-size: 12px; line-height: 1.45; white-space: pre; }}
     340.src-line {{ display: grid; grid-template-columns: 52px minmax(0, 1fr); min-height: 17px; }}
     341.src-line b {{ color: var(--faint); user-select: none; font-weight: 500; }}
     342.src-line code {{ color: #d6d3ca; }}
     343@media (max-width: 980px) {{ .shell {{ grid-template-columns: 1fr; }} .sidebar {{ position: relative; height: auto; }} main {{ padding: 24px 18px 60px; }} .arch {{ grid-template-columns: 1fr; }} }}
     344</style>
     345</head>
     346<body>
     347<div class="shell">
     348  <aside class="sidebar">
     349    <div class="logo">NIPUX ATLAS</div>
     350    <div class="subtitle">Generated from tracked source on {esc(commit)}.</div>
     351    <input id="search" class="search" placeholder="filter files, prompts, tools..." autocomplete="off">
     352    <nav class="nav">
     353      <a href="#architecture">Architecture</a>
     354      <a href="#runtime-flow">Runtime Flow</a>
     355      <a href="#prompts">Prompt Surfaces</a>
     356      <a href="#tools">Tool Registry</a>
     357      <a href="#database">Database</a>
     358      <a href="#files">Files</a>
     359      <a href="#symbols">Symbols</a>
     360      <a href="#source-browser">Source Browser</a>
     361      <a href="#tests">Tests</a>
     362      <a href="#risks">Review Points</a>
     363    </nav>
     364    <div class="stats">
     365      <div class="stat"><b>{len(files)}</b><span>tracked files</span></div>
     366      <div class="stat"><b>{total_lines:,}</b><span>tracked lines mapped</span></div>
     367      <div class="stat"><b>{len(symbols)}</b><span>python symbols</span></div>
     368      <div class="stat"><b>{len(tools)}</b><span>runtime tools</span></div>
     369    </div>
     370  </aside>
     371  <main>
     372    <section class="hero">
     373      <div class="eyebrow">Backend map / prompt audit / source index</div>
     374      <h1>Nipux Project Atlas</h1>
     375      <p class="lede">A self-contained visual map of the current backend: entrypoints, daemon loop, worker prompt assembly, durable memory, tools, SQLite schema, UI control plane, tests, and every tracked source file with parsed functions/classes and line references.</p>
     376    </section>
     377
     378    <section id="architecture" class="section">
     379      <header><div><div class="kicker">Mind map</div><h2>Architecture</h2></div><span class="muted">Click a node to jump into related detail.</span></header>
     380      <div class="arch">{architecture_nodes()}</div>
     381    </section>
     382
     383    <section id="runtime-flow" class="section">
     384      <header><div><div class="kicker">Lifecycle</div><h2>Runtime Flow</h2></div><span class="muted">What happens from terminal input to durable progress.</span></header>
     385      <ol class="flow">{runtime_flow()}</ol>
     386    </section>
     387
     388    <section id="prompts" class="section">
     389      <header><div><div class="kicker">Exact text</div><h2>Prompt Surfaces</h2></div><span class="muted">System/program prompts and instruction-like strings extracted from source.</span></header>
     390      <div class="grid">{prompt_cards}</div>
     391    </section>
     392
     393    <section id="tools" class="section">
     394      <header><div><div class="kicker">Tools</div><h2>Tool Registry</h2></div><span class="muted">Static ToolSpec definitions from nipux_cli/tools.py.</span></header>
     395      <table><thead><tr><th>Name</th><th>Description</th><th>Line</th></tr></thead><tbody>{tool_rows}</tbody></table>
     396    </section>
     397
     398    <section id="database" class="section">
     399      <header><div><div class="kicker">Persistence</div><h2>SQLite Tables</h2></div><span class="muted">CREATE TABLE blocks found in nipux_cli/db.py.</span></header>
     400      <div class="grid">{table_cards}</div>
     401    </section>
     402
     403    <section id="files" class="section">
     404      <header><div><div class="kicker">Source index</div><h2>Important Files</h2></div><span class="muted">{len(python_files)} Python modules plus docs/config files.</span></header>
     405      <div class="grid">{file_cards}</div>
     406    </section>
     407
     408    <section id="symbols" class="section">
     409      <header><div><div class="kicker">Functions and classes</div><h2>Symbol Map</h2></div><span class="muted">Parsed with Python AST.</span></header>
     410      <div class="mini-grid">{symbol_cards}</div>
     411    </section>
     412
     413    <section id="source-browser" class="section">
     414      <header><div><div class="kicker">Line-by-line</div><h2>Source Browser</h2></div><span class="muted">Collapsed raw tracked source so the backend can be inspected directly in this page.</span></header>
     415      <div class="source-list">{source_browser}</div>
     416    </section>
     417
     418    <section id="tests" class="section">
     419      <header><div><div class="kicker">Verification</div><h2>Test Coverage Map</h2></div><span class="muted">Test files included in the source index.</span></header>
     420      <div class="grid">{test_cards(files)}</div>
     421    </section>
     422
     423    <section id="risks" class="section">
     424      <header><div><div class="kicker">Audit cues</div><h2>Review Points</h2></div><span class="muted">Generated signals for where to inspect next.</span></header>
     425      <div class="grid">{risk_cards}</div>
     426    </section>
     427  </main>
     428</div>
     429<script>
     430const search = document.getElementById('search');
     431search?.addEventListener('input', () => {{
     432  const term = search.value.toLowerCase().trim();
     433  document.querySelectorAll('.searchable').forEach((node) => {{
     434    const hay = (node.getAttribute('data-search') || node.textContent || '').toLowerCase();
     435    node.classList.toggle('hidden', term && !hay.includes(term));
     436  }});
     437}});
     438document.querySelectorAll('.node[data-target]').forEach((node) => {{
     439  node.addEventListener('click', () => {{
     440    const target = document.getElementById(node.getAttribute('data-target'));
     441    if (target) target.scrollIntoView({{ behavior: 'smooth', block: 'start' }});
     442  }});
     443}});
     444</script>
     445</body>
     446</html>
     447"""
     448
     449
     450def architecture_nodes() -> str:
     451    nodes = [
     452        ("cli-tui", "CLI / TUI", "nipux_cli/cli.py", "Chat-first terminal UI, first-run menu, slash commands, job switching, event panes."),
     453        ("sqlite-state", "SQLite state", "nipux_cli/db.py", "Jobs, runs, steps, artifacts, events, ledgers, usage, and memory index."),
     454        ("daemon", "Daemon", "nipux_cli/daemon.py", "Single-instance forever loop, stale runtime fingerprint, heartbeat, work scheduling."),
     455        ("worker-loop", "Worker loop", "nipux_cli/worker.py", "Builds prompts, chooses one tool step, guards loops, records durable progress."),
     456        ("llm-adapter", "LLM adapter", "nipux_cli/llm.py", "OpenAI-compatible chat calls, usage/cost tracking, tool call parsing."),
     457        ("tool-registry", "Tool registry", "nipux_cli/tools.py", "Browser, web, shell, artifact, ledger, task, experiment, digest tools."),
     458        ("browser-web", "Browser/web", "nipux_cli/browser.py / web.py", "Visible browsing, snapshots, search/extract helpers, anti-bot source scoring."),
     459        ("artifacts-files", "Artifacts/files", "nipux_cli/artifacts.py", "Saved outputs and concrete workspace file writing."),
     460        ("memory", "Memory", "compression.py / operator_context.py", "Compact rolling memory and durable operator context."),
     461    ]
     462    return "".join(
     463        f"<button id='{esc(anchor)}' class='node searchable' data-target='source-browser' data-search='{esc(title + ' ' + path + ' ' + desc)}'>"
     464        f"<strong>{esc(title)}</strong><span>{esc(path)}</span><em>{esc(desc)}</em></button>"
     465        for anchor, title, path, desc in nodes
     466    )
     467
     468
     469def runtime_flow() -> str:
     470    steps = [
     471        ("Startup", "pyproject entrypoint calls nipux_cli.cli:main. With no args, the chat/TUI opens on the focused job or first-run workspace."),
     472        ("Operator input", "Plain chat is stored as visible events and, when relevant, durable operator context for future worker prompts."),
     473        ("Daemon scheduling", "The daemon claims runnable jobs, keeps a lock/heartbeat, starts runs, and calls one bounded worker step repeatedly."),
     474        ("Prompt assembly", "worker.build_messages layers system prompt, program template, operator context, roadmaps, tasks, ledgers, experiments, memory, timeline, and recent steps."),
     475        ("Tool call", "The LLM selects one OpenAI-style tool. The registry executes it with ToolContext and stores input/output in steps/events."),
     476        ("Progress accounting", "Guards require artifacts, findings, tasks, experiments, or milestone validation when evidence or measurements appear."),
     477        ("Persistence", "Artifacts go to the job output directory. SQLite stores steps, events, ledgers, runtime state, and usage/cost metadata."),
     478        ("UI refresh", "The TUI reads timeline/events and compact job metrics, splitting chat from worker activity and status."),
     479    ]
     480    return "".join(f"<li><strong>{esc(title)}</strong><span>{esc(body)}</span></li>" for title, body in steps)
     481
     482
     483def render_file_card(source: SourceFile, symbols: list[Symbol]) -> str:
     484    local_symbols = [symbol for symbol in symbols if symbol.path == source.path]
     485    top_names = ", ".join(symbol.name for symbol in local_symbols[:10]) or "none"
     486    doc = module_doc(source) or source.error or "No module docstring."
     487    imports = ", ".join(module_imports(source)[:12]) or "none"
     488    return f"""<article class="file-card searchable" data-search="{esc(source.path + ' ' + doc + ' ' + top_names)}">
     489<header><h3>{esc(source.path)}</h3><span>{len(source.lines)} lines</span></header>
     490<p>{esc(short(doc, 260))}</p>
     491<div class="meta"><span>{len(local_symbols)} symbols</span><span>{source.text.count('TODO')} TODOs</span></div>
     492<details><summary>Imports and top symbols</summary><p><strong>Imports:</strong> {esc(imports)}</p><p><strong>Symbols:</strong> {esc(top_names)}</p></details>
     493</article>"""
     494
     495
     496def module_doc(source: SourceFile) -> str:
     497    if source.tree is None:
     498        for line in source.lines:
     499            stripped = line.strip()
     500            if stripped and not stripped.startswith("#"):
     501                return stripped
     502        return ""
     503    return ast.get_docstring(source.tree) or ""
     504
     505
     506def module_imports(source: SourceFile) -> list[str]:
     507    if source.tree is None:
     508        return []
     509    names: list[str] = []
     510    for node in source.tree.body:
     511        if isinstance(node, ast.Import):
     512            names.extend(alias.name for alias in node.names)
     513        elif isinstance(node, ast.ImportFrom):
     514            module = "." * node.level + (node.module or "")
     515            names.append(module)
     516    return names
     517
     518
     519def render_source_file(source: SourceFile) -> str:
     520    rendered_lines = [redact_source_line(line) for line in source.lines]
     521    search_text = "\n".join(rendered_lines)
     522    code = "\n".join(
     523        f"<span class='src-line'><b>{index:>4}</b><code>{esc(line)}</code></span>"
     524        for index, line in enumerate(rendered_lines, start=1)
     525    )
     526    return f"""<details class="source-file searchable" data-search="{esc(source.path + ' ' + search_text[:4000])}">
     527<summary>{esc(source.path)} <span>{len(source.lines)} lines</span></summary>
     528<pre class="source-code">{code}</pre>
     529</details>"""
     530
     531
     532def redact_source_line(line: str) -> str:
     533    match = SENSITIVE_ASSIGNMENT_RE.match(line)
     534    if not match:
     535        return line
     536    indent, name, _space, _value = match.groups()
     537    return f"{indent}{name} = <redacted>"
     538
     539
     540def render_symbol(symbol: Symbol) -> str:
     541    doc = short(symbol.doc or "No docstring.", 180)
     542    calls = ", ".join(symbol.calls) or "none"
     543    return f"""<article class="symbol searchable" data-search="{esc(symbol.path + ' ' + symbol.name + ' ' + doc + ' ' + calls)}">
     544<h3>{esc(symbol.name)}</h3>
     545<p><span class="pill">{esc(symbol.kind)}</span> <span class="pill">{esc(symbol.path)}:{symbol.line}</span></p>
     546<p>{esc(doc)}</p>
     547<p class="calls"><strong>Calls:</strong> {esc(short(calls, 280))}</p>
     548</article>"""
     549
     550
     551def render_prompt(prompt: Prompt) -> str:
     552    title = prompt.name or "prompt"
     553    return f"""<article class="prompt searchable" data-search="{esc(prompt.path + ' ' + title + ' ' + prompt.text)}">
     554<header><h3>{esc(title)}</h3><span>{esc(prompt.path)}:{prompt.line}</span></header>
     555<p class="muted">Context: <code>{esc(prompt.context)}</code> · {len(prompt.text):,} chars</p>
     556<pre><code>{esc(prompt.text)}</code></pre>
     557</article>"""
     558
     559
     560def render_table(table: dict[str, Any]) -> str:
     561    columns = "".join(f"<li><code>{esc(column)}</code></li>" for column in table["columns"][:40])
     562    return f"""<article class="db-card searchable" data-search="{esc(table['name'] + ' ' + ' '.join(table['columns']))}">
     563<h3>{esc(table['name'])}</h3>
     564<p class="muted">nipux_cli/db.py:{table['line']}</p>
     565<ul>{columns}</ul>
     566</article>"""
     567
     568
     569def test_cards(files: list[SourceFile]) -> str:
     570    tests = [source for source in files if source.path.startswith("tests/")]
     571    cards = []
     572    for source in tests:
     573        names = []
     574        if source.tree:
     575            names = [node.name for node in ast.walk(source.tree) if isinstance(node, ast.FunctionDef) and node.name.startswith("test_")]
     576        cards.append(
     577            f"""<article class="test-card searchable" data-search="{esc(source.path + ' ' + ' '.join(names))}">
     578<h3>{esc(source.path)}</h3><p><strong>{len(names)}</strong> tests · {len(source.lines)} lines</p>
     579<p class="muted">{esc(short(', '.join(names), 320))}</p></article>"""
     580        )
     581    return "\n".join(cards)
     582
     583
     584def render_review_points(
     585    files: list[SourceFile],
     586    symbols: list[Symbol],
     587    prompts: list[Prompt],
     588    tools: list[dict[str, str]],
     589) -> str:
     590    largest = sorted(files, key=lambda source: len(source.lines), reverse=True)[:5]
     591    large_text = ", ".join(f"{source.path} ({len(source.lines)} lines)" for source in largest)
     592    prompt_text = f"{len(prompts)} prompt/instruction-like strings were extracted. Inspect this section after any agent-behavior change."
     593    tool_text = f"{len(tools)} tools are exposed to the worker. Review descriptions whenever generic behavior changes."
     594    symbol_text = f"{len(symbols)} symbols were parsed. Large modules are candidates for refactoring once behavior stabilizes."
     595    cards = [
     596        ("Large modules", large_text),
     597        ("Prompt surfaces", prompt_text),
     598        ("Tool surface", tool_text),
     599        ("Symbol map", symbol_text),
     600    ]
     601    return "\n".join(
     602        f"<article class='card warning searchable' data-search='{esc(title + ' ' + body)}'><h3>{esc(title)}</h3><p>{esc(body)}</p></article>"
     603        for title, body in cards
     604    )
     605
     606
     607def short(text: str, limit: int) -> str:
     608    clean = " ".join(str(text).split())
     609    if len(clean) <= limit:
     610        return clean
     611    return clean[: max(0, limit - 3)] + "..."
     612
     613
     614def esc(value: Any) -> str:
     615    return html.escape(str(value), quote=True)
     616
     617
     618if __name__ == "__main__":
     619    main()

scripts/live_memory_graph_smoke.py 226 lines

       1#!/usr/bin/env python3
       2"""Run an opt-in real-model smoke test for memory-graph tool calling.
       3
       4This script is intentionally outside the normal Nipux runtime path. It creates
       5an isolated temporary Nipux home, seeds generic durable job state, and verifies
       6that a configured OpenAI-compatible model can consolidate that state with the
       7`record_memory_graph` tool.
       8"""
       9
      10from __future__ import annotations
      11
      12import argparse
      13import json
      14import os
      15import shutil
      16import sys
      17import tempfile
      18from pathlib import Path
      19from typing import Any
      20
      21from nipux_cli.config import AppConfig, ModelConfig, RuntimeConfig, ToolAccessConfig
      22from nipux_cli.db import AgentDB
      23from nipux_cli.memory_graph import memory_graph_from_job
      24from nipux_cli.worker import run_one_step
      25
      26
      27DEFAULT_MODEL = "qwen/qwen3.6-27b"
      28DEFAULT_BASE_URL = "https://openrouter.ai/api/v1"
      29DEFAULT_API_KEY_ENV = <redacted>
      30
      31
      32def main() -> int:
      33    parser = argparse.ArgumentParser(description=__doc__)
      34    parser.add_argument("--model", default=DEFAULT_MODEL, help=f"OpenAI-compatible model name. Default: {DEFAULT_MODEL}")
      35    parser.add_argument("--base-url", default=DEFAULT_BASE_URL, help=f"Provider base URL. Default: {DEFAULT_BASE_URL}")
      36    parser.add_argument("--api-key-env", default=DEFAULT_API_KEY_ENV, help=f"API key env var. Default: {DEFAULT_API_KEY_ENV}")
      37    parser.add_argument("--context-length", type=int, default=262_144)
      38    parser.add_argument("--steps", type=int, default=3, help="Maximum worker turns to try.")
      39    parser.add_argument("--keep-home", action="store_true", help="Keep the temporary Nipux home for inspection.")
      40    parser.add_argument("--json", action="store_true", help="Print a machine-readable result.")
      41    args = parser.parse_args()
      42
      43    api_key = os.environ.get(args.api_key_env, "")
      44    if not api_key:
      45        return _finish(
      46            {
      47                "success": False,
      48                "error": f"{args.api_key_env} is not set",
      49                "action": f"Export {args.api_key_env} before running this live smoke. The key is never printed.",
      50            },
      51            json_output=args.json,
      52        )
      53
      54    home = Path(tempfile.mkdtemp(prefix="nipux-memory-graph-live-"))
      55    try:
      56        config = AppConfig(
      57            runtime=RuntimeConfig(home=home, max_steps_per_run=1),
      58            model=ModelConfig(
      59                model=args.model,
      60                base_url=args.base_url.rstrip("/"),
      61                api_key_env=args.api_key_env,
      62                context_length=args.context_length,
      63                request_timeout_seconds=180,
      64            ),
      65            tools=ToolAccessConfig(browser=False, web=False, shell=False, files=False),
      66        )
      67        config.ensure_dirs()
      68        db = AgentDB(config.runtime.state_db_path)
      69        try:
      70            job_id = db.create_job(
      71                "Consolidate generic durable job knowledge into an inspectable memory graph.",
      72                title="memory graph live smoke",
      73                metadata=_seed_metadata(),
      74            )
      75            db.update_job_status(job_id, "running")
      76            executions = []
      77            for _ in range(max(1, args.steps)):
      78                execution = run_one_step(job_id, config=config, db=db)
      79                executions.append(_execution_summary(execution))
      80                job = db.get_job(job_id)
      81                graph = memory_graph_from_job(job)
      82                if graph["nodes"]:
      83                    return _finish(
      84                        {
      85                            "success": True,
      86                            "home": str(home),
      87                            "model": args.model,
      88                            "base_url": args.base_url,
      89                            "job_id": job_id,
      90                            "node_count": len(graph["nodes"]),
      91                            "edge_count": len(graph["edges"]),
      92                            "executions": executions,
      93                        },
      94                        json_output=args.json,
      95                    )
      96            job = db.get_job(job_id)
      97            graph = memory_graph_from_job(job)
      98            return _finish(
      99                {
     100                    "success": False,
     101                    "home": str(home),
     102                    "model": args.model,
     103                    "base_url": args.base_url,
     104                    "job_id": job_id,
     105                    "node_count": len(graph["nodes"]),
     106                    "edge_count": len(graph["edges"]),
     107                    "executions": executions,
     108                    "error": "model did not create memory graph nodes within the step budget",
     109                },
     110                json_output=args.json,
     111            )
     112        finally:
     113            db.close()
     114    finally:
     115        if args.keep_home:
     116            print(f"kept temporary Nipux home: {home}", file=sys.stderr)
     117        else:
     118            shutil.rmtree(home, ignore_errors=True)
     119
     120
     121def _seed_metadata() -> dict[str, Any]:
     122    return {
     123        "finding_ledger": [
     124            {
     125                "name": "Durable outputs need reusable summaries",
     126                "category": "process",
     127                "reason": "Saved outputs are easier to reuse when connected to decisions and tasks.",
     128                "score": 0.82,
     129            },
     130            {
     131                "name": "Repeated branch work needs explicit rejection criteria",
     132                "category": "process",
     133                "reason": "A branch should either improve evidence, produce a deliverable, or be deprecated.",
     134                "score": 0.78,
     135            },
     136        ],
     137        "source_ledger": [
     138            {
     139                "source": "internal://recent-events",
     140                "source_type": "job_history",
     141                "usefulness_score": 0.8,
     142                "yield_count": 2,
     143                "last_outcome": "Recent events exposed reusable process knowledge.",
     144            },
     145            {
     146                "source": "internal://saved-outputs",
     147                "source_type": "artifact_index",
     148                "usefulness_score": 0.7,
     149                "yield_count": 1,
     150                "last_outcome": "Saved outputs provide evidence refs for future graph nodes.",
     151            },
     152        ],
     153        "lessons": [
     154            {
     155                "category": "strategy",
     156                "lesson": "Prefer measured or validated progress over activity counts.",
     157                "confidence": 0.86,
     158            },
     159            {
     160                "category": "memory",
     161                "lesson": "Consolidate stable findings into linked graph nodes before context grows.",
     162                "confidence": 0.9,
     163            },
     164        ],
     165        "task_queue": [
     166            {
     167                "title": "Create a connected memory graph from durable signals",
     168                "status": "open",
     169                "output_contract": "decision",
     170                "acceptance_criteria": "At least one reusable node connected to evidence or strategy.",
     171            }
     172        ],
     173        "roadmap": {
     174            "title": "Long-running job memory",
     175            "status": "active",
     176            "milestones": [
     177                {
     178                    "title": "Consolidate reusable knowledge",
     179                    "status": "open",
     180                    "validation_contract": "Future turns can retrieve the key decisions without replaying raw history.",
     181                }
     182            ],
     183        },
     184    }
     185
     186
     187def _execution_summary(execution: Any) -> dict[str, Any]:
     188    result = execution.result if isinstance(execution.result, dict) else {}
     189    return {
     190        "status": execution.status,
     191        "tool": execution.tool_name,
     192        "step_id": execution.step_id,
     193        "success": result.get("success"),
     194        "error": result.get("error"),
     195        "added_nodes": result.get("added_nodes"),
     196        "added_edges": result.get("added_edges"),
     197    }
     198
     199
     200def _finish(payload: dict[str, Any], *, json_output: bool) -> int:
     201    if json_output:
     202        print(json.dumps(payload, indent=2, sort_keys=True))
     203    else:
     204        print(_human_summary(payload))
     205    return 0 if payload.get("success") else 1
     206
     207
     208def _human_summary(payload: dict[str, Any]) -> str:
     209    lines = [f"success: {bool(payload.get('success'))}"]
     210    for key in ("model", "base_url", "home", "job_id", "node_count", "edge_count", "error", "action"):
     211        if payload.get(key) is not None:
     212            lines.append(f"{key}: {payload[key]}")
     213    executions = payload.get("executions")
     214    if isinstance(executions, list) and executions:
     215        lines.append("executions:")
     216        for item in executions:
     217            lines.append(
     218                "  - "
     219                f"status={item.get('status')} tool={item.get('tool')} "
     220                f"success={item.get('success')} error={item.get('error')}"
     221            )
     222    return "\n".join(lines)
     223
     224
     225if __name__ == "__main__":
     226    raise SystemExit(main())

scripts/render_nipux_ascii_video.py 523 lines

       1#!/usr/bin/env python3
       2"""Render a Nipux ASCII-art CLI intro as an MP4.
       3
       4The renderer is dependency-light on purpose: it draws a small embedded
       5bitmap font into raw RGB frames and pipes those frames directly to ffmpeg.
       6"""
       7
       8from __future__ import annotations
       9
      10import argparse
      11import math
      12import random
      13import shutil
      14import subprocess
      15from dataclasses import dataclass
      16from pathlib import Path
      17
      18
      19WIDTH = 1440
      20HEIGHT = 900
      21FPS = 30
      22DURATION = 8.0
      23COLS = 96
      24ROWS = 34
      25CELL_W = 13
      26CELL_H = 22
      27ORIGIN_X = (WIDTH - COLS * CELL_W) // 2
      28ORIGIN_Y = (HEIGHT - ROWS * CELL_H) // 2
      29SCALE = 2
      30
      31BG = (2, 5, 5)
      32BG_SCAN = (1, 3, 4)
      33PANEL = (4, 10, 8)
      34PANEL_EDGE = (10, 56, 37)
      35PANEL_GLOW = (4, 30, 24)
      36DIM = (33, 92, 62)
      37MID = (72, 180, 112)
      38MAIN = (126, 255, 164)
      39HOT = (104, 238, 255)
      40AMBER = (255, 184, 70)
      41MAGENTA = (255, 95, 154)
      42WHITE = (220, 255, 238)
      43
      44LOGO = [
      45    r"##   ##  ####  ######  ##   ##  ##   ##",
      46    r"###  ##   ##   ##   ## ##   ##   ## ## ",
      47    r"#### ##   ##   ##   ## ##   ##    ###  ",
      48    r"## ####   ##   ######  ##   ##    ###  ",
      49    r"##  ###   ##   ##      ##   ##   ## ## ",
      50    r"##   ##  ####  ##       #####   ##   ##",
      51]
      52
      53GLITCH_CHARS = ".:-=+*#%@/\\|<>[]{}01"
      54RAIN_CHARS = ".:-+*#01/\\<>"
      55
      56
      57GLYPHS: dict[str, tuple[str, ...]] = {
      58    " ": ("00000", "00000", "00000", "00000", "00000", "00000", "00000"),
      59    "A": ("01110", "10001", "10001", "11111", "10001", "10001", "10001"),
      60    "B": ("11110", "10001", "10001", "11110", "10001", "10001", "11110"),
      61    "C": ("01111", "10000", "10000", "10000", "10000", "10000", "01111"),
      62    "D": ("11110", "10001", "10001", "10001", "10001", "10001", "11110"),
      63    "E": ("11111", "10000", "10000", "11110", "10000", "10000", "11111"),
      64    "F": ("11111", "10000", "10000", "11110", "10000", "10000", "10000"),
      65    "G": ("01111", "10000", "10000", "10011", "10001", "10001", "01110"),
      66    "H": ("10001", "10001", "10001", "11111", "10001", "10001", "10001"),
      67    "I": ("01110", "00100", "00100", "00100", "00100", "00100", "01110"),
      68    "J": ("00111", "00010", "00010", "00010", "10010", "10010", "01100"),
      69    "K": ("10001", "10010", "10100", "11000", "10100", "10010", "10001"),
      70    "L": ("10000", "10000", "10000", "10000", "10000", "10000", "11111"),
      71    "M": ("10001", "11011", "10101", "10101", "10001", "10001", "10001"),
      72    "N": ("10001", "11001", "10101", "10011", "10001", "10001", "10001"),
      73    "O": ("01110", "10001", "10001", "10001", "10001", "10001", "01110"),
      74    "P": ("11110", "10001", "10001", "11110", "10000", "10000", "10000"),
      75    "Q": ("01110", "10001", "10001", "10001", "10101", "10010", "01101"),
      76    "R": ("11110", "10001", "10001", "11110", "10100", "10010", "10001"),
      77    "S": ("01111", "10000", "10000", "01110", "00001", "00001", "11110"),
      78    "T": ("11111", "00100", "00100", "00100", "00100", "00100", "00100"),
      79    "U": ("10001", "10001", "10001", "10001", "10001", "10001", "01110"),
      80    "V": ("10001", "10001", "10001", "10001", "10001", "01010", "00100"),
      81    "W": ("10001", "10001", "10001", "10101", "10101", "10101", "01010"),
      82    "X": ("10001", "10001", "01010", "00100", "01010", "10001", "10001"),
      83    "Y": ("10001", "10001", "01010", "00100", "00100", "00100", "00100"),
      84    "Z": ("11111", "00001", "00010", "00100", "01000", "10000", "11111"),
      85    "a": ("00000", "00000", "01110", "00001", "01111", "10001", "01111"),
      86    "b": ("10000", "10000", "10110", "11001", "10001", "10001", "11110"),
      87    "c": ("00000", "00000", "01110", "10000", "10000", "10001", "01110"),
      88    "d": ("00001", "00001", "01101", "10011", "10001", "10001", "01111"),
      89    "e": ("00000", "00000", "01110", "10001", "11111", "10000", "01110"),
      90    "f": ("00110", "01001", "01000", "11100", "01000", "01000", "01000"),
      91    "g": ("00000", "01111", "10001", "10001", "01111", "00001", "01110"),
      92    "h": ("10000", "10000", "10110", "11001", "10001", "10001", "10001"),
      93    "i": ("00100", "00000", "01100", "00100", "00100", "00100", "01110"),
      94    "j": ("00010", "00000", "00110", "00010", "00010", "10010", "01100"),
      95    "k": ("10000", "10000", "10010", "10100", "11000", "10100", "10010"),
      96    "l": ("01100", "00100", "00100", "00100", "00100", "00100", "01110"),
      97    "m": ("00000", "00000", "11010", "10101", "10101", "10101", "10101"),
      98    "n": ("00000", "00000", "10110", "11001", "10001", "10001", "10001"),
      99    "o": ("00000", "00000", "01110", "10001", "10001", "10001", "01110"),
     100    "p": ("00000", "00000", "11110", "10001", "11110", "10000", "10000"),
     101    "q": ("00000", "00000", "01101", "10011", "01111", "00001", "00001"),
     102    "r": ("00000", "00000", "10110", "11001", "10000", "10000", "10000"),
     103    "s": ("00000", "00000", "01111", "10000", "01110", "00001", "11110"),
     104    "t": ("01000", "01000", "11100", "01000", "01000", "01001", "00110"),
     105    "u": ("00000", "00000", "10001", "10001", "10001", "10011", "01101"),
     106    "v": ("00000", "00000", "10001", "10001", "10001", "01010", "00100"),
     107    "w": ("00000", "00000", "10001", "10001", "10101", "10101", "01010"),
     108    "x": ("00000", "00000", "10001", "01010", "00100", "01010", "10001"),
     109    "y": ("00000", "00000", "10001", "10001", "01111", "00001", "01110"),
     110    "z": ("00000", "00000", "11111", "00010", "00100", "01000", "11111"),
     111    "0": ("01110", "10001", "10011", "10101", "11001", "10001", "01110"),
     112    "1": ("00100", "01100", "00100", "00100", "00100", "00100", "01110"),
     113    "2": ("01110", "10001", "00001", "00010", "00100", "01000", "11111"),
     114    "3": ("11110", "00001", "00001", "01110", "00001", "00001", "11110"),
     115    "4": ("00010", "00110", "01010", "10010", "11111", "00010", "00010"),
     116    "5": ("11111", "10000", "10000", "11110", "00001", "00001", "11110"),
     117    "6": ("01110", "10000", "10000", "11110", "10001", "10001", "01110"),
     118    "7": ("11111", "00001", "00010", "00100", "01000", "01000", "01000"),
     119    "8": ("01110", "10001", "10001", "01110", "10001", "10001", "01110"),
     120    "9": ("01110", "10001", "10001", "01111", "00001", "00001", "01110"),
     121    ".": ("00000", "00000", "00000", "00000", "00000", "01100", "01100"),
     122    ",": ("00000", "00000", "00000", "00000", "00000", "01100", "01000"),
     123    ":": ("00000", "01100", "01100", "00000", "01100", "01100", "00000"),
     124    ";": ("00000", "01100", "01100", "00000", "01100", "01000", "10000"),
     125    "!": ("00100", "00100", "00100", "00100", "00100", "00000", "00100"),
     126    "?": ("01110", "10001", "00001", "00010", "00100", "00000", "00100"),
     127    "'": ("00100", "00100", "01000", "00000", "00000", "00000", "00000"),
     128    '"': ("01010", "01010", "01010", "00000", "00000", "00000", "00000"),
     129    "-": ("00000", "00000", "00000", "11111", "00000", "00000", "00000"),
     130    "_": ("00000", "00000", "00000", "00000", "00000", "00000", "11111"),
     131    "+": ("00000", "00100", "00100", "11111", "00100", "00100", "00000"),
     132    "=": ("00000", "00000", "11111", "00000", "11111", "00000", "00000"),
     133    "*": ("00000", "10101", "01110", "11111", "01110", "10101", "00000"),
     134    "#": ("01010", "11111", "01010", "01010", "11111", "01010", "01010"),
     135    "@": ("01110", "10001", "10111", "10101", "10111", "10000", "01110"),
     136    "%": ("11001", "11010", "00100", "01000", "10110", "00110", "00000"),
     137    "$": ("00100", "01111", "10100", "01110", "00101", "11110", "00100"),
     138    "&": ("01100", "10010", "10100", "01000", "10101", "10010", "01101"),
     139    "/": ("00001", "00010", "00100", "01000", "10000", "00000", "00000"),
     140    "\\": ("10000", "01000", "00100", "00010", "00001", "00000", "00000"),
     141    "|": ("00100", "00100", "00100", "00100", "00100", "00100", "00100"),
     142    "<": ("00010", "00100", "01000", "10000", "01000", "00100", "00010"),
     143    ">": ("01000", "00100", "00010", "00001", "00010", "00100", "01000"),
     144    "(": ("00010", "00100", "01000", "01000", "01000", "00100", "00010"),
     145    ")": ("01000", "00100", "00010", "00010", "00010", "00100", "01000"),
     146    "[": ("01110", "01000", "01000", "01000", "01000", "01000", "01110"),
     147    "]": ("01110", "00010", "00010", "00010", "00010", "00010", "01110"),
     148    "{": ("00010", "00100", "00100", "01000", "00100", "00100", "00010"),
     149    "}": ("01000", "00100", "00100", "00010", "00100", "00100", "01000"),
     150    "~": ("00000", "00000", "01001", "10110", "00000", "00000", "00000"),
     151    "^": ("00100", "01010", "10001", "00000", "00000", "00000", "00000"),
     152}
     153
     154
     155@dataclass(frozen=True)
     156class Cell:
     157    char: str = " "
     158    color: tuple[int, int, int] = DIM
     159
     160
     161class TextGrid:
     162    def __init__(self) -> None:
     163        self.cells = [[Cell() for _ in range(COLS)] for _ in range(ROWS)]
     164
     165    def set(self, x: int, y: int, char: str, color: tuple[int, int, int]) -> None:
     166        if 0 <= x < COLS and 0 <= y < ROWS and char:
     167            self.cells[y][x] = Cell(char[0], color)
     168
     169    def put(self, x: int, y: int, text: str, color: tuple[int, int, int]) -> None:
     170        for offset, char in enumerate(text):
     171            self.set(x + offset, y, char, color)
     172
     173    def center(self, y: int, text: str, color: tuple[int, int, int]) -> None:
     174        self.put((COLS - len(text)) // 2, y, text, color)
     175
     176    def box(self, title: str, color: tuple[int, int, int]) -> None:
     177        top = "+" + "-" * (COLS - 2) + "+"
     178        bottom = "+" + "-" * (COLS - 2) + "+"
     179        self.put(0, 0, top, color)
     180        self.put(0, ROWS - 1, bottom, color)
     181        for y in range(1, ROWS - 1):
     182            self.set(0, y, "|", color)
     183            self.set(COLS - 1, y, "|", color)
     184        label = f" {title} "
     185        self.put(3, 0, label[: COLS - 8], color)
     186
     187
     188def clamp(value: float, low: float = 0.0, high: float = 1.0) -> float:
     189    return max(low, min(high, value))
     190
     191
     192def ease(value: float) -> float:
     193    value = clamp(value)
     194    return value * value * (3.0 - 2.0 * value)
     195
     196
     197def mix(a: tuple[int, int, int], b: tuple[int, int, int], amount: float) -> tuple[int, int, int]:
     198    amount = clamp(amount)
     199    return tuple(int(a[i] + (b[i] - a[i]) * amount) for i in range(3))
     200
     201
     202def logo_origin() -> tuple[int, int]:
     203    max_width = max(len(line) for line in LOGO)
     204    return (COLS - max_width) // 2, 8
     205
     206
     207def put_logo(grid: TextGrid, frame: int, reveal: float, stable: bool = False) -> None:
     208    left, top = logo_origin()
     209    for y, line in enumerate(LOGO):
     210        for x, char in enumerate(line):
     211            if char == " ":
     212                continue
     213            rng = random.Random(frame * 1009 + x * 97 + y * 53)
     214            if stable or rng.random() < reveal:
     215                shown = char
     216                color = HOT if stable or rng.random() > 0.18 else AMBER
     217                if stable and rng.random() < 0.015:
     218                    shown = rng.choice("*+#")
     219                    color = WHITE
     220                elif not stable and rng.random() > reveal + 0.18:
     221                    shown = rng.choice(GLITCH_CHARS)
     222                    color = MAGENTA
     223                grid.set(left + x, top + y, shown, color)
     224            elif rng.random() < 0.08 + 0.22 * reveal:
     225                grid.set(left + x, top + y, rng.choice(GLITCH_CHARS), mix(DIM, HOT, 0.35))
     226
     227
     228def put_collapsing_logo(grid: TextGrid, frame: int, progress: float) -> None:
     229    left, top = logo_origin()
     230    cursor_y = 24
     231    glyph_index = 0
     232    for y, line in enumerate(LOGO):
     233        for x, char in enumerate(line):
     234            if char == " ":
     235                continue
     236            rng = random.Random(frame * 1493 + x * 31 + y * 41)
     237            target_x = 8 + (glyph_index % 30)
     238            target_y = cursor_y + (glyph_index // 30) % 2
     239            wobble = math.sin(frame * 0.35 + glyph_index * 0.9) * (1.0 - progress) * 2.0
     240            px = round((left + x) * (1.0 - progress) + target_x * progress + wobble)
     241            py = round((top + y) * (1.0 - progress) + target_y * progress)
     242            shown = char if progress < 0.68 else rng.choice("nipux$>_-/")
     243            color = mix(HOT, MAIN, progress)
     244            if rng.random() < 0.08:
     245                shown = rng.choice(GLITCH_CHARS)
     246                color = MAGENTA
     247            grid.set(px, py, shown, color)
     248            glyph_index += 1
     249
     250
     251def put_progress_bar(grid: TextGrid, x: int, y: int, width: int, progress: float) -> None:
     252    progress = clamp(progress)
     253    filled = int(round(width * progress))
     254    bar = "[" + "#" * filled + "-" * (width - filled) + "]"
     255    grid.put(x, y, bar, MID)
     256    grid.put(x + 1, y, "#" * filled, HOT if progress > 0.88 else MAIN)
     257    grid.put(x + width + 4, y, f"{int(progress * 100):03d}%", AMBER if progress < 1.0 else HOT)
     258
     259
     260def put_rain(grid: TextGrid, frame: int, intensity: float) -> None:
     261    if intensity <= 0:
     262        return
     263    for x in range(2, COLS - 2):
     264        rng = random.Random(9001 + x * 113)
     265        stream_speed = 1 + rng.randint(0, 2)
     266        head = (frame * stream_speed + rng.randint(0, ROWS * 3)) % (ROWS + 16) - 8
     267        for trail in range(5):
     268            y = head - trail
     269            if 2 <= y < ROWS - 2 and random.Random(frame * 313 + x * 17 + trail).random() < intensity:
     270                char = random.Random(frame * 997 + x * 19 + trail * 11).choice(RAIN_CHARS)
     271                fade = max(0.18, 1.0 - trail * 0.18)
     272                color = mix((8, 28, 20), MID, fade * intensity)
     273                grid.set(x, y, char, color)
     274
     275
     276def put_boot_lines(grid: TextGrid, frame: int, t: float) -> None:
     277    lines = [
     278        "$ nipux video --ascii --into-cli",
     279        "[scan] terminal cells online",
     280        "[map ] routing logo glyphs",
     281        "[sync] prompt target locked",
     282    ]
     283    start_y = 22
     284    for i, line in enumerate(lines):
     285        reveal = int(clamp((t - 0.12 - i * 0.22) / 0.32) * len(line))
     286        color = MAIN if i == 0 else MID
     287        grid.put(7, start_y + i, line[:reveal], color)
     288    put_progress_bar(grid, 7, 28, 38, clamp(t / 1.12))
     289
     290
     291def put_cli(grid: TextGrid, frame: int, t: float) -> None:
     292    command = "$ nipux enter --render ascii"
     293    start = 4.78
     294    typed = int(clamp((t - start) / 1.08) * len(command))
     295    grid.put(7, 24, command[:typed], MAIN)
     296    cursor_x = 7 + typed
     297    if frame // 8 % 2 == 0 and typed < len(command):
     298        grid.set(cursor_x, 24, "_", HOT)
     299
     300    if t > 5.9:
     301        grid.put(7, 26, "[ok] word packed into cli prompt", MID)
     302    if t > 6.26:
     303        grid.put(7, 27, "[ok] ascii signal clean", MID)
     304    if t > 6.58:
     305        put_progress_bar(grid, 7, 29, 42, clamp((t - 6.55) / 0.62))
     306    if t > 7.18:
     307        final = "nipux> "
     308        grid.put(7, 31, final, HOT)
     309        if frame // 10 % 2 == 0:
     310            grid.set(7 + len(final), 31, "_", HOT)
     311
     312
     313def build_grid(frame: int, total_frames: int) -> TextGrid:
     314    t = frame / FPS
     315    grid = TextGrid()
     316    grid.box("nipux ascii cli capture", DIM)
     317    grid.put(COLS - 25, 0, " render:rawrgb->mp4 ", DIM)
     318    grid.put(4, 2, "MODE ASCII/CLI", MID)
     319    grid.put(COLS - 23, 2, f"FRAME {frame:03d}/{total_frames - 1:03d}", DIM)
     320
     321    rain_intensity = 0.36
     322    if t > 5.0:
     323        rain_intensity *= 0.35
     324    put_rain(grid, frame, rain_intensity)
     325
     326    if t < 1.18:
     327        put_boot_lines(grid, frame, t)
     328    elif t < 2.75:
     329        put_boot_lines(grid, frame, 1.18)
     330        reveal = ease((t - 1.18) / 1.42)
     331        put_logo(grid, frame, reveal)
     332        grid.center(16, "glyphs are snapping into nipux", DIM)
     333    elif t < 3.72:
     334        put_logo(grid, frame, 1.0, stable=True)
     335        grid.center(16, "nipux", WHITE if frame // 7 % 2 == 0 else HOT)
     336        grid.center(18, "pressing the word into a command line", DIM)
     337    elif t < 5.08:
     338        progress = ease((t - 3.72) / 1.36)
     339        put_collapsing_logo(grid, frame, progress)
     340        grid.put(7, 24, "$ ", MAIN)
     341        if progress > 0.45:
     342            partial = "nipux"[: int((progress - 0.45) / 0.55 * 5)]
     343            grid.put(9, 24, partial, HOT)
     344        grid.center(18, "collapsing ascii mass -> cli input", AMBER)
     345    else:
     346        put_cli(grid, frame, t)
     347
     348    if t > 4.8:
     349        grid.put(COLS - 27, 30, "STATUS: PROMPT CONTROL", DIM)
     350    elif t > 2.0:
     351        grid.put(COLS - 24, 30, "STATUS: GLYPH LOCK", DIM)
     352    else:
     353        grid.put(COLS - 23, 30, "STATUS: BOOT RAIL", DIM)
     354
     355    return grid
     356
     357
     358def draw_rect(buf: bytearray, x: int, y: int, w: int, h: int, color: tuple[int, int, int]) -> None:
     359    x0 = max(0, x)
     360    y0 = max(0, y)
     361    x1 = min(WIDTH, x + w)
     362    y1 = min(HEIGHT, y + h)
     363    if x0 >= x1 or y0 >= y1:
     364        return
     365    row = bytes(color) * (x1 - x0)
     366    for py in range(y0, y1):
     367        start = (py * WIDTH + x0) * 3
     368        buf[start : start + len(row)] = row
     369
     370
     371def build_base_frame() -> bytearray:
     372    buf = bytearray(WIDTH * HEIGHT * 3)
     373    for y in range(HEIGHT):
     374        color = BG_SCAN if y % 4 == 0 else BG
     375        row = bytes(color) * WIDTH
     376        start = y * WIDTH * 3
     377        buf[start : start + len(row)] = row
     378
     379    panel_x = ORIGIN_X - 28
     380    panel_y = ORIGIN_Y - 28
     381    panel_w = COLS * CELL_W + 56
     382    panel_h = ROWS * CELL_H + 56
     383    draw_rect(buf, panel_x - 8, panel_y - 8, panel_w + 16, panel_h + 16, PANEL_GLOW)
     384    draw_rect(buf, panel_x, panel_y, panel_w, panel_h, PANEL)
     385    draw_rect(buf, panel_x, panel_y, panel_w, 2, PANEL_EDGE)
     386    draw_rect(buf, panel_x, panel_y + panel_h - 2, panel_w, 2, PANEL_EDGE)
     387    draw_rect(buf, panel_x, panel_y, 2, panel_h, PANEL_EDGE)
     388    draw_rect(buf, panel_x + panel_w - 2, panel_y, 2, panel_h, PANEL_EDGE)
     389
     390    for y in range(panel_y + 36, panel_y + panel_h - 20, 44):
     391        draw_rect(buf, panel_x + 18, y, panel_w - 36, 1, (5, 24, 18))
     392    return buf
     393
     394
     395BASE_FRAME = build_base_frame()
     396
     397
     398def glyph_for(char: str) -> tuple[str, ...]:
     399    return GLYPHS.get(char, GLYPHS.get(char.upper(), GLYPHS["?"]))
     400
     401
     402GLYPHS["?"] = GLYPHS["?"] if "?" in GLYPHS else ("01110", "10001", "00010", "00100", "00100", "00000", "00100")
     403
     404
     405def draw_glyph(buf: bytearray, char: str, x: int, y: int, color: tuple[int, int, int], glow: bool) -> None:
     406    glyph = glyph_for(char)
     407    if glyph is GLYPHS[" "]:
     408        return
     409    if glow:
     410        glow_color = tuple(max(0, int(c * 0.16)) for c in color)
     411    for row_i, row in enumerate(glyph):
     412        for col_i, bit in enumerate(row):
     413            if bit != "1":
     414                continue
     415            px = x + col_i * SCALE
     416            py = y + row_i * SCALE
     417            if glow:
     418                draw_rect(buf, px - 1, py - 1, SCALE + 2, SCALE + 2, glow_color)
     419            draw_rect(buf, px, py, SCALE, SCALE, color)
     420
     421
     422def render_frame(frame: int, total_frames: int) -> bytes:
     423    grid = build_grid(frame, total_frames)
     424    buf = bytearray(BASE_FRAME)
     425    jitter = 1 if frame % 37 == 0 else 0
     426    for y, row in enumerate(grid.cells):
     427        for x, cell in enumerate(row):
     428            if cell.char == " ":
     429                continue
     430            px = ORIGIN_X + x * CELL_W + 1 + jitter
     431            py = ORIGIN_Y + y * CELL_H + 4
     432            bright = sum(cell.color) > 420
     433            draw_glyph(buf, cell.char, px, py, cell.color, bright)
     434
     435    # A light CRT sweep, sparse enough to keep text readable.
     436    sweep_y = int((frame * 9) % HEIGHT)
     437    draw_rect(buf, 0, sweep_y, WIDTH, 2, (4, 18, 16))
     438    return bytes(buf)
     439
     440
     441def render_video(output: Path, poster: Path | None) -> None:
     442    ffmpeg = shutil.which("ffmpeg")
     443    if not ffmpeg:
     444        raise SystemExit("ffmpeg was not found on PATH")
     445
     446    output.parent.mkdir(parents=True, exist_ok=True)
     447    total_frames = int(FPS * DURATION)
     448    cmd = [
     449        ffmpeg,
     450        "-hide_banner",
     451        "-loglevel",
     452        "error",
     453        "-y",
     454        "-f",
     455        "rawvideo",
     456        "-pix_fmt",
     457        "rgb24",
     458        "-s",
     459        f"{WIDTH}x{HEIGHT}",
     460        "-r",
     461        str(FPS),
     462        "-i",
     463        "-",
     464        "-an",
     465        "-c:v",
     466        "libx264",
     467        "-preset",
     468        "medium",
     469        "-crf",
     470        "18",
     471        "-pix_fmt",
     472        "yuv420p",
     473        "-movflags",
     474        "+faststart",
     475        str(output),
     476    ]
     477    process = subprocess.Popen(cmd, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
     478    assert process.stdin is not None
     479    for frame in range(total_frames):
     480        process.stdin.write(render_frame(frame, total_frames))
     481        if frame % FPS == 0:
     482            print(f"rendered {frame // FPS:02d}s/{int(DURATION):02d}s")
     483    process.stdin.close()
     484    stderr = process.stderr.read().decode("utf-8", errors="replace") if process.stderr else ""
     485    return_code = process.wait()
     486    if return_code != 0:
     487        raise SystemExit(f"ffmpeg failed with exit code {return_code}\n{stderr}")
     488
     489    if poster:
     490        poster.parent.mkdir(parents=True, exist_ok=True)
     491        poster_cmd = [
     492            ffmpeg,
     493            "-hide_banner",
     494            "-loglevel",
     495            "error",
     496            "-y",
     497            "-ss",
     498            "00:00:03.05",
     499            "-i",
     500            str(output),
     501            "-frames:v",
     502            "1",
     503            str(poster),
     504        ]
     505        subprocess.run(poster_cmd, check=True)
     506
     507
     508def parse_args() -> argparse.Namespace:
     509    parser = argparse.ArgumentParser(description="Render the Nipux ASCII CLI intro video.")
     510    parser.add_argument("--output", type=Path, default=Path("docs/nipux_ascii_cli.mp4"))
     511    parser.add_argument("--poster", type=Path, default=Path("docs/nipux_ascii_cli_poster.png"))
     512    return parser.parse_args()
     513
     514
     515def main() -> None:
     516    args = parse_args()
     517    render_video(args.output, args.poster)
     518    print(f"video:  {args.output}")
     519    print(f"poster: {args.poster}")
     520
     521
     522if __name__ == "__main__":
     523    main()

tests/nipux_cli/test_artifacts.py 34 lines

       1import pytest
       2
       3from nipux_cli.artifacts import ArtifactStore
       4from nipux_cli.db import AgentDB
       5
       6
       7def test_artifact_store_writes_reads_and_searches(tmp_path):
       8    db = AgentDB(tmp_path / "state.db")
       9    try:
      10        job_id = db.create_job("Collect findings")
      11        store = ArtifactStore(tmp_path, db=db)
      12
      13        stored = store.write_text(
      14            job_id=job_id,
      15            title="Findings list",
      16            summary="contains acme finding",
      17            content="Acme Corp\ncontact: founder@example.com\n",
      18        )
      19
      20        assert store.read_text(stored.id).startswith("Acme Corp")
      21        results = store.search_text(job_id=job_id, query="founder", limit=5)
      22        assert results[0]["id"] == stored.id
      23        assert "founder@example.com" in results[0]["excerpt"]
      24    finally:
      25        db.close()
      26
      27
      28def test_artifact_store_rejects_paths_outside_home(tmp_path):
      29    store = ArtifactStore(tmp_path)
      30    outside = tmp_path.parent / "outside.txt"
      31    outside.write_text("nope", encoding="utf-8")
      32
      33    with pytest.raises(ValueError):
      34        store.read_text(str(outside))

tests/nipux_cli/test_browser_web.py 118 lines

       1import json
       2
       3from nipux_cli.browser import _annotate_source_quality, _session_name, _socket_dir
       4from nipux_cli.tools import DEFAULT_REGISTRY, ToolContext
       5from nipux_cli.web import _strip_html
       6from nipux_cli.artifacts import ArtifactStore
       7from nipux_cli.config import AppConfig, RuntimeConfig
       8from nipux_cli.db import AgentDB
       9
      10
      11def test_session_name_is_stable_and_safe():
      12    assert _session_name("job_abc/def") == "nipux_job_abc_def"
      13
      14
      15def test_long_session_name_is_short_and_hashed():
      16    task_id = "research-a-very-long-objective-title-that-needs-a-short-browser-session-name"
      17    name = _session_name(task_id)
      18    socket_dir = _socket_dir(task_id)
      19
      20    assert name.startswith("nipux_research-a-very-long")
      21    assert len(name) <= 37
      22    assert len(str(socket_dir)) < 80
      23
      24
      25def test_strip_html_removes_scripts_and_keeps_text():
      26    text = _strip_html("<html><script>bad()</script><h1>Hello</h1><p>World</p></html>")
      27    assert "bad" not in text
      28    assert "Hello" in text
      29    assert "World" in text
      30
      31
      32def test_browser_marks_anti_bot_interstitial_as_warning():
      33    result = _annotate_source_quality({
      34        "success": True,
      35        "data": {"title": "Just a moment...", "url": "https://clutch.co/example"},
      36        "snapshot": "Performing security verification. Cloudflare security challenge.",
      37    })
      38
      39    assert result["success"] is True
      40    assert "error" not in result
      41    assert result["source_warning"] == "cloudflare anti-bot challenge"
      42    assert result["warnings"][0]["type"] == "anti_bot"
      43
      44
      45def test_browser_marks_captcha_block_as_warning():
      46    result = _annotate_source_quality({
      47        "success": True,
      48        "data": {"title": "Source search", "url": "https://source.example/search"},
      49        "snapshot": 'Iframe "Security CAPTCHA" You have been blocked. You are browsing and clicking at a speed much faster than expected.',
      50    })
      51
      52    assert result["source_warning"] == "captcha/anti-bot block"
      53    assert result["warnings"][0]["type"] == "anti_bot"
      54
      55
      56def test_web_extract_marks_anti_bot_pages_as_warning(monkeypatch):
      57    from nipux_cli import web
      58
      59    def fake_request(url):
      60        del url
      61        return "<h1>Performing security verification</h1><p>Cloudflare security challenge</p>", "text/html"
      62
      63    monkeypatch.setattr(web, "_request", fake_request)
      64    result = web.web_extract(["https://clutch.co/example"])
      65
      66    page = result["pages"][0]
      67    assert "error" not in page
      68    assert page["source_warning"] == "cloudflare anti-bot challenge"
      69    assert page["warnings"][0]["type"] == "anti_bot"
      70    assert "Cloudflare security challenge" in page["text"]
      71
      72
      73def test_browser_tool_uses_native_wrapper(monkeypatch, tmp_path):
      74    from nipux_cli import browser
      75
      76    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
      77    db = AgentDB(tmp_path / "state.db")
      78    try:
      79        job_id = db.create_job("Browse")
      80        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id)
      81
      82        def fake_navigate(cfg, *, task_id, url):
      83            return {"success": True, "task_id": task_id, "url": url}
      84
      85        monkeypatch.setattr(browser, "navigate", fake_navigate)
      86        result = json.loads(DEFAULT_REGISTRY.handle("browser_navigate", {"url": "https://example.com"}, ctx))
      87
      88        assert result == {"success": True, "task_id": job_id, "url": "https://example.com"}
      89    finally:
      90        db.close()
      91
      92
      93def test_browser_click_adds_recovery_snapshot_for_stale_ref(monkeypatch, tmp_path):
      94    from nipux_cli import browser
      95
      96    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
      97    calls = []
      98
      99    def fake_command(cfg, *, task_id, command, args=None, timeout=60):
     100        del cfg, task_id, args, timeout
     101        calls.append(command)
     102        if command == "click":
     103            return {"success": False, "error": "Unknown ref: e102"}
     104        return {
     105            "success": True,
     106            "data": {
     107                "snapshot": "Directory",
     108                "refs": {"e1": {"role": "link", "name": "New Result"}},
     109            },
     110        }
     111
     112    monkeypatch.setattr(browser, "run_browser_command", fake_command)
     113    result = browser.click(config, task_id="job_abc", ref="@e102")
     114
     115    assert calls == ["click", "snapshot"]
     116    assert result["success"] is False
     117    assert result["recovery_snapshot"]["data"]["refs"]["e1"]["name"] == "New Result"
     118    assert "stale" in result["recovery_guidance"]

tests/nipux_cli/test_cli.py 4970 lines

       1import json
       2import queue
       3import subprocess
       4import time
       5from pathlib import Path
       6
       7from nipux_cli.artifacts import ArtifactStore
       8from nipux_cli import __version__
       9from nipux_cli.chat_frame_runtime import ChatFrameDeps as _ChatFrameDeps
      10from nipux_cli.chat_frame_runtime import THINKING_NOTICE as _THINKING_NOTICE
      11from nipux_cli.chat_frame_runtime import WAITING_NOTICE as _WAITING_NOTICE
      12from nipux_cli.chat_frame_runtime import _display_notices as _display_chat_notices
      13from nipux_cli.chat_frame_runtime import _drain_async_notices as _drain_chat_async_notices
      14from nipux_cli.chat_frame_runtime import _handle_edit_input as _handle_chat_edit_input
      15from nipux_cli.chat_frame_runtime import _handle_chat_submit
      16from nipux_cli.chat_frame_runtime import _safe_render_frame as _safe_chat_render_frame
      17from nipux_cli.chat_frame_runtime import frame_next_job_id as _frame_next_job_id
      18from nipux_cli.chat_frame_runtime import frame_refresh_interval as _frame_refresh_interval
      19from nipux_cli.cli import (
      20    _build_first_run_frame,
      21    _build_chat_frame,
      22    _build_chat_messages,
      23    _chat_handle_line,
      24    _chat_control_command,
      25    _capture_chat_command,
      26    _config_field_value,
      27    _emit_frame_if_changed,
      28    _first_run_click_action,
      29    _handle_first_run_action,
      30    _handle_first_run_menu_line,
      31    _handle_first_run_frame_line,
      32    _handle_chat_message,
      33    _handle_workspace_chat_message,
      34    _is_plain_chat_line,
      35    _launch_agent_plist,
      36    _load_frame_snapshot,
      37    _minimal_live_event_line,
      38    _print_shell_help,
      39    _run_shell_line,
      40    _save_config_field,
      41    _slash_suggestion_lines,
      42    _systemd_service_text,
      43    _verify_model_setup_from_first_run,
      44    _workspace_chat_job_dossier,
      45    build_parser,
      46    main,
      47)
      48from nipux_cli.config import load_config
      49from nipux_cli.cli_state import mark_model_setup_verified as _mark_model_setup_verified
      50from nipux_cli.cli_state import model_setup_verified as _model_setup_verified
      51from nipux_cli.cli_state import read_shell_state as _read_shell_state
      52from nipux_cli.cli_state import write_shell_state as _write_shell_state
      53from nipux_cli.daemon import append_daemon_event
      54from nipux_cli.db import AgentDB
      55from nipux_cli.doctor import Check
      56from nipux_cli.llm import LLMResponse
      57from nipux_cli.settings import inline_setting_notice as _inline_setting_notice
      58from nipux_cli.first_run_frame_runtime import FirstRunRuntimeDeps as _FirstRunRuntimeDeps
      59from nipux_cli.first_run_frame_runtime import _handle_edit_input as _handle_first_run_edit_input
      60from nipux_cli.first_run_frame_runtime import _safe_render_frame as _safe_first_run_render_frame
      61from nipux_cli.first_run_frame_runtime import _submit_first_run_line as _submit_first_run_line
      62from nipux_cli.first_run_frame_runtime import directional_first_run_action as _directional_first_run_action
      63from nipux_cli.frame_snapshot import WORKSPACE_CHAT_ID
      64from nipux_cli.tui_commands import (
      65    CHAT_SLASH_COMMANDS,
      66    FIRST_RUN_SLASH_COMMANDS,
      67    autocomplete_slash as _autocomplete_slash,
      68    cycle_slash as _cycle_slash,
      69    slash_completion_for_submit as _slash_completion_for_submit,
      70)
      71from nipux_cli.tui_events import chat_pane_lines
      72from nipux_cli.tui_input import decode_terminal_escape as _decode_terminal_escape
      73from nipux_cli.tui_outcomes import hourly_update_lines, recent_model_update_lines
      74from nipux_cli.updater import update_checkout as _update_checkout
      75
      76
      77def _mode(path):
      78    return path.stat().st_mode & 0o777
      79
      80
      81def _mark_test_model_ready() -> None:
      82    load_config().ensure_dirs()
      83    _mark_model_setup_verified(load_config())
      84
      85
      86def test_cli_has_operator_commands():
      87    parser = build_parser()
      88
      89    assert parser.parse_args(["shell", "--status"]).func.__name__ == "cmd_shell"
      90    assert parser.parse_args(["status", "--full"]).func.__name__ == "cmd_status"
      91    assert parser.parse_args(["health"]).func.__name__ == "cmd_health"
      92    assert parser.parse_args(["history"]).func.__name__ == "cmd_history"
      93    assert parser.parse_args(["events", "--follow"]).func.__name__ == "cmd_events"
      94    assert parser.parse_args(["activity", "--follow"]).func.__name__ == "cmd_activity"
      95    assert parser.parse_args(["feed"]).func.__name__ == "cmd_activity"
      96    assert parser.parse_args(["update"]).func.__name__ == "cmd_update"
      97    assert parser.parse_args(["update", "--no-restart"]).no_restart is True
      98    assert parser.parse_args(["uninstall", "--dry-run"]).func.__name__ == "cmd_uninstall"
      99    assert parser.parse_args(["uninstall", "--keep-tool"]).keep_tool is True
     100    assert parser.parse_args(["new", "Research topic"]).func.__name__ == "cmd_create"
     101    assert parser.parse_args(["updates"]).func.__name__ == "cmd_updates"
     102    assert parser.parse_args(["outcomes"]).func.__name__ == "cmd_updates"
     103    assert parser.parse_args(["outcomes", "--all"]).all is True
     104    assert parser.parse_args(["steer", "focus", "sources"]).func.__name__ == "cmd_steer"
     105    assert parser.parse_args(["say", "focus", "sources"]).func.__name__ == "cmd_steer"
     106    assert parser.parse_args(["pause"]).func.__name__ == "cmd_pause"
     107    assert parser.parse_args(["resume"]).func.__name__ == "cmd_resume"
     108    assert parser.parse_args(["resume", "research", "finder"]).job_id == ["research", "finder"]
     109    assert parser.parse_args(["cancel"]).func.__name__ == "cmd_cancel"
     110    assert parser.parse_args(["dashboard", "--no-follow"]).func.__name__ == "cmd_dashboard"
     111    assert parser.parse_args(["dash", "--no-follow"]).func.__name__ == "cmd_dashboard"
     112    assert parser.parse_args(["focus", "research"]).func.__name__ == "cmd_focus"
     113    assert parser.parse_args(["rename", "research", "--title", "new research"]).func.__name__ == "cmd_rename"
     114    assert parser.parse_args(["delete", "research"]).func.__name__ == "cmd_delete"
     115    assert parser.parse_args(["rm", "research"]).func.__name__ == "cmd_delete"
     116    assert parser.parse_args(["chat", "research", "finder"]).func.__name__ == "cmd_chat"
     117    assert parser.parse_args(["start", "--poll-seconds", "1"]).func.__name__ == "cmd_start"
     118    assert parser.parse_args(["stop"]).func.__name__ == "cmd_stop"
     119    assert parser.parse_args(["restart"]).func.__name__ == "cmd_restart"
     120    assert parser.parse_args(["stop", "research", "finder"]).func.__name__ == "cmd_stop"
     121    assert parser.parse_args(["stop", "research", "finder"]).job_id == ["research", "finder"]
     122    assert parser.parse_args(["ls"]).func.__name__ == "cmd_jobs"
     123    assert parser.parse_args(["autostart", "status"]).func.__name__ == "cmd_autostart"
     124    assert parser.parse_args(["browser-dashboard", "--port", "4848"]).func.__name__ == "cmd_browser_dashboard"
     125    assert parser.parse_args(["artifacts"]).func.__name__ == "cmd_artifacts"
     126    assert parser.parse_args(["artifact", "art_123"]).func.__name__ == "cmd_artifact"
     127    assert parser.parse_args(["artifact", "Findings", "Batch"]).func.__name__ == "cmd_artifact"
     128    assert parser.parse_args(["lessons"]).func.__name__ == "cmd_lessons"
     129    assert parser.parse_args(["learn", "low-evidence", "pages", "are", "bad"]).func.__name__ == "cmd_learn"
     130    assert parser.parse_args(["findings"]).func.__name__ == "cmd_findings"
     131    assert parser.parse_args(["tasks"]).func.__name__ == "cmd_tasks"
     132    assert parser.parse_args(["roadmap"]).func.__name__ == "cmd_roadmap"
     133    assert parser.parse_args(["experiments"]).func.__name__ == "cmd_experiments"
     134    assert parser.parse_args(["sources"]).func.__name__ == "cmd_sources"
     135    assert parser.parse_args(["memory"]).func.__name__ == "cmd_memory"
     136    assert parser.parse_args(["memory", "--graph"]).graph is True
     137    assert parser.parse_args(["metrics"]).func.__name__ == "cmd_metrics"
     138    assert parser.parse_args(["usage"]).func.__name__ == "cmd_usage"
     139    assert parser.parse_args(["outputs", "research", "finder"]).func.__name__ == "cmd_logs"
     140    assert parser.parse_args(["outputs"]).func.__name__ == "cmd_logs"
     141    assert parser.parse_args(["service", "status"]).func.__name__ == "cmd_service"
     142    assert parser.parse_args(["work", "--steps", "2", "--fake"]).func.__name__ == "cmd_work"
     143    assert parser.parse_args(["run", "--no-follow"]).func.__name__ == "cmd_run"
     144
     145
     146def test_cli_version_flag(capsys):
     147    try:
     148        main(["--version"])
     149    except SystemExit as exc:
     150        assert exc.code == 0
     151
     152    assert f"nipux {__version__}" in capsys.readouterr().out
     153
     154
     155def test_main_catches_keyboard_interrupt_without_traceback(monkeypatch, capsys):
     156    def interrupt(_args):
     157        raise KeyboardInterrupt
     158
     159    monkeypatch.setattr("nipux_cli.cli.cmd_home", interrupt)
     160
     161    main([])
     162
     163    assert capsys.readouterr().err == ""
     164
     165
     166def test_python_module_entrypoint_uses_cli_main():
     167    import nipux_cli.__main__ as module_entrypoint
     168
     169    assert module_entrypoint.main is main
     170
     171
     172def test_init_openrouter_writes_secret_free_config_and_env_template(monkeypatch, tmp_path, capsys):
     173    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     174
     175    main(["init", "--openrouter", "--model", "provider/model"])
     176
     177    out = capsys.readouterr().out
     178    config_text = (tmp_path / "config.yaml").read_text(encoding="utf-8")
     179    env_text = (tmp_path / ".env").read_text(encoding="utf-8")
     180    assert "Wrote" in out
     181    assert "name: provider/model" in config_text
     182    assert "base_url: https://openrouter.ai/api/v1" in config_text
     183    assert "api_key_env: OPENROUTER_API_KEY" in config_text
     184    assert "sk-" not in config_text
     185    assert env_text.strip().endswith("OPENROUTER_API_KEY" + "=")
     186    assert "sk-" not in env_text
     187    assert _mode(tmp_path / "config.yaml") == 0o600
     188    assert _mode(tmp_path / ".env") == 0o600
     189
     190
     191def test_init_defaults_to_local_endpoint(monkeypatch, tmp_path):
     192    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     193
     194    main(["init"])
     195
     196    config_text = (tmp_path / "config.yaml").read_text(encoding="utf-8")
     197    env_text = (tmp_path / ".env").read_text(encoding="utf-8")
     198    assert "name: local-model" in config_text
     199    assert "base_url: http://localhost:8000/v1" in config_text
     200    assert "api_key_env: OPENAI_API_KEY" in config_text
     201    assert env_text.strip().endswith("OPENAI_API_KEY" + "=")
     202
     203
     204def test_init_openrouter_defaults_to_generic_route(monkeypatch, tmp_path):
     205    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     206
     207    main(["init", "--openrouter"])
     208
     209    config_text = (tmp_path / "config.yaml").read_text(encoding="utf-8")
     210    assert "name: openrouter/auto" in config_text
     211    assert "base_url: https://openrouter.ai/api/v1" in config_text
     212    assert "api_key_env: OPENROUTER_API_KEY" in config_text
     213
     214
     215def test_shell_freeform_text_adds_operator_message(monkeypatch, tmp_path, capsys):
     216    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     217    db = AgentDB(tmp_path / "state.db")
     218    try:
     219        job_id = db.create_job("Research topic", title="research")
     220    finally:
     221        db.close()
     222
     223    assert _run_shell_line("focus on real evidence sources, not irrelevant sources") is True
     224
     225    out = capsys.readouterr().out
     226    db = AgentDB(tmp_path / "state.db")
     227    try:
     228        job = db.get_job(job_id)
     229        assert "waiting for research" in out
     230        assert (
     231            job["metadata"]["operator_messages"][-1]["message"]
     232            == "focus on real evidence sources, not irrelevant sources"
     233        )
     234    finally:
     235        db.close()
     236
     237
     238def test_main_no_args_enters_chat_first_home(monkeypatch, tmp_path, capsys):
     239    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     240    _mark_test_model_ready()
     241    db = AgentDB(tmp_path / "state.db")
     242    try:
     243        job_id = db.create_job("Research topic", title="research")
     244        db.append_operator_message(job_id, "remember this visible note", source="test")
     245        db.append_agent_update(job_id, "visible agent update", category="chat")
     246    finally:
     247        db.close()
     248
     249    def eof_input(_prompt):
     250        raise EOFError
     251
     252    monkeypatch.setattr("builtins.input", eof_input)
     253
     254    main([])
     255
     256    out = capsys.readouterr().out
     257    assert "WORKSPACE" in out
     258    assert "Jobs" in out
     259    assert "research" in out
     260
     261
     262def test_main_no_args_with_no_jobs_requires_setup_frame(monkeypatch, tmp_path, capsys):
     263    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     264
     265    def eof_input(_prompt):
     266        raise EOFError
     267
     268    monkeypatch.setattr("builtins.input", eof_input)
     269
     270    main([])
     271
     272    out = capsys.readouterr().out
     273    assert "Nipux setup requires an interactive terminal." in out
     274    assert "choose model, endpoint, and tool access" in out
     275    assert "first job" not in out
     276    assert "new       create a long-running job" not in out
     277    assert "doctor    check local setup" not in out
     278    assert "_   _" not in out
     279    assert "nipux menu >" not in out
     280
     281
     282def test_main_no_args_with_old_setup_marker_still_requires_model_verification(monkeypatch, tmp_path, capsys):
     283    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     284    load_config().ensure_dirs()
     285    from nipux_cli.cli_state import write_shell_state as write_shell_state
     286
     287    write_shell_state({"setup_completed": True})
     288
     289    main([])
     290
     291    out = capsys.readouterr().out
     292    assert "Nipux setup requires an interactive terminal." in out
     293    assert "No jobs are saved in this profile." not in out
     294
     295
     296def test_main_no_args_after_setup_complete_does_not_reopen_setup(monkeypatch, tmp_path, capsys):
     297    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     298    _mark_test_model_ready()
     299
     300    main([])
     301
     302    out = capsys.readouterr().out
     303    assert "No jobs are saved in this profile." in out
     304    assert "Nipux setup requires" not in out
     305    assert "Begin setup" not in out
     306
     307
     308def test_main_no_args_autoverifies_existing_model_config(monkeypatch, tmp_path, capsys):
     309    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     310    monkeypatch.setenv("TEST_MODEL_KEY", "working-key")
     311    tmp_path.mkdir(parents=True, exist_ok=True)
     312    (tmp_path / "config.yaml").write_text(
     313        "model:\n"
     314        "  name: provider/model\n"
     315        "  base_url: https://provider.example/v1\n"
     316        "  api_key_env: TEST_MODEL_KEY\n",
     317        encoding="utf-8",
     318    )
     319
     320    def fake_doctor(*, config, check_model):
     321        assert check_model is True
     322        assert config.model.model == "provider/model"
     323        return [
     324            Check("state_dir_writable", True, "ok"),
     325            Check("sqlite", True, "ok"),
     326            Check("model_config", True, "ok"),
     327            Check("model_endpoint", True, "ok"),
     328            Check("model_generation", True, "ok"),
     329        ]
     330
     331    monkeypatch.setattr("nipux_cli.cli.run_doctor", fake_doctor)
     332
     333    main([])
     334
     335    out = capsys.readouterr().out
     336    assert "No jobs are saved in this profile." in out
     337    assert "Model setup is not verified." not in out
     338    assert _model_setup_verified(load_config())
     339
     340
     341def test_main_no_args_enters_setup_when_existing_model_config_fails(monkeypatch, tmp_path, capsys):
     342    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     343    tmp_path.mkdir(parents=True, exist_ok=True)
     344    (tmp_path / "config.yaml").write_text(
     345        "model:\n"
     346        "  name: provider/model\n"
     347        "  base_url: https://provider.example/v1\n"
     348        "  api_key_env: TEST_MODEL_KEY\n",
     349        encoding="utf-8",
     350    )
     351
     352    def fake_doctor(*, config, check_model):
     353        assert check_model is True
     354        return [Check("model_generation", False, "provider rejected request")]
     355
     356    monkeypatch.setattr("nipux_cli.cli.run_doctor", fake_doctor)
     357
     358    main([])
     359
     360    out = capsys.readouterr().out
     361    assert "Nipux setup requires an interactive terminal." in out
     362    assert "No jobs are saved in this profile." not in out
     363    assert not _model_setup_verified(load_config())
     364
     365
     366def test_main_no_args_keeps_workspace_locked_after_completed_setup_if_provider_fails(monkeypatch, tmp_path, capsys):
     367    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     368    tmp_path.mkdir(parents=True, exist_ok=True)
     369    (tmp_path / "config.yaml").write_text(
     370        "model:\n"
     371        "  name: provider/model\n"
     372        "  base_url: https://provider.example/v1\n"
     373        "  api_key_env: TEST_MODEL_KEY\n",
     374        encoding="utf-8",
     375    )
     376    _write_shell_state({"setup_completed": True})
     377
     378    def fake_doctor(*, config, check_model):
     379        assert check_model is True
     380        return [Check("model_generation", False, "provider rejected request")]
     381
     382    monkeypatch.setattr("nipux_cli.cli.run_doctor", fake_doctor)
     383
     384    main([])
     385
     386    out = capsys.readouterr().out
     387    assert "Nipux setup requires an interactive terminal." in out
     388    assert "No jobs are saved in this profile." not in out
     389    assert not _model_setup_verified(load_config())
     390
     391
     392def test_first_run_refuses_job_before_model_is_verified(monkeypatch, tmp_path):
     393    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     394
     395    result = _handle_first_run_frame_line("new Build a durable workflow")
     396
     397    assert result[0] == "notice"
     398    assert "Finish setup first" in result[1]
     399    create_result = _handle_first_run_frame_line('create "Build a durable workflow"')
     400    assert create_result[0] == "notice"
     401    assert "Finish setup first" in create_result[1]
     402    jobs_result = _handle_first_run_frame_line("jobs")
     403    assert jobs_result[0] == "notice"
     404    assert "Jobs are available after Doctor verifies" in jobs_result[1]
     405    db = AgentDB(tmp_path / "state.db")
     406    try:
     407        assert db.list_jobs() == []
     408    finally:
     409        db.close()
     410
     411
     412def test_doctor_check_model_marks_model_setup_verified(monkeypatch, tmp_path, capsys):
     413    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     414
     415    def fake_doctor(*, config, check_model):
     416        assert check_model is True
     417        return [
     418            Check("state_dir_writable", True, "ok"),
     419            Check("sqlite", True, "ok"),
     420            Check("model_config", True, "ok"),
     421            Check("model_endpoint", True, "ok"),
     422            Check("model_generation", True, "ok"),
     423        ]
     424
     425    monkeypatch.setattr("nipux_cli.cli.run_doctor", fake_doctor)
     426    args = build_parser().parse_args(["doctor", "--check-model"])
     427    args.func(args)
     428
     429    out = capsys.readouterr().out
     430    assert "model_setup\tverified" in out
     431    assert _model_setup_verified(load_config())
     432
     433
     434def test_first_run_doctor_failure_shows_inline_fix_commands(monkeypatch, tmp_path):
     435    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     436
     437    def fake_doctor(*, config, check_model):
     438        assert check_model is True
     439        return [Check("model_endpoint", False, "connection refused")]
     440
     441    monkeypatch.setattr("nipux_cli.cli.run_doctor", fake_doctor)
     442
     443    lines = _verify_model_setup_from_first_run()
     444    rendered = "\n".join(lines)
     445
     446    assert "Model setup is not ready" in rendered
     447    assert "/base-url URL" in rendered
     448    assert "/api-key KEY" in rendered
     449    assert "/model MODEL" in rendered
     450    assert "local server" in rendered
     451
     452
     453def test_setting_change_clears_model_setup_verification(monkeypatch, tmp_path):
     454    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     455    _mark_test_model_ready()
     456
     457    assert _model_setup_verified(load_config())
     458    _inline_setting_notice("model.name", "provider/other-model")
     459
     460    assert not _model_setup_verified(load_config())
     461
     462
     463def test_first_run_menu_blocks_job_creation_until_workspace_chat(monkeypatch, tmp_path, capsys):
     464    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     465    _mark_test_model_ready()
     466
     467    assert _handle_first_run_menu_line("new Build a durable workflow") is True
     468
     469    out = capsys.readouterr().out
     470    db = AgentDB(tmp_path / "state.db")
     471    try:
     472        assert db.list_jobs() == []
     473        assert "Finish setup first" in out
     474    finally:
     475        db.close()
     476
     477
     478def test_first_run_plain_greeting_does_not_create_job(monkeypatch, tmp_path, capsys):
     479    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     480
     481    assert _handle_first_run_menu_line("Hello") is True
     482
     483    out = capsys.readouterr().out
     484    db = AgentDB(tmp_path / "state.db")
     485    try:
     486        assert db.list_jobs() == []
     487    finally:
     488        db.close()
     489    assert "Setup must be completed" in out
     490
     491
     492def test_first_run_frame_uses_full_screen_ui_not_banner(monkeypatch, tmp_path):
     493    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     494
     495    frame = _build_first_run_frame("", [], width=100, height=24)
     496    lines = frame.splitlines()
     497
     498    assert "workspace" not in lines[0].lower()
     499    assert "Endpoint" in lines[0]
     500    assert "Enter the endpoint first" in frame
     501    assert "Begin setup" not in frame
     502    assert "Long-running work, installed in-session." not in frame
     503    assert "Required: type an OpenAI-compatible endpoint URL" in frame
     504    assert "controls on the right" not in frame
     505    assert "Control" not in frame
     506    assert "SETUP" not in frame
     507    assert "│ SETUP" not in frame
     508    assert "daemon stopped" not in frame
     509    assert "FIRST RUN" not in frame
     510    assert "nipux menu >" not in frame
     511    assert "/shell" not in frame
     512
     513
     514def test_first_run_frame_hides_command_popup_during_setup(monkeypatch, tmp_path):
     515    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     516
     517    frame = _build_first_run_frame("/", [], width=100, height=26)
     518
     519    assert "commands" not in frame
     520    assert "/new" not in frame
     521    assert "/jobs" not in frame
     522    assert "/model" not in frame
     523    assert "/settings" not in frame
     524    assert "/shell" not in frame
     525    assert "Enter the endpoint first" in frame
     526
     527
     528def test_first_run_frame_walks_setup_screens(monkeypatch, tmp_path):
     529    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     530
     531    model = _build_first_run_frame("", [], width=100, height=26, view="model", selected=0)
     532    endpoint = _build_first_run_frame("", [], width=100, height=26, view="endpoint", selected=0)
     533    api = _build_first_run_frame("", [], width=100, height=26, view="api", selected=0)
     534    access = _build_first_run_frame("", [], width=100, height=26, view="access", selected=0)
     535    doctor = _build_first_run_frame("", [], width=100, height=28, view="doctor", selected=0)
     536    invalid = _build_first_run_frame("", [], width=100, height=26, view="settings", selected=1)
     537
     538    assert "Enter the model id" in model
     539    assert "Blank input is not accepted" in model
     540    assert "Enter the endpoint first" in endpoint
     541    assert "BASE URL" in endpoint
     542    assert "Enter the API key" in api
     543    assert "type skip" in api
     544    assert "Choose tool access" in access
     545    assert "Browser" in access
     546    assert "CLI" in access
     547    assert "Run checks" in doctor
     548    assert "/base-url" in doctor
     549    assert "/api-key" in doctor
     550    assert "/model" in doctor
     551    assert "Enter the model id" not in endpoint
     552    assert "Enter the endpoint first" not in api
     553    assert "Enter the endpoint first" in invalid
     554    assert "/shell" not in model
     555
     556
     557def test_first_run_frame_does_not_use_command_palette_for_setup(monkeypatch, tmp_path):
     558    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     559
     560    frame = _build_first_run_frame("/model", [], width=100, height=26)
     561
     562    assert "/model" in frame
     563    assert "set model" not in frame
     564    assert "Settings" not in frame
     565    assert "Enter the endpoint first" in frame
     566
     567
     568def test_settings_editor_persists_model_config(monkeypatch, tmp_path):
     569    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     570
     571    assert _save_config_field("model.name", "demo/model") == "demo/model"
     572    assert _save_config_field("model.context_length", "4096") == 4096
     573    assert _save_config_field("runtime.daily_digest_enabled", "false") is False
     574
     575    assert _config_field_value("model.name") == "demo/model"
     576    assert _config_field_value("model.context_length") == 4096
     577    assert _config_field_value("runtime.daily_digest_enabled") is False
     578    text = (tmp_path / "config.yaml").read_text(encoding="utf-8")
     579    assert "demo/model" in text
     580    assert _inline_setting_notice("model.name", "") == "kept model.name"
     581
     582
     583def test_slash_autocomplete_filters_commands():
     584    assert _autocomplete_slash("/do", FIRST_RUN_SLASH_COMMANDS) == "/doctor "
     585    assert _autocomplete_slash("/mo", FIRST_RUN_SLASH_COMMANDS) == "/model "
     586    assert _autocomplete_slash("/sta", CHAT_SLASH_COMMANDS) == "/status "
     587    assert _autocomplete_slash("/rest", CHAT_SLASH_COMMANDS) == "/restart "
     588    assert _autocomplete_slash("/step", CHAT_SLASH_COMMANDS) == "/step-limit "
     589    assert _autocomplete_slash("/out", FIRST_RUN_SLASH_COMMANDS) == "/output-chars "
     590    assert _cycle_slash("/", CHAT_SLASH_COMMANDS, direction=1) == "/new"
     591    assert _cycle_slash("/", CHAT_SLASH_COMMANDS, direction=-1) == "/exit"
     592    assert _cycle_slash("/work ", CHAT_SLASH_COMMANDS, direction=1) == "/work "
     593    assert _cycle_slash("/run", CHAT_SLASH_COMMANDS, direction=1) == "/jobs"
     594    assert _cycle_slash("/", FIRST_RUN_SLASH_COMMANDS, direction=1) == "/model"
     595    assert _cycle_slash("/", FIRST_RUN_SLASH_COMMANDS, direction=-1) == "/exit"
     596    assert _cycle_slash("/model", FIRST_RUN_SLASH_COMMANDS, direction=1) == "/base-url"
     597    assert _cycle_slash("/model", FIRST_RUN_SLASH_COMMANDS, direction=-1) == "/exit"
     598    assert _cycle_slash("/out", CHAT_SLASH_COMMANDS, direction=1) == "/outcomes"
     599    assert _cycle_slash("/out", CHAT_SLASH_COMMANDS, direction=-1) == "/output-cost"
     600    assert _slash_completion_for_submit("/", CHAT_SLASH_COMMANDS) == ("/new ", False)
     601    assert _slash_completion_for_submit("/", FIRST_RUN_SLASH_COMMANDS) == ("/model", True)
     602    assert _slash_completion_for_submit("/mo", FIRST_RUN_SLASH_COMMANDS) == ("/model", True)
     603    assert _slash_completion_for_submit("/model", FIRST_RUN_SLASH_COMMANDS) == ("/model", True)
     604    assert _slash_completion_for_submit("/new ", CHAT_SLASH_COMMANDS) == ("/new ", False)
     605    assert _slash_completion_for_submit("/new research agents", CHAT_SLASH_COMMANDS) == ("/new research agents", True)
     606    assert _slash_completion_for_submit("/model ", CHAT_SLASH_COMMANDS) == ("/model ", True)
     607    assert _slash_completion_for_submit("/j", CHAT_SLASH_COMMANDS) == ("/jobs", True)
     608    assert _slash_completion_for_submit("/set", CHAT_SLASH_COMMANDS) == ("/settings", True)
     609    assert _slash_completion_for_submit("/model", CHAT_SLASH_COMMANDS) == ("/model", True)
     610    assert _slash_completion_for_submit("/mo", CHAT_SLASH_COMMANDS) == ("/model", True)
     611    assert _slash_completion_for_submit("/run", CHAT_SLASH_COMMANDS) == ("/run", True)
     612    assert _slash_completion_for_submit("/settings", CHAT_SLASH_COMMANDS) == ("/settings", True)
     613    assert _slash_completion_for_submit("/model demo/model", CHAT_SLASH_COMMANDS) == ("/model demo/model", True)
     614    assert _autocomplete_slash("plain text", CHAT_SLASH_COMMANDS) == "plain text"
     615    lines = _slash_suggestion_lines("/art", CHAT_SLASH_COMMANDS, width=80)
     616    text = "\n".join(lines)
     617    assert "/artifacts" in text
     618    assert "/artifact" in text
     619    assert "/run" not in text
     620    hint_text = "\n".join(_slash_suggestion_lines("/model ", CHAT_SLASH_COMMANDS, width=80))
     621    assert "/model" in hint_text
     622    assert "MODEL" in hint_text
     623    partial_hint_text = "\n".join(_slash_suggestion_lines("/mo", CHAT_SLASH_COMMANDS, width=80))
     624    assert "/model MODEL" in partial_hint_text
     625    assert "↑↓ moves" in partial_hint_text
     626    full_palette_text = "\n".join(_slash_suggestion_lines("/", CHAT_SLASH_COMMANDS, width=80, limit=5))
     627    assert "enter selects" in full_palette_text
     628    assert "/new OBJECTIVE" in full_palette_text
     629    assert "/run" in full_palette_text
     630    assert "/settings" in full_palette_text
     631    assert "type OBJECTIVE" in "\n".join(_slash_suggestion_lines("/new ", CHAT_SLASH_COMMANDS, width=80))
     632    assert "/shell" not in "\n".join(_slash_suggestion_lines("/", CHAT_SLASH_COMMANDS, width=80, limit=20))
     633    assert "/restart" in "\n".join(_slash_suggestion_lines("/re", CHAT_SLASH_COMMANDS, width=80, limit=20))
     634
     635
     636def test_terminal_escape_decodes_arrows_and_mouse_click():
     637    assert _decode_terminal_escape("\x1b[A") == ("up", None)
     638    assert _decode_terminal_escape("\x1b[B") == ("down", None)
     639    assert _decode_terminal_escape("\x1b[C") == ("right", None)
     640    assert _decode_terminal_escape("\x1b[D") == ("left", None)
     641    assert _decode_terminal_escape("\x1bOB") == ("down", None)
     642    assert _decode_terminal_escape("\x1b[1;2B") == ("down", None)
     643    assert _decode_terminal_escape("\x1b[<0;88;12M") == ("click", (88, 12))
     644    assert _decode_terminal_escape("\x1b[M !!") == ("click", (1, 1))
     645
     646
     647def test_first_run_click_maps_right_pane_actions(monkeypatch):
     648    monkeypatch.setattr("shutil.get_terminal_size", lambda fallback=(100, 30): (100, 30))
     649
     650    assert _first_run_click_action(5, 15, view="endpoint") is None
     651    assert _first_run_click_action(5, 15, view="access") == 0
     652    assert _first_run_click_action(25, 15, view="access") == 1
     653    assert _first_run_click_action(1, 4, view="access") is None
     654    assert _first_run_click_action(5, 1, view="model") is None
     655
     656
     657def test_first_run_arrow_navigation_changes_setup_screens():
     658    assert _directional_first_run_action(
     659        [
     660            ("view:model", "Begin setup", "walk through setup"),
     661            ("doctor", "Doctor", "check"),
     662        ],
     663        direction=1,
     664    ) == "view:model"
     665    assert _directional_first_run_action(
     666        [
     667            ("edit:model.name", "Edit model", "set model"),
     668            ("view:connector", "Continue", "choose connector"),
     669            ("view:start", "Back", "intro"),
     670        ],
     671        direction=1,
     672    ) == "view:connector"
     673    assert _directional_first_run_action(
     674        [
     675            ("edit:model.name", "Edit model", "set model"),
     676            ("view:connector", "Continue", "choose connector"),
     677            ("view:start", "Back", "intro"),
     678        ],
     679        direction=-1,
     680    ) == "view:start"
     681
     682
     683def test_frame_next_job_cycles_jobs():
     684    snapshot = {"jobs": [{"id": "one"}, {"id": "two"}, {"id": "three"}]}
     685
     686    assert _frame_next_job_id(snapshot, "one", direction=1) == "two"
     687    assert _frame_next_job_id(snapshot, "one", direction=-1) == "three"
     688    assert _frame_next_job_id(snapshot, "missing", direction=1) == "two"
     689
     690
     691def test_frame_refresh_slows_background_updates_while_typing():
     692    assert _frame_refresh_interval("") < _frame_refresh_interval("drafting a message")
     693
     694
     695def test_first_run_empty_submit_without_actions_does_not_crash():
     696    deps = _FirstRunRuntimeDeps(
     697        render_frame=lambda _buffer, _notices, _selected, _view, _editing, _previous: "",
     698        actions=lambda _view: [],
     699        handle_action=lambda _action: ("notice", "unused"),
     700        handle_line=lambda _line: ("notice", "unused"),
     701        click_action=lambda _x, _y, _view: None,
     702    )
     703
     704    assert _submit_first_run_line("", selected=0, view="empty", deps=deps) == (
     705        "notice",
     706        "This setup step requires an explicit value.",
     707    )
     708
     709
     710def test_first_run_required_edit_cancel_and_clear_stay_on_same_field():
     711    notices: list[str] = []
     712
     713    buffer, editing_field, should_exit = _handle_first_run_edit_input(
     714        "\x15",
     715        buffer="not-a-url",
     716        editing_field="model.base_url",
     717        notices=notices,
     718        stdin_fd=0,
     719    )
     720
     721    assert buffer == ""
     722    assert editing_field == "model.base_url"
     723    assert should_exit is False
     724
     725    buffer, editing_field, should_exit = _handle_first_run_edit_input(
     726        "\x03",
     727        buffer="partial",
     728        editing_field="model.base_url",
     729        notices=notices,
     730        stdin_fd=0,
     731    )
     732
     733    assert buffer == ""
     734    assert editing_field == "model.base_url"
     735    assert should_exit is False
     736    assert "cancelled edit" in "\n".join(notices)
     737
     738
     739def test_chat_settings_edit_supports_ctrl_u_clear():
     740    buffer, editing_field, should_exit = _handle_chat_edit_input(
     741        "\x15",
     742        buffer="wrong-model",
     743        editing_field="model.name",
     744        notices=[],
     745        stdin_fd=0,
     746    )
     747
     748    assert buffer == ""
     749    assert editing_field == "model.name"
     750    assert should_exit is False
     751
     752
     753def test_first_run_render_failure_uses_safe_mode(capsys):
     754    deps = _FirstRunRuntimeDeps(
     755        render_frame=lambda *_args: (_ for _ in ()).throw(RuntimeError("bad frame")),
     756        actions=lambda _view: [],
     757        handle_action=lambda _action: ("notice", "unused"),
     758        handle_line=lambda _line: ("notice", "unused"),
     759        click_action=lambda _x, _y, _view: None,
     760    )
     761    notices: list[str] = []
     762
     763    frame = _safe_first_run_render_frame(
     764        deps,
     765        buffer="hello",
     766        notices=notices,
     767        selected=0,
     768        view="start",
     769        editing_field=None,
     770        previous_frame="",
     771    )
     772
     773    assert "safe mode" in frame
     774    assert "render failed" in "\n".join(notices)
     775    assert "bad frame" in capsys.readouterr().out
     776
     777
     778def test_chat_submit_failure_stays_in_frame():
     779    snapshot = {"job_id": "job_demo", "job": {"id": "job_demo", "title": "demo"}, "jobs": []}
     780    async_messages: queue.Queue[str] = queue.Queue()
     781
     782    deps = _ChatFrameDeps(
     783        load_snapshot=lambda _job_id, _history_limit: snapshot,
     784        render_frame=lambda *_args: "",
     785        handle_chat_message=lambda _job_id, _line: (_ for _ in ()).throw(RuntimeError("model blew up")),
     786        capture_chat_command=lambda _job_id, _line: (True, ""),
     787        write_shell_state=lambda _state: None,
     788        is_plain_chat_line=lambda _line: True,
     789        page_click=lambda _x, _y, _right_view: None,
     790    )
     791
     792    keep_running, _snapshot, job_id, notices, right_view, modal = _handle_chat_submit(
     793        "hello",
     794        job_id="job_demo",
     795        history_limit=12,
     796        snapshot=snapshot,
     797        notices=[],
     798        right_view="status",
     799        modal_view=None,
     800        deps=deps,
     801        async_messages=async_messages,
     802    )
     803
     804    assert keep_running is True
     805    assert job_id == "job_demo"
     806    assert right_view == "status"
     807    assert modal is None
     808    assert _THINKING_NOTICE in notices
     809    assert "> hello" not in "\n".join(notices)
     810    queued = async_messages.get(timeout=1)
     811    assert "message failed" in queued
     812    assert "model blew up" in queued
     813    async_messages.put(queued)
     814    assert _drain_chat_async_notices(async_messages, notices) is True
     815    assert _THINKING_NOTICE not in notices
     816    assert "message failed" in "\n".join(notices)
     817
     818
     819def test_chat_submit_plain_message_returns_without_waiting_for_model():
     820    snapshot = {"job_id": "job_demo", "job": {"id": "job_demo", "title": "demo"}, "jobs": []}
     821    async_messages: queue.Queue[str] = queue.Queue()
     822
     823    def slow_chat(_job_id, _line):
     824        time.sleep(0.3)
     825        return True, "done later"
     826
     827    deps = _ChatFrameDeps(
     828        load_snapshot=lambda _job_id, _history_limit: snapshot,
     829        render_frame=lambda *_args: "",
     830        handle_chat_message=slow_chat,
     831        capture_chat_command=lambda _job_id, _line: (True, ""),
     832        write_shell_state=lambda _state: None,
     833        is_plain_chat_line=lambda _line: True,
     834        page_click=lambda _x, _y, _right_view: None,
     835    )
     836
     837    started = time.monotonic()
     838    keep_running, _snapshot, _job_id, notices, _right_view, _modal = _handle_chat_submit(
     839        "hello",
     840        job_id="job_demo",
     841        history_limit=12,
     842        snapshot=snapshot,
     843        notices=[],
     844        right_view="status",
     845        modal_view=None,
     846        deps=deps,
     847        async_messages=async_messages,
     848    )
     849
     850    assert keep_running is True
     851    assert time.monotonic() - started < 0.1
     852    assert _THINKING_NOTICE in notices
     853    assert "> hello" not in "\n".join(notices)
     854    assert async_messages.get(timeout=1) == "__refresh__"
     855
     856
     857def test_chat_submit_plain_message_renders_thinking_notice_without_echoing_message():
     858    snapshot = {"job_id": "job_demo", "job": {"id": "job_demo", "title": "demo"}, "jobs": []}
     859    async_messages: queue.Queue[str] = queue.Queue()
     860
     861    deps = _ChatFrameDeps(
     862        load_snapshot=lambda _job_id, _history_limit: snapshot,
     863        render_frame=lambda *_args: "",
     864        handle_chat_message=lambda _job_id, _line: (True, "done"),
     865        capture_chat_command=lambda _job_id, _line: (True, ""),
     866        write_shell_state=lambda _state: None,
     867        is_plain_chat_line=lambda _line: True,
     868        page_click=lambda _x, _y, _right_view: None,
     869    )
     870
     871    _keep_running, _snapshot, _job_id, notices, _right_view, _modal = _handle_chat_submit(
     872        "Hello",
     873        job_id="job_demo",
     874        history_limit=12,
     875        snapshot=snapshot,
     876        notices=[],
     877        right_view="status",
     878        modal_view=None,
     879        deps=deps,
     880        async_messages=async_messages,
     881    )
     882
     883    rendered_notices = _display_chat_notices(notices)
     884    lines = chat_pane_lines([], rendered_notices, width=60, rows=4)
     885    rendered = "\n".join(lines)
     886    assert "thinking" in rendered
     887    assert "Hello" not in rendered
     888    assert "waiting for model" not in rendered
     889
     890
     891def test_chat_submit_waiting_command_output_becomes_animation():
     892    snapshot = {"job_id": "job_demo", "job": {"id": "job_demo", "title": "demo"}, "jobs": []}
     893
     894    deps = _ChatFrameDeps(
     895        load_snapshot=lambda _job_id, _history_limit: snapshot,
     896        render_frame=lambda *_args: "",
     897        handle_chat_message=lambda _job_id, _line: (True, ""),
     898        capture_chat_command=lambda _job_id, _line: (
     899            True,
     900            "waiting for demo: what has it done so far?\nWaiting for the next worker step.",
     901        ),
     902        write_shell_state=lambda _state: None,
     903        is_plain_chat_line=lambda _line: False,
     904        page_click=lambda _x, _y, _right_view: None,
     905    )
     906
     907    _keep_running, _snapshot, _job_id, notices, _right_view, _modal = _handle_chat_submit(
     908        "/pause",
     909        job_id="job_demo",
     910        history_limit=12,
     911        snapshot=snapshot,
     912        notices=[],
     913        right_view="status",
     914        modal_view=None,
     915        deps=deps,
     916    )
     917
     918    lines = chat_pane_lines([], _display_chat_notices(notices), width=80, rows=6)
     919    rendered = "\n".join(lines)
     920    assert "waiting" in rendered
     921    assert "waiting for demo" not in rendered
     922    assert "Waiting for the next worker step" not in rendered
     923    assert "NIPUX" not in rendered
     924
     925
     926def test_chat_submit_new_refreshes_focused_job_from_shell_state():
     927    old_snapshot = {"job_id": "old", "job": {"id": "old", "title": "old"}, "jobs": []}
     928    new_snapshot = {"job_id": "new", "job": {"id": "new", "title": "new"}, "jobs": []}
     929    loaded: list[str] = []
     930
     931    def load_snapshot(job_id, _history_limit):
     932        loaded.append(job_id)
     933        return new_snapshot if job_id == "" else old_snapshot
     934
     935    deps = _ChatFrameDeps(
     936        load_snapshot=load_snapshot,
     937        render_frame=lambda *_args: "",
     938        handle_chat_message=lambda _job_id, _line: (True, ""),
     939        capture_chat_command=lambda _job_id, _line: (True, "created new\nfocus set to new"),
     940        write_shell_state=lambda _state: None,
     941        is_plain_chat_line=lambda _line: False,
     942        page_click=lambda _x, _y, _right_view: None,
     943    )
     944
     945    keep_running, _snapshot, job_id, notices, _right_view, _modal = _handle_chat_submit(
     946        "/new Build a durable workflow",
     947        job_id="old",
     948        history_limit=12,
     949        snapshot=old_snapshot,
     950        notices=[],
     951        right_view="status",
     952        modal_view=None,
     953        deps=deps,
     954    )
     955
     956    assert keep_running is True
     957    assert job_id == "new"
     958    assert loaded == [""]
     959    assert "focus set to new" in "\n".join(notices)
     960
     961
     962def test_workspace_chat_submit_new_keeps_workspace_chat_left_pane():
     963    old_snapshot = {"job_id": WORKSPACE_CHAT_ID, "job": {"id": WORKSPACE_CHAT_ID, "title": "Nipux"}, "jobs": []}
     964    workspace_snapshot = {
     965        "job_id": WORKSPACE_CHAT_ID,
     966        "job": {"id": WORKSPACE_CHAT_ID, "title": "Nipux"},
     967        "right_job": {"id": "new", "title": "new worker"},
     968        "jobs": [],
     969    }
     970    loaded: list[str] = []
     971
     972    def load_snapshot(job_id, _history_limit):
     973        loaded.append(job_id)
     974        return workspace_snapshot
     975
     976    deps = _ChatFrameDeps(
     977        load_snapshot=load_snapshot,
     978        render_frame=lambda *_args: "",
     979        handle_chat_message=lambda _job_id, _line: (True, ""),
     980        capture_chat_command=lambda _job_id, _line: (True, "created new\nfocus set to new"),
     981        write_shell_state=lambda _state: None,
     982        is_plain_chat_line=lambda _line: False,
     983        page_click=lambda _x, _y, _right_view: None,
     984    )
     985
     986    keep_running, snapshot, job_id, notices, _right_view, _modal = _handle_chat_submit(
     987        "/new Build a durable workflow",
     988        job_id=WORKSPACE_CHAT_ID,
     989        history_limit=12,
     990        snapshot=old_snapshot,
     991        notices=[],
     992        right_view="updates",
     993        modal_view=None,
     994        deps=deps,
     995    )
     996
     997    assert keep_running is True
     998    assert job_id == WORKSPACE_CHAT_ID
     999    assert snapshot["right_job"]["title"] == "new worker"
    1000    assert loaded == [WORKSPACE_CHAT_ID]
    1001    assert "focus set to new" in "\n".join(notices)
    1002
    1003
    1004def test_chat_render_failure_uses_safe_mode(capsys):
    1005    snapshot = {"job_id": "job_demo", "job": {"id": "job_demo", "title": "demo"}, "jobs": []}
    1006    deps = _ChatFrameDeps(
    1007        load_snapshot=lambda _job_id, _history_limit: snapshot,
    1008        render_frame=lambda *_args: (_ for _ in ()).throw(RuntimeError("bad chat frame")),
    1009        handle_chat_message=lambda _job_id, _line: (True, ""),
    1010        capture_chat_command=lambda _job_id, _line: (True, ""),
    1011        write_shell_state=lambda _state: None,
    1012        is_plain_chat_line=lambda _line: True,
    1013        page_click=lambda _x, _y, _right_view: None,
    1014    )
    1015    notices: list[str] = []
    1016
    1017    frame = _safe_chat_render_frame(
    1018        deps,
    1019        snapshot=snapshot,
    1020        buffer="hello",
    1021        notices=notices,
    1022        right_view="status",
    1023        selected_control=0,
    1024        editing_field=None,
    1025        modal_view=None,
    1026        previous_frame="",
    1027    )
    1028
    1029    assert "safe mode" in frame
    1030    assert "render failed" in "\n".join(notices)
    1031    assert "bad chat frame" in capsys.readouterr().out
    1032
    1033
    1034def test_chat_help_has_config_slash_commands_without_settings_page(monkeypatch, tmp_path, capsys):
    1035    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1036    db = AgentDB(tmp_path / "state.db")
    1037    try:
    1038        job_id = db.create_job("Research topic", title="research")
    1039    finally:
    1040        db.close()
    1041
    1042    assert _chat_handle_line(job_id, "/help") is True
    1043
    1044    out = capsys.readouterr().out
    1045    assert "Core workflow:" in out
    1046    assert "/new OBJECTIVE       create a job and start work" in out
    1047    assert "/run                 resume/start the focused job" in out
    1048    assert "/activity            tool calls" in out
    1049    assert "/settings" in out
    1050    assert "/usage" in out
    1051    assert "/config" in out
    1052    assert "/outcomes" in out
    1053    assert "/model MODEL" in out
    1054    assert "/api-key KEY" in out
    1055    assert "/timeout SECONDS" in out
    1056    assert "/browser true|false" in out
    1057    assert "/cli-access true|false" in out
    1058    assert "/home PATH" in out
    1059    assert "/digest-time HH:MM" in out
    1060    assert "/shell" not in out
    1061
    1062
    1063def test_chat_slash_palette_matches_public_chat_commands():
    1064    palette = {command for command, _description in CHAT_SLASH_COMMANDS}
    1065    assert len(palette) == len(CHAT_SLASH_COMMANDS)
    1066    advertised = {
    1067        "/jobs",
    1068        "/focus",
    1069        "/switch",
    1070        "/new",
    1071        "/delete",
    1072        "/history",
    1073        "/events",
    1074        "/activity",
    1075        "/outputs",
    1076        "/updates",
    1077        "/outcomes",
    1078        "/status",
    1079        "/usage",
    1080        "/config",
    1081        "/settings",
    1082        "/health",
    1083        "/help",
    1084        "/artifacts",
    1085        "/artifact",
    1086        "/findings",
    1087        "/tasks",
    1088        "/roadmap",
    1089        "/experiments",
    1090        "/sources",
    1091        "/memory",
    1092        "/metrics",
    1093        "/lessons",
    1094        "/model",
    1095        "/base-url",
    1096        "/api-key",
    1097        "/api-key-env",
    1098        "/context",
    1099        "/input-cost",
    1100        "/output-cost",
    1101        "/timeout",
    1102        "/browser",
    1103        "/web",
    1104        "/cli-access",
    1105        "/file-access",
    1106        "/home",
    1107        "/step-limit",
    1108        "/output-chars",
    1109        "/daily-digest",
    1110        "/digest-time",
    1111        "/doctor",
    1112        "/run",
    1113        "/start",
    1114        "/restart",
    1115        "/work",
    1116        "/work-verbose",
    1117        "/stop",
    1118        "/pause",
    1119        "/resume",
    1120        "/cancel",
    1121        "/learn",
    1122        "/note",
    1123        "/follow",
    1124        "/digest",
    1125        "/clear",
    1126        "/exit",
    1127    }
    1128
    1129    assert advertised <= palette
    1130    assert "/shell" not in palette
    1131
    1132
    1133def test_workspace_help_is_minimal_and_actionable(monkeypatch, tmp_path):
    1134    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1135    _mark_test_model_ready()
    1136
    1137    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "/help")
    1138
    1139    assert keep_running is True
    1140    assert "Create: type a goal" in output
    1141    assert "/new OBJECTIVE" in output
    1142    assert "/settings" in output
    1143    assert "Navigate:" in output
    1144    assert "Common" not in output
    1145    assert "/shell" not in output
    1146
    1147
    1148def test_first_run_slash_palette_matches_setup_commands():
    1149    palette = {command for command, _description in FIRST_RUN_SLASH_COMMANDS}
    1150    assert len(palette) == len(FIRST_RUN_SLASH_COMMANDS)
    1151
    1152    advertised = {
    1153        "/model",
    1154        "/base-url",
    1155        "/api-key",
    1156        "/api-key-env",
    1157        "/config",
    1158        "/context",
    1159        "/input-cost",
    1160        "/output-cost",
    1161        "/timeout",
    1162        "/browser",
    1163        "/web",
    1164        "/cli-access",
    1165        "/file-access",
    1166        "/home",
    1167        "/step-limit",
    1168        "/output-chars",
    1169        "/daily-digest",
    1170        "/digest-time",
    1171        "/doctor",
    1172        "/init",
    1173        "/help",
    1174        "/clear",
    1175        "/exit",
    1176    }
    1177
    1178    assert advertised <= palette
    1179    assert "/new" not in palette
    1180    assert "/jobs" not in palette
    1181    assert "/shell" not in palette
    1182    assert "/settings" not in palette
    1183
    1184
    1185def test_chat_settings_slash_commands_persist_config(monkeypatch, tmp_path, capsys):
    1186    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1187    monkeypatch.setenv("NIPUX_TEST_KEY", "")
    1188    db = AgentDB(tmp_path / "state.db")
    1189    try:
    1190        job_id = db.create_job("Research topic", title="research")
    1191    finally:
    1192        db.close()
    1193
    1194    assert _chat_handle_line(job_id, "/model provider/model") is True
    1195    assert _chat_handle_line(job_id, "/base-url https://example.com/v1") is True
    1196    assert _chat_handle_line(job_id, "/context 8192") is True
    1197    assert _chat_handle_line(job_id, "/input-cost 0.10") is True
    1198    assert _chat_handle_line(job_id, "/output-cost 0.20") is True
    1199    assert _chat_handle_line(job_id, "/max-cost 15") is True
    1200    assert _chat_handle_line(job_id, "/timeout 45") is True
    1201    assert _chat_handle_line(job_id, "/browser false") is True
    1202    assert _chat_handle_line(job_id, "/web false") is True
    1203    assert _chat_handle_line(job_id, "/cli-access false") is True
    1204    assert _chat_handle_line(job_id, "/file-access false") is True
    1205    assert _chat_handle_line(job_id, "/step-limit 90") is True
    1206    assert _chat_handle_line(job_id, "/output-chars 4096") is True
    1207    assert _chat_handle_line(job_id, "/daily-digest false") is True
    1208    assert _chat_handle_line(job_id, "/digest-time 08:30") is True
    1209    assert _chat_handle_line(job_id, "/api-key-env NIPUX_TEST_KEY") is True
    1210    assert _chat_handle_line(job_id, "/api-key sk-test-value") is True
    1211    out = capsys.readouterr().out
    1212
    1213    assert "saved model.name = provider/model" in out
    1214    assert "saved model.base_url = https://example.com/v1" in out
    1215    assert "saved model.context_length = 8192" in out
    1216    assert "saved model.input_cost_per_million = 0.1" in out
    1217    assert "saved model.output_cost_per_million = 0.2" in out
    1218    assert "saved runtime.max_job_cost_usd = 15.0" in out
    1219    assert "saved model.request_timeout_seconds = 45.0" in out
    1220    assert "saved tools.browser = False" in out
    1221    assert "saved tools.web = False" in out
    1222    assert "saved tools.shell = False" in out
    1223    assert "saved tools.files = False" in out
    1224    assert "saved runtime.max_step_seconds = 90" in out
    1225    assert "saved runtime.artifact_inline_char_limit = 4096" in out
    1226    assert "saved runtime.daily_digest_enabled = False" in out
    1227    assert "saved runtime.daily_digest_time = 08:30" in out
    1228    assert "saved model.api_key_env = NIPUX_TEST_KEY" in out
    1229    assert "saved NIPUX_TEST_KEY" in out
    1230    assert "sk-test-value" not in out
    1231    assert _mode(tmp_path / "config.yaml") == 0o600
    1232    assert _mode(tmp_path / ".env") == 0o600
    1233    assert _config_field_value("model.name") == "provider/model"
    1234    assert _config_field_value("model.base_url") == "https://example.com/v1"
    1235    assert _config_field_value("model.context_length") == 8192
    1236    assert _config_field_value("model.input_cost_per_million") == 0.1
    1237    assert _config_field_value("model.output_cost_per_million") == 0.2
    1238    assert _config_field_value("runtime.max_job_cost_usd") == 15.0
    1239    assert _config_field_value("model.request_timeout_seconds") == 45.0
    1240    assert _config_field_value("tools.browser") is False
    1241    assert _config_field_value("tools.web") is False
    1242    assert _config_field_value("tools.shell") is False
    1243    assert _config_field_value("tools.files") is False
    1244    assert _config_field_value("runtime.max_step_seconds") == 90
    1245    assert _config_field_value("runtime.artifact_inline_char_limit") == 4096
    1246    assert _config_field_value("runtime.daily_digest_enabled") is False
    1247    assert _config_field_value("runtime.daily_digest_time") == "08:30"
    1248    assert "NIPUX_TEST_KEY=sk-test-value" in (tmp_path / ".env").read_text(encoding="utf-8")
    1249
    1250
    1251def test_chat_init_slash_command_does_not_crash(monkeypatch, tmp_path, capsys):
    1252    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1253    db = AgentDB(tmp_path / "state.db")
    1254    try:
    1255        job_id = db.create_job("Research topic", title="research")
    1256    finally:
    1257        db.close()
    1258
    1259    assert _chat_handle_line(job_id, "/init") is True
    1260
    1261    out = capsys.readouterr().out
    1262    assert "Wrote" in out
    1263    assert (tmp_path / "config.yaml").exists()
    1264    assert (tmp_path / ".env").exists()
    1265
    1266
    1267def test_chat_config_slash_command_summarizes_runtime_without_secret(monkeypatch, tmp_path, capsys):
    1268    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1269    monkeypatch.setenv("NIPUX_TEST_KEY", "sk-test-value")
    1270    db = AgentDB(tmp_path / "state.db")
    1271    try:
    1272        job_id = db.create_job("Research topic", title="research")
    1273    finally:
    1274        db.close()
    1275    (tmp_path / "config.yaml").write_text(
    1276        """
    1277model:
    1278  name: provider/model
    1279  base_url: https://example.com/v1
    1280  api_key_env: NIPUX_TEST_KEY
    1281  context_length: 8192
    1282  request_timeout_seconds: 45
    1283  input_cost_per_million: 0.1
    1284  output_cost_per_million: 0.2
    1285runtime:
    1286  max_step_seconds: 90
    1287  max_job_cost_usd: 15
    1288  artifact_inline_char_limit: 4096
    1289  daily_digest_enabled: false
    1290  daily_digest_time: "08:30"
    1291""",
    1292        encoding="utf-8",
    1293    )
    1294
    1295    assert _chat_handle_line(job_id, "/config") is True
    1296
    1297    out = capsys.readouterr().out
    1298    assert "config" in out
    1299    assert "model: provider/model" in out
    1300    assert "endpoint: https://example.com/v1" in out
    1301    assert "key: set (NIPUX_TEST_KEY)" in out
    1302    assert "context: 8192" in out
    1303    assert "cost rates: input $0.1 / output $0.2 per 1M tokens" in out
    1304    assert "job cost limit: $15" in out
    1305    assert "sk-test-value" not in out
    1306
    1307
    1308def test_chat_usage_slash_command_reports_tokens(monkeypatch, tmp_path, capsys):
    1309    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1310    db = AgentDB(tmp_path / "state.db")
    1311    try:
    1312        job_id = db.create_job("Research topic", title="research")
    1313        db.append_event(
    1314            job_id,
    1315            event_type="loop",
    1316            title="message_end",
    1317            metadata={
    1318                "usage": {
    1319                    "prompt_tokens": 1000,
    1320                    "completion_tokens": 250,
    1321                    "total_tokens": 1250,
    1322                    "cost": 0.0042,
    1323                }
    1324            },
    1325        )
    1326    finally:
    1327        db.close()
    1328
    1329    assert _chat_handle_line(job_id, "/usage") is True
    1330
    1331    out = capsys.readouterr().out
    1332    assert "usage research" in out
    1333    assert "tokens: total=1.2K prompt=1.0K output=250" in out
    1334    assert "cost=$0.0042" in out
    1335
    1336
    1337def test_chat_usage_estimates_cost_from_configured_rates(monkeypatch, tmp_path, capsys):
    1338    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1339    (tmp_path / "config.yaml").write_text(
    1340        """
    1341model:
    1342  name: provider/model
    1343  base_url: https://example.com/v1
    1344  input_cost_per_million: 1.0
    1345  output_cost_per_million: 2.0
    1346""",
    1347        encoding="utf-8",
    1348    )
    1349    db = AgentDB(tmp_path / "state.db")
    1350    try:
    1351        job_id = db.create_job("Research topic", title="research")
    1352        db.append_event(
    1353            job_id,
    1354            event_type="loop",
    1355            title="message_end",
    1356            metadata={
    1357                "usage": {
    1358                    "prompt_tokens": 1000,
    1359                    "completion_tokens": 500,
    1360                    "total_tokens": 1500,
    1361                    "estimated": True,
    1362                }
    1363            },
    1364        )
    1365    finally:
    1366        db.close()
    1367
    1368    assert _chat_handle_line(job_id, "/usage") is True
    1369
    1370    out = capsys.readouterr().out
    1371    assert "cost=~$0.0020" in out
    1372
    1373
    1374def test_chat_usage_shows_configured_job_cost_limit(monkeypatch, tmp_path, capsys):
    1375    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1376    (tmp_path / "config.yaml").write_text(
    1377        """
    1378model:
    1379  name: provider/model
    1380  base_url: https://example.com/v1
    1381runtime:
    1382  max_job_cost_usd: 5
    1383""",
    1384        encoding="utf-8",
    1385    )
    1386    db = AgentDB(tmp_path / "state.db")
    1387    try:
    1388        job_id = db.create_job("Research topic", title="research")
    1389        db.append_event(
    1390            job_id,
    1391            event_type="loop",
    1392            title="message_end",
    1393            metadata={"usage": {"prompt_tokens": 1000, "completion_tokens": 500, "total_tokens": 1500, "cost": 1.25}},
    1394        )
    1395    finally:
    1396        db.close()
    1397
    1398    assert _chat_handle_line(job_id, "/usage") is True
    1399
    1400    out = capsys.readouterr().out
    1401    assert "limit: max job cost=$5 remaining=$3.7500" in out
    1402
    1403
    1404def test_first_run_settings_slash_commands_persist_config(monkeypatch, tmp_path):
    1405    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1406
    1407    action, payload = _handle_first_run_frame_line("/model provider/model")
    1408
    1409    assert action == "notice"
    1410    assert isinstance(payload, list)
    1411    assert any("saved model.name = provider/model" in line for line in payload)
    1412    assert _config_field_value("model.name") == "provider/model"
    1413
    1414
    1415def test_first_run_local_connector_action_sets_generic_local_endpoint(monkeypatch, tmp_path):
    1416    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1417
    1418    action, payload = _handle_first_run_action("preset:local")
    1419
    1420    assert action == "notice"
    1421    assert isinstance(payload, list)
    1422    assert any("saved model.name = local-model" in line for line in payload)
    1423    assert any("saved model.base_url = http://localhost:8000/v1" in line for line in payload)
    1424    assert any("then run Doctor" in line for line in payload)
    1425    assert _config_field_value("model.name") == "local-model"
    1426    assert _config_field_value("model.base_url") == "http://localhost:8000/v1"
    1427
    1428
    1429def test_first_run_access_action_toggles_generic_tools(monkeypatch, tmp_path):
    1430    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1431
    1432    action, payload = _handle_first_run_action("toggle:tools.shell")
    1433
    1434    assert action == "notice"
    1435    assert isinstance(payload, list)
    1436    assert any("saved tools.shell = False" in line for line in payload)
    1437    assert _config_field_value("tools.shell") is False
    1438
    1439
    1440def test_first_run_doctor_success_opens_workspace_chat(monkeypatch, tmp_path):
    1441    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1442
    1443    def fake_verify():
    1444        _mark_test_model_ready()
    1445        return ["ok model_setup verified"]
    1446
    1447    monkeypatch.setattr("nipux_cli.cli._verify_model_setup_from_first_run", fake_verify)
    1448
    1449    action, payload = _handle_first_run_action("doctor")
    1450
    1451    assert action == "open"
    1452    assert payload == WORKSPACE_CHAT_ID
    1453
    1454
    1455def test_first_run_open_workspace_action_requires_verified_model(monkeypatch, tmp_path):
    1456    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1457
    1458    action, payload = _handle_first_run_action("open_workspace")
    1459
    1460    assert action == "notice"
    1461    assert "Run Doctor first" in str(payload)
    1462
    1463
    1464def test_first_run_open_workspace_action_opens_after_verified_model(monkeypatch, tmp_path):
    1465    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1466    _mark_test_model_ready()
    1467
    1468    action, payload = _handle_first_run_action("open_workspace")
    1469
    1470    assert action == "open"
    1471    assert payload == WORKSPACE_CHAT_ID
    1472
    1473
    1474def test_workspace_frame_snapshot_exists_without_jobs(monkeypatch, tmp_path):
    1475    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1476    _mark_test_model_ready()
    1477
    1478    snapshot = _load_frame_snapshot(WORKSPACE_CHAT_ID, history_limit=4)
    1479
    1480    assert snapshot["job_id"] == WORKSPACE_CHAT_ID
    1481    assert snapshot["job"]["kind"] == "workspace"
    1482    assert snapshot["jobs"] == []
    1483
    1484    frame = _build_chat_frame(snapshot, "", [], width=118, height=28)
    1485    assert "Type a goal in plain English to start a worker" in frame
    1486    assert "Type a goal to create the first worker" in frame
    1487    assert "Enter sends" not in frame
    1488
    1489
    1490def test_workspace_frame_right_pane_tracks_focused_worker(monkeypatch, tmp_path):
    1491    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1492    _mark_test_model_ready()
    1493    db = AgentDB(tmp_path / "state.db")
    1494    try:
    1495        job_id = db.create_job("Research browser automation libraries", title="browser research")
    1496        db.add_artifact(
    1497            job_id=job_id,
    1498            path=tmp_path / "comparison.md",
    1499            sha256="abc123",
    1500            artifact_type="text",
    1501            title="Browser Automation Comparison Draft",
    1502            summary="Checkpoint: saved comparison draft.",
    1503        )
    1504        _write_shell_state({"focus_job_id": job_id})
    1505    finally:
    1506        db.close()
    1507
    1508    snapshot = _load_frame_snapshot(WORKSPACE_CHAT_ID, history_limit=4)
    1509    frame = _build_chat_frame(snapshot, "", [], width=128, height=28, right_view="updates")
    1510
    1511    assert snapshot["job"]["kind"] == "workspace"
    1512    assert snapshot["right_job"]["title"] == "browser research"
    1513    assert "browser research" in frame
    1514    assert "Browser Automation" in frame
    1515    assert "Comparison Draft" in frame
    1516
    1517
    1518def test_workspace_slash_new_creates_and_focuses_job(monkeypatch, tmp_path):
    1519    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1520    _mark_test_model_ready()
    1521    started = {}
    1522
    1523    def fake_start(**kwargs):
    1524        started.update(kwargs)
    1525
    1526    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    1527
    1528    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "/new Build a durable workflow")
    1529
    1530    assert keep_running is True
    1531    assert "Created worker job: Build a durable workflow" in output
    1532    assert "Started worker" in output
    1533    assert started["poll_seconds"] == 0.0
    1534    assert started["quiet"] is True
    1535    db = AgentDB(tmp_path / "state.db")
    1536    try:
    1537        jobs = db.list_jobs()
    1538        assert len(jobs) == 1
    1539        assert jobs[0]["title"] == "Build a durable workflow"
    1540        assert _read_shell_state().get("focus_job_id") == jobs[0]["id"]
    1541    finally:
    1542        db.close()
    1543
    1544
    1545def test_workspace_chat_job_dossier_includes_progress_outputs_and_outcomes(tmp_path):
    1546    db = AgentDB(tmp_path / "state.db")
    1547    try:
    1548        job_id = db.create_job("Research and validate a workflow", title="workflow research")
    1549        run_id = db.start_run(job_id, model="fake")
    1550        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
    1551        db.finish_step(step_id, status="completed", output_data={"artifact_id": "art_demo"})
    1552        db.add_artifact(
    1553            job_id=job_id,
    1554            path=tmp_path / "workflow.md",
    1555            sha256="abc123",
    1556            artifact_type="text",
    1557            title="Workflow Evidence",
    1558            summary="saved research notes",
    1559        )
    1560        db.append_task_record(job_id, title="Compare source-backed options", status="active", output_contract="research")
    1561        db.append_source_record(job_id, "https://example.com/source", source_type="web")
    1562        db.append_finding_record(job_id, name="Useful research finding", source_url="https://example.com/source")
    1563        db.append_experiment_record(
    1564            job_id,
    1565            title="Validation check",
    1566            status="measured",
    1567            metric_name="pass_rate",
    1568            metric_value=0.9,
    1569        )
    1570        job = db.get_job(job_id)
    1571        dossier = _workspace_chat_job_dossier(db, [job])
    1572    finally:
    1573        db.close()
    1574
    1575    assert "workflow research" in dossier
    1576    assert "outputs=1" in dossier
    1577    assert "findings=1" in dossier
    1578    assert "sources=1" in dossier
    1579    assert "experiments=1" in dossier
    1580    assert "active task: active Compare source-backed options [research]" in dossier
    1581    assert "latest outputs: Workflow Evidence" in dossier
    1582    assert "recent outcomes:" in dossier
    1583
    1584
    1585def test_workspace_run_with_objective_creates_worker_when_no_job_matches(monkeypatch, tmp_path):
    1586    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1587    _mark_test_model_ready()
    1588    started = {}
    1589
    1590    def fake_start(**kwargs):
    1591        started.update(kwargs)
    1592
    1593    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    1594    monkeypatch.setattr(
    1595        "nipux_cli.cli._refine_job_objective_for_worker",
    1596        lambda *, message, objective: objective,
    1597    )
    1598
    1599    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "/run research browser automation libraries")
    1600
    1601    assert keep_running is True
    1602    assert "Created worker job" in output
    1603    assert started["quiet"] is True
    1604    db = AgentDB(tmp_path / "state.db")
    1605    try:
    1606        jobs = db.list_jobs()
    1607        assert len(jobs) == 1
    1608        assert jobs[0]["title"] == "research browser automation libraries"
    1609    finally:
    1610        db.close()
    1611
    1612
    1613def test_workspace_run_with_existing_job_does_not_create_duplicate(monkeypatch, tmp_path):
    1614    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1615    _mark_test_model_ready()
    1616    db = AgentDB(tmp_path / "state.db")
    1617    try:
    1618        job_id = db.create_job("research browser automation libraries", title="research browser automation libraries")
    1619    finally:
    1620        db.close()
    1621    started = {}
    1622
    1623    def fake_start(**kwargs):
    1624        started.update(kwargs)
    1625
    1626    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    1627
    1628    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "/run research browser automation libraries")
    1629
    1630    assert keep_running is True
    1631    assert "Created worker job" not in output
    1632    assert "focus set" in output
    1633    assert started["quiet"] is True
    1634    db = AgentDB(tmp_path / "state.db")
    1635    try:
    1636        assert len(db.list_jobs()) == 1
    1637        assert _read_shell_state().get("focus_job_id") == job_id
    1638    finally:
    1639        db.close()
    1640
    1641
    1642def test_workspace_start_with_existing_job_runs_without_parser_error(monkeypatch, tmp_path):
    1643    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1644    _mark_test_model_ready()
    1645    db = AgentDB(tmp_path / "state.db")
    1646    try:
    1647        job_id = db.create_job("research browser automation libraries", title="research browser automation libraries")
    1648    finally:
    1649        db.close()
    1650    started = {}
    1651
    1652    def fake_start(**kwargs):
    1653        started.update(kwargs)
    1654
    1655    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    1656
    1657    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "/start research browser automation libraries")
    1658
    1659    assert keep_running is True
    1660    assert "command exited" not in output
    1661    assert "Created worker job" not in output
    1662    assert "focus set" in output
    1663    assert started["quiet"] is True
    1664    db = AgentDB(tmp_path / "state.db")
    1665    try:
    1666        assert len(db.list_jobs()) == 1
    1667        assert _read_shell_state().get("focus_job_id") == job_id
    1668    finally:
    1669        db.close()
    1670
    1671
    1672def test_workspace_slash_new_without_objective_is_minimal(monkeypatch, tmp_path):
    1673    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1674    _mark_test_model_ready()
    1675
    1676    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "/new")
    1677
    1678    assert keep_running is True
    1679    assert output.strip() == "usage: /new OBJECTIVE"
    1680    assert "for example" not in output.lower()
    1681
    1682
    1683def test_workspace_slash_new_hides_model_preflight_noise(monkeypatch, tmp_path):
    1684    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1685    _mark_test_model_ready()
    1686
    1687    def fake_start(**_kwargs):
    1688        print("model is not ready; daemon not started")
    1689        print("  fail model_endpoint: http://localhost:8000/v1/models: connection refused")
    1690        print("Run `nipux doctor --check-model` after fixing the model configuration.")
    1691
    1692    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    1693
    1694    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "/new Build a durable workflow")
    1695
    1696    assert keep_running is True
    1697    assert "Created worker job: Build a durable workflow" in output
    1698    assert "Worker is waiting for a working model" in output
    1699    assert "model_endpoint" not in output
    1700    assert "connection refused" not in output
    1701
    1702
    1703def test_workspace_settings_slash_commands_persist_config(monkeypatch, tmp_path):
    1704    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1705    _mark_test_model_ready()
    1706
    1707    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "/model provider/model")
    1708
    1709    assert keep_running is True
    1710    assert "saved model.name = provider/model" in output
    1711    assert _config_field_value("model.name") == "provider/model"
    1712
    1713    db = AgentDB(tmp_path / "state.db")
    1714    try:
    1715        assert db.list_jobs() == []
    1716    finally:
    1717        db.close()
    1718
    1719
    1720def test_workspace_settings_slash_command_summarizes_config(monkeypatch, tmp_path):
    1721    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1722    monkeypatch.setenv("NIPUX_TEST_KEY", "sk-test-value")
    1723    _mark_test_model_ready()
    1724    (tmp_path / "config.yaml").write_text(
    1725        """
    1726model:
    1727  name: provider/model
    1728  base_url: https://example.com/v1
    1729  api_key_env: NIPUX_TEST_KEY
    1730  context_length: 8192
    1731  request_timeout_seconds: 45
    1732""",
    1733        encoding="utf-8",
    1734    )
    1735
    1736    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "/settings")
    1737
    1738    assert keep_running is True
    1739    assert "config" in output
    1740    assert "model: provider/model" in output
    1741    assert "endpoint: https://example.com/v1" in output
    1742    assert "key: set (NIPUX_TEST_KEY)" in output
    1743    assert "sk-test-value" not in output
    1744
    1745
    1746def test_workspace_natural_control_phrase_uses_mapped_command(monkeypatch, tmp_path):
    1747    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1748    _mark_test_model_ready()
    1749
    1750    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "change model")
    1751
    1752    assert keep_running is True
    1753    assert "model.name =" in output
    1754    assert "usage: /model MODEL" in output
    1755
    1756    db = AgentDB(tmp_path / "state.db")
    1757    try:
    1758        assert db.list_jobs() == []
    1759    finally:
    1760        db.close()
    1761
    1762
    1763def test_workspace_natural_settings_phrase_opens_settings_summary(monkeypatch, tmp_path):
    1764    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1765    monkeypatch.setenv("NIPUX_TEST_KEY", "sk-test-value")
    1766    _mark_test_model_ready()
    1767    (tmp_path / "config.yaml").write_text(
    1768        """
    1769model:
    1770  name: provider/model
    1771  base_url: https://example.com/v1
    1772  api_key_env: NIPUX_TEST_KEY
    1773""",
    1774        encoding="utf-8",
    1775    )
    1776
    1777    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "settings")
    1778
    1779    assert keep_running is True
    1780    assert "config" in output
    1781    assert "model: provider/model" in output
    1782    assert "key: set (NIPUX_TEST_KEY)" in output
    1783    assert "usage: /model MODEL" not in output
    1784    assert "sk-test-value" not in output
    1785
    1786
    1787def test_workspace_how_to_start_job_question_uses_local_help(monkeypatch, tmp_path):
    1788    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1789    _mark_test_model_ready()
    1790
    1791    def fail_model(_line):
    1792        raise AssertionError("model should not be called for local help")
    1793
    1794    monkeypatch.setattr("nipux_cli.cli._reply_to_workspace_chat", fail_model)
    1795
    1796    keep_running, output = _capture_chat_command(WORKSPACE_CHAT_ID, "how do I start a job?")
    1797
    1798    assert keep_running is True
    1799    assert "Create: type a goal" in output
    1800    assert "/new OBJECTIVE" in output
    1801
    1802
    1803def test_workspace_chat_connection_error_is_operator_friendly(monkeypatch, tmp_path):
    1804    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1805    _mark_test_model_ready()
    1806
    1807    def raise_connection(_line):
    1808        raise RuntimeError("APIConnectionError: Connection error.")
    1809
    1810    monkeypatch.setattr("nipux_cli.cli._reply_to_workspace_chat", raise_connection)
    1811
    1812    ok, message = _handle_workspace_chat_message("hello", quiet=True)
    1813
    1814    assert ok is True
    1815    assert message == (
    1816        "Model endpoint is unreachable. Check /base-url or start the configured model server, then run /doctor."
    1817    )
    1818    snapshot = _load_frame_snapshot(WORKSPACE_CHAT_ID, history_limit=4)
    1819    bodies = "\n".join(str(event.get("body") or "") for event in snapshot["events"])
    1820    assert "APIConnectionError" not in bodies
    1821    assert "Model endpoint is unreachable" in bodies
    1822
    1823
    1824def test_chat_start_reports_model_provider_not_ready(monkeypatch, tmp_path, capsys):
    1825    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1826    _mark_test_model_ready()
    1827    db = AgentDB(tmp_path / "state.db")
    1828    try:
    1829        job_id = db.create_job("Research topic", title="research")
    1830    finally:
    1831        db.close()
    1832
    1833    def fake_start(**_kwargs):
    1834        print("model is not ready; daemon not started")
    1835        print("  fail model_endpoint: http://localhost:8000/v1/models: connection refused")
    1836
    1837    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    1838
    1839    assert _chat_handle_line(job_id, "/start") is True
    1840
    1841    out = capsys.readouterr().out
    1842    assert "worker not started: model provider is not ready. Use /settings, then /doctor." in out
    1843    assert "model_endpoint" not in out
    1844    assert "connection refused" not in out
    1845
    1846
    1847def test_chat_doctor_checks_configured_model(monkeypatch, tmp_path, capsys):
    1848    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1849    _mark_test_model_ready()
    1850    db = AgentDB(tmp_path / "state.db")
    1851    try:
    1852        job_id = db.create_job("Research topic", title="research")
    1853    finally:
    1854        db.close()
    1855
    1856    seen = {}
    1857
    1858    def fake_doctor(*, config, check_model):
    1859        seen["check_model"] = check_model
    1860        return [Check("model_generation", True, "ok")]
    1861
    1862    monkeypatch.setattr("nipux_cli.cli.run_doctor", fake_doctor)
    1863
    1864    assert _chat_handle_line(job_id, "/doctor") is True
    1865
    1866    assert seen["check_model"] is True
    1867    assert "model_generation" in capsys.readouterr().out
    1868
    1869
    1870def test_workspace_chat_control_phrase_runs_job_command(monkeypatch, tmp_path):
    1871    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1872    _mark_test_model_ready()
    1873    db = AgentDB(tmp_path / "state.db")
    1874    try:
    1875        job_id = db.create_job("Research topic", title="research")
    1876    finally:
    1877        db.close()
    1878
    1879    keep_running, message = _handle_workspace_chat_message("stop the job", quiet=True)
    1880
    1881    assert keep_running is True
    1882    assert "paused research" in message
    1883    db = AgentDB(tmp_path / "state.db")
    1884    try:
    1885        job = db.get_job(job_id)
    1886        assert job["status"] == "paused"
    1887        events = _read_shell_state().get("workspace_chat_events") or []
    1888        assert any(
    1889            event["event_type"] == "agent_message"
    1890            and event["metadata"].get("command") == "/pause"
    1891            and "paused research" in event["body"]
    1892            for event in events
    1893        )
    1894    finally:
    1895        db.close()
    1896
    1897
    1898def test_shell_ls_alias_lists_jobs_instead_of_steering(monkeypatch, tmp_path, capsys):
    1899    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1900    db = AgentDB(tmp_path / "state.db")
    1901    try:
    1902        db.create_job("Research topic", title="research")
    1903    finally:
    1904        db.close()
    1905
    1906    assert _run_shell_line("ls") is True
    1907
    1908    out = capsys.readouterr().out
    1909    assert "research" in out
    1910    assert "queued for" not in out
    1911
    1912
    1913def test_roadmap_command_renders_roadmap(monkeypatch, tmp_path, capsys):
    1914    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1915    db = AgentDB(tmp_path / "state.db")
    1916    try:
    1917        job_id = db.create_job("Broad work", title="broad")
    1918        db.append_roadmap_record(
    1919            job_id,
    1920            title="Broad Roadmap",
    1921            status="active",
    1922            current_milestone="Foundation",
    1923            milestones=[{
    1924                "title": "Foundation",
    1925                "status": "validating",
    1926                "validation_status": "pending",
    1927                "features": [{"title": "First feature", "status": "done"}],
    1928            }],
    1929        )
    1930    finally:
    1931        db.close()
    1932
    1933    main(["roadmap", "broad"])
    1934
    1935    out = capsys.readouterr().out
    1936    assert "roadmap broad" in out
    1937    assert "Broad Roadmap" in out
    1938    assert "Foundation" in out
    1939    assert "validation=pending" in out
    1940
    1941
    1942def test_shell_focus_controls_default_steering_job(monkeypatch, tmp_path, capsys):
    1943    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1944    db = AgentDB(tmp_path / "state.db")
    1945    try:
    1946        first = db.create_job("Research topic", title="first research")
    1947        second = db.create_job("Find investors", title="investor search")
    1948    finally:
    1949        db.close()
    1950
    1951    assert _run_shell_line("focus investor") is True
    1952    assert _run_shell_line("prioritize Toronto findings") is True
    1953
    1954    out = capsys.readouterr().out
    1955    db = AgentDB(tmp_path / "state.db")
    1956    try:
    1957        first_job = db.get_job(first)
    1958        second_job = db.get_job(second)
    1959        assert "focus set:" in out
    1960        assert first_job["metadata"].get("operator_messages") is None
    1961        assert second_job["metadata"]["operator_messages"][-1]["message"] == "prioritize Toronto findings"
    1962    finally:
    1963        db.close()
    1964
    1965
    1966def test_shell_rename_updates_job_title_and_program(monkeypatch, tmp_path, capsys):
    1967    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1968    db = AgentDB(tmp_path / "state.db")
    1969    try:
    1970        job_id = db.create_job("Research topic", title="old title")
    1971        program = tmp_path / "jobs" / job_id / "program.md"
    1972        program.parent.mkdir(parents=True, exist_ok=True)
    1973        program.write_text("# old title\n\nBody\n", encoding="utf-8")
    1974    finally:
    1975        db.close()
    1976
    1977    assert _run_shell_line("rename old title --title new title") is True
    1978
    1979    out = capsys.readouterr().out
    1980    db = AgentDB(tmp_path / "state.db")
    1981    try:
    1982        job = db.get_job(job_id)
    1983        assert "renamed old title -> new title" in out
    1984        assert job["title"] == "new title"
    1985        assert program.read_text(encoding="utf-8").startswith("# new title\n")
    1986    finally:
    1987        db.close()
    1988
    1989
    1990def test_shell_delete_removes_job_and_artifact_dir(monkeypatch, tmp_path, capsys):
    1991    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    1992    db = AgentDB(tmp_path / "state.db")
    1993    try:
    1994        job_id = db.create_job("Research topic", title="delete me")
    1995        run_id = db.start_run(job_id, model="fake")
    1996        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
    1997        store = ArtifactStore(tmp_path, db=db)
    1998        stored = store.write_text(
    1999            job_id=job_id,
    2000            run_id=run_id,
    2001            step_id=step_id,
    2002            title="Artifact",
    2003            summary="saved",
    2004            content="content",
    2005        )
    2006        artifact_path = stored.path
    2007    finally:
    2008        db.close()
    2009
    2010    assert artifact_path.exists()
    2011    assert _run_shell_line("delete delete me") is True
    2012
    2013    out = capsys.readouterr().out
    2014    db = AgentDB(tmp_path / "state.db")
    2015    try:
    2016        assert "deleted delete me" in out
    2017        try:
    2018            db.get_job(job_id)
    2019        except KeyError:
    2020            pass
    2021        else:
    2022            raise AssertionError("job still exists after shell delete")
    2023        assert not artifact_path.exists()
    2024        assert not (tmp_path / "jobs" / job_id).exists()
    2025    finally:
    2026        db.close()
    2027
    2028
    2029def test_shell_help_has_no_examples_or_control_run_sections(capsys):
    2030    _print_shell_help()
    2031
    2032    out = capsys.readouterr().out
    2033    assert "Examples:" not in out
    2034    assert "\nControl\n" not in out
    2035    assert "\nRun\n" not in out
    2036    assert "delete JOB_TITLE" in out
    2037    assert "usage [JOB_TITLE]" in out
    2038    assert "update" in out
    2039    assert "Jobs" in out
    2040    assert "Worker" in out
    2041
    2042
    2043def test_update_checkout_falls_back_to_tool_install_for_non_git_path(monkeypatch, tmp_path):
    2044    monkeypatch.setattr("shutil.which", lambda name: "/usr/bin/uv" if name == "uv" else None)
    2045
    2046    def runner(command):
    2047        assert command == [
    2048            "/usr/bin/uv",
    2049            "tool",
    2050            "install",
    2051            "--force",
    2052            "--upgrade",
    2053            "--reinstall",
    2054            "--refresh",
    2055            "git+https://github.com/nipuxx/agent-cli.git@main",
    2056        ]
    2057        return subprocess.CompletedProcess(command, 0, stdout="Installed nipux\n")
    2058
    2059    code, lines = _update_checkout(path=tmp_path, command_runner=runner)
    2060
    2061    assert code == 0
    2062    rendered = "\n".join(lines)
    2063    assert "is not a source checkout; updating the installed Nipux tool instead" in rendered
    2064    assert "not a git checkout" not in rendered
    2065    assert "Update complete" in rendered
    2066
    2067
    2068def test_update_checkout_upgrades_uv_tool_when_installed_package(monkeypatch):
    2069    monkeypatch.setattr("nipux_cli.updater.find_checkout_root", lambda: None)
    2070    monkeypatch.setattr("shutil.which", lambda name: "/usr/bin/uv" if name == "uv" else None)
    2071    calls: list[tuple[str, ...]] = []
    2072
    2073    def runner(command):
    2074        calls.append(tuple(command))
    2075        return subprocess.CompletedProcess(command, 0, stdout="Resolved 1 package\nInstalled nipux\n")
    2076
    2077    code, lines = _update_checkout(command_runner=runner)
    2078
    2079    assert code == 0
    2080    assert calls == [
    2081        (
    2082            "/usr/bin/uv",
    2083            "tool",
    2084            "install",
    2085            "--force",
    2086            "--upgrade",
    2087            "--reinstall",
    2088            "--refresh",
    2089            "git+https://github.com/nipuxx/agent-cli.git@main",
    2090        )
    2091    ]
    2092    rendered = "\n".join(lines)
    2093    assert "not a git checkout" not in rendered
    2094    assert "Updating installed Nipux command" in rendered
    2095    assert "Nipux command refreshed from source" in rendered
    2096    assert "Update complete" in rendered
    2097
    2098
    2099def test_update_checkout_fast_forwards_git_checkout(tmp_path):
    2100    repo = tmp_path / "repo"
    2101    repo.mkdir()
    2102    (repo / ".git").mkdir()
    2103    calls: list[tuple[str, ...]] = []
    2104    rev_calls = 0
    2105
    2106    def runner(command, cwd):
    2107        nonlocal rev_calls
    2108        assert cwd == repo
    2109        calls.append(tuple(command))
    2110        if command == ["git", "rev-parse", "--show-toplevel"]:
    2111            return subprocess.CompletedProcess(command, 0, stdout=str(repo) + "\n")
    2112        if command == ["git", "rev-parse", "--short", "HEAD"]:
    2113            rev_calls += 1
    2114            return subprocess.CompletedProcess(command, 0, stdout=("aaa111\n" if rev_calls == 1 else "bbb222\n"))
    2115        if command == ["git", "branch", "--show-current"]:
    2116            return subprocess.CompletedProcess(command, 0, stdout="main\n")
    2117        if command == ["git", "status", "--porcelain"]:
    2118            return subprocess.CompletedProcess(command, 0, stdout="")
    2119        if command == ["git", "pull", "--ff-only"]:
    2120            return subprocess.CompletedProcess(command, 0, stdout="Fast-forward\n")
    2121        raise AssertionError(f"unexpected command: {command}")
    2122
    2123    code, lines = _update_checkout(path=repo, runner=runner)
    2124
    2125    assert code == 0
    2126    assert ("git", "pull", "--ff-only") in calls
    2127    rendered = "\n".join(lines)
    2128    assert "Fast-forward" in rendered
    2129    assert "aaa111 -> bbb222" in rendered
    2130
    2131
    2132def test_update_checkout_verifies_installed_command(monkeypatch):
    2133    monkeypatch.setattr("nipux_cli.updater.find_checkout_root", lambda: None)
    2134
    2135    def which(name):
    2136        if name == "uv":
    2137            return "/usr/bin/uv"
    2138        if name == "nipux":
    2139            return "/Users/me/.local/bin/nipux"
    2140        return None
    2141
    2142    monkeypatch.setattr("shutil.which", which)
    2143    calls: list[tuple[str, ...]] = []
    2144
    2145    def runner(command):
    2146        calls.append(tuple(command))
    2147        if command == ["/Users/me/.local/bin/nipux", "--version"]:
    2148            return subprocess.CompletedProcess(command, 0, stdout="nipux 0.1.0\n")
    2149        return subprocess.CompletedProcess(command, 0, stdout="Installed nipux\n")
    2150
    2151    code, lines = _update_checkout(command_runner=runner)
    2152
    2153    assert code == 0
    2154    assert calls[-1] == ("/Users/me/.local/bin/nipux", "--version")
    2155    assert "Verified: nipux 0.1.0" in "\n".join(lines)
    2156
    2157
    2158def test_update_command_reports_no_restart_when_daemon_is_stopped(monkeypatch, tmp_path, capsys):
    2159    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    2160    monkeypatch.setattr("nipux_cli.cli.update_checkout", lambda **_kwargs: (0, ["Update complete."]))
    2161    monkeypatch.setattr(
    2162        "nipux_cli.cli.daemon_lock_status",
    2163        lambda _path: {"running": False, "metadata": {}},
    2164    )
    2165
    2166    args = build_parser().parse_args(["update"])
    2167    args.func(args)
    2168
    2169    out = capsys.readouterr().out
    2170    assert "Update complete." in out
    2171    assert "No daemon is running; no restart needed." in out
    2172
    2173
    2174def test_update_command_restarts_running_daemon(monkeypatch, tmp_path, capsys):
    2175    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    2176    monkeypatch.setattr("nipux_cli.cli.update_checkout", lambda **_kwargs: (0, ["Update complete."]))
    2177    monkeypatch.setattr(
    2178        "nipux_cli.cli.daemon_lock_status",
    2179        lambda _path: {"running": True, "metadata": {"pid": 123}},
    2180    )
    2181    restarted = {}
    2182
    2183    def fake_restart(args):
    2184        restarted["wait"] = args.wait
    2185        restarted["quiet"] = args.quiet
    2186        print("restart ok")
    2187
    2188    monkeypatch.setattr("nipux_cli.cli.cmd_restart", fake_restart)
    2189
    2190    args = build_parser().parse_args(["update"])
    2191    args.func(args)
    2192
    2193    out = capsys.readouterr().out
    2194    assert "Restarting running daemon" in out
    2195    assert "restart ok" in out
    2196    assert restarted == {"wait": 5.0, "quiet": True}
    2197
    2198
    2199def test_update_command_no_restart_flag_skips_running_daemon(monkeypatch, tmp_path, capsys):
    2200    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    2201    monkeypatch.setattr("nipux_cli.cli.update_checkout", lambda **_kwargs: (0, ["Update complete."]))
    2202    monkeypatch.setattr(
    2203        "nipux_cli.cli.daemon_lock_status",
    2204        lambda _path: {"running": True, "metadata": {"pid": 123}},
    2205    )
    2206    monkeypatch.setattr("nipux_cli.cli.cmd_restart", lambda _args: (_ for _ in ()).throw(AssertionError("restart")))
    2207
    2208    args = build_parser().parse_args(["update", "--no-restart"])
    2209    args.func(args)
    2210
    2211    assert "Daemon restart skipped by --no-restart." in capsys.readouterr().out
    2212
    2213
    2214def test_uninstall_dry_run_removes_installed_tool_by_default(monkeypatch, tmp_path, capsys):
    2215    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    2216
    2217    args = build_parser().parse_args(["uninstall", "--dry-run"])
    2218    args.func(args)
    2219
    2220    out = capsys.readouterr().out
    2221    assert "would remove" in out
    2222    assert "would run uv tool uninstall nipux" in out
    2223    assert "runtime removed" not in out
    2224
    2225
    2226def test_uninstall_keep_tool_skips_tool_removal(monkeypatch, tmp_path, capsys):
    2227    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    2228
    2229    args = build_parser().parse_args(["uninstall", "--dry-run", "--keep-tool"])
    2230    args.func(args)
    2231
    2232    out = capsys.readouterr().out
    2233    assert "would remove" in out
    2234    assert "uv tool uninstall nipux" not in out
    2235
    2236
    2237def test_uninstall_runtime_skips_missing_systemd_service_without_runner_noise(monkeypatch, tmp_path):
    2238    from nipux_cli.uninstall import uninstall_runtime
    2239
    2240    monkeypatch.setenv("HOME", str(tmp_path))
    2241    runtime = tmp_path / ".nipux"
    2242    runtime.mkdir()
    2243    calls = []
    2244
    2245    def which(name):
    2246        if name == "systemctl":
    2247            return "/bin/systemctl"
    2248        return None
    2249
    2250    def runner(command, **_kwargs):
    2251        calls.append(command)
    2252        return subprocess.CompletedProcess(command, 0, stdout="", stderr="")
    2253
    2254    monkeypatch.setattr("nipux_cli.uninstall.shutil.which", which)
    2255
    2256    lines = uninstall_runtime(runtime_home=runtime, dry_run=False, runner=runner)
    2257
    2258    assert calls == []
    2259    rendered = "\n".join(lines)
    2260    assert "disabled systemd" not in rendered
    2261    assert "no installed service files found" in rendered
    2262    assert f"removed {runtime}" in rendered
    2263
    2264
    2265def test_chat_clear_does_not_queue_operator_message(monkeypatch, tmp_path, capsys):
    2266    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    2267    db = AgentDB(tmp_path / "state.db")
    2268    try:
    2269        job_id = db.create_job("Research topic", title="research")
    2270    finally:
    2271        db.close()
    2272
    2273    assert _chat_handle_line(job_id, "clear") is True
    2274
    2275    out = capsys.readouterr().out
    2276    db = AgentDB(tmp_path / "state.db")
    2277    try:
    2278        job = db.get_job(job_id)
    2279        assert "\033[2J\033[H" in out
    2280        assert job["metadata"].get("operator_messages") is None
    2281    finally:
    2282        db.close()
    2283
    2284
    2285def test_minimal_live_event_line_summarizes_tool_steps():
    2286    line = _minimal_live_event_line(
    2287        {
    2288            "event_type": "tool_call",
    2289            "title": "shell_exec",
    2290            "body": "",
    2291            "metadata": {"input": {"arguments": {"command": "ssh server nvidia-smi"}}},
    2292        }
    2293    )
    2294
    2295    assert line == "start shell ssh server nvidia-smi"
    2296
    2297
    2298def test_chat_frame_is_bounded_and_has_composer():
    2299    snapshot = {
    2300        "job_id": "job_demo",
    2301        "job": {
    2302            "id": "job_demo",
    2303            "title": "demo job",
    2304            "objective": "keep a generic long-running job visible",
    2305            "status": "running",
    2306            "kind": "generic",
    2307            "metadata": {"task_queue": [{"status": "active", "title": "Draft next deliverable", "priority": 7}]},
    2308        },
    2309        "jobs": [{"id": "job_demo", "title": "demo job", "status": "running", "kind": "generic", "metadata": {}}],
    2310        "steps": [
    2311            {
    2312                "step_no": 3,
    2313                "status": "completed",
    2314                "kind": "tool",
    2315                "tool_name": "web_search",
    2316                "summary": "web_search returned sources",
    2317            }
    2318        ],
    2319        "artifacts": [{"id": "art_demo"}],
    2320        "memory_entries": [{}],
    2321        "events": [
    2322            {
    2323                "event_type": "agent_message",
    2324                "title": "plan",
    2325                "body": "I will plan this.\nPlan:\n- one\n- two\nQuestions:\n- answer?",
    2326                "metadata": {},
    2327            },
    2328            {
    2329                "event_type": "task",
    2330                "title": "internal task",
    2331                "body": "internal task body",
    2332                "metadata": {},
    2333            },
    2334            {
    2335                "event_type": "tool_result",
    2336                "title": "web_search",
    2337                "body": "web_search query='demo' returned 1 results",
    2338                "metadata": {"status": "completed", "input": {"arguments": {"query": "demo"}}},
    2339            }
    2340        ],
    2341        "daemon": {"running": True, "metadata": {"pid": 123}},
    2342        "model": "model/demo",
    2343        "base_url": "https://openrouter.ai/api/v1",
    2344        "context_length": 8192,
    2345        "token_usage": {
    2346            "calls": 2,
    2347            "latest_prompt_tokens": 4096,
    2348            "completion_tokens": 1234,
    2349            "total_tokens": 5330,
    2350            "cost": 0.0123,
    2351            "has_cost": True,
    2352        },
    2353    }
    2354
    2355    frame = _build_chat_frame(snapshot, "hello", [], width=100, height=22)
    2356    wide_frame = _build_chat_frame(snapshot, "", [], width=140, height=22)
    2357
    2358    assert len(frame.splitlines()) <= 22
    2359    assert "NIPUX" in frame
    2360    assert "CHAT" in frame
    2361    assert "MODEL UPDATES" in frame
    2362    assert "NAV" not in frame
    2363    assert "Visible" in frame
    2364    assert "#3" not in frame
    2365    assert "Jobs" in frame
    2366    assert "Recent outcomes" not in frame
    2367    assert "ctx" in frame
    2368    assert "4.1K/8.2K" in frame
    2369    assert "out" in frame
    2370    assert "1.2K" in frame
    2371    assert "tok" in frame
    2372    assert "5.3K" in frame
    2373    assert "$0.0123" in frame
    2374    assert "NIPUX" in wide_frame
    2375    assert "model model/demo  ctx 4.1K/8.2K" in wide_frame
    2376    assert "daemon running" not in wide_frame
    2377    assert wide_frame.splitlines()[1].startswith("━")
    2378    assert "Enter sends" in frame
    2379    assert "❯ hello" in frame
    2380    jobs = _build_chat_frame(snapshot, "", [], width=100, height=26, right_view="status")
    2381    assert "Draft next deli" in jobs
    2382
    2383    legacy_work = _build_chat_frame(snapshot, "", [], width=100, height=24, right_view="work")
    2384    assert "MODEL UPDATES" in legacy_work
    2385    assert "Tool / console" not in legacy_work
    2386
    2387    updates = _build_chat_frame(snapshot, "", [], width=100, height=24, right_view="updates")
    2388    assert "MODEL UPDATES" in updates
    2389    assert "Visible" in updates
    2390
    2391    settings = _build_chat_frame(snapshot, "", [], width=120, height=30, modal_view="settings")
    2392    assert "Settings" in settings
    2393    assert "/model MODEL" in settings
    2394    assert "/api-key KEY" in settings
    2395    assert "/base-url URL" in settings
    2396    assert "Esc closes" in settings
    2397    assert "NAV" not in settings
    2398
    2399    secret = _build_chat_frame(
    2400        snapshot,
    2401        "secret-value",
    2402        [],
    2403        width=100,
    2404        height=24,
    2405        editing_field="secret:model.api_key",
    2406    )
    2407    assert "Editing API key" in secret
    2408    assert "secret-value" not in secret
    2409    assert "••••" in secret
    2410
    2411
    2412def test_chat_frame_separates_chat_from_worker_activity():
    2413    snapshot = {
    2414        "job_id": "job_demo",
    2415        "job": {
    2416            "id": "job_demo",
    2417            "title": "demo job",
    2418            "objective": "keep chat separate",
    2419            "status": "running",
    2420            "kind": "generic",
    2421            "metadata": {},
    2422        },
    2423        "jobs": [{"id": "job_demo", "title": "demo job", "status": "running", "kind": "generic", "metadata": {}}],
    2424        "steps": [],
    2425        "artifacts": [],
    2426        "memory_entries": [],
    2427        "events": [
    2428            {"event_type": "operator_message", "body": "start a benchmark job", "metadata": {}},
    2429            {"event_type": "agent_message", "title": "chat", "body": "I created the job and started it.", "metadata": {}},
    2430            {"event_type": "tool_call", "title": "shell_exec", "body": "", "metadata": {"input": {"arguments": {"command": "python bench.py"}}}},
    2431            {"event_type": "tool_result", "title": "shell_exec", "body": "shell_exec rc=0", "metadata": {"status": "completed", "input": {"arguments": {"command": "python bench.py"}}}},
    2432        ],
    2433        "daemon": {"running": True, "metadata": {"pid": 123}},
    2434        "model": "model/demo",
    2435    }
    2436
    2437    frame = _build_chat_frame(snapshot, "", [], width=130, height=24, right_view="updates")
    2438    chat_side = frame.split(" │ ", 1)[0]
    2439
    2440    assert "start a benchmark job" in frame
    2441    assert "I created the job" in frame
    2442    assert "Tool / console" not in frame
    2443    assert "python bench.py" not in frame
    2444    assert "python bench.py" not in chat_side
    2445
    2446
    2447def test_chat_frame_empty_state_is_minimal_and_actionable():
    2448    snapshot = {
    2449        "job_id": "job_demo",
    2450        "job": {
    2451            "id": "job_demo",
    2452            "title": "demo job",
    2453            "objective": "keep chat visible",
    2454            "status": "running",
    2455            "kind": "generic",
    2456            "metadata": {},
    2457        },
    2458        "jobs": [{"id": "job_demo", "title": "demo job", "status": "running", "kind": "generic", "metadata": {}}],
    2459        "steps": [],
    2460        "artifacts": [],
    2461        "memory_entries": [],
    2462        "events": [],
    2463        "daemon": {"running": True, "metadata": {"pid": 123}},
    2464        "model": "model/demo",
    2465    }
    2466
    2467    frame = _build_chat_frame(snapshot, "", [], width=120, height=28)
    2468
    2469    assert "NIPUX" in frame
    2470    assert "plain English" in frame
    2471    assert "/new OBJECTIVE" in frame
    2472    assert "/settings" in frame
    2473    assert "███" not in frame
    2474    assert "No chat yet." not in frame
    2475    assert "star..." not in frame
    2476
    2477
    2478def test_frame_emit_skips_unchanged_render(capsys):
    2479    first = _emit_frame_if_changed("line one\nline two")
    2480    second = _emit_frame_if_changed("frame", first)
    2481    third = _emit_frame_if_changed("frame\nline three", second)
    2482
    2483    out = capsys.readouterr().out
    2484    assert first == "line one\nline two"
    2485    assert second == "frame"
    2486    assert third == "frame\nline three"
    2487    assert out.count("\033[H") == 1
    2488    assert "\033[1;1H\033[2Kframe" in out
    2489    assert "\033[2K" in out
    2490    assert "\033[J" not in out
    2491
    2492
    2493def test_chat_frame_does_not_cap_long_agent_messages():
    2494    long_reply = (
    2495        "**Completed Work:** "
    2496        "1. Test suite analysis finished. "
    2497        "2. Code analysis findings documented. "
    2498        "3. Market readiness gaps identified. "
    2499        "4. Packaging risks summarized. "
    2500        "5. Daemon reliability checked. "
    2501        "6. UI ergonomics reviewed. "
    2502        "7. Final recommendation included."
    2503    )
    2504    snapshot = {
    2505        "job_id": "job_demo",
    2506        "job": {
    2507            "id": "job_demo",
    2508            "title": "demo job",
    2509            "objective": "keep chat readable",
    2510            "status": "running",
    2511            "kind": "generic",
    2512            "metadata": {},
    2513        },
    2514        "jobs": [{"id": "job_demo", "title": "demo job", "status": "running", "kind": "generic", "metadata": {}}],
    2515        "steps": [],
    2516        "artifacts": [],
    2517        "memory_entries": [],
    2518        "events": [
    2519            {"event_type": "operator_message", "body": "what have you done so far", "metadata": {}},
    2520            {"event_type": "agent_message", "title": "chat", "body": long_reply, "metadata": {}},
    2521        ],
    2522        "daemon": {"running": True, "metadata": {"pid": 123}},
    2523        "model": "model/demo",
    2524    }
    2525
    2526    frame = _build_chat_frame(snapshot, "", [], width=118, height=32)
    2527
    2528    assert "Completed Work:" in frame
    2529    assert "Final recommendation included" in frame
    2530    assert "…" not in frame
    2531
    2532
    2533def test_plain_chat_control_intents_map_to_commands():
    2534    assert _chat_control_command("how is it going?") == "/status"
    2535    assert _chat_control_command("what is blocking it?") == "/status"
    2536    assert _chat_control_command("check status") == "/status"
    2537    assert _chat_control_command("what's happening?") == "/status"
    2538    assert _chat_control_command("start working") == "/run"
    2539    assert _chat_control_command("run it") == "/run"
    2540    assert _chat_control_command("start worker") == "/run"
    2541    assert _chat_control_command("pause this job") == "/pause"
    2542    assert _chat_control_command("pause the job") == "/pause"
    2543    assert _chat_control_command("pause it") == "/pause"
    2544    assert _chat_control_command("stop the job") == "/pause"
    2545    assert _chat_control_command("stop worker") == "/pause"
    2546    assert _chat_control_command("resume the job") == "/resume"
    2547    assert _chat_control_command("resume it") == "/resume"
    2548    assert _chat_control_command("show jobs") == "/jobs"
    2549    assert _chat_control_command("change model") == "/model"
    2550    assert _chat_control_command("settings") == "/settings"
    2551    assert _chat_control_command("show settings") == "/settings"
    2552    assert _chat_control_command("how do I start a job?") == "/help"
    2553    assert _chat_control_command("how much did it cost") == "/usage"
    2554    assert _chat_control_command("what has it done") == "/outcomes"
    2555    assert _chat_control_command("what have you done so far") == "/outcomes"
    2556    assert _chat_control_command("what did the model do") == "/outcomes"
    2557    assert _chat_control_command("what have all jobs done") == "/outcomes all"
    2558    assert _chat_control_command("what files did it create") == "/artifacts"
    2559    assert _chat_control_command("show me the saved files") == "/artifacts"
    2560    assert _chat_control_command("what tool calls did it run") == "/activity"
    2561    assert _chat_control_command("show console output") == "/outputs"
    2562    assert _chat_control_command("what tasks are open") == "/tasks"
    2563    assert _chat_control_command("show the current plan") == "/roadmap"
    2564    assert _chat_control_command("show benchmarks") == "/experiments"
    2565    assert _chat_control_command("how many tokens did it use") == "/usage"
    2566    assert _chat_control_command("restart daemon") == "/restart"
    2567    assert _chat_control_command("prefer artifact-backed findings") == ""
    2568
    2569
    2570def test_plain_chat_classifier_keeps_natural_controls_out_of_model_path():
    2571    assert _is_plain_chat_line("hello there") is True
    2572    assert _is_plain_chat_line("stop the job") is False
    2573    assert _is_plain_chat_line("what has it done") is False
    2574    assert _is_plain_chat_line("show me the saved files") is False
    2575
    2576
    2577def test_plain_chat_control_intent_does_not_queue_operator_context(monkeypatch, tmp_path):
    2578    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    2579    _mark_test_model_ready()
    2580    db = AgentDB(tmp_path / "state.db")
    2581    try:
    2582        job_id = db.create_job("Research topic", title="research")
    2583    finally:
    2584        db.close()
    2585
    2586    captured = {}
    2587
    2588    def fake_capture(job_id_arg, command):
    2589        captured["job_id"] = job_id_arg
    2590        captured["command"] = command
    2591        return True, "status output\n"
    2592
    2593    monkeypatch.setattr("nipux_cli.cli._capture_chat_command", fake_capture)
    2594
    2595    keep_running, message = _handle_chat_message(job_id, "how is it going?", quiet=True)
    2596
    2597    assert keep_running is True
    2598    assert message == "status output"
    2599    assert captured == {"job_id": job_id, "command": "/status"}
    2600    db = AgentDB(tmp_path / "state.db")
    2601    try:
    2602        job = db.get_job(job_id)
    2603        assert job["metadata"].get("operator_messages") is None
    2604    finally:
    2605        db.close()
    2606
    2607
    2608def test_plain_chat_reply_usage_is_recorded(monkeypatch, tmp_path):
    2609    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    2610    _mark_test_model_ready()
    2611    db = AgentDB(tmp_path / "state.db")
    2612    try:
    2613        job_id = db.create_job("Research topic", title="research")
    2614    finally:
    2615        db.close()
    2616
    2617    keep_running, message = _handle_chat_message(
    2618        job_id,
    2619        "hello",
    2620        quiet=True,
    2621        reply_fn=lambda _job_id, _line: LLMResponse(
    2622            content="reply",
    2623            usage={"prompt_tokens": 120, "completion_tokens": 20, "total_tokens": 140, "cost": 0.001},
    2624            model="provider/model",
    2625            response_id="gen_chat",
    2626        ),
    2627    )
    2628
    2629    db = AgentDB(tmp_path / "state.db")
    2630    try:
    2631        usage = db.job_token_usage(job_id)
    2632        events = db.list_events(job_id=job_id, event_types=["loop"], limit=5)
    2633    finally:
    2634        db.close()
    2635    assert keep_running is True
    2636    assert message == ""
    2637    assert usage["calls"] == 1
    2638    assert usage["prompt_tokens"] == 120
    2639    assert usage["completion_tokens"] == 20
    2640    assert usage["cost"] == 0.001
    2641    assert events[-1]["metadata"]["source"] == "chat"
    2642    assert events[-1]["metadata"]["response_id"] == "gen_chat"
    2643
    2644
    2645def test_chat_frame_surfaces_actual_work_events():
    2646    snapshot = {
    2647        "job_id": "job_demo",
    2648        "job": {
    2649            "id": "job_demo",
    2650            "title": "demo job",
    2651            "objective": "produce visible work",
    2652            "status": "running",
    2653            "kind": "generic",
    2654            "metadata": {
    2655                "task_queue": [{"status": "open"}],
    2656                "roadmap": {"milestones": [{"title": "Draft", "status": "active"}]},
    2657            },
    2658        },
    2659        "jobs": [{"id": "job_demo", "title": "demo job", "status": "running", "kind": "generic", "metadata": {}}],
    2660        "steps": [],
    2661        "artifacts": [{"id": "art_demo"}],
    2662        "memory_entries": [{}],
    2663        "events": [
    2664            {"event_type": "operator_message", "body": "please keep improving", "metadata": {"mode": "steer"}},
    2665            {"event_type": "tool_call", "title": "web_search", "body": "", "metadata": {"input": {"arguments": {"query": "agent harness distillation"}}}},
    2666            {"event_type": "tool_result", "title": "web_search", "body": "web_search query='agent harness distillation' returned 5 results", "metadata": {"status": "completed", "input": {"arguments": {"query": "agent harness distillation"}}}},
    2667            {"event_type": "artifact", "title": "Research Paper Draft", "body": "", "metadata": {"summary": "saved first complete draft"}},
    2668            {"event_type": "finding", "title": "Distillation finding", "body": "tool traces improve student behavior", "metadata": {}},
    2669            {"event_type": "task", "title": "Compare methods", "body": "", "metadata": {"status": "open"}},
    2670            {"event_type": "roadmap", "title": "Paper roadmap", "body": "", "metadata": {"status": "active"}},
    2671            {"event_type": "milestone_validation", "title": "Draft", "body": "", "metadata": {"validation_status": "passed"}},
    2672            {"event_type": "experiment", "title": "Citation coverage check", "body": "", "metadata": {"metric_name": "sources", "metric_value": 18, "metric_unit": ""}},
    2673            {"event_type": "lesson", "title": "strategy", "body": "prefer measured updates", "metadata": {}},
    2674            {"event_type": "reflection", "title": "reflection", "body": "Reflection through step #10: next branch is evaluation.", "metadata": {}},
    2675        ],
    2676        "daemon": {"running": True, "metadata": {"pid": 123}},
    2677        "model": "model/demo",
    2678        "counts": {"steps": 10, "artifacts": 1, "memory": 1},
    2679    }
    2680
    2681    updates = _build_chat_frame(snapshot, "", [], width=150, height=34, right_view="updates")
    2682    jobs = _build_chat_frame(snapshot, "", [], width=150, height=34, right_view="status")
    2683    frame = updates + "\n" + jobs
    2684
    2685    assert "Research Paper Draft" in frame
    2686    assert "Distillation finding" in frame
    2687    assert "Compare methods" in frame
    2688    assert "Paper roadmap" in frame
    2689    assert "passed Draft" in frame
    2690    assert "Citation coverage check" in frame
    2691    assert "LEARN" in frame
    2692    assert "strategy" in frame
    2693
    2694
    2695def test_chat_frame_has_model_updates_page():
    2696    snapshot = {
    2697        "job_id": "job_demo",
    2698        "job": {
    2699            "id": "job_demo",
    2700            "title": "paper job",
    2701            "objective": "write a paper",
    2702            "status": "running",
    2703            "kind": "generic",
    2704            "metadata": {},
    2705        },
    2706        "jobs": [{"id": "job_demo", "title": "paper job", "status": "running", "kind": "generic", "metadata": {}}],
    2707        "steps": [],
    2708        "artifacts": [],
    2709        "memory_entries": [],
    2710        "events": [
    2711            {"event_type": "tool_result", "title": "web_search", "body": "web_search query='distillation agents' returned 5 results", "metadata": {"status": "completed", "input": {"arguments": {"query": "distillation agents"}}}},
    2712            {"event_type": "artifact", "title": "Literature Review Draft", "body": "saved draft", "metadata": {}},
    2713            {"event_type": "finding", "title": "Trajectory distillation", "body": "teacher traces improve tool use", "metadata": {}},
    2714            {"event_type": "experiment", "title": "Citation density check", "body": "", "metadata": {"metric_name": "citations", "metric_value": 12, "metric_unit": "count"}},
    2715            {"event_type": "tool_result", "title": "write_file", "body": "write_file overwrite /tmp/paper.md", "metadata": {"status": "completed", "input": {"arguments": {"path": "/tmp/paper.md"}}, "output": {"path": "/tmp/paper.md"}}},
    2716            {"event_type": "tool_result", "title": "shell_exec", "body": "shell_exec rc=0", "metadata": {"status": "completed", "input": {"arguments": {"command": "printf draft | tee /tmp/outline.md"}}}},
    2717        ],
    2718        "daemon": {"running": True, "metadata": {"pid": 123}},
    2719        "model": "model/demo",
    2720    }
    2721
    2722    frame = _build_chat_frame(snapshot, "", [], width=132, height=28, right_view="updates")
    2723
    2724    assert "MODEL UPDATES" in frame
    2725    assert "Page" in frame
    2726    assert "1 outputs" in frame
    2727    assert "measurements" in frame
    2728    assert "Literature Review Draft" in frame
    2729    assert "Trajectory distillation" in frame
    2730    assert "Citation density check" in frame
    2731    assert "paper.md" in frame
    2732    assert "outline.md" in frame
    2733
    2734
    2735def test_workspace_status_page_does_not_render_fake_worker_when_no_jobs():
    2736    snapshot = {
    2737        "job_id": WORKSPACE_CHAT_ID,
    2738        "job": {
    2739            "id": WORKSPACE_CHAT_ID,
    2740            "title": "Nipux",
    2741            "objective": "Chat with Nipux to create, start, inspect, and steer long-running worker jobs.",
    2742            "status": "ready",
    2743            "kind": "workspace",
    2744            "metadata": {},
    2745        },
    2746        "right_job": {
    2747            "id": WORKSPACE_CHAT_ID,
    2748            "title": "Nipux",
    2749            "objective": "Chat with Nipux to create, start, inspect, and steer long-running worker jobs.",
    2750            "status": "ready",
    2751            "kind": "workspace",
    2752            "metadata": {},
    2753        },
    2754        "right_job_id": WORKSPACE_CHAT_ID,
    2755        "jobs": [],
    2756        "steps": [],
    2757        "artifacts": [],
    2758        "job_artifacts": {},
    2759        "job_summary_events": {},
    2760        "job_counts": {},
    2761        "memory_entries": [],
    2762        "events": [],
    2763        "right_events": [],
    2764        "summary_events": [],
    2765        "daemon": {"running": False, "metadata": {}},
    2766        "model": "model/demo",
    2767        "base_url": "http://127.0.0.1:8000/v1",
    2768        "context_length": 0,
    2769        "token_usage": {},
    2770        "counts": {"steps": 0, "artifacts": 0, "memory": 0},
    2771    }
    2772
    2773    frame = _build_chat_frame(snapshot, "", [], width=132, height=28, right_view="status")
    2774
    2775    assert "No workers yet" in frame
    2776    assert "plain English goal" in frame
    2777    assert "/new OBJECTIVE" in frame
    2778    assert "Chat with Nipux to create" not in frame
    2779    assert "actions:0" not in frame
    2780
    2781
    2782def test_status_job_cards_show_durable_work_mix():
    2783    events = [
    2784        {"event_type": "artifact", "title": "Paper draft", "body": "", "metadata": {}},
    2785        {"event_type": "finding", "title": "Method taxonomy", "body": "", "metadata": {}},
    2786        {
    2787            "event_type": "experiment",
    2788            "title": "Citation coverage check",
    2789            "body": "",
    2790            "metadata": {"metric_name": "citations", "metric_value": 12, "metric_unit": "count"},
    2791        },
    2792    ]
    2793    snapshot = {
    2794        "job_id": "job_demo",
    2795        "job": {
    2796            "id": "job_demo",
    2797            "title": "paper job",
    2798            "objective": "write a paper",
    2799            "status": "running",
    2800            "kind": "generic",
    2801            "metadata": {},
    2802        },
    2803        "jobs": [{"id": "job_demo", "title": "paper job", "status": "running", "kind": "generic", "metadata": {}}],
    2804        "steps": [],
    2805        "artifacts": [{"id": "art_1", "title": "Paper draft"}],
    2806        "job_artifacts": {"job_demo": [{"id": "art_1", "title": "Paper draft"}]},
    2807        "job_summary_events": {"job_demo": events},
    2808        "job_counts": {"job_demo": {"artifacts": 1}},
    2809        "memory_entries": [],
    2810        "events": events,
    2811        "summary_events": events,
    2812        "daemon": {"running": True, "metadata": {"pid": 123}},
    2813        "model": "model/demo",
    2814    }
    2815
    2816    frame = _build_chat_frame(snapshot, "", [], width=150, height=34, right_view="status")
    2817
    2818    assert "work 1 outputs 1 findings 1 measurements" in frame
    2819    assert "made 1 output" in frame
    2820    assert "Paper draft" in frame
    2821
    2822
    2823def test_recent_outcome_lines_wrap_long_updates():
    2824    lines = recent_model_update_lines(
    2825        [
    2826            {
    2827                "event_type": "finding",
    2828                "title": "Trajectory distillation improves agentic tool selection when teacher traces include failures and recovery actions",
    2829                "body": "",
    2830                "metadata": {},
    2831                "created_at": "2026-05-01T12:34:00+00:00",
    2832            }
    2833        ],
    2834        width=62,
    2835        limit=4,
    2836    )
    2837
    2838    rendered = "\n".join(lines)
    2839    assert len(lines) >= 2
    2840    assert "Trajectory distillation improves" in rendered
    2841    assert "teacher traces include" in rendered
    2842    assert "failures" in rendered
    2843
    2844
    2845def test_recent_outcome_lines_do_not_pretruncate_actual_work():
    2846    events = [
    2847        {
    2848            "event_type": "artifact",
    2849            "title": (
    2850                "Research paper draft rewritten with a new methods section, expanded evaluation table, "
    2851                "and integrated citations from teacher trajectory distillation, agent workflow distillation, "
    2852                "and self-improvement harness papers"
    2853            ),
    2854            "body": "",
    2855            "metadata": {},
    2856            "created_at": "2026-05-01T12:34:00+00:00",
    2857        }
    2858    ]
    2859
    2860    rendered = "\n".join(recent_model_update_lines(events, width=72, limit=6))
    2861
    2862    assert "methods" in rendered
    2863    assert "section" in rendered
    2864    assert "integrated" in rendered
    2865    assert "citations" in rendered
    2866    assert "self-improvement harness" in rendered
    2867    assert "papers" in rendered
    2868    assert "..." not in rendered
    2869
    2870
    2871def test_chat_updates_page_keeps_updates_to_one_line_each():
    2872    long_task = (
    2873        "open Publish a concise progress update and keep working on the next useful branch. "
    2874        "This task title is deliberately long enough to wrap several times if the compact pane does not constrain it."
    2875    )
    2876    events = [
    2877        {
    2878            "event_type": "task",
    2879            "title": "Publish progress",
    2880            "body": long_task,
    2881            "metadata": {"status": "open"},
    2882            "created_at": "2026-04-25T20:00:00+00:00",
    2883        }
    2884    ]
    2885
    2886    lines = recent_model_update_lines(events, width=56, limit=4, wrap=False)
    2887
    2888    assert len(lines) == 1
    2889    assert "Publish progress" in lines[0]
    2890
    2891
    2892def test_chat_pane_marks_hidden_overflow():
    2893    events = [
    2894        {
    2895            "event_type": "agent_message",
    2896            "title": "chat",
    2897            "body": " ".join(f"word{i}" for i in range(80)),
    2898            "metadata": {},
    2899            "created_at": "2026-04-25T12:00:00Z",
    2900        }
    2901    ]
    2902
    2903    lines = chat_pane_lines(events, [], width=48, rows=4)
    2904
    2905    assert "nipux" in lines[0]
    2906    assert "middle lines hidden" in "\n".join(lines)
    2907    assert "word" in lines[-1]
    2908    assert len(lines) == 4
    2909
    2910
    2911def test_chat_pane_groups_multiline_command_output_under_one_label():
    2912    lines = chat_pane_lines(
    2913        [],
    2914        [
    2915            "> /help",
    2916            "Create: type a goal, or /new OBJECTIVE.\n"
    2917            "Run: /run, /pause, /resume. Inspect: /jobs, /outcomes, /artifacts, /activity.\n"
    2918            "Config: /settings, /model, /base-url, /api-key. Navigate: ←→ pages, ↑↓ jobs.",
    2919        ],
    2920        width=78,
    2921        rows=12,
    2922    )
    2923
    2924    rendered = "\n".join(lines)
    2925    assert rendered.count("nipux") == 1
    2926    assert "Create: type a goal" in rendered
    2927    assert "Inspect: /jobs" in rendered
    2928    assert "Config: /settings" in rendered
    2929
    2930
    2931def test_chat_pane_suppresses_transient_duplicates_after_events_arrive():
    2932    events = [
    2933        {
    2934            "event_type": "operator_message",
    2935            "title": "chat",
    2936            "body": "Hello",
    2937            "metadata": {},
    2938            "created_at": "2026-04-25T20:00:00+00:00",
    2939        },
    2940        {
    2941            "event_type": "agent_message",
    2942            "title": "chat",
    2943            "body": "Hello! I can help with worker jobs.",
    2944            "metadata": {},
    2945            "created_at": "2026-04-25T20:00:01+00:00",
    2946        },
    2947    ]
    2948
    2949    lines = chat_pane_lines(
    2950        events,
    2951        ["> Hello", "sent; waiting for model", "Hello! I can help with worker jobs."],
    2952        width=80,
    2953        rows=12,
    2954    )
    2955
    2956    rendered = "\n".join(lines)
    2957    assert rendered.count("Hello") == 2
    2958    assert "waiting for model" not in rendered
    2959
    2960
    2961def test_chat_pane_hides_persisted_legacy_waiting_notice():
    2962    lines = chat_pane_lines(
    2963        [
    2964            {
    2965                "event_type": "operator_message",
    2966                "title": "chat",
    2967                "body": "Hello",
    2968                "metadata": {},
    2969                "created_at": "2026-04-25T20:00:00+00:00",
    2970            },
    2971            {
    2972                "event_type": "agent_message",
    2973                "title": "chat",
    2974                "body": "sent; waiting for model",
    2975                "metadata": {},
    2976                "created_at": "2026-04-25T20:00:01+00:00",
    2977            },
    2978            {
    2979                "event_type": "agent_message",
    2980                "title": "chat",
    2981                "body": "Hello! I can help with worker jobs.",
    2982                "metadata": {},
    2983                "created_at": "2026-04-25T20:00:02+00:00",
    2984            },
    2985        ],
    2986        [],
    2987        width=80,
    2988        rows=12,
    2989    )
    2990
    2991    rendered = "\n".join(lines)
    2992    assert "waiting for model" not in rendered
    2993    assert "Hello! I can help with worker jobs." in rendered
    2994
    2995
    2996def test_chat_pane_renders_waiting_notice_as_animation_only():
    2997    rendered_notices = _display_chat_notices([_WAITING_NOTICE])
    2998    lines = chat_pane_lines([], rendered_notices, width=64, rows=4)
    2999
    3000    rendered = "\n".join(lines)
    3001    assert "AGENT" in rendered
    3002    assert "waiting" in rendered
    3003    assert "Waiting for the next worker step" not in rendered
    3004    assert "waiting for model" not in rendered
    3005
    3006
    3007def test_chat_pane_hides_persisted_worker_waiting_text():
    3008    lines = chat_pane_lines(
    3009        [
    3010            {
    3011                "event_type": "operator_message",
    3012                "title": "chat",
    3013                "body": "what has it done so far?",
    3014                "metadata": {},
    3015                "created_at": "2026-04-25T20:00:00+00:00",
    3016            },
    3017            {
    3018                "event_type": "agent_message",
    3019                "title": "chat",
    3020                "body": "waiting for demo job: what has it done so far?",
    3021                "metadata": {},
    3022                "created_at": "2026-04-25T20:00:01+00:00",
    3023            },
    3024            {
    3025                "event_type": "agent_message",
    3026                "title": "chat",
    3027                "body": "Waiting for the next worker step.",
    3028                "metadata": {},
    3029                "created_at": "2026-04-25T20:00:02+00:00",
    3030            },
    3031        ],
    3032        [],
    3033        width=80,
    3034        rows=12,
    3035    )
    3036
    3037    rendered = "\n".join(lines)
    3038    assert "what has it done so far?" in rendered
    3039    assert "waiting for demo job" not in rendered
    3040    assert "Waiting for the next worker step" not in rendered
    3041    assert "NIPUX" not in rendered
    3042
    3043
    3044def test_chat_pane_renders_stored_provider_errors_as_actions():
    3045    lines = chat_pane_lines(
    3046        [
    3047            {
    3048                "event_type": "agent_message",
    3049                "title": "chat",
    3050                "body": "APIConnectionError: Connection error.",
    3051                "metadata": {"error": True},
    3052                "created_at": "2026-04-25T20:00:00+00:00",
    3053            }
    3054        ],
    3055        [],
    3056        width=80,
    3057        rows=6,
    3058    )
    3059
    3060    rendered = "\n".join(lines)
    3061    assert "APIConnectionError" not in rendered
    3062    assert "Model endpoint is unreachable" in rendered
    3063    assert "/doctor" in rendered
    3064
    3065
    3066def test_chat_updates_page_uses_deeper_summary_events():
    3067    snapshot = {
    3068        "job_id": "job_demo",
    3069        "job": {
    3070            "id": "job_demo",
    3071            "title": "paper job",
    3072            "objective": "write a paper",
    3073            "status": "running",
    3074            "kind": "generic",
    3075            "metadata": {},
    3076        },
    3077        "jobs": [{"id": "job_demo", "title": "paper job", "status": "running", "kind": "generic", "metadata": {}}],
    3078        "steps": [],
    3079        "artifacts": [],
    3080        "memory_entries": [],
    3081        "events": [
    3082            {"event_type": "tool_call", "title": "web_search", "body": "", "metadata": {}},
    3083        ],
    3084        "summary_events": [
    3085            {"event_type": "artifact", "title": "Full Paper Draft", "body": "saved draft", "metadata": {}},
    3086            {"event_type": "finding", "title": "Distillation method map", "body": "", "metadata": {}},
    3087        ],
    3088        "daemon": {"running": True, "metadata": {"pid": 123}},
    3089        "model": "model/demo",
    3090    }
    3091
    3092    frame = _build_chat_frame(snapshot, "", [], width=132, height=26, right_view="updates")
    3093
    3094    assert "Full Paper Draft" in frame
    3095    assert "Distillation method map" in frame
    3096
    3097
    3098def test_hourly_outcomes_prioritize_durable_work_over_research_noise():
    3099    events = [
    3100        {
    3101            "event_type": "tool_result",
    3102            "title": "web_search",
    3103            "body": "web_search query='generic harness patterns' returned 5 results",
    3104            "metadata": {"status": "completed", "input": {"arguments": {"query": "generic harness patterns"}}},
    3105            "created_at": "2026-05-01T12:05:00+00:00",
    3106        },
    3107        {
    3108            "event_type": "tool_result",
    3109            "title": "web_extract",
    3110            "body": "web_extract fetched 3/3 pages",
    3111            "metadata": {"status": "completed"},
    3112            "created_at": "2026-05-01T12:08:00+00:00",
    3113        },
    3114        {
    3115            "event_type": "artifact",
    3116            "title": "Harness Architecture Notes",
    3117            "body": "saved design notes",
    3118            "metadata": {},
    3119            "created_at": "2026-05-01T12:20:00+00:00",
    3120        },
    3121        {
    3122            "event_type": "experiment",
    3123            "title": "Context budget check",
    3124            "body": "",
    3125            "metadata": {"metric_name": "prompt_tokens", "metric_value": 4200, "metric_unit": "tokens"},
    3126            "created_at": "2026-05-01T12:30:00+00:00",
    3127        },
    3128    ]
    3129
    3130    rendered = "\n".join(hourly_update_lines(events, width=96, limit=8))
    3131
    3132    assert "2 research" in rendered
    3133    assert "1 outputs" in rendered
    3134    assert "1 measurements" in rendered
    3135    assert "Harness Architecture Notes" in rendered
    3136    assert "Context budget check" in rendered
    3137    assert "generic harness patterns" not in rendered
    3138
    3139
    3140def test_status_recent_outcomes_hide_research_noise():
    3141    events = [
    3142        {
    3143            "event_type": "tool_result",
    3144            "title": "web_search",
    3145            "body": "web_search query='generic harness patterns' returned 5 results",
    3146            "metadata": {"status": "completed", "input": {"arguments": {"query": "generic harness patterns"}}},
    3147            "created_at": "2026-05-01T12:05:00+00:00",
    3148        },
    3149        {
    3150            "event_type": "artifact",
    3151            "title": "Harness Architecture Notes",
    3152            "body": "saved design notes",
    3153            "metadata": {},
    3154            "created_at": "2026-05-01T12:20:00+00:00",
    3155        },
    3156    ]
    3157
    3158    rendered = "\n".join(recent_model_update_lines(events, width=96, limit=4))
    3159
    3160    assert "Harness Architecture Notes" in rendered
    3161    assert "generic harness patterns" not in rendered
    3162
    3163
    3164def test_status_recent_outcomes_hide_plan_update_noise():
    3165    events = [
    3166        {
    3167            "event_type": "reflection",
    3168            "title": "reflection",
    3169            "body": "summarized current counts",
    3170            "metadata": {},
    3171            "created_at": "2026-05-01T12:05:00+00:00",
    3172        },
    3173        {
    3174            "event_type": "agent_message",
    3175            "title": "progress",
    3176            "body": "Checkpoint at step #100.",
    3177            "metadata": {},
    3178            "created_at": "2026-05-01T12:08:00+00:00",
    3179        },
    3180        {
    3181            "event_type": "finding",
    3182            "title": "Teacher trace distillation pattern",
    3183            "body": "",
    3184            "metadata": {},
    3185            "created_at": "2026-05-01T12:20:00+00:00",
    3186        },
    3187    ]
    3188
    3189    rendered = "\n".join(recent_model_update_lines(events, width=96, limit=5))
    3190
    3191    assert "Teacher trace distillation pattern" in rendered
    3192    assert "Checkpoint at step" not in rendered
    3193    assert "summarized current counts" not in rendered
    3194
    3195
    3196def test_status_recent_outcomes_show_durable_checkpoint_updates():
    3197    events = [
    3198        {
    3199            "event_type": "agent_message",
    3200            "title": "progress",
    3201            "body": "Checkpoint step #90: ~1 task updated, 1 task resolved.",
    3202            "metadata": {
    3203                "updates": {"tasks": 1},
    3204                "resolutions": {"tasks": 1},
    3205                "deltas": {"findings": 0},
    3206            },
    3207            "created_at": "2026-05-01T12:08:00+00:00",
    3208        }
    3209    ]
    3210
    3211    rendered = "\n".join(recent_model_update_lines(events, width=96, limit=4))
    3212
    3213    assert "TASK" in rendered
    3214    assert "~1 task updated" in rendered
    3215    assert "1 task resolved" in rendered
    3216    assert "Checkpoint step #90" in rendered
    3217
    3218
    3219def test_status_recent_outcomes_compact_repeated_updates():
    3220    events = [
    3221        {
    3222            "event_type": "agent_message",
    3223            "title": "error",
    3224            "body": "Model provider requires operator action.",
    3225            "metadata": {},
    3226            "created_at": f"2026-05-01T12:0{index}:00+00:00",
    3227        }
    3228        for index in range(3)
    3229    ]
    3230
    3231    rendered = "\n".join(recent_model_update_lines(events, width=96, limit=4))
    3232
    3233    assert rendered.count("Model provider requires operator action") == 1
    3234    assert "x3" in rendered
    3235
    3236
    3237def test_hourly_outcomes_hide_plan_update_noise():
    3238    events = [
    3239        {
    3240            "event_type": "reflection",
    3241            "title": "reflection",
    3242            "body": "summarized current counts",
    3243            "metadata": {},
    3244            "created_at": "2026-05-01T12:05:00+00:00",
    3245        },
    3246        {
    3247            "event_type": "agent_message",
    3248            "title": "progress",
    3249            "body": "Checkpoint at step #100.",
    3250            "metadata": {},
    3251            "created_at": "2026-05-01T12:08:00+00:00",
    3252        },
    3253        {
    3254            "event_type": "artifact",
    3255            "title": "Saved research draft",
    3256            "body": "",
    3257            "metadata": {},
    3258            "created_at": "2026-05-01T12:20:00+00:00",
    3259        },
    3260    ]
    3261
    3262    rendered = "\n".join(hourly_update_lines(events, width=96, limit=6))
    3263
    3264    assert "Saved research draft" in rendered
    3265    assert "Checkpoint at step" not in rendered
    3266    assert "summarized current counts" not in rendered
    3267
    3268
    3269def test_hourly_outcomes_count_durable_checkpoint_updates():
    3270    events = [
    3271        {
    3272            "event_type": "agent_message",
    3273            "title": "progress",
    3274            "body": "Checkpoint step #110: ~1 experiment updated, 1 experiment resolved.",
    3275            "metadata": {
    3276                "updates": {"experiments": 1},
    3277                "resolutions": {"experiments": 1},
    3278            },
    3279            "created_at": "2026-05-01T12:08:00+00:00",
    3280        }
    3281    ]
    3282
    3283    rendered = "\n".join(hourly_update_lines(events, width=96, limit=6))
    3284
    3285    assert "1 measurements" in rendered
    3286    assert "~1 measurement updated" in rendered
    3287    assert "1 measurement resolved" in rendered
    3288
    3289
    3290def test_hourly_outcome_summary_uses_progress_order():
    3291    events = [
    3292        {
    3293            "event_type": "source",
    3294            "title": "source scored",
    3295            "body": "",
    3296            "metadata": {},
    3297            "created_at": "2026-05-01T12:01:00+00:00",
    3298        },
    3299        {
    3300            "event_type": "artifact",
    3301            "title": "draft saved",
    3302            "body": "",
    3303            "metadata": {},
    3304            "created_at": "2026-05-01T12:02:00+00:00",
    3305        },
    3306        {
    3307            "event_type": "experiment",
    3308            "title": "metric checked",
    3309            "body": "",
    3310            "metadata": {"metric_name": "score", "metric_value": 1, "metric_unit": "point"},
    3311            "created_at": "2026-05-01T12:03:00+00:00",
    3312        },
    3313    ]
    3314
    3315    rendered = "\n".join(hourly_update_lines(events, width=96, limit=8))
    3316
    3317    assert "1 outputs 1 measurements 1 sources" in rendered
    3318
    3319
    3320def test_hourly_outcomes_wrap_long_durable_updates_without_pre_truncation():
    3321    events = [
    3322        {
    3323            "event_type": "finding",
    3324            "title": (
    3325                "Distillation survey breakthrough: teacher trajectories should include failed tool calls, "
    3326                "operator corrections, recovery steps, and measured validation so the student learns the "
    3327                "whole harness loop instead of only final answers"
    3328            ),
    3329            "body": "",
    3330            "metadata": {},
    3331            "created_at": "2026-05-01T12:05:00+00:00",
    3332        },
    3333    ]
    3334
    3335    rendered = "\n".join(hourly_update_lines(events, width=82, limit=6))
    3336
    3337    assert "operator corrections" in rendered
    3338    assert "measured" in rendered
    3339    assert "validation" in rendered
    3340    assert "only" in rendered
    3341    assert "final answers" in rendered
    3342    assert "..." not in rendered
    3343
    3344
    3345def test_hourly_outcomes_limit_visible_hours_without_losing_headers():
    3346    events = []
    3347    for hour in range(8):
    3348        events.extend(
    3349            [
    3350                {
    3351                    "event_type": "artifact",
    3352                    "title": f"Draft saved hour {hour}",
    3353                    "body": "",
    3354                    "metadata": {},
    3355                    "created_at": f"2026-05-01T{hour:02d}:05:00+00:00",
    3356                },
    3357                {
    3358                    "event_type": "finding",
    3359                    "title": f"Finding hour {hour}",
    3360                    "body": "",
    3361                    "metadata": {},
    3362                    "created_at": f"2026-05-01T{hour:02d}:20:00+00:00",
    3363                },
    3364            ]
    3365        )
    3366
    3367    rendered = "\n".join(hourly_update_lines(events, width=96, limit=8))
    3368
    3369    assert "2026-05-01 06:00" in rendered
    3370    assert "2026-05-01 07:00" in rendered
    3371    assert "Draft saved hour 7" in rendered
    3372    assert "Finding hour 7" in rendered
    3373    assert "Draft saved hour 0" not in rendered
    3374
    3375
    3376def test_chat_updates_page_includes_agent_error_updates():
    3377    snapshot = {
    3378        "job_id": "job_demo",
    3379        "job": {
    3380            "id": "job_demo",
    3381            "title": "provider job",
    3382            "objective": "keep provider state visible",
    3383            "status": "paused",
    3384            "kind": "generic",
    3385            "metadata": {},
    3386        },
    3387        "jobs": [{"id": "job_demo", "title": "provider job", "status": "paused", "kind": "generic", "metadata": {}}],
    3388        "steps": [],
    3389        "artifacts": [],
    3390        "memory_entries": [],
    3391        "events": [],
    3392        "summary_events": [
    3393            {
    3394                "event_type": "agent_message",
    3395                "title": "error",
    3396                "body": "Model provider requires operator action.",
    3397                "metadata": {"reason": "llm_provider_blocked"},
    3398            },
    3399        ],
    3400        "daemon": {"running": True, "metadata": {"pid": 123}},
    3401        "model": "model/demo",
    3402    }
    3403
    3404    updates = _build_chat_frame(snapshot, "", [], width=132, height=34, right_view="updates")
    3405    status = _build_chat_frame(snapshot, "", [], width=132, height=34, right_view="status")
    3406
    3407    assert "Model provider requires" in updates
    3408    assert "operator action" in updates
    3409    assert "Outcome" in status
    3410    assert "Model provider re" in status
    3411
    3412
    3413def test_chat_status_marks_provider_blocked_jobs_before_daemon_retry():
    3414    job = {
    3415        "id": "job_demo",
    3416        "title": "provider job",
    3417        "objective": "keep provider state visible",
    3418        "status": "running",
    3419        "kind": "generic",
    3420        "metadata": {"provider_blocked_at": "2026-05-01T00:00:00+00:00"},
    3421    }
    3422    snapshot = {
    3423        "job_id": "job_demo",
    3424        "job": job,
    3425        "jobs": [job],
    3426        "steps": [],
    3427        "artifacts": [],
    3428        "memory_entries": [],
    3429        "events": [],
    3430        "summary_events": [],
    3431        "daemon": {"running": True, "metadata": {"pid": 123}},
    3432        "model": "model/demo",
    3433    }
    3434
    3435    frame = _build_chat_frame(snapshot, "", [], width=132, height=30, right_view="status")
    3436
    3437    assert "provider wait" in frame
    3438    assert "Provider" in frame
    3439    assert "action needed" in frame
    3440    assert "advancing" not in frame
    3441
    3442
    3443def test_chat_status_page_surfaces_context_pressure():
    3444    snapshot = {
    3445        "job_id": "job_demo",
    3446        "job": {
    3447            "id": "job_demo",
    3448            "title": "context job",
    3449            "objective": "keep context pressure visible",
    3450            "status": "running",
    3451            "kind": "generic",
    3452            "metadata": {},
    3453        },
    3454        "jobs": [{"id": "job_demo", "title": "context job", "status": "running", "kind": "generic", "metadata": {}}],
    3455        "steps": [],
    3456        "artifacts": [],
    3457        "memory_entries": [],
    3458        "events": [],
    3459        "daemon": {"running": True, "metadata": {"pid": 123}},
    3460        "model": "model/demo",
    3461        "context_length": 8192,
    3462        "token_usage": {"calls": 3, "latest_prompt_tokens": 7000, "total_tokens": 9000, "completion_tokens": 2000},
    3463    }
    3464
    3465    frame = _build_chat_frame(snapshot, "", [], width=132, height=30, right_view="status")
    3466
    3467    assert "Context" in frame
    3468    assert "7.0K/8.2K" in frame
    3469    assert "85%" in frame
    3470    assert "high" in frame
    3471
    3472
    3473def test_chat_status_page_surfaces_low_durable_yield():
    3474    snapshot = {
    3475        "job_id": "job_demo",
    3476        "job": {
    3477            "id": "job_demo",
    3478            "title": "yield job",
    3479            "objective": "keep durable progress visible",
    3480            "status": "running",
    3481            "kind": "generic",
    3482            "metadata": {},
    3483        },
    3484        "jobs": [{"id": "job_demo", "title": "yield job", "status": "running", "kind": "generic", "metadata": {}}],
    3485        "steps": [],
    3486        "artifacts": [{"id": "art_demo", "title": "Only Saved Output"}],
    3487        "memory_entries": [],
    3488        "events": [],
    3489        "daemon": {"running": True, "metadata": {"pid": 123}},
    3490        "model": "model/demo",
    3491        "counts": {"steps": 120, "artifacts": 1, "memory": 0},
    3492    }
    3493
    3494    frame = _build_chat_frame(snapshot, "", [], width=132, height=30, right_view="status")
    3495
    3496    assert "Yield" in frame
    3497    assert "watch" in frame
    3498    assert "120.0 actions/outcome" in frame
    3499
    3500
    3501def test_chat_status_page_shows_job_outputs():
    3502    snapshot = {
    3503        "job_id": "job_demo",
    3504        "job": {
    3505            "id": "job_demo",
    3506            "title": "demo job",
    3507            "objective": "show created outputs per job",
    3508            "status": "running",
    3509            "kind": "generic",
    3510            "metadata": {},
    3511        },
    3512        "jobs": [
    3513            {"id": "job_demo", "title": "demo job", "status": "running", "kind": "generic", "metadata": {}},
    3514            {"id": "job_other", "title": "other job", "status": "queued", "kind": "generic", "metadata": {}},
    3515        ],
    3516        "steps": [],
    3517        "artifacts": [
    3518            {"id": "art_demo", "title": "Primary Saved Draft"},
    3519            {"id": "art_second", "title": "Secondary Saved Note"},
    3520        ],
    3521        "job_artifacts": {
    3522            "job_demo": [
    3523                {"id": "art_demo", "title": "Primary Saved Draft"},
    3524                {"id": "art_second", "title": "Secondary Saved Note"},
    3525            ],
    3526            "job_other": [{"id": "art_other", "title": "Other Job Deliverable"}],
    3527        },
    3528        "job_counts": {
    3529            "job_demo": {"artifacts": 2},
    3530            "job_other": {"artifacts": 4},
    3531        },
    3532        "job_summary_events": {
    3533            "job_demo": [
    3534                {"event_type": "artifact", "title": "Primary Saved Draft", "body": "", "metadata": {}},
    3535                {"event_type": "experiment", "title": "Primary quality check", "body": "", "metadata": {"metric_name": "score", "metric_value": 8}},
    3536            ],
    3537            "job_other": [
    3538                {"event_type": "finding", "title": "Other job durable finding", "body": "", "metadata": {}},
    3539            ],
    3540        },
    3541        "memory_entries": [],
    3542        "events": [],
    3543        "summary_events": [
    3544            {"event_type": "finding", "title": "Latest durable milestone", "body": "", "metadata": {}},
    3545        ],
    3546        "daemon": {"running": True, "metadata": {"pid": 123}},
    3547        "model": "model/demo",
    3548        "counts": {"steps": 0, "artifacts": 1, "memory": 0},
    3549    }
    3550
    3551    frame = _build_chat_frame(snapshot, "", [], width=132, height=34, right_view="status")
    3552
    3553    assert "Jobs" in frame
    3554    assert "Latest hour" in frame
    3555    assert "1 findings" in frame
    3556    assert "Outcome" in frame
    3557    assert "Latest durable milestone" in frame
    3558    assert "2 outputs" in frame
    3559    assert "Primary Saved Draft" in frame
    3560    assert "Secondary Saved Note" in frame
    3561    assert "Primary quality check" in frame
    3562    assert "4 outputs" in frame
    3563    assert "Other Job Deliverable" in frame
    3564    assert "Other job durable finding" in frame
    3565
    3566
    3567def test_frame_snapshot_keeps_summary_events_durable(monkeypatch, tmp_path):
    3568    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    3569    db = AgentDB(tmp_path / "state.db")
    3570    try:
    3571        job_id = db.create_job("keep frame refresh focused", title="focused")
    3572        for index in range(30):
    3573            db.append_event(
    3574                job_id=job_id,
    3575                event_type="tool_result",
    3576                title="web_search",
    3577                body=f"search noise {index}",
    3578                metadata={"status": "completed"},
    3579            )
    3580        db.append_event(
    3581            job_id=job_id,
    3582            event_type="tool_result",
    3583            title="write_file",
    3584            body="write_file overwrite /tmp/paper.md",
    3585            metadata={"status": "completed", "input": {"arguments": {"path": "/tmp/paper.md"}}, "output": {"path": "/tmp/paper.md"}},
    3586        )
    3587        db.append_event(
    3588            job_id=job_id,
    3589            event_type="tool_result",
    3590            title="shell_exec",
    3591            body="shell_exec rc=0",
    3592            metadata={"status": "completed", "input": {"arguments": {"command": "printf draft | tee /tmp/outline.md"}}},
    3593        )
    3594        db.append_event(job_id=job_id, event_type="artifact", title="Durable Paper Draft", body="", metadata={})
    3595        db.append_event(job_id=job_id, event_type="finding", title="Actual finding", body="", metadata={})
    3596    finally:
    3597        db.close()
    3598
    3599    snapshot = _load_frame_snapshot(job_id, history_limit=4)
    3600    summary_text = "\n".join(str(event.get("title") or event.get("body") or "") for event in snapshot["summary_events"])
    3601
    3602    assert "Durable Paper Draft" in summary_text
    3603    assert "Actual finding" in summary_text
    3604    assert "write_file" in summary_text
    3605    assert "shell_exec" in summary_text
    3606    assert "web_search" not in summary_text
    3607    assert "search noise" not in summary_text
    3608
    3609
    3610def test_frame_snapshot_respects_explicit_job_over_saved_focus(monkeypatch, tmp_path):
    3611    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    3612    db = AgentDB(tmp_path / "state.db")
    3613    try:
    3614        focused_id = db.create_job("saved focus", title="saved focus")
    3615        requested_id = db.create_job("requested focus", title="requested focus")
    3616        (tmp_path / "shell_state.json").write_text(json.dumps({"focus_job_id": focused_id}), encoding="utf-8")
    3617    finally:
    3618        db.close()
    3619
    3620    snapshot = _load_frame_snapshot(requested_id, history_limit=4)
    3621
    3622    assert snapshot["job_id"] == requested_id
    3623    assert snapshot["job"]["title"] == "requested focus"
    3624
    3625
    3626def test_chat_status_page_marks_deferred_jobs_waiting():
    3627    snapshot = {
    3628        "job_id": "job_demo",
    3629        "job": {
    3630            "id": "job_demo",
    3631            "title": "deferred job",
    3632            "objective": "check a long-running process later",
    3633            "status": "running",
    3634            "kind": "generic",
    3635            "metadata": {"defer_until": "2999-01-01T00:00:00+00:00", "defer_reason": "external process running"},
    3636        },
    3637        "jobs": [
    3638            {
    3639                "id": "job_demo",
    3640                "title": "deferred job",
    3641                "status": "running",
    3642                "kind": "generic",
    3643                "metadata": {"defer_until": "2999-01-01T00:00:00+00:00", "defer_reason": "external process running"},
    3644            }
    3645        ],
    3646        "steps": [],
    3647        "artifacts": [],
    3648        "job_artifacts": {},
    3649        "memory_entries": [],
    3650        "events": [],
    3651        "daemon": {"running": True, "metadata": {"pid": 123}},
    3652        "model": "model/demo",
    3653    }
    3654
    3655    frame = _build_chat_frame(snapshot, "", [], width=132, height=28, right_view="status")
    3656
    3657    assert "waiting" in frame
    3658    assert "Wait" in frame
    3659    assert "next check" in frame
    3660    assert "external" in frame
    3661    assert "active" not in frame
    3662
    3663
    3664def test_chat_frame_collapses_repeated_failures_and_hides_memory_noise():
    3665    repeated_error = {
    3666        "event_type": "error",
    3667        "title": "llm",
    3668        "body": "Error code: 403 - {'error': {'message': 'Key limit exceeded (total limit).'}}",
    3669        "metadata": {},
    3670    }
    3671    snapshot = {
    3672        "job_id": "job_demo",
    3673        "job": {
    3674            "id": "job_demo",
    3675            "title": "demo job",
    3676            "objective": "stay readable",
    3677            "status": "running",
    3678            "kind": "generic",
    3679            "metadata": {"task_queue": []},
    3680        },
    3681        "jobs": [{"id": "job_demo", "title": "demo job", "status": "running", "kind": "generic", "metadata": {}}],
    3682        "steps": [],
    3683        "artifacts": [],
    3684        "memory_entries": [{}],
    3685        "events": [
    3686            repeated_error,
    3687            {
    3688                "event_type": "compaction",
    3689                "title": "rolling_state",
    3690                "body": "very long compact memory " * 80,
    3691                "metadata": {},
    3692            },
    3693            repeated_error,
    3694            repeated_error,
    3695        ],
    3696        "daemon": {"running": True, "metadata": {"pid": 123}},
    3697        "model": "model/demo",
    3698        "counts": {"steps": 3, "artifacts": 0, "memory": 1},
    3699    }
    3700
    3701    frame = _build_chat_frame(snapshot, "", [], width=120, height=24, right_view="updates")
    3702
    3703    assert "3 blocks" in frame
    3704    assert "FAIL" in frame
    3705    assert "very long compact memory" not in frame
    3706
    3707
    3708def test_work_pane_uses_badges_without_duplicate_action_verbs():
    3709    snapshot = {
    3710        "job_id": "job_demo",
    3711        "job": {
    3712            "id": "job_demo",
    3713            "title": "demo job",
    3714            "objective": "stay readable",
    3715            "status": "running",
    3716            "kind": "generic",
    3717            "metadata": {"task_queue": []},
    3718        },
    3719        "jobs": [{"id": "job_demo", "title": "demo job", "status": "running", "kind": "generic", "metadata": {}}],
    3720        "steps": [],
    3721        "artifacts": [],
    3722        "memory_entries": [],
    3723        "events": [
    3724            {"event_type": "artifact", "title": "Demo Output", "body": "", "metadata": {}},
    3725            {"event_type": "finding", "title": "Demo Finding", "body": "", "metadata": {}},
    3726            {"event_type": "experiment", "title": "Demo Measurement", "body": "", "metadata": {}},
    3727        ],
    3728        "daemon": {"running": True, "metadata": {"pid": 123}},
    3729        "model": "model/demo",
    3730    }
    3731
    3732    frame = _build_chat_frame(snapshot, "", [], width=120, height=24, right_view="updates")
    3733
    3734    assert "Demo Output" in frame
    3735    assert "Demo Finding" in frame
    3736    assert "TEST" in frame
    3737    assert "Demo Measurement" in frame
    3738    assert "save saved" not in frame
    3739    assert "find finding" not in frame
    3740    assert "test experiment" not in frame
    3741
    3742
    3743def test_run_reopens_completed_focused_job(monkeypatch, tmp_path, capsys):
    3744    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    3745    _mark_test_model_ready()
    3746    parser = build_parser()
    3747    db = AgentDB(tmp_path / "state.db")
    3748    try:
    3749        job_id = db.create_job("Keep improving", title="perpetual")
    3750        db.update_job_status(job_id, "completed")
    3751    finally:
    3752        db.close()
    3753    started = {}
    3754
    3755    def fake_start(**kwargs):
    3756        started.update(kwargs)
    3757
    3758    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    3759    args = parser.parse_args(["run", "perpetual", "--no-follow"])
    3760
    3761    args.func(args)
    3762
    3763    out = capsys.readouterr().out
    3764    db = AgentDB(tmp_path / "state.db")
    3765    try:
    3766        job = db.get_job(job_id)
    3767        assert "focus set: perpetual" in out
    3768        assert job["status"] == "queued"
    3769        assert job["metadata"]["last_note"] == "reopened from completed by operator run command"
    3770        assert started["poll_seconds"] == 0.0
    3771    finally:
    3772        db.close()
    3773
    3774
    3775def test_run_delegates_unverified_provider_state_to_daemon_start(monkeypatch, tmp_path, capsys):
    3776    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    3777    parser = build_parser()
    3778    db = AgentDB(tmp_path / "state.db")
    3779    try:
    3780        db.create_job("Keep checking provider recovery", title="provider recovery")
    3781    finally:
    3782        db.close()
    3783    started = {}
    3784
    3785    def fake_start(**kwargs):
    3786        started.update(kwargs)
    3787        print("model provider is not ready; starting daemon in recovery monitor mode")
    3788
    3789    monkeypatch.setattr("nipux_cli.cli._remote_model_preflight_failures", lambda _config: [])
    3790    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    3791    args = parser.parse_args(["run", "provider recovery", "--no-follow"])
    3792
    3793    args.func(args)
    3794
    3795    out = capsys.readouterr().out
    3796    assert "Model setup is not verified." not in out
    3797    assert "recovery monitor mode" in out
    3798    assert started["poll_seconds"] == 0.0
    3799
    3800
    3801def test_run_marks_job_waiting_when_provider_recovery_is_needed(monkeypatch, tmp_path, capsys):
    3802    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    3803    parser = build_parser()
    3804    db = AgentDB(tmp_path / "state.db")
    3805    try:
    3806        job_id = db.create_job("Keep checking provider recovery", title="provider recovery")
    3807    finally:
    3808        db.close()
    3809
    3810    monkeypatch.setattr("nipux_cli.cli._remote_model_preflight_failures", lambda _config: ["model_generation: key limit exceeded"])
    3811
    3812    def fake_start(**_kwargs):
    3813        print("model provider is not ready; starting daemon in recovery monitor mode")
    3814
    3815    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    3816    args = parser.parse_args(["run", "provider recovery", "--no-follow"])
    3817
    3818    args.func(args)
    3819
    3820    out = capsys.readouterr().out
    3821    assert "recovery monitor mode" in out
    3822    db = AgentDB(tmp_path / "state.db")
    3823    try:
    3824        job = db.get_job(job_id)
    3825        assert job["status"] == "paused"
    3826        assert job["metadata"]["provider_blocked_at"]
    3827        assert "monitor and resume" in job["metadata"]["last_note"]
    3828    finally:
    3829        db.close()
    3830
    3831
    3832def test_run_does_not_reopen_already_provider_blocked_job(monkeypatch, tmp_path, capsys):
    3833    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    3834    parser = build_parser()
    3835    db = AgentDB(tmp_path / "state.db")
    3836    blocked_at = "2026-05-01T00:00:00+00:00"
    3837    try:
    3838        job_id = db.create_job("Keep checking provider recovery", title="provider recovery")
    3839        db.update_job_status(
    3840            job_id,
    3841            "paused",
    3842            metadata_patch={
    3843                "provider_blocked_at": blocked_at,
    3844                "last_note": "Model provider is unavailable; daemon will monitor and resume this job when calls succeed.",
    3845            },
    3846        )
    3847        event_count = len(db.list_events(job_id=job_id, limit=20))
    3848    finally:
    3849        db.close()
    3850
    3851    monkeypatch.setattr("nipux_cli.cli._remote_model_preflight_failures", lambda _config: ["model_generation: key limit exceeded"])
    3852
    3853    def fake_start(**_kwargs):
    3854        print("model provider is not ready; starting daemon in recovery monitor mode")
    3855
    3856    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    3857    args = parser.parse_args(["run", "provider recovery", "--no-follow"])
    3858
    3859    args.func(args)
    3860
    3861    out = capsys.readouterr().out
    3862    assert "recovery monitor mode" in out
    3863    db = AgentDB(tmp_path / "state.db")
    3864    try:
    3865        job = db.get_job(job_id)
    3866        assert job["status"] == "paused"
    3867        assert job["metadata"]["provider_blocked_at"] == blocked_at
    3868        assert "still unavailable" in job["metadata"]["last_note"]
    3869        events = db.list_events(job_id=job_id, limit=20)
    3870        assert len(events) == event_count
    3871        assert all("Reopened from" not in str(event.get("body") or "") for event in events)
    3872    finally:
    3873        db.close()
    3874
    3875
    3876def test_run_does_not_reopen_job_when_provider_preflight_is_hard_failure(monkeypatch, tmp_path):
    3877    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    3878    parser = build_parser()
    3879    db = AgentDB(tmp_path / "state.db")
    3880    try:
    3881        job_id = db.create_job("Keep checking provider recovery", title="provider recovery")
    3882        db.update_job_status(job_id, "paused", metadata_patch={"last_note": "operator paused"})
    3883    finally:
    3884        db.close()
    3885
    3886    monkeypatch.setattr("nipux_cli.cli._remote_model_preflight_failures", lambda _config: ["model_auth: user not found"])
    3887    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", lambda **_kwargs: None)
    3888    args = parser.parse_args(["run", "provider recovery", "--no-follow"])
    3889
    3890    args.func(args)
    3891
    3892    db = AgentDB(tmp_path / "state.db")
    3893    try:
    3894        job = db.get_job(job_id)
    3895        assert job["status"] == "paused"
    3896        assert job["metadata"]["last_note"] == "operator paused"
    3897    finally:
    3898        db.close()
    3899
    3900
    3901def test_create_sets_new_job_as_shell_focus(monkeypatch, tmp_path, capsys):
    3902    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    3903    _mark_test_model_ready()
    3904    parser = build_parser()
    3905    args = parser.parse_args(["create", "Research new topic", "--title", "new research", "--kind", "generic"])
    3906
    3907    args.func(args)
    3908    created = capsys.readouterr().out.strip()
    3909    assert _run_shell_line("focus") is True
    3910
    3911    out = capsys.readouterr().out
    3912    assert created == "created new research"
    3913    assert "new research" in out
    3914    assert (tmp_path / "jobs" / "new-research" / "program.md").exists()
    3915    db = AgentDB(tmp_path / "state.db")
    3916    try:
    3917        job = db.get_job("new-research")
    3918        assert job["status"] == "queued"
    3919        assert job["metadata"]["planning_status"] == "auto_accepted"
    3920        assert job["metadata"]["planning"]["questions"]
    3921        tasks = job["metadata"]["task_queue"]
    3922        assert tasks
    3923        assert all(task["output_contract"] for task in tasks)
    3924        assert all(task["acceptance_criteria"] for task in tasks)
    3925        assert all(task["evidence_needed"] for task in tasks)
    3926        assert all(task["stall_behavior"] for task in tasks)
    3927    finally:
    3928        db.close()
    3929
    3930
    3931def test_commands_accept_unquoted_job_titles_in_shell(monkeypatch, tmp_path, capsys):
    3932    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    3933    db = AgentDB(tmp_path / "state.db")
    3934    try:
    3935        db.create_job("Research topic", title="nightly research")
    3936    finally:
    3937        db.close()
    3938
    3939    assert _run_shell_line("status nightly research") is True
    3940
    3941    out = capsys.readouterr().out
    3942    assert "focus: nightly research" in out
    3943    assert "state: open" in out
    3944    assert "job_" not in out
    3945
    3946
    3947def test_shell_stop_job_title_pauses_job_instead_of_stopping_daemon(monkeypatch, tmp_path, capsys):
    3948    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    3949    db = AgentDB(tmp_path / "state.db")
    3950    try:
    3951        job_id = db.create_job("Research topic", title="nightly research")
    3952        db.update_job_status(job_id, "running")
    3953    finally:
    3954        db.close()
    3955
    3956    assert _run_shell_line("stop nightly research") is True
    3957
    3958    out = capsys.readouterr().out
    3959    db = AgentDB(tmp_path / "state.db")
    3960    try:
    3961        job = db.get_job(job_id)
    3962        assert "stopped nightly research" in out
    3963        assert job["status"] == "paused"
    3964        assert job["metadata"]["last_note"] == "stopped by operator"
    3965    finally:
    3966        db.close()
    3967
    3968
    3969def test_resume_clears_provider_block_before_retry(monkeypatch, tmp_path, capsys):
    3970    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    3971    db = AgentDB(tmp_path / "state.db")
    3972    try:
    3973        job_id = db.create_job("Research topic", title="nightly research")
    3974        db.update_job_status(
    3975            job_id,
    3976            "paused",
    3977            metadata_patch={
    3978                "provider_blocked_at": "2026-05-01T00:00:00+00:00",
    3979                "defer_until": "2999-01-01T00:00:00+00:00",
    3980                "defer_reason": "waiting for a monitor interval",
    3981                "defer_next_action": "check later",
    3982            },
    3983        )
    3984    finally:
    3985        db.close()
    3986
    3987    main(["resume", "nightly research"])
    3988
    3989    out = capsys.readouterr().out
    3990    db = AgentDB(tmp_path / "state.db")
    3991    try:
    3992        job = db.get_job(job_id)
    3993        assert "resumed nightly research" in out
    3994        assert job["status"] == "queued"
    3995        assert job["metadata"]["provider_blocked_at"] == ""
    3996        assert job["metadata"]["provider_unblocked_at"]
    3997        assert job["metadata"]["defer_until"] == ""
    3998        assert job["metadata"]["defer_reason"] == ""
    3999        assert job["metadata"]["defer_next_action"] == ""
    4000    finally:
    4001        db.close()
    4002
    4003
    4004def test_shell_cancel_prefers_multiword_job_title_over_note(monkeypatch, tmp_path, capsys):
    4005    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4006    db = AgentDB(tmp_path / "state.db")
    4007    try:
    4008        job_id = db.create_job("Research topic", title="nightly research")
    4009        db.update_job_status(job_id, "running")
    4010    finally:
    4011        db.close()
    4012
    4013    assert _run_shell_line("cancel nightly research") is True
    4014
    4015    out = capsys.readouterr().out
    4016    db = AgentDB(tmp_path / "state.db")
    4017    try:
    4018        job = db.get_job(job_id)
    4019        assert "cancelled nightly research" in out
    4020        assert ": finder" not in out
    4021        assert job["status"] == "cancelled"
    4022        assert "last_note" not in job["metadata"]
    4023    finally:
    4024        db.close()
    4025
    4026
    4027def test_shell_pause_splits_note_after_longest_matching_job_title(monkeypatch, tmp_path, capsys):
    4028    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4029    db = AgentDB(tmp_path / "state.db")
    4030    try:
    4031        job_id = db.create_job("Research topic", title="nightly research")
    4032        db.update_job_status(job_id, "running")
    4033    finally:
    4034        db.close()
    4035
    4036    assert _run_shell_line("pause nightly research checking costs") is True
    4037
    4038    out = capsys.readouterr().out
    4039    db = AgentDB(tmp_path / "state.db")
    4040    try:
    4041        job = db.get_job(job_id)
    4042        assert "paused nightly research: checking costs" in out
    4043        assert job["status"] == "paused"
    4044        assert job["metadata"]["last_note"] == "checking costs"
    4045    finally:
    4046        db.close()
    4047
    4048
    4049def test_chat_handle_line_adds_operator_message(monkeypatch, tmp_path, capsys):
    4050    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4051    _mark_test_model_ready()
    4052    db = AgentDB(tmp_path / "state.db")
    4053    try:
    4054        job_id = db.create_job("Research topic", title="nightly research")
    4055    finally:
    4056        db.close()
    4057
    4058    assert (
    4059        _chat_handle_line(
    4060            job_id, "prefer artifact-backed findings", reply_fn=lambda _job_id, _message: "Okay, I will focus there."
    4061        )
    4062        is True
    4063    )
    4064
    4065    out = capsys.readouterr().out
    4066    db = AgentDB(tmp_path / "state.db")
    4067    try:
    4068        job = db.get_job(job_id)
    4069        assert "waiting:" in out
    4070        assert "Okay, I will focus there." in out
    4071        assert job["metadata"]["operator_messages"][-1]["source"] == "chat"
    4072        assert job["metadata"]["operator_messages"][-1]["message"] == "prefer artifact-backed findings"
    4073        assert job["metadata"]["last_agent_update"]["category"] == "chat"
    4074    finally:
    4075        db.close()
    4076
    4077
    4078def test_chat_can_spawn_new_job_from_plain_message(monkeypatch, tmp_path, capsys):
    4079    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4080    _mark_test_model_ready()
    4081    db = AgentDB(tmp_path / "state.db")
    4082    try:
    4083        original_id = db.create_job("Research topic", title="nightly research")
    4084    finally:
    4085        db.close()
    4086    started = {}
    4087
    4088    def fake_start(**kwargs):
    4089        started.update(kwargs)
    4090
    4091    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    4092
    4093    assert (
    4094        _chat_handle_line(
    4095            original_id,
    4096            "create a job to monitor nightly benchmarks and report regressions",
    4097            reply_fn=lambda _job_id, _message: "should not call model",
    4098        )
    4099        is True
    4100    )
    4101
    4102    out = capsys.readouterr().out
    4103    db = AgentDB(tmp_path / "state.db")
    4104    try:
    4105        jobs = db.list_jobs()
    4106        assert len(jobs) == 2
    4107        created = [job for job in jobs if job["id"] != original_id][0]
    4108        assert "monitor nightly benchmarks" in created["objective"]
    4109        assert created["status"] == "queued"
    4110        assert created["metadata"]["planning_status"] == "auto_accepted"
    4111        assert "should not call model" not in out
    4112        assert "Created job" in out
    4113        assert "Started worker" in out
    4114        assert started["poll_seconds"] == 0.0
    4115        assert started["quiet"] is True
    4116    finally:
    4117        db.close()
    4118
    4119
    4120def test_workspace_chat_can_create_refined_worker_job(monkeypatch, tmp_path):
    4121    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4122    _mark_test_model_ready()
    4123    started = {}
    4124
    4125    def fake_start(**kwargs):
    4126        started.update(kwargs)
    4127
    4128    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    4129    monkeypatch.setattr(
    4130        "nipux_cli.cli._refine_job_objective_for_worker",
    4131        lambda *, message, objective: f"{objective}\n\nRefined durable objective with success criteria and artifacts.",
    4132    )
    4133
    4134    ok, message = _handle_chat_message(
    4135        WORKSPACE_CHAT_ID,
    4136        "create a job to research browser automation libraries",
    4137        quiet=True,
    4138    )
    4139
    4140    assert ok is True
    4141    assert "Created worker job" in message
    4142    assert started["quiet"] is True
    4143    db = AgentDB(tmp_path / "state.db")
    4144    try:
    4145        jobs = db.list_jobs()
    4146        assert len(jobs) == 1
    4147        assert "Refined durable objective" in jobs[0]["objective"]
    4148        snapshot = _load_frame_snapshot(WORKSPACE_CHAT_ID, history_limit=4)
    4149        bodies = "\n".join(str(event.get("body") or "") for event in snapshot["events"])
    4150        assert "create a job" in bodies
    4151        assert "Created worker job" in bodies
    4152    finally:
    4153        db.close()
    4154
    4155
    4156def test_workspace_chat_start_objective_creates_worker_without_model_reply(monkeypatch, tmp_path):
    4157    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4158    _mark_test_model_ready()
    4159    started = {}
    4160
    4161    def fake_start(**kwargs):
    4162        started.update(kwargs)
    4163
    4164    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    4165    monkeypatch.setattr(
    4166        "nipux_cli.cli._refine_job_objective_for_worker",
    4167        lambda *, message, objective: objective,
    4168    )
    4169
    4170    ok, message = _handle_workspace_chat_message("start research browser automation libraries", quiet=True)
    4171
    4172    assert ok is True
    4173    assert "Created worker job" in message
    4174    assert started["quiet"] is True
    4175    db = AgentDB(tmp_path / "state.db")
    4176    try:
    4177        jobs = db.list_jobs()
    4178        assert len(jobs) == 1
    4179        assert jobs[0]["title"] == "research browser automation libraries"
    4180        assert _read_shell_state().get("focus_job_id") == jobs[0]["id"]
    4181    finally:
    4182        db.close()
    4183
    4184
    4185def test_workspace_chat_accepts_natural_worker_and_task_phrasing(monkeypatch, tmp_path):
    4186    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4187    _mark_test_model_ready()
    4188    started = {}
    4189
    4190    def fake_start(**kwargs):
    4191        started.setdefault("calls", []).append(kwargs)
    4192
    4193    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    4194    monkeypatch.setattr(
    4195        "nipux_cli.cli._refine_job_objective_for_worker",
    4196        lambda *, message, objective: objective,
    4197    )
    4198
    4199    ok, worker_message = _handle_workspace_chat_message(
    4200        "spin up a worker to monitor docs and report changes",
    4201        quiet=True,
    4202    )
    4203    ok2, task_message = _handle_workspace_chat_message(
    4204        "run a task to audit onboarding and write a report",
    4205        quiet=True,
    4206    )
    4207
    4208    assert ok is True
    4209    assert ok2 is True
    4210    assert "Created worker job" in worker_message
    4211    assert "Created worker job" in task_message
    4212    assert len(started["calls"]) == 2
    4213    db = AgentDB(tmp_path / "state.db")
    4214    try:
    4215        titles = [job["title"] for job in db.list_jobs()]
    4216        assert "monitor docs and report changes" in titles
    4217        assert "audit onboarding and write a report" in titles
    4218    finally:
    4219        db.close()
    4220
    4221
    4222def test_chat_can_queue_new_job_without_starting(monkeypatch, tmp_path, capsys):
    4223    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4224    _mark_test_model_ready()
    4225    db = AgentDB(tmp_path / "state.db")
    4226    try:
    4227        original_id = db.create_job("Research topic", title="nightly research")
    4228    finally:
    4229        db.close()
    4230    started = {}
    4231
    4232    def fake_start(**kwargs):
    4233        started.update(kwargs)
    4234
    4235    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    4236
    4237    assert (
    4238        _chat_handle_line(
    4239            original_id,
    4240            "create only a job to monitor nightly benchmarks and report regressions",
    4241            reply_fn=lambda _job_id, _message: "should not call model",
    4242        )
    4243        is True
    4244    )
    4245
    4246    out = capsys.readouterr().out
    4247    assert "Created job" in out
    4248    assert "Started worker" not in out
    4249    assert started == {}
    4250
    4251
    4252def test_chat_can_spawn_generic_deliverable_job_from_plain_message(monkeypatch, tmp_path):
    4253    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4254    _mark_test_model_ready()
    4255    db = AgentDB(tmp_path / "state.db")
    4256    try:
    4257        original_id = db.create_job("Research topic", title="nightly research")
    4258    finally:
    4259        db.close()
    4260    started = {}
    4261
    4262    def fake_start(**kwargs):
    4263        started.update(kwargs)
    4264
    4265    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    4266
    4267    assert (
    4268        _chat_handle_line(
    4269            original_id,
    4270            "generate a polished launch checklist for this repository",
    4271            reply_fn=lambda _job_id, _message: "should not call model",
    4272        )
    4273        is True
    4274    )
    4275
    4276    db = AgentDB(tmp_path / "state.db")
    4277    try:
    4278        jobs = db.list_jobs()
    4279        assert len(jobs) == 2
    4280        created = [job for job in jobs if job["id"] != original_id][0]
    4281        assert "launch checklist" in created["objective"]
    4282        assert started["quiet"] is True
    4283    finally:
    4284        db.close()
    4285
    4286
    4287def test_chat_start_job_message_starts_daemon(monkeypatch, tmp_path, capsys):
    4288    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4289    _mark_test_model_ready()
    4290    db = AgentDB(tmp_path / "state.db")
    4291    try:
    4292        original_id = db.create_job("Research topic", title="nightly research")
    4293    finally:
    4294        db.close()
    4295    started = {}
    4296
    4297    def fake_start(**kwargs):
    4298        started.update(kwargs)
    4299
    4300    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    4301
    4302    assert (
    4303        _chat_handle_line(
    4304            original_id,
    4305            "start a job to monitor nightly benchmarks and report regressions",
    4306            reply_fn=lambda _job_id, _message: "should not call model",
    4307        )
    4308        is True
    4309    )
    4310
    4311    out = capsys.readouterr().out
    4312    assert started["poll_seconds"] == 0.0
    4313    assert started["quiet"] is True
    4314    assert "Started worker" in out
    4315
    4316
    4317def test_chat_create_job_and_run_it_starts_daemon(monkeypatch, tmp_path, capsys):
    4318    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4319    _mark_test_model_ready()
    4320    db = AgentDB(tmp_path / "state.db")
    4321    try:
    4322        original_id = db.create_job("Research topic", title="nightly research")
    4323    finally:
    4324        db.close()
    4325    started = {}
    4326
    4327    def fake_start(**kwargs):
    4328        started.update(kwargs)
    4329
    4330    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    4331
    4332    assert (
    4333        _chat_handle_line(
    4334            original_id,
    4335            "create a job to monitor nightly benchmarks and then run it",
    4336            reply_fn=lambda _job_id, _message: "should not call model",
    4337        )
    4338        is True
    4339    )
    4340
    4341    out = capsys.readouterr().out
    4342    assert started["poll_seconds"] == 0.0
    4343    assert started["quiet"] is True
    4344    assert "Started worker" in out
    4345
    4346
    4347def test_chat_jobs_command_lists_jobs_instead_of_steering(monkeypatch, tmp_path, capsys):
    4348    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4349    db = AgentDB(tmp_path / "state.db")
    4350    try:
    4351        job_id = db.create_job("Research topic", title="nightly research")
    4352    finally:
    4353        db.close()
    4354
    4355    assert _chat_handle_line(job_id, "/jobs", reply_fn=lambda _job_id, _message: "should not run") is True
    4356
    4357    out = capsys.readouterr().out
    4358    db = AgentDB(tmp_path / "state.db")
    4359    try:
    4360        job = db.get_job(job_id)
    4361        assert "nightly research" in out
    4362        assert "should not run" not in out
    4363        assert job["metadata"].get("operator_messages") is None
    4364    finally:
    4365        db.close()
    4366
    4367
    4368def test_chat_command_inside_chat_is_not_queued(monkeypatch, tmp_path, capsys):
    4369    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4370    db = AgentDB(tmp_path / "state.db")
    4371    try:
    4372        job_id = db.create_job("Research topic", title="nightly research")
    4373    finally:
    4374        db.close()
    4375
    4376    assert (
    4377        _chat_handle_line(job_id, 'chat "nightly research"', reply_fn=lambda _job_id, _message: "should not run")
    4378        is True
    4379    )
    4380
    4381    out = capsys.readouterr().out
    4382    db = AgentDB(tmp_path / "state.db")
    4383    try:
    4384        job = db.get_job(job_id)
    4385        assert "already chatting with nightly research" in out
    4386        assert "should not run" not in out
    4387        assert job["metadata"].get("operator_messages") is None
    4388    finally:
    4389        db.close()
    4390
    4391
    4392def test_chat_run_accepts_initial_plan_before_starting(monkeypatch, tmp_path):
    4393    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4394    _mark_test_model_ready()
    4395    parser = build_parser()
    4396    args = parser.parse_args(["create", "Research new topic", "--title", "new research", "--kind", "generic"])
    4397    args.func(args)
    4398    job_id = "new-research"
    4399    captured = {}
    4400
    4401    def fake_run(run_args):
    4402        captured["job_id"] = run_args.job_id
    4403
    4404    monkeypatch.setattr("nipux_cli.cli.cmd_run", fake_run)
    4405
    4406    assert _chat_handle_line(job_id, "/run") is True
    4407
    4408    db = AgentDB(tmp_path / "state.db")
    4409    try:
    4410        job = db.get_job(job_id)
    4411        assert job["status"] == "queued"
    4412        assert captured["job_id"] == job_id
    4413    finally:
    4414        db.close()
    4415
    4416
    4417def test_run_without_jobs_does_not_start_empty_daemon(monkeypatch, tmp_path, capsys):
    4418    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4419    _mark_test_model_ready()
    4420    started = {}
    4421
    4422    def fake_start(**kwargs):
    4423        started.update(kwargs)
    4424
    4425    monkeypatch.setattr("nipux_cli.cli._start_daemon_if_needed", fake_start)
    4426
    4427    assert _run_shell_line("run") is True
    4428
    4429    out = capsys.readouterr().out
    4430    assert "No jobs found. Create one with /new OBJECTIVE." in out
    4431    assert started == {}
    4432
    4433
    4434def test_build_chat_messages_includes_recent_job_state(tmp_path):
    4435    db = AgentDB(tmp_path / "state.db")
    4436    try:
    4437        job_id = db.create_job("Research topic", title="nightly research", kind="generic")
    4438        db.create_job("Monitor another branch", title="other branch", kind="generic")
    4439        run_id = db.start_run(job_id, model="fake")
    4440        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="web_search")
    4441        db.finish_step(step_id, status="completed", summary="web_search returned useful sources")
    4442        job = db.get_job(job_id)
    4443
    4444        messages = _build_chat_messages(db, job, "what is going on?")
    4445
    4446        content = messages[-1]["content"]
    4447        assert "Job title: nightly research" in content
    4448        assert "Jobs:" in content
    4449        assert "* 1. nightly research" in content
    4450        assert "- 2. other branch" in content
    4451        assert "web_search returned useful sources" in content
    4452        assert "what is going on?" in content
    4453    finally:
    4454        db.close()
    4455
    4456
    4457def test_build_chat_messages_includes_durable_outcome_summary(tmp_path):
    4458    db = AgentDB(tmp_path / "state.db")
    4459    try:
    4460        job_id = db.create_job("Research topic", title="nightly research", kind="generic")
    4461        db.append_event(job_id=job_id, event_type="artifact", title="First draft", body="saved report", metadata={})
    4462        db.append_event(job_id=job_id, event_type="finding", title="Evidence map", body="", metadata={})
    4463        db.append_event(
    4464            job_id=job_id,
    4465            event_type="experiment",
    4466            title="Citation coverage",
    4467            body="",
    4468            metadata={"metric_name": "citations", "metric_value": 12, "metric_unit": "count"},
    4469        )
    4470        job = db.get_job(job_id)
    4471
    4472        messages = _build_chat_messages(db, job, "what has it actually done?")
    4473
    4474        content = messages[-1]["content"]
    4475        assert "Durable outcomes:" in content
    4476        assert "summary: 1 outputs 1 findings 1 measurements" in content
    4477        assert "save: First draft" in content
    4478        assert "find: Evidence map" in content
    4479        assert "test: Citation coverage" in content
    4480    finally:
    4481        db.close()
    4482
    4483
    4484def test_build_chat_messages_does_not_include_local_machine_context(monkeypatch, tmp_path):
    4485    monkeypatch.setenv("HOME", str(tmp_path))
    4486    ssh_dir = tmp_path / ".ssh"
    4487    ssh_dir.mkdir()
    4488    (ssh_dir / "config").write_text("Host private-box\n  HostName 10.9.8.7\n  User private\n", encoding="utf-8")
    4489    db = AgentDB(tmp_path / "state.db")
    4490    try:
    4491        job_id = db.create_job("Research topic", title="nightly research", kind="generic")
    4492        job = db.get_job(job_id)
    4493
    4494        messages = _build_chat_messages(db, job, "what is going on?")
    4495
    4496        content = messages[-1]["content"]
    4497        assert "Local CLI context" not in content
    4498        assert "private-box" not in content
    4499        assert "10.9.8.7" not in content
    4500    finally:
    4501        db.close()
    4502
    4503
    4504def test_build_chat_messages_points_to_artifact_and_lessons(tmp_path):
    4505    db = AgentDB(tmp_path / "state.db")
    4506    try:
    4507        job_id = db.create_job("Research topic", title="nightly research", kind="generic")
    4508        run_id = db.start_run(job_id, model="fake")
    4509        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
    4510        ArtifactStore(tmp_path, db=db).write_text(
    4511            job_id=job_id,
    4512            run_id=run_id,
    4513            step_id=step_id,
    4514            title="Findings Batch",
    4515            summary="15 reusable findings",
    4516            content="Acme",
    4517        )
    4518        db.append_lesson(job_id, "Prefer actual evidence sources over low-evidence pages.", category="strategy")
    4519        job = db.get_job(job_id)
    4520
    4521        messages = _build_chat_messages(db, job, "where are the findings?")
    4522
    4523        content = messages[-1]["content"]
    4524        assert "/artifact 1" in content
    4525        assert "Prefer actual evidence sources over low-evidence pages" in content
    4526    finally:
    4527        db.close()
    4528
    4529
    4530def test_build_chat_messages_clip_large_visible_state(tmp_path):
    4531    db = AgentDB(tmp_path / "state.db")
    4532    try:
    4533        job_id = db.create_job("Research topic", title="nightly research", kind="generic")
    4534        for index in range(30):
    4535            db.append_event(
    4536                job_id=job_id,
    4537                event_type="finding",
    4538                title=f"large finding {index}",
    4539                body="evidence " * 400,
    4540                metadata={},
    4541            )
    4542        job = db.get_job(job_id)
    4543
    4544        messages = _build_chat_messages(db, job, "keep this exact operator question visible")
    4545
    4546        content = messages[-1]["content"]
    4547        assert len(content) < 14_000
    4548        assert "clipped" in content
    4549        assert "keep this exact operator question visible" in content
    4550    finally:
    4551        db.close()
    4552
    4553
    4554def test_artifact_command_resolves_title_query(monkeypatch, tmp_path, capsys):
    4555    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4556    db = AgentDB(tmp_path / "state.db")
    4557    try:
    4558        job_id = db.create_job("Research topic", title="nightly research")
    4559        run_id = db.start_run(job_id, model="fake")
    4560        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
    4561        ArtifactStore(tmp_path, db=db).write_text(
    4562            job_id=job_id,
    4563            run_id=run_id,
    4564            step_id=step_id,
    4565            title="Findings Batch",
    4566            summary="saved findings",
    4567            content="Acme Corp\n",
    4568        )
    4569    finally:
    4570        db.close()
    4571
    4572    parser = build_parser()
    4573    args = parser.parse_args(["artifact", "Findings", "Batch"])
    4574    args.func(args)
    4575
    4576    out = capsys.readouterr().out
    4577    assert "artifact: Findings Batch" in out
    4578    assert "Acme Corp" in out
    4579
    4580
    4581def test_artifacts_command_prints_compact_view_command(monkeypatch, tmp_path, capsys):
    4582    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4583    db = AgentDB(tmp_path / "state.db")
    4584    try:
    4585        job_id = db.create_job("Research topic", title="nightly research")
    4586        run_id = db.start_run(job_id, model="fake")
    4587        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
    4588        ArtifactStore(tmp_path, db=db).write_text(
    4589            job_id=job_id,
    4590            run_id=run_id,
    4591            step_id=step_id,
    4592            title="Findings Batch",
    4593            summary="saved findings",
    4594            content="Acme Corp\n",
    4595        )
    4596    finally:
    4597        db.close()
    4598
    4599    parser = build_parser()
    4600    args = parser.parse_args(["artifacts"])
    4601    args.func(args)
    4602
    4603    out = capsys.readouterr().out
    4604    assert "saved outputs nightly research" in out
    4605    assert "view: artifact 1" in out
    4606    assert "/jobs/" not in out
    4607
    4608
    4609def test_artifact_command_opens_recent_output_by_number(monkeypatch, tmp_path, capsys):
    4610    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4611    db = AgentDB(tmp_path / "state.db")
    4612    try:
    4613        job_id = db.create_job("Research topic", title="nightly research")
    4614        run_id = db.start_run(job_id, model="fake")
    4615        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
    4616        ArtifactStore(tmp_path, db=db).write_text(
    4617            job_id=job_id,
    4618            run_id=run_id,
    4619            step_id=step_id,
    4620            title="Findings Batch",
    4621            summary="saved findings",
    4622            content="Acme Corp\n",
    4623        )
    4624    finally:
    4625        db.close()
    4626
    4627    parser = build_parser()
    4628    args = parser.parse_args(["artifact", "1"])
    4629    args.func(args)
    4630
    4631    out = capsys.readouterr().out
    4632    assert "artifact: Findings Batch" in out
    4633    assert "Acme Corp" in out
    4634
    4635
    4636def test_chat_work_defaults_to_compact_output(monkeypatch, tmp_path):
    4637    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4638    db = AgentDB(tmp_path / "state.db")
    4639    try:
    4640        job_id = db.create_job("Research topic", title="nightly research")
    4641    finally:
    4642        db.close()
    4643    captured = {}
    4644
    4645    def fake_work(args):
    4646        captured["verbose"] = args.verbose
    4647        captured["chars"] = args.chars
    4648
    4649    monkeypatch.setattr("nipux_cli.cli.cmd_work", fake_work)
    4650
    4651    assert _chat_handle_line(job_id, "/work") is True
    4652
    4653    assert captured == {"verbose": False, "chars": 260}
    4654
    4655
    4656def test_chat_learn_adds_lesson(monkeypatch, tmp_path, capsys):
    4657    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4658    db = AgentDB(tmp_path / "state.db")
    4659    try:
    4660        job_id = db.create_job("Research topic", title="nightly research")
    4661    finally:
    4662        db.close()
    4663
    4664    assert _chat_handle_line(job_id, "/learn low-evidence pages are not research findings") is True
    4665
    4666    out = capsys.readouterr().out
    4667    db = AgentDB(tmp_path / "state.db")
    4668    try:
    4669        job = db.get_job(job_id)
    4670        assert "learned for nightly research" in out
    4671        assert job["metadata"]["last_lesson"]["lesson"] == "low-evidence pages are not research findings"
    4672    finally:
    4673        db.close()
    4674
    4675
    4676def test_chat_follow_queues_follow_up_message(monkeypatch, tmp_path, capsys):
    4677    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4678    db = AgentDB(tmp_path / "state.db")
    4679    try:
    4680        job_id = db.create_job("Research topic", title="nightly research")
    4681    finally:
    4682        db.close()
    4683
    4684    assert _chat_handle_line(job_id, "/follow after this branch, check another source") is True
    4685
    4686    out = capsys.readouterr().out
    4687    db = AgentDB(tmp_path / "state.db")
    4688    try:
    4689        job = db.get_job(job_id)
    4690        message = job["metadata"]["operator_messages"][-1]
    4691        assert "waiting after current branch" in out
    4692        assert message["mode"] == "follow_up"
    4693        assert message["message"] == "after this branch, check another source"
    4694    finally:
    4695        db.close()
    4696
    4697
    4698def test_findings_sources_memory_metrics_commands(monkeypatch, tmp_path, capsys):
    4699    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4700    db = AgentDB(tmp_path / "state.db")
    4701    try:
    4702        job_id = db.create_job("Research topic", title="nightly research")
    4703        db.append_finding_record(job_id, name="Acme Finding", category="example category", score=0.8)
    4704        db.append_task_record(job_id, title="Explore primary sources", status="open", priority=5)
    4705        db.append_experiment_record(job_id, title="Variant A", status="measured", metric_name="score", metric_value=1.5)
    4706        db.append_source_record(job_id, "https://example.com", usefulness_score=0.9, yield_count=1)
    4707        db.append_lesson(job_id, "Source indexes work.", category="strategy")
    4708        db.append_reflection(job_id, "Keep using source indexes.", strategy="Try primary records.")
    4709        db.append_memory_graph_records(
    4710            job_id,
    4711            nodes=[
    4712                {
    4713                    "key": "source-indexes-work",
    4714                    "kind": "strategy",
    4715                    "title": "Source indexes work",
    4716                    "summary": "Use source indexes when they produce durable records.",
    4717                    "salience": 0.8,
    4718                    "confidence": 0.9,
    4719                }
    4720            ],
    4721        )
    4722    finally:
    4723        db.close()
    4724
    4725    parser = build_parser()
    4726    for command in (["findings"], ["tasks"], ["experiments"], ["sources"], ["memory"], ["metrics"]):
    4727        args = parser.parse_args(command)
    4728        args.func(args)
    4729
    4730    out = capsys.readouterr().out
    4731    assert "Acme Finding" in out
    4732    assert "Explore primary sources" in out
    4733    assert "Variant A" in out
    4734    assert "https://example.com" in out
    4735    assert "Keep using source indexes" in out
    4736    assert "graph_nodes=1" in out
    4737    assert "Source indexes work" in out
    4738    assert "tasks: 1" in out
    4739    assert "experiments: 1" in out
    4740    assert "findings: 1" in out
    4741
    4742
    4743def test_memory_graph_html_command_writes_clickable_artifact(monkeypatch, tmp_path, capsys):
    4744    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4745    db = AgentDB(tmp_path / "state.db")
    4746    try:
    4747        job_id = db.create_job("Research topic", title="nightly research")
    4748        db.append_memory_graph_records(
    4749            job_id,
    4750            nodes=[
    4751                {
    4752                    "key": "validated-loop",
    4753                    "kind": "skill",
    4754                    "status": "stable",
    4755                    "title": "Validated loop",
    4756                    "summary": "Check progress against measured evidence before expanding scope.",
    4757                    "tags": ["validation", "progress"],
    4758                    "evidence_refs": ["artifact:report"],
    4759                    "salience": 0.9,
    4760                    "confidence": 0.8,
    4761                },
    4762                {
    4763                    "key": "open-risk",
    4764                    "kind": "question",
    4765                    "status": "open",
    4766                    "title": "Open risk",
    4767                    "summary": "Needs another validation pass.",
    4768                },
    4769            ],
    4770            edges=[{"from_key": "validated-loop", "to_key": "open-risk", "relation": "raises"}],
    4771        )
    4772    finally:
    4773        db.close()
    4774
    4775    args = build_parser().parse_args(["memory", "--graph"])
    4776    args.func(args)
    4777
    4778    out = capsys.readouterr().out
    4779    assert "memory graph written:" in out
    4780    db = AgentDB(tmp_path / "state.db")
    4781    try:
    4782        artifacts = db.list_artifacts(job_id)
    4783        assert artifacts[0]["type"] == "html"
    4784        html = Path(artifacts[0]["path"]).read_text(encoding="utf-8")
    4785        assert "<canvas id=\"graph\"" in html
    4786        assert "click a node" in html
    4787        assert "Validated loop" in html
    4788        assert "open-risk" in html
    4789    finally:
    4790        db.close()
    4791
    4792
    4793def test_shell_natural_update_phrase_shows_updates(monkeypatch, tmp_path, capsys):
    4794    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4795    db = AgentDB(tmp_path / "state.db")
    4796    try:
    4797        db.create_job("Research topic", title="research")
    4798    finally:
    4799        db.close()
    4800
    4801    assert _run_shell_line("tell me updates") is True
    4802    assert _run_shell_line("show outcomes") is True
    4803
    4804    out = capsys.readouterr().out
    4805    assert "updates" in out
    4806    assert "queued for" not in out
    4807
    4808
    4809def test_updates_command_summarizes_durable_outcomes(monkeypatch, tmp_path, capsys):
    4810    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4811    db = AgentDB(tmp_path / "state.db")
    4812    try:
    4813        job_id = db.create_job("Research topic", title="research")
    4814        artifact_path = tmp_path / "artifact.md"
    4815        artifact_path.write_text("saved", encoding="utf-8")
    4816        db.append_event(
    4817            job_id,
    4818            event_type="tool_call",
    4819            title="web_search",
    4820            metadata={"input": {"arguments": {"query": "raw search"}}},
    4821        )
    4822        db.append_finding_record(job_id, name="Durable Result", category="evidence", reason="real outcome", score=0.7)
    4823        db.add_artifact(
    4824            job_id=job_id,
    4825            path=artifact_path,
    4826            sha256="abc",
    4827            artifact_type="text",
    4828            title="Saved Report",
    4829            summary="durable output",
    4830        )
    4831    finally:
    4832        db.close()
    4833
    4834    args = build_parser().parse_args(["updates", "research", "--limit", "3", "--chars", "120"])
    4835    args.func(args)
    4836
    4837    out = capsys.readouterr().out
    4838    assert "outcomes by hour:" in out
    4839    assert "Durable Result" in out
    4840    assert "Saved Report" in out
    4841    assert "latest saved outputs:" in out
    4842    assert "raw tool stream: activity" in out
    4843    assert "recent tool calls:" not in out
    4844
    4845
    4846def test_updates_all_summarizes_durable_work_across_jobs(monkeypatch, tmp_path, capsys):
    4847    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4848    db = AgentDB(tmp_path / "state.db")
    4849    try:
    4850        first_id = db.create_job("Research first topic", title="first")
    4851        second_id = db.create_job("Research second topic", title="second")
    4852        first_path = tmp_path / "first.md"
    4853        first_path.write_text("first", encoding="utf-8")
    4854        second_path = tmp_path / "second.md"
    4855        second_path.write_text("second", encoding="utf-8")
    4856        db.append_finding_record(first_id, name="First durable finding", category="evidence")
    4857        db.add_artifact(
    4858            job_id=first_id,
    4859            path=first_path,
    4860            sha256="abc",
    4861            artifact_type="text",
    4862            title="First saved output",
    4863            summary="first summary",
    4864        )
    4865        db.append_experiment_record(
    4866            second_id,
    4867            title="Second measured result",
    4868            status="measured",
    4869            metric_name="quality",
    4870            metric_value=9,
    4871            metric_unit="points",
    4872        )
    4873        db.add_artifact(
    4874            job_id=second_id,
    4875            path=second_path,
    4876            sha256="def",
    4877            artifact_type="text",
    4878            title="Second saved output",
    4879            summary="second summary",
    4880        )
    4881    finally:
    4882        db.close()
    4883
    4884    args = build_parser().parse_args(["outcomes", "--all", "--limit", "5", "--chars", "120"])
    4885    args.func(args)
    4886
    4887    out = capsys.readouterr().out
    4888    assert "outcomes all jobs | 2 tracked" in out
    4889    assert "first |" in out
    4890    assert "second |" in out
    4891    assert "First durable finding" in out
    4892    assert "First saved output" in out
    4893    assert "Second measured result" in out
    4894    assert "Second saved output" in out
    4895
    4896
    4897def test_history_and_events_commands_render_visible_timeline(monkeypatch, tmp_path, capsys):
    4898    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4899    db = AgentDB(tmp_path / "state.db")
    4900    try:
    4901        job_id = db.create_job("Research topic", title="research")
    4902        db.append_operator_message(job_id, "operator timeline note", source="test")
    4903        db.append_agent_update(job_id, "agent timeline note", category="chat")
    4904    finally:
    4905        db.close()
    4906
    4907    parser = build_parser()
    4908    parser.parse_args(["history", "research"]).func(parser.parse_args(["history", "research"]))
    4909    parser.parse_args(["events", "research"]).func(parser.parse_args(["events", "research"]))
    4910
    4911    out = capsys.readouterr().out
    4912    assert "history research" in out
    4913    assert "events research" in out
    4914    assert "operator timeline note" in out
    4915    assert "agent timeline note" in out
    4916
    4917
    4918def test_shell_natural_health_phrase_shows_health(monkeypatch, tmp_path, capsys):
    4919    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4920    db = AgentDB(tmp_path / "state.db")
    4921    try:
    4922        db.create_job("Research topic", title="research")
    4923    finally:
    4924        db.close()
    4925
    4926    assert _run_shell_line("is it running") is True
    4927
    4928    out = capsys.readouterr().out
    4929    assert "Nipux Health" in out
    4930    assert "queued for" not in out
    4931
    4932
    4933def test_health_prints_recent_daemon_events(monkeypatch, tmp_path, capsys):
    4934    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4935
    4936    config = load_config()
    4937    append_daemon_event(config, "daemon_error", error_type="RuntimeError", error="provider fell over")
    4938
    4939    parser = build_parser()
    4940    args = parser.parse_args(["health", "--limit", "3"])
    4941    args.func(args)
    4942
    4943    out = capsys.readouterr().out
    4944    assert "Nipux Health" in out
    4945    assert "daemon_error" in out
    4946    assert "RuntimeError" in out
    4947
    4948
    4949def test_launch_agent_plist_contains_daemon_command(monkeypatch, tmp_path):
    4950    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4951
    4952    plist = _launch_agent_plist(poll_seconds=7, quiet=True)
    4953
    4954    assert "com.nipux.agent" in plist
    4955    assert "<string>daemon</string>" in plist
    4956    assert "<string>--poll-seconds</string>" in plist
    4957    assert "<string>7</string>" in plist
    4958    assert str(tmp_path) in plist
    4959
    4960
    4961def test_systemd_service_text_contains_daemon_command(monkeypatch, tmp_path):
    4962    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
    4963
    4964    service = _systemd_service_text(poll_seconds=0, quiet=True)
    4965
    4966    assert "[Service]" in service
    4967    assert "ExecStart=" in service
    4968    assert "daemon --poll-seconds 0" in service
    4969    assert f"Environment=NIPUX_HOME={tmp_path}" in service
    4970    assert "Restart=always" in service

tests/nipux_cli/test_cli_model_preflight.py 86 lines

       1from types import SimpleNamespace
       2
       3from nipux_cli.cli import _ensure_remote_model_ready_for_worker, build_parser
       4from nipux_cli.doctor import Check
       5
       6
       7def _config(base_url: str):
       8    return SimpleNamespace(
       9        model=SimpleNamespace(
      10            model="provider/model",
      11            base_url=base_url,
      12            api_key="",
      13            api_key_env="TEST_API_KEY",
      14        )
      15    )
      16
      17
      18def test_remote_model_preflight_blocks_rejected_auth(monkeypatch, capsys):
      19    def fake_doctor(*, config, check_model):
      20        assert check_model is True
      21        return [Check("model_auth", False, "OpenRouter rejected API key: User not found")]
      22
      23    monkeypatch.setattr("nipux_cli.cli.run_doctor", fake_doctor)
      24
      25    assert _ensure_remote_model_ready_for_worker(_config("https://openrouter.ai/api/v1"), fake=False) is False
      26
      27    out = capsys.readouterr().out
      28    assert "model is not ready; daemon not started" in out
      29    assert "model_auth: OpenRouter rejected API key" in out
      30    assert "doctor --check-model" in out
      31
      32
      33def test_remote_model_preflight_allows_recovery_monitor_for_quota(monkeypatch, capsys):
      34    def fake_doctor(*, config, check_model):
      35        assert check_model is True
      36        return [Check("model_generation", False, "Key limit exceeded (total limit) (code=403)")]
      37
      38    monkeypatch.setattr("nipux_cli.cli.run_doctor", fake_doctor)
      39
      40    assert _ensure_remote_model_ready_for_worker(_config("https://openrouter.ai/api/v1"), fake=False) is True
      41
      42    out = capsys.readouterr().out
      43    assert "recovery monitor mode" in out
      44    assert "Key limit exceeded" in out
      45
      46
      47def test_remote_model_preflight_skips_fake_runs(monkeypatch):
      48    def fake_doctor(*, config, check_model):
      49        raise AssertionError("fake runs should not need remote model auth")
      50
      51    monkeypatch.setattr("nipux_cli.cli.run_doctor", fake_doctor)
      52
      53    assert _ensure_remote_model_ready_for_worker(_config("https://openrouter.ai/api/v1"), fake=True) is True
      54
      55
      56def test_model_preflight_checks_local_endpoints(monkeypatch):
      57    called = {}
      58
      59    def fake_doctor(*, config, check_model):
      60        called["check_model"] = check_model
      61        return []
      62
      63    monkeypatch.setattr("nipux_cli.cli.run_doctor", fake_doctor)
      64
      65    assert _ensure_remote_model_ready_for_worker(_config("http://localhost:11434/v1"), fake=False) is True
      66    assert called["check_model"] is True
      67
      68
      69def test_start_does_not_spawn_daemon_when_model_preflight_fails(monkeypatch, tmp_path):
      70    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
      71    checked = {}
      72
      73    def fake_ready(config, *, fake):
      74        checked["fake"] = fake
      75        return False
      76
      77    def fake_popen(*args, **kwargs):
      78        raise AssertionError("daemon should not spawn when model preflight fails")
      79
      80    monkeypatch.setattr("nipux_cli.cli._ensure_remote_model_ready_for_worker", fake_ready)
      81    monkeypatch.setattr("subprocess.Popen", fake_popen)
      82
      83    args = build_parser().parse_args(["start", "--quiet"])
      84    args.func(args)
      85
      86    assert checked["fake"] is False

tests/nipux_cli/test_compression.py 101 lines

       1from nipux_cli.compression import refresh_memory_index
       2from nipux_cli.db import AgentDB
       3
       4
       5def test_refresh_memory_index_includes_durable_progress_ledgers(tmp_path):
       6    db = AgentDB(tmp_path / "state.db")
       7    try:
       8        job_id = db.create_job(
       9            "Keep improving a report",
      10            title="report",
      11            metadata={
      12                "task_queue": [
      13                    {
      14                        "title": "Draft evidence-backed section",
      15                        "status": "active",
      16                        "priority": 10,
      17                        "output_contract": "report",
      18                    }
      19                ],
      20                "finding_ledger": [{"name": "Teacher traces improve tool use"}],
      21                "source_ledger": [{"source": "https://example.test/paper", "usefulness_score": 0.8}],
      22                "experiment_ledger": [
      23                    {
      24                        "title": "Citation density check",
      25                        "status": "measured",
      26                        "metric_name": "citations",
      27                        "metric_value": 12,
      28                        "metric_unit": "count",
      29                    }
      30                ],
      31                "roadmap": {
      32                    "title": "Research paper roadmap",
      33                    "status": "active",
      34                    "current_milestone": "Improve literature review",
      35                },
      36                "memory_graph": {
      37                    "nodes": [
      38                        {
      39                            "title": "Validated evidence loop",
      40                            "kind": "strategy",
      41                            "status": "active",
      42                            "summary": "Evidence-backed checkpoints should drive the next branch.",
      43                            "salience": 0.9,
      44                        }
      45                    ],
      46                    "edges": [
      47                        {
      48                            "from_key": "validated-evidence-loop",
      49                            "to_key": "research-paper-roadmap",
      50                            "relation": "supports",
      51                        }
      52                    ],
      53                },
      54                "pending_measurement_obligation": {
      55                    "source_step_no": 42,
      56                    "tool": "shell_exec",
      57                    "summary": "benchmark output needs accounting",
      58                    "metric_candidates": ["latency 120ms", "throughput 9 req/s"],
      59                },
      60            },
      61        )
      62        db.append_event(
      63            job_id,
      64            event_type="loop",
      65            title="message_end",
      66            metadata={
      67                "usage": {
      68                    "prompt_tokens": 1200,
      69                    "completion_tokens": 300,
      70                    "total_tokens": 1500,
      71                    "estimated": True,
      72                    "context_length": 1600,
      73                    "context_fraction": 0.75,
      74                }
      75            },
      76        )
      77
      78        refresh_memory_index(db, job_id)
      79
      80        memory = db.list_memory(job_id)[0]["summary"]
      81        assert "Durable progress ledgers:" in memory
      82        assert "tasks=1" in memory
      83        assert "findings=1" in memory
      84        assert "sources=1" in memory
      85        assert "experiments=1" in memory
      86        assert "memory_nodes=1" in memory
      87        assert "Validated evidence loop" in memory
      88        assert "memory_links=1" in memory
      89        assert "Draft evidence-backed section" in memory
      90        assert "Citation density check" in memory
      91        assert "Teacher traces improve tool use" in memory
      92        assert "Research paper roadmap" in memory
      93        assert "pending_measurement step=#42 tool=shell_exec" in memory
      94        assert "latency 120ms" in memory
      95        assert "Model usage:" in memory
      96        assert "total_tokens=1.5K" in memory
      97        assert "estimated_calls=1" in memory
      98        assert "context_pressure" in memory
      99        assert "latest_context=1.2K/1.6K" in memory
     100    finally:
     101        db.close()

tests/nipux_cli/test_config.py 143 lines

       1from pathlib import Path
       2
       3from nipux_cli.config import DEFAULT_CONTEXT_LENGTH, default_config_yaml, load_config
       4
       5
       6def _mode(path):
       7    return path.stat().st_mode & 0o777
       8
       9
      10def test_load_config_defaults_to_local_endpoint(tmp_path, monkeypatch):
      11    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
      12
      13    config = load_config()
      14
      15    assert config.runtime.home == tmp_path
      16    assert config.model.model == "local-model"
      17    assert config.model.base_url == "http://localhost:8000/v1"
      18    assert config.model.api_key_env == "OPENAI_API_KEY"
      19    assert config.model.context_length == DEFAULT_CONTEXT_LENGTH
      20    assert config.runtime.state_db_path == tmp_path / "state.db"
      21    assert config.runtime.daily_digest_enabled is True
      22    assert config.runtime.daily_digest_time == "08:00"
      23    assert config.runtime.max_job_cost_usd is None
      24    assert config.tools.browser is True
      25    assert config.tools.web is True
      26    assert config.tools.shell is True
      27    assert config.tools.files is True
      28
      29
      30def test_load_config_from_yaml(tmp_path, monkeypatch):
      31    monkeypatch.setenv("NIPUX_HOME", str(tmp_path / "home"))
      32    cfg = tmp_path / "config.yaml"
      33    cfg.write_text(
      34        """
      35model:
      36  name: local-test
      37  base_url: http://127.0.0.1:9999/v1/
      38  context_length: 12345
      39  input_cost_per_million: 0.1
      40  output_cost_per_million: 0.2
      41runtime:
      42  home: ./agent-home
      43  max_step_seconds: 42
      44  max_job_cost_usd: 12.5
      45  daily_digest_enabled: false
      46  daily_digest_time: "07:30"
      47tools:
      48  browser: false
      49  web: true
      50  shell: false
      51  files: true
      52email:
      53  enabled: true
      54  to_addr: kai@example.com
      55""",
      56        encoding="utf-8",
      57    )
      58
      59    config = load_config(cfg)
      60
      61    assert config.model.model == "local-test"
      62    assert config.model.base_url == "http://127.0.0.1:9999/v1"
      63    assert config.model.context_length == 12345
      64    assert config.model.input_cost_per_million == 0.1
      65    assert config.model.output_cost_per_million == 0.2
      66    assert config.runtime.home == Path("./agent-home")
      67    assert config.runtime.max_step_seconds == 42
      68    assert config.runtime.max_job_cost_usd == 12.5
      69    assert config.runtime.daily_digest_enabled is False
      70    assert config.runtime.daily_digest_time == "07:30"
      71    assert config.tools.browser is False
      72    assert config.tools.web is True
      73    assert config.tools.shell is False
      74    assert config.tools.files is True
      75    assert config.email.enabled is True
      76    assert config.email.to_addr == "kai@example.com"
      77
      78
      79def test_load_config_reads_local_env_file(tmp_path, monkeypatch):
      80    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
      81    monkeypatch.delenv("OPENROUTER_API_KEY", raising=False)
      82    (tmp_path / ".env").write_text("OPENROUTER_API_KEY" + "=secret-test-key\n", encoding="utf-8")
      83    (tmp_path / "config.yaml").write_text(
      84        """
      85model:
      86  name: provider/test-model
      87  base_url: https://openrouter.ai/api/v1
      88  api_key_env: OPENROUTER_API_KEY
      89""",
      90        encoding="utf-8",
      91    )
      92
      93    config = load_config()
      94
      95    assert config.model.api_key == "secret-test-key"
      96
      97
      98def test_load_config_tightens_local_env_permissions(tmp_path, monkeypatch):
      99    monkeypatch.setenv("NIPUX_HOME", str(tmp_path))
     100    monkeypatch.delenv("OPENROUTER_API_KEY", raising=False)
     101    env_path = tmp_path / ".env"
     102    env_path.write_text("OPENROUTER_API_KEY" + "=secret-test-key\n", encoding="utf-8")
     103    env_path.chmod(0o644)
     104
     105    load_config()
     106
     107    assert _mode(env_path) == 0o600
     108
     109
     110def test_default_config_yaml_allows_provider_template_without_secret():
     111    text = default_config_yaml(
     112        model="provider/model",
     113        base_url="https://openrouter.ai/api/v1/",
     114        api_key_env="OPENROUTER_API_KEY",
     115        context_length=8192,
     116    )
     117
     118    assert "name: provider/model" in text
     119    assert "base_url: https://openrouter.ai/api/v1" in text
     120    assert "api_key_env: OPENROUTER_API_KEY" in text
     121    assert "context_length: 8192" in text
     122    assert "input_cost_per_million: null" in text
     123    assert "output_cost_per_million: null" in text
     124    assert "max_job_cost_usd: null" in text
     125    assert "tools:" in text
     126    assert "browser: true" in text
     127    assert "shell: true" in text
     128    assert "sk-" not in text
     129
     130
     131def test_config_example_matches_default_local_endpoint():
     132    root = Path(__file__).resolve().parents[2]
     133    text = (root / "config.example.yaml").read_text(encoding="utf-8")
     134
     135    assert "name: local-model" in text
     136    assert "base_url: http://localhost:8000/v1" in text
     137    assert "api_key_env: OPENAI_API_KEY" in text
     138    assert "input_cost_per_million: null" in text
     139    assert "output_cost_per_million: null" in text
     140    assert "max_job_cost_usd: null" in text
     141    assert "tools:" in text
     142    assert "browser: true" in text
     143    assert "shell: true" in text

tests/nipux_cli/test_daemon.py 560 lines

       1from datetime import datetime, timedelta, timezone
       2import json
       3import threading
       4import time
       5
       6import pytest
       7
       8from nipux_cli.config import AppConfig, RuntimeConfig
       9from nipux_cli.daemon import (
      10    Daemon,
      11    DaemonAlreadyRunning,
      12    RUNTIME_CODE_FILES,
      13    append_daemon_event,
      14    current_runtime_fingerprint,
      15    daemon_lock_status,
      16    read_daemon_events,
      17    runtime_stale,
      18    single_instance_lock,
      19    update_lock_metadata,
      20    _exception_backoff,
      21    _parse_retry_after,
      22    _step_failure_backoff,
      23)
      24from nipux_cli.daemon_control import stop_daemon_process_impl
      25from nipux_cli.db import AgentDB
      26from nipux_cli.worker import StepExecution
      27from nipux_cli.doctor import Check
      28
      29
      30def test_single_instance_lock_rejects_second_holder(tmp_path):
      31    lock_path = tmp_path / "agentd.lock"
      32    with single_instance_lock(lock_path):
      33        with pytest.raises(DaemonAlreadyRunning):
      34            with single_instance_lock(lock_path):
      35                pass
      36
      37
      38def test_daemon_lock_status_reports_free_lock(tmp_path):
      39    status = daemon_lock_status(tmp_path / "agentd.lock")
      40
      41    assert status["running"] is False
      42    assert status["detail"] == "daemon lock is free"
      43
      44
      45def test_lock_metadata_can_be_updated_while_held(tmp_path):
      46    lock_path = tmp_path / "agentd.lock"
      47    with single_instance_lock(lock_path) as handle:
      48        update_lock_metadata(handle, last_state="step", consecutive_failures=2)
      49        status = daemon_lock_status(lock_path)
      50
      51    assert status["running"] is True
      52    assert status["metadata"]["last_state"] == "step"
      53    assert status["metadata"]["consecutive_failures"] == 2
      54    assert status["metadata"]["pid"]
      55    assert status["metadata"]["started_at"]
      56
      57
      58def test_lock_metadata_update_restores_missing_process_fields(tmp_path):
      59    lock_path = tmp_path / "agentd.lock"
      60    with single_instance_lock(lock_path) as handle:
      61        handle.seek(0)
      62        handle.truncate()
      63        handle.write(json.dumps({"last_state": "idle"}))
      64        handle.flush()
      65
      66        update_lock_metadata(handle, last_state="step")
      67        status = daemon_lock_status(lock_path)
      68
      69    assert status["running"] is True
      70    assert status["metadata"]["pid"]
      71    assert status["metadata"]["started_at"]
      72    assert status["metadata"]["last_state"] == "step"
      73
      74
      75def test_daemon_lock_heartbeat_updates_while_worker_turn_runs(monkeypatch, tmp_path):
      76    monkeypatch.setattr("nipux_cli.daemon.WORK_HEARTBEAT_INTERVAL_SECONDS", 0.01)
      77    monkeypatch.setattr("nipux_cli.daemon.signal.getsignal", lambda _sig: None)
      78    monkeypatch.setattr("nipux_cli.daemon.signal.signal", lambda _sig, _handler: None)
      79
      80    class SlowDaemon(Daemon):
      81        def run_once(self, *, fake: bool = False, verbose: bool = False):  # noqa: ARG002
      82            time.sleep(0.2)
      83            return None
      84
      85    config = AppConfig(runtime=RuntimeConfig(home=tmp_path, daily_digest_enabled=False))
      86    db = AgentDB(tmp_path / "state.db")
      87    try:
      88        daemon = SlowDaemon(config=config, db=db)
      89        thread = threading.Thread(
      90            target=daemon.run_forever,
      91            kwargs={"poll_seconds": 0, "quiet": True, "max_iterations": 1},
      92            daemon=True,
      93        )
      94        thread.start()
      95        seen_working: dict | None = None
      96        deadline = time.time() + 1.0
      97        while time.time() < deadline:
      98            status = daemon_lock_status(tmp_path / "agentd.lock")
      99            metadata = status.get("metadata") or {}
     100            if status.get("running") and metadata.get("last_state") == "working":
     101                seen_working = metadata
     102                break
     103            time.sleep(0.01)
     104
     105        thread.join(timeout=2.0)
     106
     107        assert seen_working is not None
     108        assert seen_working["last_heartbeat"]
     109        assert seen_working["runtime"]["runtime_hash"]
     110        assert not thread.is_alive()
     111    finally:
     112        db.close()
     113
     114
     115def test_stop_daemon_recovers_pidless_lock_from_process_list(tmp_path, monkeypatch):
     116    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     117    lock_path = tmp_path / "agentd.lock"
     118    killed = []
     119
     120    class PsResult:
     121        returncode = 0
     122        stdout = " 99999 /venv/bin/python -m nipux_cli.cli daemon --poll-seconds 0.0 --quiet\n"
     123
     124    monkeypatch.setattr("nipux_cli.daemon_control.subprocess.run", lambda *_args, **_kwargs: PsResult())
     125    monkeypatch.setattr("nipux_cli.daemon_control.os.kill", lambda pid, sig: killed.append((pid, sig)))
     126
     127    with single_instance_lock(lock_path) as handle:
     128        handle.seek(0)
     129        handle.truncate()
     130        handle.write(json.dumps({"last_state": "idle"}))
     131        handle.flush()
     132
     133        stopped = stop_daemon_process_impl(config, wait=0.1, quiet=True, pid_alive=lambda _pid: False)
     134
     135    assert stopped is True
     136    assert killed and killed[0][0] == 99999
     137
     138
     139def test_daemon_lock_status_detects_stale_runtime(tmp_path):
     140    lock_path = tmp_path / "agentd.lock"
     141    with single_instance_lock(lock_path) as handle:
     142        update_lock_metadata(handle, runtime={"runtime_hash": "old"})
     143        status = daemon_lock_status(lock_path)
     144
     145    assert status["running"] is True
     146    assert status["stale"] is True
     147    assert status["current_runtime"]["code_hash"]
     148    assert status["current_runtime"]["code_mtime"]
     149    assert runtime_stale({"runtime": {"runtime_hash": "old"}}) is True
     150    assert runtime_stale({"runtime": current_runtime_fingerprint()}) is False
     151
     152
     153def test_runtime_fingerprint_tracks_progress_code():
     154    assert "progress.py" in RUNTIME_CODE_FILES
     155    assert "parser_builder.py" in RUNTIME_CODE_FILES
     156
     157
     158def test_rate_limit_backoff_uses_retry_after_header():
     159    class RateLimit(Exception):
     160        status_code = 429
     161        response = type("Response", (), {"headers": {"Retry-After": "42"}})()
     162
     163    assert _exception_backoff(RateLimit("too many requests"), poll_seconds=0, consecutive_failures=1) == 42
     164
     165
     166def test_rate_limit_backoff_has_conservative_fallback():
     167    class RateLimit(Exception):
     168        status_code = 429
     169
     170    assert _exception_backoff(RateLimit("rate limit exceeded"), poll_seconds=0, consecutive_failures=1) == 10
     171
     172
     173def test_failed_step_provider_config_error_uses_normal_backoff():
     174    result = StepExecution(
     175        job_id="job",
     176        run_id="run",
     177        step_id="step",
     178        tool_name=None,
     179        status="failed",
     180        result={
     181            "error_type": "PermissionDeniedError",
     182            "error": "Error code: 403 - key limit exceeded",
     183        },
     184    )
     185
     186    assert _step_failure_backoff(result, poll_seconds=0, consecutive_failures=1) == 1
     187
     188
     189def test_failed_tool_auth_error_uses_normal_backoff():
     190    result = StepExecution(
     191        job_id="job",
     192        run_id="run",
     193        step_id="step",
     194        tool_name="shell_exec",
     195        status="failed",
     196        result={
     197            "error": "command output indicates authentication or authorization failure: permission denied",
     198        },
     199    )
     200
     201    assert _step_failure_backoff(result, poll_seconds=3, consecutive_failures=1) == 3
     202
     203
     204def test_failed_step_rate_limit_uses_normal_backoff():
     205    result = StepExecution(
     206        job_id="job",
     207        run_id="run",
     208        step_id="step",
     209        tool_name=None,
     210        status="failed",
     211        result={
     212            "error_type": "RateLimitError",
     213            "error": "429 too many requests",
     214        },
     215    )
     216
     217    assert _step_failure_backoff(result, poll_seconds=0, consecutive_failures=1) == 1
     218
     219
     220def test_failed_step_provider_timeout_uses_normal_backoff():
     221    result = StepExecution(
     222        job_id="job",
     223        run_id="run",
     224        step_id="step",
     225        tool_name=None,
     226        status="failed",
     227        result={
     228            "error_type": "APITimeoutError",
     229            "error": "Request timed out.",
     230        },
     231    )
     232
     233    assert _step_failure_backoff(result, poll_seconds=0, consecutive_failures=1) == 1
     234
     235
     236def test_retry_after_parses_epoch_milliseconds():
     237    future_ms = str(int((time.time() + 5) * 1000))
     238
     239    parsed = _parse_retry_after(future_ms)
     240
     241    assert parsed is not None
     242    assert 0 <= parsed <= 6
     243
     244
     245def test_daemon_run_once_claims_next_job_with_fake_step(tmp_path):
     246    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     247    db = AgentDB(tmp_path / "state.db")
     248    try:
     249        job_id = db.create_job("Run forever in small steps")
     250        daemon = Daemon(config=config, db=db)
     251
     252        result = daemon.run_once(fake=True)
     253
     254        assert result is not None
     255        assert result.job_id == job_id
     256        assert result.status == "completed"
     257        assert db.list_artifacts(job_id)[0]["title"] == "daemon-fake-step"
     258    finally:
     259        db.close()
     260
     261
     262def test_daemon_ignores_ui_focus_for_worker_scheduling(tmp_path):
     263    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     264    db = AgentDB(tmp_path / "state.db")
     265    try:
     266        first = db.create_job("First job", title="first")
     267        second = db.create_job("Second job", title="second")
     268        (tmp_path / "shell_state.json").write_text(json.dumps({"focus_job_id": second}), encoding="utf-8")
     269        daemon = Daemon(config=config, db=db)
     270
     271        job = daemon.next_runnable_job()
     272
     273        assert first != second
     274        assert job is not None
     275        assert job["id"] == first
     276    finally:
     277        db.close()
     278
     279
     280def test_daemon_skips_deferred_jobs_until_due(tmp_path):
     281    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     282    db = AgentDB(tmp_path / "state.db")
     283    try:
     284        deferred = db.create_job("Deferred job", title="deferred")
     285        ready = db.create_job("Ready job", title="ready")
     286        db.update_job_status(
     287            deferred,
     288            "queued",
     289            metadata_patch={"defer_until": (datetime.now(timezone.utc) + timedelta(hours=1)).isoformat()},
     290        )
     291        daemon = Daemon(config=config, db=db)
     292
     293        job = daemon.next_runnable_job()
     294
     295        assert job is not None
     296        assert job["id"] == ready
     297    finally:
     298        db.close()
     299
     300
     301def test_daemon_quarantines_provider_blocked_jobs(tmp_path):
     302    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     303    db = AgentDB(tmp_path / "state.db")
     304    try:
     305        blocked = db.create_job("Provider blocked job", title="blocked")
     306        ready = db.create_job("Ready job", title="ready")
     307        db.update_job_status(
     308            blocked,
     309            "running",
     310            metadata_patch={"provider_blocked_at": datetime.now(timezone.utc).isoformat()},
     311        )
     312        daemon = Daemon(config=config, db=db)
     313
     314        job = daemon.next_runnable_job()
     315
     316        assert job is not None
     317        assert job["id"] == ready
     318        blocked_job = db.get_job(blocked)
     319        assert blocked_job["status"] == "paused"
     320        assert "provider" in blocked_job["metadata"]["last_note"].lower()
     321        events = db.list_events(job_id=blocked, limit=10)
     322        assert any(event["event_type"] == "agent_message" and event["metadata"].get("reason") == "llm_provider_blocked" for event in events)
     323    finally:
     324        db.close()
     325
     326
     327def test_daemon_leaves_provider_blocked_job_paused_until_model_recovers(monkeypatch, tmp_path):
     328    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     329    db = AgentDB(tmp_path / "state.db")
     330    try:
     331        blocked_at = (datetime.now(timezone.utc) - timedelta(hours=1)).isoformat()
     332        job_id = db.create_job("Provider blocked job", title="blocked")
     333        db.update_job_status(
     334            job_id,
     335            "paused",
     336            metadata_patch={"provider_blocked_at": blocked_at},
     337        )
     338
     339        def fake_doctor(*, config, check_model):
     340            assert check_model is True
     341            return [Check("model_generation", False, "key limit exceeded")]
     342
     343        monkeypatch.setattr("nipux_cli.daemon.run_doctor", fake_doctor)
     344        daemon = Daemon(config=config, db=db)
     345
     346        assert daemon.next_runnable_job() is None
     347        job = db.get_job(job_id)
     348        assert job["status"] == "paused"
     349        assert job["metadata"]["provider_last_probe_detail"].startswith("model_generation")
     350        assert read_daemon_events(config, limit=1)[0]["event"] == "provider_recovery_wait"
     351    finally:
     352        db.close()
     353
     354
     355def test_daemon_resumes_provider_blocked_job_when_model_recovers(monkeypatch, tmp_path):
     356    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     357    db = AgentDB(tmp_path / "state.db")
     358    try:
     359        blocked_at = (datetime.now(timezone.utc) - timedelta(hours=1)).isoformat()
     360        job_id = db.create_job("Provider blocked job", title="blocked")
     361        db.update_job_status(
     362            job_id,
     363            "paused",
     364            metadata_patch={"provider_blocked_at": blocked_at},
     365        )
     366
     367        def fake_doctor(*, config, check_model):
     368            assert check_model is True
     369            return []
     370
     371        monkeypatch.setattr("nipux_cli.daemon.run_doctor", fake_doctor)
     372        daemon = Daemon(config=config, db=db)
     373
     374        job = daemon.next_runnable_job()
     375        stored = db.get_job(job_id)
     376
     377        assert job is not None
     378        assert job["id"] == job_id
     379        assert stored["status"] == "queued"
     380        assert stored["metadata"]["provider_unblocked_at"]
     381        events = db.list_events(job_id=job_id, limit=10)
     382        assert any(event["event_type"] == "agent_message" and event["metadata"].get("reason") == "llm_provider_recovered" for event in events)
     383    finally:
     384        db.close()
     385
     386
     387def test_daemon_idle_sleep_wakes_for_deferred_job(tmp_path):
     388    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     389    db = AgentDB(tmp_path / "state.db")
     390    try:
     391        now = datetime.now(timezone.utc)
     392        job_id = db.create_job("Deferred job", title="deferred")
     393        db.update_job_status(
     394            job_id,
     395            "queued",
     396            metadata_patch={"defer_until": (now + timedelta(seconds=2)).isoformat()},
     397        )
     398        daemon = Daemon(config=config, db=db)
     399
     400        sleep_seconds = daemon.idle_sleep_seconds(poll_seconds=30, now=now)
     401
     402        assert 1.9 <= sleep_seconds <= 2.1
     403    finally:
     404        db.close()
     405
     406
     407def test_daemon_idle_sleep_uses_poll_when_no_deferred_jobs(tmp_path):
     408    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     409    db = AgentDB(tmp_path / "state.db")
     410    try:
     411        db.create_job("Ready job", title="ready")
     412        daemon = Daemon(config=config, db=db)
     413
     414        assert daemon.idle_sleep_seconds(poll_seconds=30) == 30
     415        assert daemon.idle_sleep_seconds(poll_seconds=0) == 5.0
     416    finally:
     417        db.close()
     418
     419
     420def test_daemon_advances_multiple_runnable_jobs_without_focus_starvation(tmp_path):
     421    config = AppConfig(runtime=RuntimeConfig(home=tmp_path, daily_digest_enabled=False))
     422    db = AgentDB(tmp_path / "state.db")
     423    try:
     424        first = db.create_job("First job", title="first")
     425        second = db.create_job("Second job", title="second")
     426        (tmp_path / "shell_state.json").write_text(json.dumps({"focus_job_id": second}), encoding="utf-8")
     427        daemon = Daemon(config=config, db=db)
     428
     429        daemon.run_forever(poll_seconds=0, quiet=True, max_iterations=4, fake=True)
     430
     431        assert db.list_steps(job_id=first)
     432        assert db.list_steps(job_id=second)
     433    finally:
     434        db.close()
     435
     436
     437def test_daemon_writes_due_daily_digest_once(tmp_path):
     438    config = AppConfig(runtime=RuntimeConfig(home=tmp_path, daily_digest_time="00:00"))
     439    db = AgentDB(tmp_path / "state.db")
     440    try:
     441        db.create_job("Keep finding findings", title="findings")
     442        daemon = Daemon(config=config, db=db)
     443        now = datetime(2026, 4, 23, 8, 30)
     444
     445        first = daemon.send_due_daily_digest(now=now)
     446        second = daemon.send_due_daily_digest(now=now)
     447
     448        assert first is not None
     449        assert first["status"] == "dry_run"
     450        assert second is None
     451        assert (tmp_path / "digests" / "2026-04-23-daily.md").exists()
     452    finally:
     453        db.close()
     454
     455
     456def test_daemon_event_log_round_trips_jsonl(tmp_path):
     457    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     458
     459    path = append_daemon_event(config, "step", job_id="job_1", status="completed")
     460    events = read_daemon_events(config, limit=3)
     461
     462    assert path.name == "daemon-events.jsonl"
     463    assert events[-1]["event"] == "step"
     464    assert events[-1]["job_id"] == "job_1"
     465
     466
     467def test_daemon_recovers_stale_running_steps_on_start(tmp_path):
     468    config = AppConfig(runtime=RuntimeConfig(home=tmp_path, daily_digest_enabled=False))
     469    db = AgentDB(tmp_path / "state.db")
     470    try:
     471        job_id = db.create_job("Recover stale work", title="stale")
     472        run_id = db.start_run(job_id, model="fake")
     473        stale_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="browser_navigate")
     474        daemon = Daemon(config=config, db=db)
     475
     476        daemon.run_forever(poll_seconds=0, quiet=True, max_iterations=1, fake=True)
     477
     478        steps = db.list_steps(job_id=job_id)
     479        stale = next(step for step in steps if step["id"] == stale_step)
     480        events = read_daemon_events(config, limit=5)
     481        assert stale["status"] == "failed"
     482        assert stale["error"] == "daemon recovered abandoned running work from a previous process"
     483        assert db.list_runs(job_id, limit=10)[-1]["status"] == "failed"
     484        assert any(event.get("event") == "stale_work_recovered" for event in events)
     485    finally:
     486        db.close()
     487
     488
     489def test_daemon_survives_unexpected_step_exception(tmp_path):
     490    class ExplodingDaemon(Daemon):
     491        def run_once(self, *, fake: bool = False, verbose: bool = False):  # noqa: ARG002
     492            raise RuntimeError("provider fell over")
     493
     494    config = AppConfig(runtime=RuntimeConfig(home=tmp_path, daily_digest_enabled=False))
     495    db = AgentDB(tmp_path / "state.db")
     496    try:
     497        daemon = ExplodingDaemon(config=config, db=db)
     498
     499        daemon.run_forever(poll_seconds=0, quiet=True, max_iterations=1)
     500
     501        status = daemon_lock_status(tmp_path / "agentd.lock")
     502        events = read_daemon_events(config, limit=5)
     503        assert status["metadata"]["last_state"] == "error"
     504        assert status["metadata"]["consecutive_failures"] == 1
     505        assert any(event.get("event") == "daemon_error" for event in events)
     506    finally:
     507        db.close()
     508
     509
     510def test_daemon_treats_blocked_steps_as_recoverable(tmp_path):
     511    class BlockedDaemon(Daemon):
     512        def run_once(self, *, fake: bool = False, verbose: bool = False):  # noqa: ARG002
     513            return StepExecution(
     514                job_id="job",
     515                run_id="run",
     516                step_id="step",
     517                tool_name="web_search",
     518                status="blocked",
     519                result={"error": "search loop blocked", "recoverable": True},
     520            )
     521
     522    config = AppConfig(runtime=RuntimeConfig(home=tmp_path, daily_digest_enabled=False))
     523    db = AgentDB(tmp_path / "state.db")
     524    try:
     525        daemon = BlockedDaemon(config=config, db=db)
     526
     527        daemon.run_forever(poll_seconds=0, quiet=True, max_iterations=3)
     528
     529        status = daemon_lock_status(tmp_path / "agentd.lock")
     530        events = read_daemon_events(config, limit=10)
     531        assert status["metadata"]["consecutive_failures"] == 0
     532        assert sum(1 for event in events if event.get("event") == "step") == 3
     533        assert not any(event.get("event") == "daemon_error" for event in events)
     534    finally:
     535        db.close()
     536
     537
     538def test_fake_daemon_can_run_100_iterations_without_auto_stop(tmp_path):
     539    config = AppConfig(runtime=RuntimeConfig(home=tmp_path, daily_digest_enabled=False))
     540    db = AgentDB(tmp_path / "state.db")
     541    try:
     542        job_id = db.create_job("Run a long fake worker", title="long")
     543        daemon = Daemon(config=config, db=db)
     544
     545        daemon.run_forever(poll_seconds=0, quiet=True, max_iterations=100, fake=True)
     546
     547        steps = db.list_steps(job_id=job_id)
     548        assert len(steps) == 100
     549        assert any(step["kind"] == "reflection" for step in steps)
     550        assert db.list_artifacts(job_id)
     551        memory = db.list_memory(job_id)
     552        assert memory
     553        assert memory[0]["key"] == "rolling_state"
     554        assert "Recent steps:" in memory[0]["summary"]
     555        assert db.get_job(job_id)["status"] in {"queued", "running"}
     556        step_events = [event for event in read_daemon_events(config, limit=120) if event.get("event") == "step"]
     557        assert len(step_events) == 100
     558        assert daemon_lock_status(tmp_path / "agentd.lock")["running"] is False
     559    finally:
     560        db.close()

tests/nipux_cli/test_dashboard.py 86 lines

       1from datetime import datetime, timedelta, timezone
       2
       3from nipux_cli.artifacts import ArtifactStore
       4from nipux_cli.config import AppConfig, RuntimeConfig
       5from nipux_cli.dashboard import collect_dashboard_state, render_dashboard, render_overview
       6from nipux_cli.db import AgentDB
       7
       8
       9def test_dashboard_collects_jobs_steps_and_artifacts(tmp_path):
      10    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
      11    db = AgentDB(tmp_path / "state.db")
      12    try:
      13        job_id = db.create_job("Research topic every morning", title="research", kind="generic")
      14        run_id = db.start_run(job_id, model="fake-model")
      15        step_id = db.add_step(
      16            job_id=job_id,
      17            run_id=run_id,
      18            kind="tool",
      19            tool_name="write_artifact",
      20            input_data={"arguments": {"title": "Findings"}},
      21        )
      22        ArtifactStore(tmp_path, db=db).write_text(
      23            job_id=job_id,
      24            run_id=run_id,
      25            step_id=step_id,
      26            title="Findings",
      27            summary="first saved finding",
      28            content="Acme Corp",
      29        )
      30        db.finish_step(step_id, status="completed", summary="saved finding", output_data={"success": True})
      31        db.finish_run(run_id, "completed")
      32        db.append_lesson(job_id, "Low-evidence summaries are not finding batches.", category="source_quality")
      33        db.append_task_record(job_id, title="Explore primary sources", status="open", priority=5)
      34
      35        state = collect_dashboard_state(db, config, job_id=job_id)
      36        rendered = render_dashboard(state, width=100)
      37        overview = render_overview(state, width=100)
      38
      39        assert state["daemon"]["running"] is False
      40        assert state["focus"]["counts"]["artifacts"] == 1
      41        assert state["focus"]["counts"]["tasks"] == 1
      42        assert "Nipux CLI Dashboard" in rendered
      43        assert "research" in rendered
      44        assert "write_artifact" in rendered
      45        assert "Findings" in rendered
      46        assert "Low-evidence summaries are not finding batches" in rendered
      47        assert "Explore primary sources" in rendered
      48        assert "Nipux Status" in overview
      49        assert "latest artifact: Findings" in overview
      50        assert "latest lesson:" in overview
      51    finally:
      52        db.close()
      53
      54
      55def test_overview_marks_idle_daemon_as_ready_for_work(tmp_path):
      56    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
      57    db = AgentDB(tmp_path / "state.db")
      58    try:
      59        db.create_job("Research topic", title="research")
      60        state = collect_dashboard_state(db, config)
      61        overview = render_overview(state, width=100)
      62
      63        assert "ready when work starts" in overview
      64    finally:
      65        db.close()
      66
      67
      68def test_overview_marks_old_heartbeat_as_busy_for_running_step(tmp_path):
      69    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
      70    db = AgentDB(tmp_path / "state.db")
      71    try:
      72        job_id = db.create_job("Measure a process", title="measure")
      73        run_id = db.start_run(job_id, model="fake-model")
      74        db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec", status="running")
      75        state = collect_dashboard_state(db, config, job_id=job_id)
      76        state["daemon"]["running"] = True
      77        state["daemon"]["metadata"] = {
      78            "last_heartbeat": (datetime.now(timezone.utc) - timedelta(seconds=180)).isoformat(),
      79        }
      80
      81        overview = render_overview(state, width=100)
      82
      83        assert "busy #1 shell_exec" in overview
      84        assert "heartbeat 180s ago (stale)" not in overview
      85    finally:
      86        db.close()

tests/nipux_cli/test_db.py 609 lines

       1from nipux_cli.db import AgentDB
       2
       3
       4def test_db_job_run_step_and_artifact_roundtrip(tmp_path):
       5    db = AgentDB(tmp_path / "state.db")
       6    try:
       7        job_id = db.create_job("Research topic every day", title="research", kind="generic")
       8        assert job_id == "research"
       9        job = db.get_job(job_id)
      10        assert job["status"] == "queued"
      11        assert job["kind"] == "generic"
      12
      13        run_id = db.start_run(job_id, model="local-test-model")
      14        step_id = db.add_step(
      15            job_id=job_id,
      16            run_id=run_id,
      17            kind="tool",
      18            tool_name="write_artifact",
      19            input_data={"x": 1},
      20        )
      21        db.finish_step(step_id, status="completed", summary="wrote artifact", output_data={"ok": True})
      22        artifact_id = db.add_artifact(
      23            job_id=job_id,
      24            run_id=run_id,
      25            step_id=step_id,
      26            path=tmp_path / "artifact.md",
      27            sha256="abc",
      28            artifact_type="text",
      29            title="A",
      30        )
      31        db.finish_run(run_id, "completed")
      32
      33        assert db.get_job(job_id)["status"] == "running"
      34        assert db.list_steps(run_id=run_id)[0]["output"]["ok"] is True
      35        assert db.list_runs(job_id)[0]["id"] == run_id
      36        assert db.get_artifact(artifact_id)["title"] == "A"
      37        assert db.list_artifacts(job_id)[0]["id"] == artifact_id
      38    finally:
      39        db.close()
      40
      41
      42def test_create_job_uses_unique_readable_slug_ids(tmp_path):
      43    db = AgentDB(tmp_path / "state.db")
      44    try:
      45        first = db.create_job("Research topic", title="Nightly Research")
      46        second = db.create_job("Research more topics", title="Nightly Research")
      47
      48        assert first == "nightly-research"
      49        assert second == "nightly-research-2"
      50    finally:
      51        db.close()
      52
      53
      54def test_step_numbers_increment_across_runs_for_a_job(tmp_path):
      55    db = AgentDB(tmp_path / "state.db")
      56    try:
      57        job_id = db.create_job("Long job")
      58        run_1 = db.start_run(job_id, model="fake")
      59        step_1 = db.add_step(job_id=job_id, run_id=run_1, kind="tool")
      60        db.finish_step(step_1, status="completed")
      61        db.finish_run(run_1, "completed")
      62
      63        run_2 = db.start_run(job_id, model="fake")
      64        step_2 = db.add_step(job_id=job_id, run_id=run_2, kind="tool")
      65
      66        steps = db.list_steps(job_id=job_id)
      67        assert step_2 != step_1
      68        assert [step["step_no"] for step in steps] == [1, 2]
      69    finally:
      70        db.close()
      71
      72
      73def test_job_token_usage_aggregates_message_usage(tmp_path):
      74    db = AgentDB(tmp_path / "state.db")
      75    try:
      76        job_id = db.create_job("Long job")
      77        db.append_event(
      78            job_id,
      79            event_type="loop",
      80            title="message_end",
      81            metadata={
      82                "usage": {
      83                    "prompt_tokens": 100,
      84                    "completion_tokens": 25,
      85                    "total_tokens": 125,
      86                    "cost": 0.001,
      87                    "prompt_tokens_details": {"cached_tokens": 10},
      88                    "completion_tokens_details": {"reasoning_tokens": 3},
      89                }
      90            },
      91        )
      92        db.append_event(
      93            job_id,
      94            event_type="loop",
      95            title="message_end",
      96            metadata={
      97                "usage": {
      98                    "prompt_tokens": 150,
      99                    "completion_tokens": 50,
     100                    "total_tokens": 200,
     101                    "estimated": True,
     102                    "context_length": 1000,
     103                    "context_fraction": 0.15,
     104                }
     105            },
     106        )
     107
     108        usage = db.job_token_usage(job_id)
     109
     110        assert usage["prompt_tokens"] == 250
     111        assert usage["completion_tokens"] == 75
     112        assert usage["total_tokens"] == 325
     113        assert usage["latest_prompt_tokens"] == 150
     114        assert usage["cost"] == 0.001
     115        assert usage["has_cost"] is True
     116        assert usage["estimated_calls"] == 1
     117        assert usage["reasoning_tokens"] == 3
     118        assert usage["cached_tokens"] == 10
     119        assert usage["latest_context_length"] == 1000
     120        assert usage["latest_context_fraction"] == 0.15
     121    finally:
     122        db.close()
     123
     124
     125def test_append_operator_message_roundtrip(tmp_path):
     126    db = AgentDB(tmp_path / "state.db")
     127    try:
     128        job_id = db.create_job("Research topic")
     129
     130        entry = db.append_operator_message(job_id, "Focus on artifact-backed findings", source="shell")
     131        job = db.get_job(job_id)
     132
     133        assert entry["message"] == "Focus on artifact-backed findings"
     134        assert job["metadata"]["operator_messages"][0]["source"] == "shell"
     135        assert job["metadata"]["operator_messages"][0]["mode"] == "steer"
     136        assert job["metadata"]["operator_messages"][0]["message"] == "Focus on artifact-backed findings"
     137        assert job["metadata"]["last_operator_message"]["message"] == "Focus on artifact-backed findings"
     138        events = db.list_timeline_events(job_id)
     139        assert events[-1]["event_type"] == "operator_message"
     140        assert events[-1]["body"] == "Focus on artifact-backed findings"
     141    finally:
     142        db.close()
     143
     144
     145def test_claim_operator_messages_marks_one_message_at_a_time(tmp_path):
     146    db = AgentDB(tmp_path / "state.db")
     147    try:
     148        job_id = db.create_job("Research topic")
     149        first = db.append_operator_message(job_id, "first steer", source="chat")
     150        db.append_operator_message(job_id, "second steer", source="chat")
     151
     152        claimed = db.claim_operator_messages(job_id, modes=("steer",), limit=1)
     153        second_claim = db.claim_operator_messages(job_id, modes=("steer",), limit=1)
     154
     155        job = db.get_job(job_id)
     156        messages = job["metadata"]["operator_messages"]
     157        events = db.list_timeline_events(job_id, limit=20)
     158
     159        assert [item["message"] for item in claimed] == ["first steer"]
     160        assert claimed[0]["event_id"] == first["event_id"]
     161        assert [item["message"] for item in second_claim] == ["second steer"]
     162        assert all(message.get("claimed_at") for message in messages)
     163        assert any(event["event_type"] == "loop" and event["title"] == "steering claimed" for event in events)
     164    finally:
     165        db.close()
     166
     167
     168def test_acknowledge_operator_messages_marks_delivered_context(tmp_path):
     169    db = AgentDB(tmp_path / "state.db")
     170    try:
     171        job_id = db.create_job("Research topic")
     172        entry = db.append_operator_message(job_id, "correct the target before continuing", source="chat")
     173        db.claim_operator_messages(job_id, modes=("steer",), limit=1)
     174
     175        result = db.acknowledge_operator_messages(
     176            job_id,
     177            message_ids=[entry["event_id"]],
     178            summary="target correction incorporated",
     179        )
     180
     181        job = db.get_job(job_id)
     182        message = job["metadata"]["operator_messages"][0]
     183        events = db.list_timeline_events(job_id, limit=20)
     184
     185        assert result["count"] == 1
     186        assert message["acknowledged_at"]
     187        assert job["metadata"]["last_operator_context_ack"]["summary"] == "target correction incorporated"
     188        assert any(event["event_type"] == "operator_context" for event in events)
     189    finally:
     190        db.close()
     191
     192
     193def test_rename_job_updates_title_without_changing_id(tmp_path):
     194    db = AgentDB(tmp_path / "state.db")
     195    try:
     196        job_id = db.create_job("Research topic", title="old title")
     197
     198        renamed = db.rename_job(job_id, "new title")
     199        job = db.get_job(job_id)
     200
     201        assert renamed["id"] == job_id
     202        assert renamed["title"] == "new title"
     203        assert job["title"] == "new title"
     204    finally:
     205        db.close()
     206
     207
     208def test_delete_job_removes_related_rows(tmp_path):
     209    db = AgentDB(tmp_path / "state.db")
     210    try:
     211        job_id = db.create_job("Research topic", title="delete me")
     212        run_id = db.start_run(job_id, model="fake")
     213        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
     214        artifact_path = tmp_path / "artifact.md"
     215        artifact_path.write_text("artifact", encoding="utf-8")
     216        db.add_artifact(
     217            job_id=job_id,
     218            run_id=run_id,
     219            step_id=step_id,
     220            path=artifact_path,
     221            sha256="abc",
     222            artifact_type="text",
     223            title="Artifact",
     224        )
     225        db.upsert_memory(job_id=job_id, key="rolling_state", summary="summary")
     226
     227        result = db.delete_job(job_id)
     228
     229        assert result["job"]["title"] == "delete me"
     230        assert result["counts"]["runs"] == 1
     231        assert result["counts"]["steps"] == 1
     232        assert result["counts"]["artifacts"] == 1
     233        assert result["counts"]["memory"] == 1
     234        try:
     235            db.get_job(job_id)
     236        except KeyError:
     237            pass
     238        else:
     239            raise AssertionError("job still exists after delete")
     240        assert db.list_steps(job_id=job_id) == []
     241        assert db.list_artifacts(job_id) == []
     242        assert db.list_memory(job_id) == []
     243    finally:
     244        db.close()
     245
     246
     247def test_append_lesson_roundtrip(tmp_path):
     248    db = AgentDB(tmp_path / "state.db")
     249    try:
     250        job_id = db.create_job("Research topic")
     251
     252        entry = db.append_lesson(
     253            job_id,
     254            "Low-evidence pages are not useful evidence sources.",
     255            category="source_quality",
     256            confidence=0.9,
     257        )
     258        job = db.get_job(job_id)
     259
     260        assert entry["category"] == "source_quality"
     261        assert job["metadata"]["lessons"][0]["lesson"] == "Low-evidence pages are not useful evidence sources."
     262        assert job["metadata"]["last_lesson"]["confidence"] == 0.9
     263    finally:
     264        db.close()
     265
     266
     267def test_append_lesson_dedupes_repeated_memory(tmp_path):
     268    db = AgentDB(tmp_path / "state.db")
     269    try:
     270        job_id = db.create_job("Research topic")
     271
     272        first = db.append_lesson(job_id, "Use primary source indexes.", category="strategy", metadata={"step": 1})
     273        second = db.append_lesson(job_id, "Use primary source indexes.", category="strategy", metadata={"step": 2})
     274        job = db.get_job(job_id)
     275
     276        assert first["lesson"] == second["lesson"]
     277        assert len(job["metadata"]["lessons"]) == 1
     278        assert job["metadata"]["lessons"][0]["seen_count"] == 2
     279        assert job["metadata"]["lessons"][0]["metadata"]["step"] == 2
     280        assert first["created"] is True
     281        assert second["created"] is False
     282        assert second["substantive_update"] is False
     283        assert len(db.list_events(job_id=job_id, event_types=["lesson"])) == 1
     284    finally:
     285        db.close()
     286
     287
     288def test_source_and_finding_ledgers_dedupe_and_update(tmp_path):
     289    db = AgentDB(tmp_path / "state.db")
     290    try:
     291        job_id = db.create_job("Research topic")
     292
     293        source = db.append_source_record(
     294            job_id,
     295            "https://example.com/source",
     296            source_type="web_source",
     297            usefulness_score=0.7,
     298            yield_count=3,
     299            outcome="yielded reusable findings",
     300        )
     301        updated_source = db.append_source_record(
     302            job_id,
     303            "https://example.com/source",
     304            usefulness_score=0.9,
     305            yield_count=2,
     306            fail_count_delta=1,
     307        )
     308        finding = db.append_finding_record(
     309            job_id,
     310            name="Acme Finding",
     311            url="https://acme.example",
     312            category="example category",
     313            score=0.8,
     314        )
     315        updated_finding = db.append_finding_record(
     316            job_id,
     317            name="Acme Finding",
     318            url="https://acme.example",
     319            contact="source note",
     320            score=0.85,
     321        )
     322        reflection = db.append_reflection(job_id, "Keep using source indexes", strategy="Prioritize primary records")
     323        job = db.get_job(job_id)
     324
     325        assert source["key"] == updated_source["key"]
     326        assert source["created"] is True
     327        assert updated_source["created"] is False
     328        assert updated_source["yield_count"] == 5
     329        assert updated_source["fail_count"] == 1
     330        assert finding["created"] is True
     331        assert updated_finding["created"] is False
     332        assert updated_finding["contact"] == "source note"
     333        assert len(job["metadata"]["source_ledger"]) == 1
     334        assert len(job["metadata"]["finding_ledger"]) == 1
     335        assert job["metadata"]["last_reflection"]["summary"] == reflection["summary"]
     336    finally:
     337        db.close()
     338
     339
     340def test_repeated_source_and_finding_records_mark_non_substantive_touches(tmp_path):
     341    db = AgentDB(tmp_path / "state.db")
     342    try:
     343        job_id = db.create_job("Research topic")
     344
     345        db.append_source_record(
     346            job_id,
     347            "https://example.com/source",
     348            source_type="web_source",
     349            usefulness_score=0.7,
     350            outcome="yielded reusable findings",
     351        )
     352        repeated_source = db.append_source_record(
     353            job_id,
     354            "https://example.com/source",
     355            source_type="web_source",
     356            usefulness_score=0.7,
     357            outcome="yielded reusable findings",
     358        )
     359        changed_source = db.append_source_record(
     360            job_id,
     361            "https://example.com/source",
     362            source_type="web_source",
     363            usefulness_score=0.8,
     364            outcome="yielded reusable findings",
     365        )
     366
     367        db.append_finding_record(job_id, name="Acme Finding", url="https://acme.example", score=0.8)
     368        repeated_finding = db.append_finding_record(job_id, name="Acme Finding", url="https://acme.example", score=0.8)
     369        changed_finding = db.append_finding_record(
     370            job_id,
     371            name="Acme Finding",
     372            url="https://acme.example",
     373            score=0.9,
     374        )
     375
     376        assert repeated_source["created"] is False
     377        assert repeated_source["substantive_update"] is False
     378        assert changed_source["substantive_update"] is True
     379        assert repeated_finding["created"] is False
     380        assert repeated_finding["substantive_update"] is False
     381        assert changed_finding["substantive_update"] is True
     382    finally:
     383        db.close()
     384
     385
     386def test_task_queue_dedupes_and_updates(tmp_path):
     387    db = AgentDB(tmp_path / "state.db")
     388    try:
     389        job_id = db.create_job("Research topic")
     390
     391        first = db.append_task_record(
     392            job_id,
     393            title="Explore primary sources",
     394            status="open",
     395            priority=3,
     396            goal="Find direct evidence",
     397        )
     398        second = db.append_task_record(
     399            job_id,
     400            title="Explore primary sources",
     401            status="done",
     402            priority=5,
     403            result="Saved source artifact",
     404        )
     405        job = db.get_job(job_id)
     406
     407        assert first["created"] is True
     408        assert second["created"] is False
     409        assert len(job["metadata"]["task_queue"]) == 1
     410        assert job["metadata"]["task_queue"][0]["status"] == "done"
     411        assert job["metadata"]["task_queue"][0]["priority"] == 5
     412        assert job["metadata"]["task_queue"][0]["result"] == "Saved source artifact"
     413    finally:
     414        db.close()
     415
     416
     417def test_repeated_task_and_experiment_records_mark_non_substantive_touches(tmp_path):
     418    db = AgentDB(tmp_path / "state.db")
     419    try:
     420        job_id = db.create_job("Research topic")
     421
     422        db.append_task_record(job_id, title="Explore primary sources", status="open", priority=3)
     423        repeated_task = db.append_task_record(job_id, title="Explore primary sources", status="open", priority=3)
     424        changed_task = db.append_task_record(job_id, title="Explore primary sources", status="done", priority=3)
     425
     426        db.append_experiment_record(
     427            job_id,
     428            title="Trial",
     429            status="measured",
     430            metric_name="score",
     431            metric_value=0.8,
     432        )
     433        repeated_experiment = db.append_experiment_record(
     434            job_id,
     435            title="Trial",
     436            status="measured",
     437            metric_name="score",
     438            metric_value=0.8,
     439        )
     440        changed_experiment = db.append_experiment_record(
     441            job_id,
     442            title="Trial",
     443            status="measured",
     444            metric_name="score",
     445            metric_value=0.9,
     446        )
     447
     448        assert repeated_task["created"] is False
     449        assert repeated_task["substantive_update"] is False
     450        assert changed_task["substantive_update"] is True
     451        assert repeated_experiment["created"] is False
     452        assert repeated_experiment["substantive_update"] is False
     453        assert changed_experiment["substantive_update"] is True
     454    finally:
     455        db.close()
     456
     457
     458def test_non_substantive_ledger_touches_do_not_emit_visible_events(tmp_path):
     459    db = AgentDB(tmp_path / "state.db")
     460    try:
     461        job_id = db.create_job("Research topic")
     462
     463        db.append_source_record(job_id, "https://example.com", outcome="useful")
     464        db.append_source_record(job_id, "https://example.com", outcome="useful")
     465        db.append_finding_record(job_id, name="Reusable finding", reason="evidence")
     466        db.append_finding_record(job_id, name="Reusable finding", reason="evidence")
     467        db.append_task_record(job_id, title="Explore primary sources", status="open", priority=3)
     468        db.append_task_record(job_id, title="Explore primary sources", status="open", priority=3)
     469        db.append_roadmap_record(
     470            job_id,
     471            title="Roadmap",
     472            milestones=[{"title": "Foundation", "status": "active", "priority": 5}],
     473        )
     474        db.append_roadmap_record(
     475            job_id,
     476            title="Roadmap",
     477            milestones=[{"title": "Foundation", "status": "active", "priority": 5}],
     478        )
     479        db.append_experiment_record(
     480            job_id,
     481            title="Trial",
     482            status="measured",
     483            metric_name="score",
     484            metric_value=0.8,
     485        )
     486        db.append_experiment_record(
     487            job_id,
     488            title="Trial",
     489            status="measured",
     490            metric_name="score",
     491            metric_value=0.8,
     492        )
     493
     494        events = db.list_timeline_events(job_id, limit=50)
     495        counts = {event_type: sum(1 for event in events if event["event_type"] == event_type) for event_type in {
     496            "source",
     497            "finding",
     498            "task",
     499            "roadmap",
     500            "experiment",
     501        }}
     502
     503        assert counts == {
     504            "source": 1,
     505            "finding": 1,
     506            "task": 1,
     507            "roadmap": 1,
     508            "experiment": 1,
     509        }
     510    finally:
     511        db.close()
     512
     513
     514def test_roadmap_last_records_include_progress_accounting_metadata(tmp_path):
     515    db = AgentDB(tmp_path / "state.db")
     516    try:
     517        job_id = db.create_job("Research topic")
     518        db.append_roadmap_record(
     519            job_id,
     520            title="Roadmap",
     521            milestones=[{"title": "Foundation", "status": "active", "priority": 5}],
     522        )
     523        db.append_roadmap_record(
     524            job_id,
     525            title="Roadmap",
     526            milestones=[{"title": "Foundation", "status": "validating", "priority": 6}],
     527        )
     528        db.append_milestone_validation_record(
     529            job_id,
     530            milestone="Foundation",
     531            validation_status="passed",
     532            result="Evidence satisfies acceptance criteria.",
     533        )
     534        metadata = db.get_job(job_id)["metadata"]
     535        roadmap_record = metadata["last_roadmap_record"]
     536        validation_record = metadata["last_milestone_validation"]
     537
     538        assert roadmap_record["created"] is False
     539        assert roadmap_record["updated_at"]
     540        assert roadmap_record["added_milestones"] == 0
     541        assert roadmap_record["updated_milestones"] == 1
     542        assert validation_record["validated_at"]
     543        assert validation_record["validation_status"] == "passed"
     544    finally:
     545        db.close()
     546
     547
     548def test_repeated_roadmap_records_do_not_create_fake_milestone_updates(tmp_path):
     549    db = AgentDB(tmp_path / "state.db")
     550    try:
     551        job_id = db.create_job("Research topic")
     552        milestone = {"title": "Foundation", "status": "active", "priority": 5}
     553        db.append_roadmap_record(job_id, title="Roadmap", milestones=[milestone])
     554        repeated = db.append_roadmap_record(job_id, title="Roadmap", milestones=[milestone])
     555        metadata = db.get_job(job_id)["metadata"]
     556        roadmap_record = metadata["last_roadmap_record"]
     557
     558        assert repeated["created"] is False
     559        assert repeated["substantive_update"] is False
     560        assert roadmap_record["updated_milestones"] == 0
     561        assert roadmap_record["updated_features"] == 0
     562        assert roadmap_record["roadmap_updated"] is False
     563    finally:
     564        db.close()
     565
     566
     567def test_timeline_events_cover_visible_activity(tmp_path):
     568    db = AgentDB(tmp_path / "state.db")
     569    try:
     570        job_id = db.create_job("Research topic", title="research")
     571        db.append_operator_message(job_id, "operator note", source="test")
     572        db.append_agent_update(job_id, "agent note", category="chat")
     573        db.append_lesson(job_id, "durable lesson", category="strategy")
     574        db.append_source_record(job_id, "https://example.com", usefulness_score=0.7, outcome="useful")
     575        db.append_finding_record(job_id, name="Reusable finding", reason="evidence")
     576        db.append_task_record(job_id, title="Explore branch", status="open")
     577        db.append_reflection(job_id, "reflect summary", strategy="next strategy")
     578        run_id = db.start_run(job_id, model="fake")
     579        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="web_search", input_data={"query": "x"})
     580        db.finish_step(step_id, status="completed", summary="searched", output_data={"ok": True})
     581        db.add_artifact(
     582            job_id=job_id,
     583            run_id=run_id,
     584            step_id=step_id,
     585            path=tmp_path / "artifact.md",
     586            sha256="abc",
     587            artifact_type="text",
     588            title="Artifact",
     589            summary="saved",
     590        )
     591        db.upsert_memory(job_id=job_id, key="rolling_state", summary="compact state")
     592
     593        events = db.list_timeline_events(job_id, limit=50)
     594        event_types = {event["event_type"] for event in events}
     595
     596        assert "operator_message" in event_types
     597        assert "agent_message" in event_types
     598        assert "lesson" in event_types
     599        assert "source" in event_types
     600        assert "finding" in event_types
     601        assert "task" in event_types
     602        assert "reflection" in event_types
     603        assert "tool_call" in event_types
     604        assert "tool_result" in event_types
     605        assert "artifact" in event_types
     606        assert "compaction" in event_types
     607        assert any(event["body"] == "operator note" for event in events)
     608    finally:
     609        db.close()

tests/nipux_cli/test_digest.py 43 lines

       1from nipux_cli.config import AppConfig, RuntimeConfig
       2from nipux_cli.db import AgentDB
       3from nipux_cli.digest import render_daily_digest, render_job_digest, write_daily_digest
       4
       5
       6def test_daily_digest_includes_ledgers_lessons_sources_and_strategy(tmp_path):
       7    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
       8    db = AgentDB(tmp_path / "state.db")
       9    try:
      10        job_id = db.create_job("Research topic", title="research", kind="generic")
      11        db.append_finding_record(job_id, name="Acme Finding", category="example category", reason="reusable result", score=0.8)
      12        db.append_task_record(job_id, title="Explore primary sources", status="open", priority=5)
      13        db.append_source_record(job_id, "https://example.com", usefulness_score=0.9, yield_count=1, outcome="yielded findings")
      14        db.append_lesson(job_id, "Low-evidence pages are not finding sources.", category="source_quality")
      15        db.append_reflection(job_id, "Primary source map is working.", strategy="Try archival sources next.")
      16        db.start_run(job_id, model="test-model")
      17        db.append_event(
      18            job_id,
      19            event_type="loop",
      20            title="message_end",
      21            metadata={"usage": {"prompt_tokens": 1200, "completion_tokens": 300, "total_tokens": 1500, "cost": 0.0025}},
      22        )
      23
      24        body = render_daily_digest(db)
      25        job_body = render_job_digest(db, job_id)
      26        result = write_daily_digest(config, db, day="2026-04-25")
      27
      28        assert "Counts: 1 findings, 1 sources, 1 tasks, 0 experiments, 1 lessons" in body
      29        assert "Model usage:" in body
      30        assert "1.5K tokens" in body
      31        assert "cost=$0.0025" in body
      32        assert "## Model Usage" in job_body
      33        assert "test-model: 1 calls" in job_body
      34        assert "Experiments:" in body
      35        assert "Acme Finding" in body
      36        assert "Explore primary sources" in body
      37        assert "Low-evidence pages are not finding sources." in body
      38        assert "https://example.com" in body
      39        assert "Try archival sources next." in body
      40        assert result["status"] == "dry_run"
      41        assert (tmp_path / "digests" / "2026-04-25-daily.md").exists()
      42    finally:
      43        db.close()

tests/nipux_cli/test_doctor.py 157 lines

       1import io
       2import json
       3import urllib.error
       4
       5from nipux_cli.config import AppConfig, ModelConfig, RuntimeConfig
       6from nipux_cli.doctor import run_doctor
       7
       8
       9class FakeHTTPResponse:
      10    def __init__(self, payload: dict):
      11        self.payload = payload
      12
      13    def __enter__(self):
      14        return self
      15
      16    def __exit__(self, exc_type, exc, tb):
      17        return False
      18
      19    def read(self, _limit=-1):
      20        return json.dumps(self.payload).encode("utf-8")
      21
      22
      23def test_doctor_checks_local_runtime_without_model_call(tmp_path):
      24    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
      25
      26    checks = run_doctor(config=config, check_model=False)
      27
      28    assert {check.name for check in checks} == {"state_dir_writable", "sqlite", "model_config", "tool_surface", "browser_runtime"}
      29    assert all(check.ok for check in checks)
      30
      31
      32def test_doctor_warns_when_remote_model_key_is_missing(tmp_path, monkeypatch):
      33    monkeypatch.delenv("OPENROUTER_API_KEY", raising=False)
      34    config = AppConfig(
      35        runtime=RuntimeConfig(home=tmp_path),
      36        model=ModelConfig(
      37            model="provider/model",
      38            base_url="https://openrouter.ai/api/v1",
      39            api_key_env="OPENROUTER_API_KEY",
      40        ),
      41    )
      42
      43    checks = run_doctor(config=config, check_model=False)
      44    model_check = next(check for check in checks if check.name == "model_config")
      45
      46    assert not model_check.ok
      47    assert "OPENROUTER_API_KEY is not set" in model_check.detail
      48    assert "sk-" not in model_check.detail
      49
      50
      51def test_doctor_reports_openrouter_auth_failure(tmp_path, monkeypatch):
      52    monkeypatch.setenv("TEST_OPENROUTER_KEY", "bad-key")
      53    config = AppConfig(
      54        runtime=RuntimeConfig(home=tmp_path),
      55        model=ModelConfig(
      56            model="nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free",
      57            base_url="https://openrouter.ai/api/v1",
      58            api_key_env="TEST_OPENROUTER_KEY",
      59        ),
      60    )
      61
      62    def fake_urlopen(_request, timeout):
      63        raise urllib.error.HTTPError(
      64            "https://openrouter.ai/api/v1/key",
      65            401,
      66            "Unauthorized",
      67            hdrs=None,
      68            fp=None,
      69        )
      70
      71    monkeypatch.setattr("urllib.request.urlopen", fake_urlopen)
      72
      73    checks = run_doctor(config=config, check_model=True)
      74    model_check = checks[-1]
      75
      76    assert model_check.name == "model_auth"
      77    assert model_check.ok is False
      78    assert "OpenRouter rejected API key" in model_check.detail
      79
      80
      81def test_doctor_reports_generation_limit_after_model_listing(tmp_path, monkeypatch):
      82    monkeypatch.setenv("TEST_OPENROUTER_KEY", "limited-key")
      83    config = AppConfig(
      84        runtime=RuntimeConfig(home=tmp_path),
      85        model=ModelConfig(
      86            model="provider/test-model",
      87            base_url="https://openrouter.ai/api/v1",
      88            api_key_env="TEST_OPENROUTER_KEY",
      89        ),
      90    )
      91
      92    def fake_urlopen(request, timeout):
      93        url = request.full_url
      94        if url.endswith("/key"):
      95            return FakeHTTPResponse({})
      96        if url.endswith("/models"):
      97            return FakeHTTPResponse({"data": [{"id": "provider/test-model"}]})
      98        if url.endswith("/chat/completions"):
      99            body = b'{"error":{"message":"Key limit exceeded (total limit).","code":403}}'
     100            raise urllib.error.HTTPError(url, 403, "Forbidden", hdrs=None, fp=io.BytesIO(body))
     101        raise AssertionError(url)
     102
     103    monkeypatch.setattr("urllib.request.urlopen", fake_urlopen)
     104
     105    checks = run_doctor(config=config, check_model=True)
     106    model_check = checks[-1]
     107
     108    assert model_check.name == "model_generation"
     109    assert model_check.ok is False
     110    assert "Key limit exceeded" in model_check.detail
     111
     112
     113def test_doctor_reports_nested_provider_generation_error(tmp_path, monkeypatch):
     114    monkeypatch.setenv("TEST_OPENROUTER_KEY", "limited-key")
     115    config = AppConfig(
     116        runtime=RuntimeConfig(home=tmp_path),
     117        model=ModelConfig(
     118            model="provider/test-model",
     119            base_url="https://openrouter.ai/api/v1",
     120            api_key_env="TEST_OPENROUTER_KEY",
     121        ),
     122    )
     123
     124    def fake_urlopen(request, timeout):
     125        url = request.full_url
     126        if url.endswith("/key"):
     127            return FakeHTTPResponse({})
     128        if url.endswith("/models"):
     129            return FakeHTTPResponse({"data": [{"id": "provider/test-model"}]})
     130        if url.endswith("/chat/completions"):
     131            body = json.dumps(
     132                {
     133                    "error": {
     134                        "message": "Provider returned error",
     135                        "code": 429,
     136                        "metadata": {
     137                            "raw": "provider/test-model is temporarily rate-limited upstream.",
     138                            "provider_name": "ExampleProvider",
     139                            "is_byok": False,
     140                        },
     141                    }
     142                }
     143            ).encode("utf-8")
     144            raise urllib.error.HTTPError(url, 429, "Too Many Requests", hdrs=None, fp=io.BytesIO(body))
     145        raise AssertionError(url)
     146
     147    monkeypatch.setattr("urllib.request.urlopen", fake_urlopen)
     148
     149    checks = run_doctor(config=config, check_model=True)
     150    model_check = checks[-1]
     151
     152    assert model_check.name == "model_generation"
     153    assert model_check.ok is False
     154    assert "Provider returned error" in model_check.detail
     155    assert "temporarily rate-limited upstream" in model_check.detail
     156    assert "provider=ExampleProvider" in model_check.detail
     157    assert "byok=False" in model_check.detail

tests/nipux_cli/test_generic_runtime_audit.py 35 lines

       1from pathlib import Path
       2
       3
       4FORBIDDEN_RUNTIME_LITERALS = {
       5    "192.168",
       6    "9060 xt",
       7    "canadian",
       8    "client finder",
       9    "lead batch",
      10    "lead ledger",
      11    "client prospect",
      12    "edmonton",
      13    "home ssh",
      14    "home-ssh.local",
      15    "home@",
      16    "huggingface.co/qwen",
      17    "livebusiness",
      18    "qwen_qwen",
      19    "ssh home",
      20    "treefrog",
      21    "yelp",
      22    "llama.cpp",
      23}
      24
      25
      26def test_runtime_code_has_no_task_specific_literals():
      27    root = Path(__file__).resolve().parents[2] / "nipux_cli"
      28    haystack = "\n".join(
      29        path.read_text(encoding="utf-8", errors="replace").lower()
      30        for path in sorted(root.glob("*.py"))
      31        if path.name != "__init__.py"
      32    )
      33
      34    for literal in FORBIDDEN_RUNTIME_LITERALS:
      35        assert literal not in haystack

tests/nipux_cli/test_live_memory_graph_smoke.py 37 lines

       1import importlib.util
       2import sys
       3from pathlib import Path
       4
       5from nipux_cli.memory_graph import memory_graph_for_prompt
       6
       7
       8def _load_live_smoke():
       9    path = Path(__file__).resolve().parents[2] / "scripts" / "live_memory_graph_smoke.py"
      10    spec = importlib.util.spec_from_file_location("live_memory_graph_smoke", path)
      11    assert spec is not None
      12    module = importlib.util.module_from_spec(spec)
      13    assert spec.loader is not None
      14    sys.modules[spec.name] = module
      15    spec.loader.exec_module(module)
      16    return module
      17
      18
      19def test_live_memory_graph_smoke_fails_cleanly_without_key(monkeypatch, capsys):
      20    smoke = _load_live_smoke()
      21    monkeypatch.delenv("NIPUX_LIVE_TEST_KEY", raising=False)
      22    monkeypatch.setattr(sys, "argv", ["live_memory_graph_smoke.py", "--api-key-env", "NIPUX_LIVE_TEST_KEY", "--json"])
      23
      24    assert smoke.main() == 1
      25
      26    out = capsys.readouterr().out
      27    assert '"success": false' in out
      28    assert "NIPUX_LIVE_TEST_KEY is not set" in out
      29    assert "secret" not in out.lower()
      30
      31
      32def test_live_memory_graph_smoke_seed_pushes_generic_consolidation():
      33    smoke = _load_live_smoke()
      34    prompt = memory_graph_for_prompt({"metadata": smoke._seed_metadata()})
      35
      36    assert "No memory graph yet" in prompt
      37    assert "Durable ledgers already contain" in prompt

tests/nipux_cli/test_llm.py 151 lines

       1from types import SimpleNamespace
       2
       3from nipux_cli.config import ModelConfig
       4from nipux_cli.llm import OpenAIChatLLM, _enrich_openrouter_generation_usage
       5
       6
       7class _FakeCompletions:
       8    def __init__(self):
       9        self.kwargs = None
      10        self.calls = []
      11
      12    def create(self, **kwargs):
      13        self.kwargs = kwargs
      14        self.calls.append(kwargs)
      15        usage = SimpleNamespace(prompt_tokens=11, completion_tokens=7, total_tokens=18, cost=0.00042)
      16        message = SimpleNamespace(content="ok", tool_calls=[])
      17        choice = SimpleNamespace(message=message)
      18        return SimpleNamespace(id="gen_test", model="provider/model", choices=[choice], usage=usage)
      19
      20
      21def test_chat_llm_requires_tool_choice_for_worker_actions(monkeypatch):
      22    fake_completions = _FakeCompletions()
      23    monkeypatch.setenv("TEST_API_KEY", "test")
      24
      25    class FakeOpenAI:
      26        def __init__(self, **kwargs):
      27            pass
      28
      29        chat = SimpleNamespace(completions=fake_completions)
      30
      31    monkeypatch.setattr("nipux_cli.llm.OpenAI", FakeOpenAI)
      32
      33    llm = OpenAIChatLLM(ModelConfig(model="test/model", base_url="https://example.test/v1", api_key_env="TEST_API_KEY"))
      34    response = llm.next_action(messages=[{"role": "user", "content": "hi"}], tools=[{"type": "function", "function": {"name": "noop"}}])
      35
      36    assert response.content == "ok"
      37    assert response.usage["prompt_tokens"] == 11
      38    assert response.usage["completion_tokens"] == 7
      39    assert response.usage["cost"] == 0.00042
      40    assert response.model == "provider/model"
      41    assert response.response_id == "gen_test"
      42    assert fake_completions.kwargs["tools"]
      43    assert fake_completions.kwargs["tool_choice"] == "required"
      44
      45
      46def test_chat_llm_retries_without_tool_choice_when_provider_rejects_it(monkeypatch):
      47    monkeypatch.setenv("TEST_API_KEY", "test")
      48
      49    class RejectingCompletions(_FakeCompletions):
      50        def create(self, **kwargs):
      51            self.calls.append(kwargs)
      52            if kwargs.get("tool_choice") == "required":
      53                raise RuntimeError("unsupported parameter: tool_choice")
      54            return super().create(**kwargs)
      55
      56    fake_completions = RejectingCompletions()
      57
      58    class FakeOpenAI:
      59        def __init__(self, **kwargs):
      60            pass
      61
      62        chat = SimpleNamespace(completions=fake_completions)
      63
      64    monkeypatch.setattr("nipux_cli.llm.OpenAI", FakeOpenAI)
      65
      66    llm = OpenAIChatLLM(ModelConfig(model="test/model", base_url="https://example.test/v1", api_key_env="TEST_API_KEY"))
      67    response = llm.next_action(messages=[{"role": "user", "content": "hi"}], tools=[{"type": "function", "function": {"name": "noop"}}])
      68
      69    assert response.content == "ok"
      70    assert fake_completions.calls[0]["tool_choice"] == "required"
      71    assert "tool_choice" not in fake_completions.calls[-1]
      72
      73
      74def test_chat_llm_complete_response_returns_usage(monkeypatch):
      75    fake_completions = _FakeCompletions()
      76    monkeypatch.setenv("TEST_API_KEY", "test")
      77
      78    class FakeOpenAI:
      79        def __init__(self, **kwargs):
      80            pass
      81
      82        chat = SimpleNamespace(completions=fake_completions)
      83
      84    monkeypatch.setattr("nipux_cli.llm.OpenAI", FakeOpenAI)
      85
      86    llm = OpenAIChatLLM(ModelConfig(model="test/model", base_url="https://example.test/v1", api_key_env="TEST_API_KEY"))
      87    response = llm.complete_response(messages=[{"role": "user", "content": "hi"}])
      88
      89    assert response.content == "ok"
      90    assert response.usage["prompt_tokens"] == 11
      91    assert response.usage["completion_tokens"] == 7
      92    assert response.usage["cost"] == 0.00042
      93    assert response.model == "provider/model"
      94    assert response.response_id == "gen_test"
      95    assert fake_completions.kwargs["model"] == "test/model"
      96
      97
      98def test_chat_llm_disables_provider_sdk_retries(monkeypatch):
      99    captured = {}
     100    monkeypatch.setenv("TEST_API_KEY", "test")
     101
     102    class FakeOpenAI:
     103        def __init__(self, **kwargs):
     104            captured.update(kwargs)
     105
     106        chat = SimpleNamespace(completions=_FakeCompletions())
     107
     108    monkeypatch.setattr("nipux_cli.llm.OpenAI", FakeOpenAI)
     109
     110    OpenAIChatLLM(ModelConfig(model="test/model", base_url="https://example.test/v1", api_key_env="TEST_API_KEY", request_timeout_seconds=37))
     111
     112    assert captured["timeout"] == 37
     113    assert captured["max_retries"] == 0
     114
     115
     116def test_openrouter_generation_usage_enriches_cost_and_tokens(monkeypatch):
     117    class FakeHTTPResponse:
     118        def __enter__(self):
     119            return self
     120
     121        def __exit__(self, *_args):
     122            return False
     123
     124        def read(self):
     125            return (
     126                b'{"data":{"total_cost":"0.0042","native_tokens_prompt":123,'
     127                b'"native_tokens_completion":45,"native_tokens_total":168}}'
     128            )
     129
     130    captured = {}
     131
     132    def fake_urlopen(request, timeout):
     133        captured["url"] = request.full_url
     134        captured["timeout"] = timeout
     135        return FakeHTTPResponse()
     136
     137    monkeypatch.setattr("nipux_cli.llm.urllib.request.urlopen", fake_urlopen)
     138
     139    usage = _enrich_openrouter_generation_usage(
     140        {"prompt_tokens": 10, "completion_tokens": 5, "total_tokens": 15, "estimated": False},
     141        response_id="gen_123",
     142        base_url="https://openrouter.ai/api/v1",
     143        api_key="sk-test",
     144    )
     145
     146    assert captured["url"] == "https://openrouter.ai/api/v1/generation?id=gen_123"
     147    assert captured["timeout"] == 5
     148    assert usage["cost"] == 0.0042
     149    assert usage["prompt_tokens"] == 123
     150    assert usage["completion_tokens"] == 45
     151    assert usage["total_tokens"] == 168

tests/nipux_cli/test_measurement.py 32 lines

       1from nipux_cli.measurement import measurement_candidates, measurement_candidates_are_diagnostic_only
       2
       3
       4def test_measurement_candidates_extract_markdown_table_unit_columns():
       5    output = {
       6        "stdout": (
       7            "| model                          |       size | backend    | threads |            test |                  t/s |\n"
       8            "| ------------------------------ | ---------: | ---------- | ------: | --------------: | -------------------: |\n"
       9            "| example model                  |  11.71 GiB | CPU        |      24 |            pp32 |          5.48 ± 0.11 |\n"
      10            "| example model                  |  11.71 GiB | CPU        |      24 |           tg128 |          3.44 ± 0.05 |\n"
      11        )
      12    }
      13
      14    candidates = measurement_candidates(output, command="run benchmark")
      15
      16    assert "pp32 5.48 ± 0.11 t/s" in candidates
      17    assert "tg128 3.44 ± 0.05 t/s" in candidates
      18    assert not measurement_candidates_are_diagnostic_only(candidates, command="run benchmark")
      19
      20
      21def test_measurement_candidates_extract_generic_table_metrics():
      22    output = {
      23        "stdout": (
      24            "| benchmark | latency | req/s |\n"
      25            "| --- | ---: | ---: |\n"
      26            "| warm path | 18.4 | 42.7 |\n"
      27        )
      28    }
      29
      30    candidates = measurement_candidates(output, command="profile throughput")
      31
      32    assert "warm path 42.7 req/s" in candidates

tests/nipux_cli/test_metric_format.py 11 lines

       1from nipux_cli.metric_format import format_metric_value
       2
       3
       4def test_format_metric_value_spaces_named_units():
       5    assert format_metric_value("citations", 42, "count") == "citations=42 count"
       6    assert format_metric_value("speed", 2.7, "tokens/s") == "speed=2.7 tokens/s"
       7
       8
       9def test_format_metric_value_keeps_attached_symbol_units():
      10    assert format_metric_value("accuracy", 98.2, "%") == "accuracy=98.2%"
      11    assert format_metric_value("throughput", 120, "/s") == "throughput=120/s"

tests/nipux_cli/test_operator_context.py 30 lines

       1from nipux_cli.operator_context import inactive_prompt_operator_ids, operator_entry_is_prompt_relevant
       2
       3
       4def _entry(message: str, *, event_id: str = "op_1", mode: str = "steer") -> dict:
       5    return {"event_id": event_id, "mode": mode, "message": message}
       6
       7
       8def test_conversation_only_operator_messages_do_not_enter_worker_prompt():
       9    for message in ("hello", "how is it going?", "clear", "stop 1", "jobs"):
      10        assert not operator_entry_is_prompt_relevant(_entry(message))
      11
      12
      13def test_actionable_operator_messages_remain_worker_constraints():
      14    for message in (
      15        "do not run local testing on my computer",
      16        "use the corrected target from the chat",
      17        "focus on measured results instead of saved notes",
      18        "the address is wrong, use `target-box`",
      19    ):
      20        assert operator_entry_is_prompt_relevant(_entry(message))
      21
      22
      23def test_inactive_prompt_operator_ids_returns_only_conversation_active_messages():
      24    messages = [
      25        _entry("hello", event_id="op_chat"),
      26        _entry("use the corrected target", event_id="op_use"),
      27        {**_entry("clear", event_id="op_done"), "acknowledged_at": "2026-04-26T00:00:00+00:00"},
      28    ]
      29
      30    assert inactive_prompt_operator_ids(messages) == ["op_chat"]

tests/nipux_cli/test_planning.py 90 lines

       1from nipux_cli.planning import initial_plan_for_objective, initial_roadmap_for_objective, initial_task_contract, objective_profiles
       2
       3
       4def test_initial_task_contracts_are_generic_and_complete():
       5    for title in [
       6        "Clarify the exact success criteria and constraints.",
       7        "Map the first research or execution branches.",
       8        "Collect evidence and save outputs as files.",
       9        "Reflect on what worked, update memory, and continue with the next branch.",
      10    ]:
      11        contract = initial_task_contract(title)
      12
      13        assert contract["output_contract"] in {"research", "artifact", "experiment", "action", "monitor", "decision", "report"}
      14        assert contract["acceptance_criteria"]
      15        assert contract["evidence_needed"]
      16        assert contract["stall_behavior"]
      17
      18
      19def test_initial_roadmap_uses_valid_generic_contracts():
      20    roadmap = initial_roadmap_for_objective(title="paper", objective="write a paper")
      21
      22    for milestone in roadmap["milestones"]:
      23        for feature in milestone["features"]:
      24            assert feature["output_contract"] in {
      25                "research",
      26                "artifact",
      27                "experiment",
      28                "action",
      29                "monitor",
      30                "decision",
      31                "report",
      32            }
      33
      34
      35def test_initial_plan_adapts_to_measurable_objectives():
      36    plan = initial_plan_for_objective("optimize a generic process for lower latency and higher throughput")
      37    contracts = [initial_task_contract(title)["output_contract"] for title in plan["tasks"]]
      38
      39    assert plan["profile"] == "measured"
      40    assert "experiment" in contracts
      41    assert any("baseline" in title.lower() for title in plan["tasks"])
      42    assert any("metric" in question.lower() for question in plan["questions"])
      43
      44
      45def test_initial_plan_adapts_to_deliverable_objectives():
      46    plan = initial_plan_for_objective("write a full research paper from evidence")
      47    contracts = [initial_task_contract(title)["output_contract"] for title in plan["tasks"]]
      48
      49    assert plan["profile"] == "deliverable"
      50    assert "report" in contracts
      51    assert any("draft" in title.lower() or "report" in title.lower() for title in plan["tasks"])
      52    assert any("revise" in title.lower() and "evidence" in title.lower() for title in plan["tasks"])
      53
      54
      55def test_initial_plan_treats_generated_files_as_deliverables():
      56    plan = initial_plan_for_objective("generate a polished launch checklist for this repository")
      57    contracts = [initial_task_contract(title)["output_contract"] for title in plan["tasks"]]
      58
      59    assert plan["profile"] == "deliverable"
      60    assert "report" in contracts
      61    assert any("audience" in question.lower() for question in plan["questions"])
      62
      63
      64def test_initial_plan_adapts_to_monitoring_objectives():
      65    plan = initial_plan_for_objective("monitor a recurring process and report important changes")
      66    contracts = [initial_task_contract(title)["output_contract"] for title in plan["tasks"]]
      67
      68    assert plan["profile"] == "monitor"
      69    assert "monitor" in contracts
      70    assert any("cadence" in question.lower() or "check" in question.lower() for question in plan["questions"])
      71
      72
      73def test_initial_plan_does_not_add_meta_progress_update_task():
      74    for objective in [
      75        "optimize a generic process for lower latency and higher throughput",
      76        "write a full research paper from evidence",
      77        "monitor a recurring process and report important changes",
      78        "investigate build quality and compare output changes",
      79    ]:
      80        plan = initial_plan_for_objective(objective)
      81
      82        assert all("progress update" not in title.lower() for title in plan["tasks"])
      83        assert all("keep working on the next useful branch" not in title.lower() for title in plan["tasks"])
      84
      85
      86def test_objective_profiles_stay_generic():
      87    profiles = objective_profiles("investigate build quality and compare output changes")
      88
      89    assert profiles
      90    assert all(profile in {"measured", "deliverable", "monitor", "implementation", "research", "general"} for profile in profiles)

tests/nipux_cli/test_progress.py 214 lines

       1from nipux_cli.progress import build_progress_checkpoint, ledger_counts, recent_progress_bits
       2
       3
       4def test_progress_checkpoint_reports_deltas_and_recent_durable_work():
       5    metadata = {
       6        "finding_ledger": [{"title": "First finding"}, {"title": "Better branch"}],
       7        "source_ledger": [{"url": "https://example.test"}],
       8        "task_queue": [
       9            {"title": "Draft report", "status": "done", "priority": 2},
      10            {"title": "Validate report", "status": "open", "priority": 8},
      11        ],
      12        "experiment_ledger": [{"title": "Quality check", "metric_name": "score", "metric_value": 0.82}],
      13        "lessons": [{"lesson": "Prefer measured output"}],
      14        "roadmap": {"milestones": [{"title": "Publishable draft", "status": "validating"}]},
      15    }
      16
      17    checkpoint = build_progress_checkpoint(
      18        metadata,
      19        previous_counts={"findings": 1, "sources": 0, "tasks": 2, "experiments": 0, "lessons": 1, "milestones": 0},
      20        step_no=40,
      21        tool_name="record_findings",
      22    )
      23
      24    assert checkpoint.counts == {
      25        "findings": 2,
      26        "sources": 1,
      27        "tasks": 2,
      28        "experiments": 1,
      29        "lessons": 1,
      30        "milestones": 1,
      31    }
      32    assert checkpoint.deltas["findings"] == 1
      33    assert checkpoint.deltas["sources"] == 1
      34    assert checkpoint.deltas["tasks"] == 0
      35    assert checkpoint.category == "progress"
      36    assert "+1 finding" in checkpoint.message
      37    assert "+1 source" in checkpoint.message
      38    assert "+1 experiment" in checkpoint.message
      39    assert "finding=Better branch" in checkpoint.message
      40    assert "task=Validate report" in checkpoint.message
      41    assert "measurement=score=0.82" in checkpoint.message
      42    assert "milestone=Publishable draft" in checkpoint.message
      43
      44
      45def test_progress_checkpoint_for_saved_output_is_concise():
      46    metadata = {"finding_ledger": [{}], "source_ledger": [{}, {}], "task_queue": [{}], "experiment_ledger": []}
      47
      48    checkpoint = build_progress_checkpoint(
      49        metadata,
      50        step_no=12,
      51        tool_name="write_artifact",
      52        artifact_id="art_123",
      53        is_finding_output=True,
      54    )
      55
      56    assert checkpoint.category == "finding"
      57    assert checkpoint.message.startswith("Saved output art_123")
      58    assert "1 findings, 2 sources, 1 tasks, and 0 experiments" in checkpoint.message
      59
      60
      61def test_progress_checkpoint_without_delta_is_activity_not_progress():
      62    metadata = {"finding_ledger": [{}], "source_ledger": [{}], "task_queue": [{}], "experiment_ledger": []}
      63
      64    checkpoint = build_progress_checkpoint(
      65        metadata,
      66        previous_counts={"findings": 1, "sources": 1, "tasks": 1, "experiments": 0, "lessons": 0, "milestones": 0},
      67        step_no=50,
      68        tool_name="web_extract",
      69    )
      70
      71    assert checkpoint.category == "activity"
      72    assert "no new durable ledger entries" in checkpoint.message
      73
      74
      75def test_progress_checkpoint_counts_existing_record_updates_as_progress():
      76    metadata = {
      77        "last_checkpoint_at": "2026-01-01T00:00:00+00:00",
      78        "finding_ledger": [{}],
      79        "source_ledger": [{}],
      80        "task_queue": [{"title": "Existing branch", "status": "done"}],
      81        "experiment_ledger": [{"title": "Trial", "status": "measured"}],
      82        "last_task_record": {
      83            "title": "Existing branch",
      84            "status": "done",
      85            "result": "Validated the branch.",
      86            "created": False,
      87            "updated_at": "2026-01-01T00:01:00+00:00",
      88        },
      89        "last_source_record": {
      90            "source": "https://example.test",
      91            "created": False,
      92            "last_seen": "2026-01-01T00:01:30+00:00",
      93        },
      94        "last_experiment_record": {
      95            "title": "Trial",
      96            "status": "measured",
      97            "metric_name": "score",
      98            "metric_value": 0.9,
      99            "created": False,
     100            "updated_at": "2026-01-01T00:02:00+00:00",
     101        },
     102    }
     103
     104    checkpoint = build_progress_checkpoint(
     105        metadata,
     106        previous_counts={"findings": 1, "sources": 1, "tasks": 1, "experiments": 1, "lessons": 0, "milestones": 0},
     107        step_no=60,
     108        tool_name="record_tasks",
     109    )
     110
     111    assert checkpoint.category == "progress"
     112    assert checkpoint.deltas["tasks"] == 0
     113    assert checkpoint.updates["tasks"] == 1
     114    assert checkpoint.updates["sources"] == 1
     115    assert checkpoint.resolutions["tasks"] == 1
     116    assert checkpoint.updates["experiments"] == 1
     117    assert checkpoint.resolutions["experiments"] == 1
     118    assert "~1 task updated" in checkpoint.message
     119    assert "~1 source updated" in checkpoint.message
     120    assert "1 task resolved" in checkpoint.message
     121    assert "~1 experiment updated" in checkpoint.message
     122
     123
     124def test_progress_checkpoint_ignores_non_substantive_record_touches():
     125    metadata = {
     126        "last_checkpoint_at": "2026-01-01T00:00:00+00:00",
     127        "finding_ledger": [{}],
     128        "source_ledger": [{}],
     129        "task_queue": [{"title": "Existing branch", "status": "open"}],
     130        "experiment_ledger": [{"title": "Trial", "status": "planned"}],
     131        "last_task_record": {
     132            "title": "Existing branch",
     133            "status": "open",
     134            "created": False,
     135            "substantive_update": False,
     136            "updated_at": "2026-01-01T00:01:00+00:00",
     137        },
     138        "last_source_record": {
     139            "source": "https://example.test",
     140            "created": False,
     141            "substantive_update": False,
     142            "last_seen": "2026-01-01T00:01:30+00:00",
     143        },
     144        "last_experiment_record": {
     145            "title": "Trial",
     146            "status": "planned",
     147            "created": False,
     148            "substantive_update": False,
     149            "updated_at": "2026-01-01T00:02:00+00:00",
     150        },
     151    }
     152
     153    checkpoint = build_progress_checkpoint(
     154        metadata,
     155        previous_counts={"findings": 1, "sources": 1, "tasks": 1, "experiments": 1, "lessons": 0, "milestones": 0},
     156        step_no=61,
     157        tool_name="record_tasks",
     158    )
     159
     160    assert checkpoint.category == "activity"
     161    assert checkpoint.updates["tasks"] == 0
     162    assert checkpoint.updates["sources"] == 0
     163    assert checkpoint.updates["experiments"] == 0
     164    assert "no new durable ledger entries" in checkpoint.message
     165
     166
     167def test_progress_checkpoint_counts_roadmap_updates_and_validations():
     168    metadata = {
     169        "last_checkpoint_at": "2026-01-01T00:00:00+00:00",
     170        "roadmap": {"milestones": [{"title": "Foundation", "status": "validating"}]},
     171        "last_roadmap_record": {
     172            "title": "Roadmap",
     173            "created": False,
     174            "updated_at": "2026-01-01T00:01:00+00:00",
     175            "added_milestones": 0,
     176            "updated_milestones": 1,
     177            "added_features": 0,
     178            "updated_features": 0,
     179        },
     180        "last_milestone_validation": {
     181            "milestone": "Foundation",
     182            "validation_status": "passed",
     183            "validated_at": "2026-01-01T00:02:00+00:00",
     184        },
     185    }
     186
     187    checkpoint = build_progress_checkpoint(
     188        metadata,
     189        previous_counts={"findings": 0, "sources": 0, "tasks": 0, "experiments": 0, "lessons": 0, "milestones": 1},
     190        step_no=70,
     191        tool_name="record_milestone_validation",
     192    )
     193
     194    assert checkpoint.category == "progress"
     195    assert checkpoint.deltas["milestones"] == 0
     196    assert checkpoint.updates["milestones"] == 2
     197    assert checkpoint.resolutions["milestones"] == 1
     198    assert "~2 milestones updated" in checkpoint.message
     199    assert "1 milestone resolved" in checkpoint.message
     200
     201
     202def test_progress_helpers_ignore_malformed_metadata():
     203    metadata = {
     204        "finding_ledger": "bad",
     205        "source_ledger": [None, {"url": "ok"}],
     206        "task_queue": [{"title": "Task", "status": "blocked", "priority": "bad"}],
     207        "roadmap": {"milestones": ["bad", {"title": "Milestone", "status": "active"}]},
     208    }
     209
     210    assert ledger_counts(metadata)["sources"] == 1
     211    assert ledger_counts(metadata)["milestones"] == 2
     212    bits = recent_progress_bits(metadata)
     213    assert "task=Task" in bits
     214    assert "milestone=Milestone" in bits

tests/nipux_cli/test_project_atlas.py 46 lines

       1import importlib.util
       2import sys
       3from pathlib import Path
       4
       5
       6def _load_generator():
       7    path = Path(__file__).resolve().parents[2] / "scripts" / "generate_project_atlas.py"
       8    spec = importlib.util.spec_from_file_location("generate_project_atlas", path)
       9    assert spec is not None
      10    module = importlib.util.module_from_spec(spec)
      11    assert spec.loader is not None
      12    sys.modules[spec.name] = module
      13    spec.loader.exec_module(module)
      14    return module
      15
      16
      17def test_project_atlas_generator_maps_prompts_tools_and_source_without_self_embedding():
      18    generator = _load_generator()
      19
      20    files = generator.load_source_files()
      21    prompts = generator.extract_prompts(files)
      22    tools = generator.extract_tools(files)
      23
      24    assert "docs/project-atlas.html" not in {source.path for source in files}
      25    assert any(prompt.path == "nipux_cli/worker_policy.py" and "SYSTEM_PROMPT" in prompt.name for prompt in prompts)
      26    assert any(tool["name"] == "web_search" for tool in tools)
      27
      28
      29def test_project_atlas_redacts_secret_assignments_from_rendered_source():
      30    generator = _load_generator()
      31    openrouter_key = "OPENROUTER_API_KEY"
      32    openai_key = "OPENAI_API_KEY"
      33    source = generator.SourceFile(
      34        path=".env.example",
      35        text=f"{openrouter_key}=\n{openai_key}=secret\nNORMAL=value",
      36        lines=[openrouter_key + "=", openai_key + "=secret", "NORMAL=value"],
      37        tree=None,
      38    )
      39
      40    rendered = generator.render_source_file(source)
      41
      42    assert openrouter_key + "=" not in rendered
      43    assert openai_key + "=secret" not in rendered
      44    assert f"{openrouter_key} = &lt;redacted&gt;" in rendered
      45    assert f"{openai_key} = &lt;redacted&gt;" in rendered
      46    assert "NORMAL=value" in rendered

tests/nipux_cli/test_provider_errors.py 21 lines

       1from nipux_cli.provider_errors import (
       2    provider_action_required,
       3    provider_action_required_note,
       4    provider_rate_limited,
       5)
       6
       7
       8class ProviderPayloadError(Exception):
       9    payload = {"error": {"message": "Key limit exceeded", "code": 403}}
      10
      11
      12def test_provider_action_required_detects_payload_and_status_text():
      13    assert provider_action_required(ProviderPayloadError("provider rejected request"))
      14    assert provider_action_required("PermissionDeniedError: Error code: 403")
      15    assert "operator action" in provider_action_required_note("invalid api key")
      16
      17
      18def test_provider_rate_limited_detects_transient_rate_text():
      19    assert provider_rate_limited("429 too many requests")
      20    assert provider_rate_limited("provider temporarily over capacity")
      21    assert not provider_rate_limited("invalid api key")

tests/nipux_cli/test_templates.py 15 lines

       1from nipux_cli.templates import program_for_job
       2
       3
       4def test_generic_template_pushes_artifacts_and_updates():
       5    program = program_for_job(kind="generic", title="research", objective="Find findings")
       6
       7    assert "Save important observations as artifacts" in program
       8    assert "Use report_update" in program
       9    assert "Use record_lesson" in program
      10    assert "record_source" in program
      11    assert "record_findings" in program
      12    assert "record_tasks" in program
      13    assert "record_roadmap" in program
      14    assert "record_milestone_validation" in program
      15    assert "record_findings" in program

tests/nipux_cli/test_tools.py 2166 lines

       1import json
       2import os
       3import signal
       4import subprocess
       5import time
       6
       7from nipux_cli.artifacts import ArtifactStore
       8from nipux_cli.config import AppConfig, RuntimeConfig, ToolAccessConfig
       9from nipux_cli.db import AgentDB
      10from nipux_cli.shell_tools import cleanup_registered_shell_processes
      11from nipux_cli.tools import APPROVED_TOOL_NAMES, DEFAULT_REGISTRY, ToolContext
      12
      13
      14def test_static_tool_surface_is_focused():
      15    assert tuple(DEFAULT_REGISTRY.names()) == tuple(sorted(APPROVED_TOOL_NAMES))
      16    assert "terminal" not in DEFAULT_REGISTRY.names()
      17    assert "delegate_task" not in DEFAULT_REGISTRY.names()
      18    assert "skill_manage" not in DEFAULT_REGISTRY.names()
      19    assert "browser_navigate" in DEFAULT_REGISTRY.names()
      20    assert "shell_exec" in DEFAULT_REGISTRY.names()
      21    assert "write_file" in DEFAULT_REGISTRY.names()
      22    assert "write_artifact" in DEFAULT_REGISTRY.names()
      23    assert "defer_job" in DEFAULT_REGISTRY.names()
      24    assert "report_update" in DEFAULT_REGISTRY.names()
      25    assert "record_lesson" in DEFAULT_REGISTRY.names()
      26    assert "record_memory_graph" in DEFAULT_REGISTRY.names()
      27    assert "search_memory_graph" in DEFAULT_REGISTRY.names()
      28    assert "record_source" in DEFAULT_REGISTRY.names()
      29    assert "record_findings" in DEFAULT_REGISTRY.names()
      30    assert "record_tasks" in DEFAULT_REGISTRY.names()
      31    assert "record_roadmap" in DEFAULT_REGISTRY.names()
      32    assert "record_milestone_validation" in DEFAULT_REGISTRY.names()
      33    assert "record_experiment" in DEFAULT_REGISTRY.names()
      34    assert "acknowledge_operator_context" in DEFAULT_REGISTRY.names()
      35
      36
      37def test_tool_registry_validates_required_arguments(tmp_path):
      38    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
      39
      40    missing = DEFAULT_REGISTRY.validate_arguments("shell_exec", {}, config)
      41    assert missing is not None
      42    assert missing["missing_arguments"] == ["command"]
      43    assert missing["recoverable"] is True
      44
      45    artifact_ref = DEFAULT_REGISTRY.validate_arguments("read_artifact", {}, config)
      46    assert artifact_ref is not None
      47    assert artifact_ref["missing_arguments"] == ["artifact reference"]
      48
      49    graph = DEFAULT_REGISTRY.validate_arguments("record_memory_graph", {}, config)
      50    assert graph is not None
      51    assert graph["missing_arguments"] == ["nodes or edges"]
      52
      53    experiment = DEFAULT_REGISTRY.validate_arguments("record_experiment", {"metric_name": "throughput"}, config)
      54    assert experiment is None
      55
      56    nested = DEFAULT_REGISTRY.validate_arguments("record_findings", {"findings": [{}]}, config)
      57    assert nested is not None
      58    assert nested["missing_arguments"] == ["findings[0].name"]
      59
      60    nested_task = DEFAULT_REGISTRY.validate_arguments("record_tasks", {"tasks": [{"goal": "do work"}]}, config)
      61    assert nested_task is not None
      62    assert nested_task["missing_arguments"] == ["tasks[0].title"]
      63
      64
      65def test_tool_registry_blocks_truncated_reference_arguments(tmp_path):
      66    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
      67
      68    experiment = DEFAULT_REGISTRY.validate_arguments(
      69        "record_experiment",
      70        {
      71            "title": "Measure local files",
      72            "evidence_artifact": "art_fb73...",
      73            "next_action": "validate the exact artifact",
      74        },
      75        config,
      76    )
      77
      78    assert experiment is not None
      79    assert experiment["error"] == "placeholder tool arguments"
      80    assert experiment["placeholder_arguments"] == ["evidence_artifact"]
      81
      82
      83def test_tool_access_config_filters_worker_schema_and_blocks_calls(tmp_path):
      84    config = AppConfig(runtime=RuntimeConfig(home=tmp_path), tools=ToolAccessConfig(browser=False, web=False, shell=False, files=False))
      85    db = AgentDB(tmp_path / "state.db")
      86    try:
      87        job_id = db.create_job("Restricted tools")
      88        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id)
      89
      90        names = {tool["function"]["name"] for tool in DEFAULT_REGISTRY.openai_tools(config=config)}
      91        assert "browser_navigate" not in names
      92        assert "web_search" not in names
      93        assert "shell_exec" not in names
      94        assert "write_file" not in names
      95        assert "write_artifact" in names
      96
      97        result = json.loads(DEFAULT_REGISTRY.handle("shell_exec", {"command": "printf no"}, ctx))
      98        assert result["success"] is False
      99        assert result["tool_access"] == "shell"
     100    finally:
     101        db.close()
     102
     103
     104def test_artifact_tools_roundtrip(tmp_path):
     105    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     106    db = AgentDB(tmp_path / "state.db")
     107    try:
     108        job_id = db.create_job("Save evidence")
     109        run_id = db.start_run(job_id, model="fake")
     110        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
     111        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     112
     113        raw = DEFAULT_REGISTRY.handle("write_artifact", {"content": "needle text", "title": "Evidence"}, ctx)
     114        result = json.loads(raw)
     115        assert result["success"] is True
     116
     117        read_raw = DEFAULT_REGISTRY.handle("read_artifact", {"artifact_id": result["artifact_id"]}, ctx)
     118        assert json.loads(read_raw)["content"] == "needle text"
     119
     120        path_raw = DEFAULT_REGISTRY.handle("read_artifact", {"artifact_id": result["path"]}, ctx)
     121        assert json.loads(path_raw)["artifact_id"] == result["artifact_id"]
     122
     123        title_raw = DEFAULT_REGISTRY.handle("read_artifact", {"title": "Evidence"}, ctx)
     124        assert json.loads(title_raw)["content"] == "needle text"
     125
     126        number_raw = DEFAULT_REGISTRY.handle("read_artifact", {"artifact_id": "1"}, ctx)
     127        assert json.loads(number_raw)["content"] == "needle text"
     128
     129        search_raw = DEFAULT_REGISTRY.handle("search_artifacts", {"query": "needle"}, ctx)
     130        assert json.loads(search_raw)["results"][0]["id"] == result["artifact_id"]
     131    finally:
     132        db.close()
     133
     134
     135def test_read_artifact_missing_ref_returns_valid_recent_refs(tmp_path):
     136    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     137    db = AgentDB(tmp_path / "state.db")
     138    try:
     139        job_id = db.create_job("Save evidence")
     140        run_id = db.start_run(job_id, model="fake")
     141        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
     142        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     143        stored = ctx.artifacts.write_text(job_id=job_id, run_id=run_id, step_id=step_id, title="Useful Evidence", content="saved")
     144
     145        raw = DEFAULT_REGISTRY.handle("read_artifact", {"artifact_id": "art_missing"}, ctx)
     146        result = json.loads(raw)
     147
     148        assert result["success"] is False
     149        assert result["recoverable"] is True
     150        assert result["error"] == "artifact not found: art_missing"
     151        assert "search_artifacts" in result["guidance"]
     152        assert result["recent_artifacts"][0]["id"] == stored.id
     153        assert result["recent_artifacts"][0]["title"] == "Useful Evidence"
     154    finally:
     155        db.close()
     156
     157
     158def test_defer_job_records_resume_time_without_pausing(tmp_path):
     159    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     160    db = AgentDB(tmp_path / "state.db")
     161    try:
     162        job_id = db.create_job("Monitor a long process")
     163        run_id = db.start_run(job_id, model="fake")
     164        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="defer_job")
     165        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     166
     167        raw = DEFAULT_REGISTRY.handle(
     168            "defer_job",
     169            {"seconds": 60, "reason": "process is still running", "next_action": "check status"},
     170            ctx,
     171        )
     172        result = json.loads(raw)
     173
     174        assert result["success"] is True
     175        assert result["status"] == "running"
     176        job = db.get_job(job_id)
     177        assert job["status"] == "running"
     178        assert job["metadata"]["defer_until"]
     179        assert job["metadata"]["defer_reason"] == "process is still running"
     180        assert job["metadata"]["defer_next_action"] == "check status"
     181        assert any(event["event_type"] == "agent_message" for event in db.list_events(job_id=job_id, limit=10))
     182    finally:
     183        db.close()
     184
     185
     186def test_shell_exec_tool_runs_bounded_command(tmp_path):
     187    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     188    db = AgentDB(tmp_path / "state.db")
     189    try:
     190        job_id = db.create_job("Run command")
     191        run_id = db.start_run(job_id, model="fake")
     192        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     193        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     194
     195        raw = DEFAULT_REGISTRY.handle("shell_exec", {"command": "printf hello", "timeout_seconds": 5}, ctx)
     196        result = json.loads(raw)
     197
     198        assert result["success"] is True
     199        assert result["returncode"] == 0
     200        assert result["stdout"] == "hello"
     201    finally:
     202        db.close()
     203
     204
     205def test_shell_exec_flags_masked_auth_failure_output(tmp_path):
     206    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     207    db = AgentDB(tmp_path / "state.db")
     208    try:
     209        job_id = db.create_job("Run command")
     210        run_id = db.start_run(job_id, model="fake")
     211        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     212        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     213
     214        raw = DEFAULT_REGISTRY.handle(
     215            "shell_exec",
     216            {
     217                "command": (
     218                    "printf 'HTTP request sent, awaiting response... 401 Unauthorized\\n"
     219                    "Username/Password Authentication Failed.\\nDownloaded: file.bin (29 bytes)\\n'"
     220                ),
     221                "timeout_seconds": 5,
     222            },
     223            ctx,
     224        )
     225        result = json.loads(raw)
     226
     227        assert result["returncode"] == 0
     228        assert result["success"] is False
     229        assert "authentication or authorization failure" in result["error"]
     230    finally:
     231        db.close()
     232
     233
     234def test_write_file_tool_writes_and_appends_workspace_file(tmp_path, monkeypatch):
     235    monkeypatch.chdir(tmp_path)
     236    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     237    db = AgentDB(tmp_path / "state.db")
     238    try:
     239        job_id = db.create_job("Write deliverable")
     240        run_id = db.start_run(job_id, model="fake")
     241        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_file")
     242        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     243
     244        raw = DEFAULT_REGISTRY.handle("write_file", {"path": "out/report.md", "content": "one\n"}, ctx)
     245        result = json.loads(raw)
     246        append_raw = DEFAULT_REGISTRY.handle(
     247            "write_file",
     248            {"path": "out/report.md", "content": "two\n", "mode": "append"},
     249            ctx,
     250        )
     251        append_result = json.loads(append_raw)
     252
     253        assert result["success"] is True
     254        assert append_result["success"] is True
     255        assert (tmp_path / "out" / "report.md").read_text() == "one\ntwo\n"
     256    finally:
     257        db.close()
     258
     259
     260def test_shell_exec_timeout_kills_process_group(tmp_path):
     261    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     262    db = AgentDB(tmp_path / "state.db")
     263    try:
     264        job_id = db.create_job("Run command")
     265        run_id = db.start_run(job_id, model="fake")
     266        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     267        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     268
     269        raw = DEFAULT_REGISTRY.handle("shell_exec", {"command": "sleep 5 | cat", "timeout_seconds": 1}, ctx)
     270        result = json.loads(raw)
     271
     272        assert result["success"] is False
     273        assert result["timed_out"] is True
     274        assert result["duration_seconds"] < 4
     275    finally:
     276        db.close()
     277
     278
     279def test_cleanup_registered_shell_processes_kills_orphaned_group(tmp_path):
     280    process = subprocess.Popen("sleep 30", shell=True, start_new_session=True)
     281    for _ in range(20):
     282        if process.poll() is None:
     283            try:
     284                os.kill(process.pid, 0)
     285                break
     286            except ProcessLookupError:
     287                pass
     288        time.sleep(0.02)
     289    registry = tmp_path / "runtime" / "shell_processes.jsonl"
     290    registry.parent.mkdir(parents=True)
     291    registry.write_text(json.dumps({"pid": process.pid, "command": "sleep 30"}) + "\n", encoding="utf-8")
     292    try:
     293        cleaned = cleanup_registered_shell_processes(tmp_path)
     294
     295        assert cleaned and cleaned[0]["pid"] == process.pid
     296        process.wait(timeout=3)
     297        assert not registry.exists()
     298    finally:
     299        if process.poll() is None:
     300            try:
     301                os.killpg(process.pid, signal.SIGKILL)
     302            except ProcessLookupError:
     303                pass
     304            process.wait(timeout=3)
     305
     306
     307def test_shell_exec_does_not_attach_local_ssh_config(tmp_path):
     308    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     309    db = AgentDB(tmp_path / "state.db")
     310    try:
     311        job_id = db.create_job("Run command")
     312        run_id = db.start_run(job_id, model="fake")
     313        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     314        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     315
     316        raw = DEFAULT_REGISTRY.handle("shell_exec", {"command": "ssh -V", "timeout_seconds": 5}, ctx)
     317        result = json.loads(raw)
     318
     319        assert "ssh_config" not in result
     320    finally:
     321        db.close()
     322
     323
     324def test_shell_exec_reports_nonzero_stderr_as_error(tmp_path):
     325    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     326    db = AgentDB(tmp_path / "state.db")
     327    try:
     328        job_id = db.create_job("Run command")
     329        run_id = db.start_run(job_id, model="fake")
     330        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     331        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     332
     333        raw = DEFAULT_REGISTRY.handle(
     334            "shell_exec",
     335            {"command": "printf 'sudo: a terminal is required to read the password\\n' >&2; exit 1", "timeout_seconds": 5},
     336            ctx,
     337        )
     338        result = json.loads(raw)
     339
     340        assert result["success"] is False
     341        assert "interactive sudo/password" in result["error"]
     342    finally:
     343        db.close()
     344
     345
     346def test_shell_exec_flags_sudo_password_hidden_by_success_status(tmp_path):
     347    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     348    db = AgentDB(tmp_path / "state.db")
     349    try:
     350        job_id = db.create_job("Run command")
     351        run_id = db.start_run(job_id, model="fake")
     352        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     353        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     354
     355        raw = DEFAULT_REGISTRY.handle(
     356            "shell_exec",
     357            {
     358                "command": (
     359                    "printf 'sudo: a terminal is required to read the password\\n"
     360                    "sudo: a password is required\\n'"
     361                ),
     362                "timeout_seconds": 5,
     363            },
     364            ctx,
     365        )
     366        result = json.loads(raw)
     367
     368        assert result["returncode"] == 0
     369        assert result["success"] is False
     370        assert "interactive sudo/password" in result["error"]
     371    finally:
     372        db.close()
     373
     374
     375def test_shell_exec_flags_missing_command_hidden_by_success_status(tmp_path):
     376    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     377    db = AgentDB(tmp_path / "state.db")
     378    try:
     379        job_id = db.create_job("Run command")
     380        run_id = db.start_run(job_id, model="fake")
     381        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     382        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     383
     384        raw = DEFAULT_REGISTRY.handle(
     385            "shell_exec",
     386            {"command": "printf '/bin/sh: 1: build-tool: not found\\n'", "timeout_seconds": 5},
     387            ctx,
     388        )
     389        result = json.loads(raw)
     390
     391        assert result["success"] is False
     392        assert "missing command" in result["error"]
     393    finally:
     394        db.close()
     395
     396
     397def test_shell_exec_flags_missing_absolute_executable_hidden_by_success_status(tmp_path):
     398    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     399    db = AgentDB(tmp_path / "state.db")
     400    try:
     401        job_id = db.create_job("Run command")
     402        run_id = db.start_run(job_id, model="fake")
     403        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     404        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     405
     406        raw = DEFAULT_REGISTRY.handle(
     407            "shell_exec",
     408            {"command": "printf '/bin/sh: 1: /tmp/tools/build-tool: not found\\n'", "timeout_seconds": 5},
     409            ctx,
     410        )
     411        result = json.loads(raw)
     412
     413        assert result["returncode"] == 0
     414        assert result["success"] is False
     415        assert "missing command" in result["error"]
     416        assert "/tmp/tools/build-tool: not found" in result["error"]
     417    finally:
     418        db.close()
     419
     420
     421def test_shell_exec_reports_empty_which_probe_as_missing_executable(tmp_path):
     422    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     423    db = AgentDB(tmp_path / "state.db")
     424    try:
     425        job_id = db.create_job("Run command")
     426        run_id = db.start_run(job_id, model="fake")
     427        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     428        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     429
     430        raw = DEFAULT_REGISTRY.handle(
     431            "shell_exec",
     432            {"command": "which definitely-missing-nipux-test-command", "timeout_seconds": 5},
     433            ctx,
     434        )
     435        result = json.loads(raw)
     436
     437        assert result["success"] is False
     438        assert result["returncode"] == 1
     439        assert result["error"] == "command probe found no executable: definitely-missing-nipux-test-command"
     440    finally:
     441        db.close()
     442
     443
     444def test_shell_exec_flags_empty_successful_probe_as_no_observation(tmp_path):
     445    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     446    db = AgentDB(tmp_path / "state.db")
     447    try:
     448        job_id = db.create_job("Run command")
     449        run_id = db.start_run(job_id, model="fake")
     450        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     451        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     452
     453        raw = DEFAULT_REGISTRY.handle(
     454            "shell_exec",
     455            {"command": "find /tmp/definitely-missing-nipux-test-path -maxdepth 1 2>/dev/null || true", "timeout_seconds": 5},
     456            ctx,
     457        )
     458        result = json.loads(raw)
     459
     460        assert result["returncode"] == 0
     461        assert result["success"] is False
     462        assert "produced no output" in result["error"]
     463        assert "filesystem probe" in result["error"]
     464    finally:
     465        db.close()
     466
     467
     468def test_shell_exec_flags_missing_which_probe_hidden_by_true(tmp_path):
     469    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     470    db = AgentDB(tmp_path / "state.db")
     471    try:
     472        job_id = db.create_job("Run command")
     473        run_id = db.start_run(job_id, model="fake")
     474        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     475        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     476
     477        raw = DEFAULT_REGISTRY.handle(
     478            "shell_exec",
     479            {"command": "which definitely-missing-nipux-test-command || true", "timeout_seconds": 5},
     480            ctx,
     481        )
     482        result = json.loads(raw)
     483
     484        assert result["returncode"] == 0
     485        assert result["success"] is False
     486        assert "probe found no executable" in result["error"]
     487    finally:
     488        db.close()
     489
     490
     491def test_shell_exec_flags_make_failure_hidden_by_pipe_status(tmp_path):
     492    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     493    db = AgentDB(tmp_path / "state.db")
     494    try:
     495        job_id = db.create_job("Run command")
     496        run_id = db.start_run(job_id, model="fake")
     497        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
     498        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     499
     500        raw = DEFAULT_REGISTRY.handle(
     501            "shell_exec",
     502            {"command": "printf 'Makefile:6: *** Build system changed:\\n.  Stop.\\n'", "timeout_seconds": 5},
     503            ctx,
     504        )
     505        result = json.loads(raw)
     506
     507        assert result["success"] is False
     508        assert "build/tool failure" in result["error"]
     509    finally:
     510        db.close()
     511
     512
     513def test_update_job_state_keeps_terminal_statuses_operator_only(tmp_path):
     514    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     515    db = AgentDB(tmp_path / "state.db")
     516    try:
     517        job_id = db.create_job("Keep running")
     518        run_id = db.start_run(job_id, model="fake")
     519        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="update_job_state")
     520        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     521
     522        for requested in ("paused", "cancelled", "completed", "failed"):
     523            raw = DEFAULT_REGISTRY.handle("update_job_state", {"status": requested}, ctx)
     524            result = json.loads(raw)
     525
     526            assert result["success"] is True
     527            assert result["requested_status"] == requested
     528            assert result["kept_running"] is True
     529            assert db.get_job(job_id)["status"] == "running"
     530            if requested == "completed":
     531                assert result["follow_up_task"]["title"] == "Audit latest checkpoint against objective"
     532                assert result["follow_up_task"]["status"] == "open"
     533                assert result["follow_up_task"]["output_contract"] == "decision"
     534                assert "prompt-to-artifact checklist" in result["follow_up_task"]["acceptance_criteria"]
     535                assert result["follow_up_task"]["evidence_needed"]
     536                assert result["follow_up_task"]["stall_behavior"]
     537                assert result["follow_up_task"]["metadata"]["source"] == "update_job_state"
     538                assert result["follow_up_task"]["metadata"]["completion_audit_required"] is True
     539            else:
     540                assert "follow_up_task" not in result
     541
     542        tasks = db.get_job(job_id)["metadata"]["task_queue"]
     543        assert [task["title"] for task in tasks] == ["Audit latest checkpoint against objective"]
     544    finally:
     545        db.close()
     546
     547
     548def test_report_update_tool_records_operator_visible_note(tmp_path):
     549    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     550    db = AgentDB(tmp_path / "state.db")
     551    try:
     552        job_id = db.create_job("Research topic")
     553        run_id = db.start_run(job_id, model="fake")
     554        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="report_update")
     555        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     556
     557        raw = DEFAULT_REGISTRY.handle("report_update", {"message": "Found a usable finding source", "category": "finding"}, ctx)
     558        result = json.loads(raw)
     559        job = db.get_job(job_id)
     560
     561        assert result["success"] is True
     562        assert job["metadata"]["agent_updates"][-1]["message"] == "Found a usable finding source"
     563        assert job["metadata"]["last_agent_update"]["category"] == "finding"
     564    finally:
     565        db.close()
     566
     567
     568def test_record_lesson_tool_records_durable_learning(tmp_path):
     569    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     570    db = AgentDB(tmp_path / "state.db")
     571    try:
     572        job_id = db.create_job("Research topic")
     573        run_id = db.start_run(job_id, model="fake")
     574        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_lesson")
     575        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     576
     577        raw = DEFAULT_REGISTRY.handle(
     578            "record_lesson",
     579            {"lesson": "Competitor low-evidence lists are not finding sources.", "category": "source_quality", "confidence": 0.8},
     580            ctx,
     581        )
     582        result = json.loads(raw)
     583        job = db.get_job(job_id)
     584
     585        assert result["success"] is True
     586        assert job["metadata"]["lessons"][-1]["lesson"] == "Competitor low-evidence lists are not finding sources."
     587        assert job["metadata"]["last_lesson"]["category"] == "source_quality"
     588    finally:
     589        db.close()
     590
     591
     592def test_record_lesson_cannot_clear_measurement_obligation_with_vague_lesson(tmp_path):
     593    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     594    db = AgentDB(tmp_path / "state.db")
     595    try:
     596        job_id = db.create_job(
     597            "Improve a measurable process",
     598            metadata={
     599                "pending_measurement_obligation": {
     600                    "source_step_no": 4,
     601                    "tool": "shell_exec",
     602                    "metric_candidates": ["42 units/s"],
     603                    "command": "run trial",
     604                }
     605            },
     606        )
     607        run_id = db.start_run(job_id, model="fake")
     608        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_lesson")
     609        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     610
     611        raw = DEFAULT_REGISTRY.handle(
     612            "record_lesson",
     613            {"lesson": "continue focused work", "category": "strategy"},
     614            ctx,
     615        )
     616        result = json.loads(raw)
     617        job = db.get_job(job_id)
     618
     619        assert result["success"] is False
     620        assert result["error"] == "measurement explanation required"
     621        assert job["metadata"]["pending_measurement_obligation"]["source_step_no"] == 4
     622        assert "lessons" not in job["metadata"]
     623    finally:
     624        db.close()
     625
     626
     627def test_record_lesson_can_explain_invalid_measurement_obligation(tmp_path):
     628    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     629    db = AgentDB(tmp_path / "state.db")
     630    try:
     631        job_id = db.create_job(
     632            "Improve a measurable process",
     633            metadata={
     634                "pending_measurement_obligation": {
     635                    "source_step_no": 4,
     636                    "tool": "shell_exec",
     637                    "metric_candidates": ["42 units/s"],
     638                    "command": "run trial",
     639                }
     640            },
     641        )
     642        run_id = db.start_run(job_id, model="fake")
     643        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_lesson")
     644        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     645
     646        raw = DEFAULT_REGISTRY.handle(
     647            "record_lesson",
     648            {
     649                "lesson": (
     650                    "The output was diagnostic only and did not contain a valid metric; "
     651                    "rerun the branch with a measured trial."
     652                ),
     653                "category": "mistake",
     654            },
     655            ctx,
     656        )
     657        result = json.loads(raw)
     658        job = db.get_job(job_id)
     659
     660        assert result["success"] is True
     661        assert job["metadata"].get("pending_measurement_obligation") == {}
     662        assert job["metadata"]["last_measurement_obligation"]["resolution_status"] == "explained"
     663        assert job["metadata"]["last_measurement_obligation"]["resolution_tool"] == "record_lesson"
     664    finally:
     665        db.close()
     666
     667
     668def test_memory_graph_tools_roundtrip(tmp_path):
     669    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     670    db = AgentDB(tmp_path / "state.db")
     671    try:
     672        job_id = db.create_job("Build durable project understanding")
     673        run_id = db.start_run(job_id, model="fake")
     674        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_memory_graph")
     675        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     676
     677        raw = DEFAULT_REGISTRY.handle(
     678            "record_memory_graph",
     679            {
     680                "nodes": [
     681                    {
     682                        "title": "Use measured checkpoints before expanding scope",
     683                        "kind": "strategy",
     684                        "status": "active",
     685                        "summary": "Convert branch outcomes into evidence-backed decisions before opening more work.",
     686                        "salience": 0.9,
     687                        "tags": ["progress", "validation"],
     688                        "evidence_refs": ["art_123"],
     689                    },
     690                    {
     691                        "title": "Open question: missing evaluator",
     692                        "kind": "question",
     693                        "status": "open",
     694                        "summary": "The job needs a concrete validation signal for the next branch.",
     695                    },
     696                ],
     697                "edges": [
     698                    {
     699                        "from_key": "Use measured checkpoints before expanding scope",
     700                        "to_key": "Open question: missing evaluator",
     701                        "relation": "raises",
     702                    }
     703                ],
     704            },
     705            ctx,
     706        )
     707        result = json.loads(raw)
     708
     709        assert result["success"] is True
     710        assert result["added_nodes"] == 2
     711        assert result["added_edges"] == 1
     712        job = db.get_job(job_id)
     713        graph = job["metadata"]["memory_graph"]
     714        assert len(graph["nodes"]) == 2
     715        assert graph["nodes"][0]["kind"] == "strategy"
     716        assert graph["nodes"][0]["evidence_refs"] == ["art_123"]
     717        assert db.list_events(job_id=job_id, event_types=["memory_node"])[0]["title"] == "memory graph"
     718
     719        search_raw = DEFAULT_REGISTRY.handle("search_memory_graph", {"query": "evaluator"}, ctx)
     720        search = json.loads(search_raw)
     721        assert search["success"] is True
     722        assert search["nodes"][0]["title"] == "Open question: missing evaluator"
     723        assert search["edges"][0]["relation"] == "raises"
     724    finally:
     725        db.close()
     726
     727
     728def test_record_source_and_findings_tools_update_ledgers(tmp_path):
     729    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     730    db = AgentDB(tmp_path / "state.db")
     731    try:
     732        job_id = db.create_job("Research topic")
     733        run_id = db.start_run(job_id, model="fake")
     734        source_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_source")
     735        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=source_step)
     736
     737        source_raw = DEFAULT_REGISTRY.handle(
     738            "record_source",
     739            {"source": "https://example.com", "source_type": "web_source", "usefulness_score": 0.8, "yield_count": 2},
     740            ctx,
     741        )
     742        finding_raw = DEFAULT_REGISTRY.handle(
     743            "record_findings",
     744            {
     745                "findings": [
     746                    {
     747                        "name": "Acme Finding",
     748                        "url": "https://acme.example",
     749                        "source_url": "https://example-source.com/acme",
     750                        "location": "Toronto",
     751                        "category": "example category",
     752                        "reason": "reusable result",
     753                        "score": 0.75,
     754                    }
     755                ]
     756            },
     757            ctx,
     758        )
     759        job = db.get_job(job_id)
     760
     761        assert json.loads(source_raw)["source"]["yield_count"] == 2
     762        finding_result = json.loads(finding_raw)
     763        assert finding_result["added"] == 1
     764        assert finding_result["sources_updated"] == 1
     765        assert job["metadata"]["source_ledger"][0]["source"] == "https://example.com"
     766        assert any(source["source"] == "https://example-source.com/acme" for source in job["metadata"]["source_ledger"])
     767        assert job["metadata"]["finding_ledger"][0]["name"] == "Acme Finding"
     768        assert job["metadata"]["last_agent_update"]["category"] == "finding"
     769    finally:
     770        db.close()
     771
     772
     773def test_record_source_requires_assessment(tmp_path):
     774    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     775    db = AgentDB(tmp_path / "state.db")
     776    try:
     777        job_id = db.create_job("Research topic")
     778        run_id = db.start_run(job_id, model="fake")
     779        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_source")
     780        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     781
     782        raw = DEFAULT_REGISTRY.handle("record_source", {"source": "https://example.com"}, ctx)
     783        result = json.loads(raw)
     784
     785        assert result["success"] is False
     786        assert result["error"] == "source assessment is required"
     787        assert db.get_job(job_id)["metadata"].get("source_ledger") is None
     788    finally:
     789        db.close()
     790
     791
     792def test_record_source_does_not_accept_type_without_assessment(tmp_path):
     793    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     794    db = AgentDB(tmp_path / "state.db")
     795    try:
     796        job_id = db.create_job("Research topic")
     797        run_id = db.start_run(job_id, model="fake")
     798        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_source")
     799        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     800
     801        raw = DEFAULT_REGISTRY.handle(
     802            "record_source",
     803            {"source": "https://example.com", "source_type": "web_source"},
     804            ctx,
     805        )
     806        result = json.loads(raw)
     807
     808        assert result["success"] is False
     809        assert result["error"] == "source assessment is required"
     810    finally:
     811        db.close()
     812
     813
     814def test_record_findings_reports_unchanged_duplicates_without_agent_update_noise(tmp_path):
     815    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     816    db = AgentDB(tmp_path / "state.db")
     817    try:
     818        job_id = db.create_job("Research topic")
     819        run_id = db.start_run(job_id, model="fake")
     820        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_findings")
     821        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     822        args = {
     823            "findings": [
     824                {
     825                    "name": "Reusable finding",
     826                    "source_url": "https://example-source.com/finding",
     827                    "reason": "Evidence-backed result",
     828                    "score": 0.75,
     829                }
     830            ]
     831        }
     832
     833        first = json.loads(DEFAULT_REGISTRY.handle("record_findings", args, ctx))
     834        agent_events_after_first = len(db.list_events(job_id=job_id, event_types=["agent_message"]))
     835        repeated = json.loads(DEFAULT_REGISTRY.handle("record_findings", args, ctx))
     836        agent_events_after_repeat = len(db.list_events(job_id=job_id, event_types=["agent_message"]))
     837
     838        assert first["added"] == 1
     839        assert first["updated"] == 0
     840        assert first["unchanged"] == 0
     841        assert repeated["added"] == 0
     842        assert repeated["updated"] == 0
     843        assert repeated["unchanged"] == 1
     844        assert agent_events_after_repeat == agent_events_after_first
     845    finally:
     846        db.close()
     847
     848
     849def test_record_findings_requires_evidence_anchor(tmp_path):
     850    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     851    db = AgentDB(tmp_path / "state.db")
     852    try:
     853        job_id = db.create_job("Research topic")
     854        run_id = db.start_run(job_id, model="fake")
     855        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_findings")
     856        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     857
     858        raw = DEFAULT_REGISTRY.handle(
     859            "record_findings",
     860            {"findings": [{"name": "Unsupported label", "category": "candidate"}]},
     861            ctx,
     862        )
     863        result = json.loads(raw)
     864
     865        assert result["success"] is False
     866        assert result["error"] == "no valid finding with name/title and evidence was provided"
     867        assert result["rejected"] == [{"name": "Unsupported label", "reason": "missing_evidence"}]
     868        assert db.get_job(job_id)["metadata"].get("finding_ledger") is None
     869    finally:
     870        db.close()
     871
     872
     873def test_record_findings_reports_rejected_unevidenced_items_in_mixed_batch(tmp_path):
     874    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     875    db = AgentDB(tmp_path / "state.db")
     876    try:
     877        job_id = db.create_job("Research topic")
     878        run_id = db.start_run(job_id, model="fake")
     879        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_findings")
     880        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     881
     882        raw = DEFAULT_REGISTRY.handle(
     883            "record_findings",
     884            {
     885                "findings": [
     886                    {"name": "Unsupported label"},
     887                    {"name": "Evidence-backed result", "metadata": {"source_url": "file:///tmp/evidence.txt"}},
     888                ]
     889            },
     890            ctx,
     891        )
     892        result = json.loads(raw)
     893        job = db.get_job(job_id)
     894
     895        assert result["success"] is True
     896        assert result["added"] == 1
     897        assert result["rejected"] == [{"name": "Unsupported label", "reason": "missing_evidence"}]
     898        assert job["metadata"]["finding_ledger"][0]["name"] == "Evidence-backed result"
     899        assert job["metadata"]["last_agent_update"]["metadata"]["rejected"] == 1
     900    finally:
     901        db.close()
     902
     903
     904def test_record_tasks_tool_updates_task_queue(tmp_path):
     905    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     906    db = AgentDB(tmp_path / "state.db")
     907    try:
     908        job_id = db.create_job("Research topic")
     909        run_id = db.start_run(job_id, model="fake")
     910        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
     911        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     912
     913        raw = DEFAULT_REGISTRY.handle(
     914            "record_tasks",
     915            {
     916                "tasks": [
     917                    {
     918                        "title": "Explore primary sources",
     919                        "status": "open",
     920                        "priority": 5,
     921                        "goal": "Find artifact-backed evidence",
     922                        "source_hint": "official docs",
     923                    }
     924                ]
     925            },
     926            ctx,
     927        )
     928        result = json.loads(raw)
     929        job = db.get_job(job_id)
     930
     931        assert result["success"] is True
     932        assert result["added"] == 1
     933        task = job["metadata"]["task_queue"][0]
     934        assert task["title"] == "Explore primary sources"
     935        assert task["priority"] == 5
     936        assert task["output_contract"] == "research"
     937        assert task["acceptance_criteria"]
     938        assert task["evidence_needed"]
     939        assert task["stall_behavior"]
     940        assert task["metadata"]["contract_inferred_fields"] == [
     941            "acceptance_criteria",
     942            "evidence_needed",
     943            "output_contract",
     944            "stall_behavior",
     945        ]
     946        assert job["metadata"]["last_agent_update"]["category"] == "plan"
     947    finally:
     948        db.close()
     949
     950
     951def test_record_tasks_dedupes_semantic_task_under_backlog_pressure(tmp_path):
     952    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     953    db = AgentDB(tmp_path / "state.db")
     954    try:
     955        job_id = db.create_job(
     956            "Keep a long-running job focused",
     957            metadata={
     958                "task_queue": [
     959                    {
     960                        "title": "Validate model files and run baseline benchmark",
     961                        "status": "open",
     962                        "priority": 5,
     963                        "goal": "Get a measured baseline.",
     964                    },
     965                    *[
     966                        {"title": f"Done branch {index}", "status": "done", "priority": 0}
     967                        for index in range(81)
     968                    ],
     969                ]
     970            },
     971        )
     972        run_id = db.start_run(job_id, model="fake")
     973        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
     974        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
     975
     976        raw = DEFAULT_REGISTRY.handle(
     977            "record_tasks",
     978            {
     979                "tasks": [
     980                    {
     981                        "title": "Validate candidate model files and run baseline benchmark",
     982                        "status": "active",
     983                        "priority": 10,
     984                        "goal": "Use the existing validation branch for the first measured run.",
     985                    }
     986                ]
     987            },
     988            ctx,
     989        )
     990        result = json.loads(raw)
     991        job = db.get_job(job_id)
     992        task_queue = job["metadata"]["task_queue"]
     993
     994        assert result["success"] is True
     995        assert result["added"] == 0
     996        assert result["updated"] == 1
     997        assert len(task_queue) == 82
     998        task = task_queue[0]
     999        assert task["title"] == "Validate model files and run baseline benchmark"
    1000        assert task["status"] == "active"
    1001        assert task["metadata"]["original_title"] == "Validate candidate model files and run baseline benchmark"
    1002        assert task["metadata"]["matched_existing_task"]["title"] == "Validate model files and run baseline benchmark"
    1003    finally:
    1004        db.close()
    1005
    1006
    1007def test_record_tasks_reports_unchanged_duplicates_without_agent_update_noise(tmp_path):
    1008    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1009    db = AgentDB(tmp_path / "state.db")
    1010    try:
    1011        job_id = db.create_job("Research topic")
    1012        run_id = db.start_run(job_id, model="fake")
    1013        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1014        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1015        args = {
    1016            "tasks": [
    1017                {
    1018                    "title": "Explore primary sources",
    1019                    "status": "open",
    1020                    "priority": 5,
    1021                    "goal": "Find artifact-backed evidence",
    1022                }
    1023            ]
    1024        }
    1025
    1026        first = json.loads(DEFAULT_REGISTRY.handle("record_tasks", args, ctx))
    1027        agent_events_after_first = len(db.list_events(job_id=job_id, event_types=["agent_message"]))
    1028        db.update_job_metadata(
    1029            job_id,
    1030            {"pending_measurement_obligation": {"source_step_no": 1, "metric_candidates": ["score"]}},
    1031        )
    1032        repeated = json.loads(DEFAULT_REGISTRY.handle("record_tasks", args, ctx))
    1033        agent_events_after_repeat = len(db.list_events(job_id=job_id, event_types=["agent_message"]))
    1034
    1035        assert first["added"] == 1
    1036        assert first["updated"] == 0
    1037        assert first["unchanged"] == 0
    1038        assert repeated["added"] == 0
    1039        assert repeated["updated"] == 0
    1040        assert repeated["unchanged"] == 1
    1041        assert agent_events_after_repeat == agent_events_after_first
    1042        assert db.get_job(job_id)["metadata"]["pending_measurement_obligation"]["source_step_no"] == 1
    1043    finally:
    1044        db.close()
    1045
    1046
    1047def test_record_tasks_cannot_defer_measurement_with_unrelated_task(tmp_path):
    1048    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1049    db = AgentDB(tmp_path / "state.db")
    1050    try:
    1051        job_id = db.create_job(
    1052            "Improve measurable process",
    1053            metadata={
    1054                "pending_measurement_obligation": {
    1055                    "source_step_no": 8,
    1056                    "tool": "shell_exec",
    1057                    "metric_candidates": ["42 units/s"],
    1058                }
    1059            },
    1060        )
    1061        run_id = db.start_run(job_id, model="fake")
    1062        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1063        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1064
    1065        raw = DEFAULT_REGISTRY.handle(
    1066            "record_tasks",
    1067            {"tasks": [{"title": "Read more background sources", "status": "open", "output_contract": "research"}]},
    1068            ctx,
    1069        )
    1070        result = json.loads(raw)
    1071        job = db.get_job(job_id)
    1072
    1073        assert result["success"] is False
    1074        assert result["error"] == "measurement task required"
    1075        assert job["metadata"]["pending_measurement_obligation"]["source_step_no"] == 8
    1076        assert "task_queue" not in job["metadata"]
    1077    finally:
    1078        db.close()
    1079
    1080
    1081def test_record_tasks_can_defer_measurement_with_explicit_measurement_task(tmp_path):
    1082    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1083    db = AgentDB(tmp_path / "state.db")
    1084    try:
    1085        job_id = db.create_job(
    1086            "Improve measurable process",
    1087            metadata={
    1088                "pending_measurement_obligation": {
    1089                    "source_step_no": 8,
    1090                    "tool": "shell_exec",
    1091                    "metric_candidates": ["42 units/s"],
    1092                }
    1093            },
    1094        )
    1095        run_id = db.start_run(job_id, model="fake")
    1096        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1097        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1098
    1099        raw = DEFAULT_REGISTRY.handle(
    1100            "record_tasks",
    1101            {
    1102                "tasks": [{
    1103                    "title": "Rerun the branch and record the missing measurement",
    1104                    "status": "open",
    1105                    "output_contract": "experiment",
    1106                    "acceptance_criteria": "valid metric recorded",
    1107                    "evidence_needed": "measured command output",
    1108                    "stall_behavior": "record blocker if measurement cannot be obtained",
    1109                }]
    1110            },
    1111            ctx,
    1112        )
    1113        result = json.loads(raw)
    1114        job = db.get_job(job_id)
    1115
    1116        assert result["success"] is True
    1117        assert result["added"] == 1
    1118        assert job["metadata"].get("pending_measurement_obligation") == {}
    1119        assert job["metadata"]["last_measurement_obligation"]["resolution_status"] == "deferred"
    1120        assert job["metadata"]["last_measurement_obligation"]["resolution_tool"] == "record_tasks"
    1121    finally:
    1122        db.close()
    1123
    1124
    1125def test_record_roadmap_tool_updates_roadmap(tmp_path):
    1126    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1127    db = AgentDB(tmp_path / "state.db")
    1128    try:
    1129        job_id = db.create_job("Build a broad generic outcome")
    1130        run_id = db.start_run(job_id, model="fake")
    1131        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_roadmap")
    1132        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1133
    1134        raw = DEFAULT_REGISTRY.handle(
    1135            "record_roadmap",
    1136            {
    1137                "title": "Generic Roadmap",
    1138                "status": "active",
    1139                "scope": "Coordinate broad work through milestones.",
    1140                "current_milestone": "Foundation",
    1141                "validation_contract": "Each milestone needs evidence.",
    1142                "milestones": [{
    1143                    "title": "Foundation",
    1144                    "status": "active",
    1145                    "priority": 7,
    1146                    "acceptance_criteria": "first durable output exists",
    1147                    "evidence_needed": "artifact and ledger update",
    1148                    "features": [{
    1149                        "title": "Create first checkpoint",
    1150                        "status": "active",
    1151                        "output_contract": "artifact",
    1152                    }],
    1153                }],
    1154            },
    1155            ctx,
    1156        )
    1157        result = json.loads(raw)
    1158        job = db.get_job(job_id)
    1159        roadmap = job["metadata"]["roadmap"]
    1160
    1161        assert result["success"] is True
    1162        assert roadmap["title"] == "Generic Roadmap"
    1163        assert roadmap["status"] == "active"
    1164        assert roadmap["milestones"][0]["title"] == "Foundation"
    1165        assert roadmap["milestones"][0]["features"][0]["title"] == "Create first checkpoint"
    1166        assert job["metadata"]["last_agent_update"]["metadata"]["roadmap_status"] == "active"
    1167    finally:
    1168        db.close()
    1169
    1170
    1171def test_record_roadmap_dedupes_milestone_titles_even_when_keys_change(tmp_path):
    1172    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1173    db = AgentDB(tmp_path / "state.db")
    1174    try:
    1175        job_id = db.create_job("Keep broad work coordinated")
    1176        run_id = db.start_run(job_id, model="fake")
    1177        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_roadmap")
    1178        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1179
    1180        DEFAULT_REGISTRY.handle(
    1181            "record_roadmap",
    1182            {
    1183                "title": "Generic Roadmap",
    1184                "milestones": [{
    1185                    "key": "initial-key",
    1186                    "title": "Foundation",
    1187                    "status": "planned",
    1188                    "features": [{"key": "feature-a", "title": "First feature", "status": "planned"}],
    1189                }],
    1190            },
    1191            ctx,
    1192        )
    1193        DEFAULT_REGISTRY.handle(
    1194            "record_roadmap",
    1195            {
    1196                "title": "Generic Roadmap",
    1197                "milestones": [{
    1198                    "key": "model-invented-key",
    1199                    "title": "Foundation",
    1200                    "status": "active",
    1201                    "features": [{"key": "different-feature-key", "title": "First feature", "status": "done"}],
    1202                }],
    1203            },
    1204            ctx,
    1205        )
    1206        roadmap = db.get_job(job_id)["metadata"]["roadmap"]
    1207
    1208        assert len(roadmap["milestones"]) == 1
    1209        assert roadmap["milestones"][0]["status"] == "active"
    1210        assert len(roadmap["milestones"][0]["features"]) == 1
    1211        assert roadmap["milestones"][0]["features"][0]["status"] == "done"
    1212    finally:
    1213        db.close()
    1214
    1215
    1216def test_record_milestone_validation_creates_follow_up_tasks(tmp_path):
    1217    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1218    db = AgentDB(tmp_path / "state.db")
    1219    try:
    1220        job_id = db.create_job("Validate broad work")
    1221        run_id = db.start_run(job_id, model="fake")
    1222        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_milestone_validation")
    1223        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1224
    1225        raw = DEFAULT_REGISTRY.handle(
    1226            "record_milestone_validation",
    1227            {
    1228                "milestone": "Foundation",
    1229                "validation_status": "failed",
    1230                "result": "Missing durable evidence.",
    1231                "issues": ["no artifact"],
    1232                "next_action": "Create evidence.",
    1233                "follow_up_tasks": [{
    1234                    "title": "Produce missing evidence",
    1235                    "output_contract": "artifact",
    1236                    "acceptance_criteria": "saved output exists",
    1237                }],
    1238            },
    1239            ctx,
    1240        )
    1241        result = json.loads(raw)
    1242        job = db.get_job(job_id)
    1243        roadmap = job["metadata"]["roadmap"]
    1244
    1245        assert result["success"] is True
    1246        assert result["validation"]["validation_status"] == "failed"
    1247        assert result["follow_up_tasks"][0]["title"] == "Produce missing evidence"
    1248        assert roadmap["milestones"][0]["status"] == "blocked"
    1249        assert job["metadata"]["task_queue"][0]["parent"] == "Foundation"
    1250        assert job["metadata"]["last_agent_update"]["metadata"]["validation_status"] == "failed"
    1251    finally:
    1252        db.close()
    1253
    1254
    1255def test_record_milestone_validation_requires_evidence_for_passed_status(tmp_path):
    1256    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1257    db = AgentDB(tmp_path / "state.db")
    1258    try:
    1259        job_id = db.create_job("Validate broad work")
    1260        run_id = db.start_run(job_id, model="fake")
    1261        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_milestone_validation")
    1262        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1263
    1264        raw = DEFAULT_REGISTRY.handle(
    1265            "record_milestone_validation",
    1266            {
    1267                "milestone": "Foundation",
    1268                "validation_status": "passed",
    1269            },
    1270            ctx,
    1271        )
    1272        result = json.loads(raw)
    1273
    1274        assert result["success"] is False
    1275        assert result["error"] == "passed milestone validation requires evidence or result"
    1276        assert db.get_job(job_id)["metadata"].get("roadmap") is None
    1277    finally:
    1278        db.close()
    1279
    1280
    1281def test_record_milestone_validation_allows_passed_status_with_metadata_evidence(tmp_path):
    1282    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1283    db = AgentDB(tmp_path / "state.db")
    1284    try:
    1285        job_id = db.create_job("Validate broad work")
    1286        run_id = db.start_run(job_id, model="fake")
    1287        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_milestone_validation")
    1288        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1289
    1290        raw = DEFAULT_REGISTRY.handle(
    1291            "record_milestone_validation",
    1292            {
    1293                "milestone": "Foundation",
    1294                "validation_status": "passed",
    1295                "metadata": {"artifact_id": "art_123"},
    1296            },
    1297            ctx,
    1298        )
    1299        result = json.loads(raw)
    1300
    1301        assert result["success"] is True
    1302        assert result["validation"]["validation_status"] == "passed"
    1303    finally:
    1304        db.close()
    1305
    1306
    1307def test_record_milestone_validation_requires_gap_for_failed_or_blocked_status(tmp_path):
    1308    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1309    db = AgentDB(tmp_path / "state.db")
    1310    try:
    1311        job_id = db.create_job("Validate broad work")
    1312        run_id = db.start_run(job_id, model="fake")
    1313        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_milestone_validation")
    1314        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1315
    1316        failed = json.loads(DEFAULT_REGISTRY.handle(
    1317            "record_milestone_validation",
    1318            {
    1319                "milestone": "Foundation",
    1320                "validation_status": "failed",
    1321            },
    1322            ctx,
    1323        ))
    1324        blocked = json.loads(DEFAULT_REGISTRY.handle(
    1325            "record_milestone_validation",
    1326            {
    1327                "milestone": "Foundation",
    1328                "validation_status": "blocked",
    1329            },
    1330            ctx,
    1331        ))
    1332
    1333        assert failed["success"] is False
    1334        assert failed["error"] == "failed milestone validation requires a gap, issue, evidence, next_action, or follow-up task"
    1335        assert blocked["success"] is False
    1336        assert blocked["error"] == "blocked milestone validation requires a gap, issue, evidence, next_action, or follow-up task"
    1337        assert db.get_job(job_id)["metadata"].get("roadmap") is None
    1338    finally:
    1339        db.close()
    1340
    1341
    1342def test_record_experiment_tool_tracks_best_measured_result(tmp_path):
    1343    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1344    db = AgentDB(tmp_path / "state.db")
    1345    try:
    1346        job_id = db.create_job("Improve a measurable process")
    1347        run_id = db.start_run(job_id, model="fake")
    1348        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    1349        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1350
    1351        first = DEFAULT_REGISTRY.handle(
    1352            "record_experiment",
    1353            {
    1354                "title": "baseline attempt",
    1355                "status": "measured",
    1356                "metric_name": "score",
    1357                "metric_value": 2.0,
    1358                "metric_unit": "units",
    1359                "higher_is_better": True,
    1360                "config": {"variant": "a"},
    1361                "result": "baseline measured",
    1362                "next_action": "try variant b",
    1363            },
    1364            ctx,
    1365        )
    1366        second = DEFAULT_REGISTRY.handle(
    1367            "record_experiment",
    1368            {
    1369                "title": "second attempt",
    1370                "status": "measured",
    1371                "metric_name": "score",
    1372                "metric_value": 3.5,
    1373                "metric_unit": "units",
    1374                "higher_is_better": True,
    1375                "config": {"variant": "b"},
    1376                "result": "improved",
    1377                "next_action": "test a different branch",
    1378            },
    1379            ctx,
    1380        )
    1381        job = db.get_job(job_id)
    1382        experiments = job["metadata"]["experiment_ledger"]
    1383
    1384        assert json.loads(first)["experiment"]["best_observed"] is True
    1385        assert json.loads(second)["experiment"]["best_observed"] is True
    1386        assert experiments[0]["best_observed"] is False
    1387        assert experiments[1]["best_observed"] is True
    1388        assert experiments[1]["delta_from_previous_best"] == 1.5
    1389        assert job["metadata"]["best_experiment_record"]["title"] == "second attempt"
    1390        assert job["metadata"]["last_agent_update"]["metadata"]["best_observed"] is True
    1391    finally:
    1392        db.close()
    1393
    1394
    1395def test_record_experiment_synthesizes_missing_title(tmp_path):
    1396    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1397    db = AgentDB(tmp_path / "state.db")
    1398    try:
    1399        job_id = db.create_job("Improve a measurable process")
    1400        run_id = db.start_run(job_id, model="fake")
    1401        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    1402        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1403
    1404        raw = DEFAULT_REGISTRY.handle(
    1405            "record_experiment",
    1406            {
    1407                "status": "planned",
    1408                "metric_name": "download_progress_bytes",
    1409                "result": "download incomplete",
    1410            },
    1411            ctx,
    1412        )
    1413        result = json.loads(raw)
    1414
    1415        assert result["success"] is True
    1416        assert result["experiment"]["title"] == "download_progress_bytes"
    1417    finally:
    1418        db.close()
    1419
    1420
    1421def test_record_experiment_requires_next_action_for_closed_trials(tmp_path):
    1422    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1423    db = AgentDB(tmp_path / "state.db")
    1424    try:
    1425        job_id = db.create_job("Improve a measurable process")
    1426        run_id = db.start_run(job_id, model="fake")
    1427        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    1428        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1429
    1430        raw = DEFAULT_REGISTRY.handle(
    1431            "record_experiment",
    1432            {
    1433                "title": "blocked attempt",
    1434                "status": "blocked",
    1435                "metric_name": "score",
    1436                "result": "no valid measurement",
    1437            },
    1438            ctx,
    1439        )
    1440        result = json.loads(raw)
    1441
    1442        assert result["success"] is False
    1443        assert result["error"] == "next_action is required for measured, failed, blocked, or skipped experiments"
    1444        assert db.get_job(job_id)["metadata"].get("experiment_ledger") is None
    1445    finally:
    1446        db.close()
    1447
    1448
    1449def test_record_experiment_requires_context_for_closed_non_measured_trials(tmp_path):
    1450    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1451    db = AgentDB(tmp_path / "state.db")
    1452    try:
    1453        job_id = db.create_job("Improve a measurable process")
    1454        run_id = db.start_run(job_id, model="fake")
    1455        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    1456        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1457
    1458        raw = DEFAULT_REGISTRY.handle(
    1459            "record_experiment",
    1460            {
    1461                "title": "blocked attempt",
    1462                "status": "blocked",
    1463                "metric_name": "score",
    1464                "next_action": "try a different branch",
    1465            },
    1466            ctx,
    1467        )
    1468        result = json.loads(raw)
    1469
    1470        assert result["success"] is False
    1471        assert result["error"] == "blocked experiments require result, evidence, config, or metadata"
    1472        assert db.get_job(job_id)["metadata"].get("experiment_ledger") is None
    1473    finally:
    1474        db.close()
    1475
    1476
    1477def test_record_experiment_accepts_blocked_trial_with_context(tmp_path):
    1478    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1479    db = AgentDB(tmp_path / "state.db")
    1480    try:
    1481        job_id = db.create_job("Improve a measurable process")
    1482        run_id = db.start_run(job_id, model="fake")
    1483        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    1484        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1485
    1486        raw = DEFAULT_REGISTRY.handle(
    1487            "record_experiment",
    1488            {
    1489                "title": "blocked attempt",
    1490                "status": "blocked",
    1491                "metric_name": "score",
    1492                "result": "required input was unavailable",
    1493                "next_action": "try a different branch",
    1494            },
    1495            ctx,
    1496        )
    1497        result = json.loads(raw)
    1498
    1499        assert result["success"] is True
    1500        assert result["experiment"]["status"] == "blocked"
    1501        assert result["experiment"]["result"] == "required input was unavailable"
    1502    finally:
    1503        db.close()
    1504
    1505
    1506def test_record_experiment_requires_metric_for_measured_trials(tmp_path):
    1507    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1508    db = AgentDB(tmp_path / "state.db")
    1509    try:
    1510        job_id = db.create_job("Improve a measurable process")
    1511        run_id = db.start_run(job_id, model="fake")
    1512        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    1513        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1514
    1515        missing_value = json.loads(DEFAULT_REGISTRY.handle(
    1516            "record_experiment",
    1517            {
    1518                "title": "invalid measurement",
    1519                "status": "measured",
    1520                "metric_name": "score",
    1521                "result": "looked better but no numeric metric",
    1522                "next_action": "run a real measurement",
    1523            },
    1524            ctx,
    1525        ))
    1526        missing_name = json.loads(DEFAULT_REGISTRY.handle(
    1527            "record_experiment",
    1528            {
    1529                "title": "invalid measurement",
    1530                "status": "measured",
    1531                "metric_value": 2.7,
    1532                "result": "numeric result with no metric name",
    1533                "next_action": "label the metric and retry",
    1534            },
    1535            ctx,
    1536        ))
    1537
    1538        assert missing_value["success"] is False
    1539        assert missing_value["error"] == "measured experiments require metric_name and numeric metric_value"
    1540        assert missing_name["success"] is False
    1541        assert missing_name["error"] == "measured experiments require metric_name and numeric metric_value"
    1542        assert db.get_job(job_id)["metadata"].get("experiment_ledger") is None
    1543    finally:
    1544        db.close()
    1545
    1546
    1547def test_record_experiment_accepts_numeric_metric_strings(tmp_path):
    1548    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1549    db = AgentDB(tmp_path / "state.db")
    1550    try:
    1551        job_id = db.create_job("Improve a measurable process")
    1552        run_id = db.start_run(job_id, model="fake")
    1553        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    1554        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1555
    1556        raw = DEFAULT_REGISTRY.handle(
    1557            "record_experiment",
    1558            {
    1559                "title": "string metric",
    1560                "status": "measured",
    1561                "metric_name": "score",
    1562                "metric_value": "2.7",
    1563                "metric_unit": "units",
    1564                "result": "measured from output",
    1565                "next_action": "try the next branch",
    1566            },
    1567            ctx,
    1568        )
    1569        result = json.loads(raw)
    1570
    1571        assert result["success"] is True
    1572        assert result["experiment"]["metric_value"] == 2.7
    1573    finally:
    1574        db.close()
    1575
    1576
    1577def test_acknowledge_operator_context_tool_marks_context(tmp_path):
    1578    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1579    db = AgentDB(tmp_path / "state.db")
    1580    try:
    1581        job_id = db.create_job("Run with operator corrections")
    1582        entry = db.append_operator_message(job_id, "use the corrected target", source="chat")
    1583        db.claim_operator_messages(job_id, modes=("steer",), limit=1)
    1584        run_id = db.start_run(job_id, model="fake")
    1585        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="acknowledge_operator_context")
    1586        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1587
    1588        raw = DEFAULT_REGISTRY.handle(
    1589            "acknowledge_operator_context",
    1590            {"message_ids": [entry["event_id"]], "summary": "correction incorporated"},
    1591            ctx,
    1592        )
    1593        result = json.loads(raw)
    1594        job = db.get_job(job_id)
    1595
    1596        assert result["success"] is True
    1597        assert result["count"] == 1
    1598        assert job["metadata"]["operator_messages"][0]["acknowledged_at"]
    1599        assert job["metadata"]["last_operator_context_ack"]["summary"] == "correction incorporated"
    1600    finally:
    1601        db.close()
    1602
    1603
    1604def test_acknowledge_operator_context_requires_active_context(tmp_path):
    1605    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1606    db = AgentDB(tmp_path / "state.db")
    1607    try:
    1608        job_id = db.create_job("Run without operator corrections")
    1609        run_id = db.start_run(job_id, model="fake")
    1610        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="acknowledge_operator_context")
    1611        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1612
    1613        raw = DEFAULT_REGISTRY.handle(
    1614            "acknowledge_operator_context",
    1615            {"summary": "ordinary progress note"},
    1616            ctx,
    1617        )
    1618        result = json.loads(raw)
    1619
    1620        assert result["success"] is False
    1621        assert result["recoverable"] is True
    1622        assert result["error"] == "no active operator context to acknowledge"
    1623        assert "report_update" in result["guidance"]
    1624        assert "last_operator_context_ack" not in db.get_job(job_id)["metadata"]
    1625    finally:
    1626        db.close()
    1627
    1628
    1629def test_record_tasks_accepts_generic_output_contracts(tmp_path):
    1630    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1631    db = AgentDB(tmp_path / "state.db")
    1632    try:
    1633        job_id = db.create_job("Improve measurable process")
    1634        run_id = db.start_run(job_id, model="fake")
    1635        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1636        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1637
    1638        raw = DEFAULT_REGISTRY.handle(
    1639            "record_tasks",
    1640            {
    1641                "tasks": [{
    1642                    "title": "Run one comparison",
    1643                    "status": "open",
    1644                    "output_contract": "experiment",
    1645                    "acceptance_criteria": "metric recorded",
    1646                    "evidence_needed": "command output or artifact",
    1647                    "stall_behavior": "record blocker and pivot",
    1648                }]
    1649            },
    1650            ctx,
    1651        )
    1652        result = json.loads(raw)
    1653        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    1654
    1655        assert result["success"] is True
    1656        assert task["output_contract"] == "experiment"
    1657        assert task["acceptance_criteria"] == "metric recorded"
    1658        assert task["evidence_needed"] == "command output or artifact"
    1659        assert task["stall_behavior"] == "record blocker and pivot"
    1660    finally:
    1661        db.close()
    1662
    1663
    1664def test_record_tasks_promotes_output_contract_from_metadata(tmp_path):
    1665    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1666    db = AgentDB(tmp_path / "state.db")
    1667    try:
    1668        job_id = db.create_job("Improve measurable process")
    1669        run_id = db.start_run(job_id, model="fake")
    1670        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1671        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1672
    1673        raw = DEFAULT_REGISTRY.handle(
    1674            "record_tasks",
    1675            {
    1676                "tasks": [{
    1677                    "title": "Validate concrete candidate",
    1678                    "status": "open",
    1679                    "metadata": {"output_contract": "action", "source": "planner"},
    1680                    "acceptance_criteria": "candidate is tested",
    1681                }]
    1682            },
    1683            ctx,
    1684        )
    1685        result = json.loads(raw)
    1686        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    1687
    1688        assert result["success"] is True
    1689        assert task["output_contract"] == "action"
    1690        assert task["metadata"]["source"] == "planner"
    1691    finally:
    1692        db.close()
    1693
    1694
    1695def test_record_tasks_downgrades_done_artifact_without_delivery_evidence(tmp_path):
    1696    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1697    db = AgentDB(tmp_path / "state.db")
    1698    try:
    1699        job_id = db.create_job("Update a deliverable")
    1700        run_id = db.start_run(job_id, model="fake")
    1701        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1702        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1703
    1704        raw = DEFAULT_REGISTRY.handle(
    1705            "record_tasks",
    1706            {
    1707                "tasks": [{
    1708                    "title": "Update report draft",
    1709                    "status": "done",
    1710                    "output_contract": "artifact",
    1711                    "result": "Updated the report",
    1712                }]
    1713            },
    1714            ctx,
    1715        )
    1716        result = json.loads(raw)
    1717        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    1718
    1719        assert result["success"] is True
    1720        assert task["status"] == "active"
    1721        assert task["metadata"]["completion_validation"] == "missing_recent_deliverable_evidence"
    1722        assert task["metadata"]["claimed_result"] == "Updated the report"
    1723    finally:
    1724        db.close()
    1725
    1726
    1727def test_record_tasks_downgrades_done_without_result_evidence(tmp_path):
    1728    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1729    db = AgentDB(tmp_path / "state.db")
    1730    try:
    1731        job_id = db.create_job("Validate generic work")
    1732        run_id = db.start_run(job_id, model="fake")
    1733        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1734        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1735
    1736        raw = DEFAULT_REGISTRY.handle(
    1737            "record_tasks",
    1738            {
    1739                "tasks": [{
    1740                    "title": "Check current branch",
    1741                    "status": "done",
    1742                    "output_contract": "decision",
    1743                }]
    1744            },
    1745            ctx,
    1746        )
    1747        result = json.loads(raw)
    1748        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    1749
    1750        assert result["success"] is True
    1751        assert task["status"] == "active"
    1752        assert task["metadata"]["completion_validation"] == "missing_result_evidence"
    1753    finally:
    1754        db.close()
    1755
    1756
    1757def test_record_tasks_downgrades_done_research_without_durable_evidence(tmp_path):
    1758    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1759    db = AgentDB(tmp_path / "state.db")
    1760    try:
    1761        job_id = db.create_job("Research a topic")
    1762        run_id = db.start_run(job_id, model="fake")
    1763        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1764        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1765
    1766        raw = DEFAULT_REGISTRY.handle(
    1767            "record_tasks",
    1768            {
    1769                "tasks": [{
    1770                    "title": "Synthesize source evidence",
    1771                    "status": "done",
    1772                    "output_contract": "research",
    1773                    "result": "Found useful background.",
    1774                }]
    1775            },
    1776            ctx,
    1777        )
    1778        result = json.loads(raw)
    1779        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    1780
    1781        assert result["success"] is True
    1782        assert task["status"] == "active"
    1783        assert task["metadata"]["completion_validation"] == "missing_research_evidence"
    1784        assert task["metadata"]["claimed_result"] == "Found useful background."
    1785    finally:
    1786        db.close()
    1787
    1788
    1789def test_record_tasks_allows_done_research_after_source_evidence(tmp_path):
    1790    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1791    db = AgentDB(tmp_path / "state.db")
    1792    try:
    1793        job_id = db.create_job("Research a topic")
    1794        run_id = db.start_run(job_id, model="fake")
    1795        source_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_source")
    1796        db.finish_step(source_step, status="completed", summary="source recorded", output_data={"success": True})
    1797        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1798        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1799
    1800        raw = DEFAULT_REGISTRY.handle(
    1801            "record_tasks",
    1802            {
    1803                "tasks": [{
    1804                    "title": "Synthesize source evidence",
    1805                    "status": "done",
    1806                    "output_contract": "research",
    1807                    "result": "Source ledger records the useful branch.",
    1808                }]
    1809            },
    1810            ctx,
    1811        )
    1812        result = json.loads(raw)
    1813        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    1814
    1815        assert result["success"] is True
    1816        assert task["status"] == "done"
    1817        assert "completion_validation" not in task.get("metadata", {})
    1818    finally:
    1819        db.close()
    1820
    1821
    1822def test_record_tasks_allows_done_research_with_metadata_evidence(tmp_path):
    1823    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1824    db = AgentDB(tmp_path / "state.db")
    1825    try:
    1826        job_id = db.create_job("Research a topic")
    1827        run_id = db.start_run(job_id, model="fake")
    1828        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1829        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1830
    1831        raw = DEFAULT_REGISTRY.handle(
    1832            "record_tasks",
    1833            {
    1834                "tasks": [{
    1835                    "title": "Synthesize source evidence",
    1836                    "status": "done",
    1837                    "output_contract": "research",
    1838                    "metadata": {"source_url": "https://example.com/source"},
    1839                }]
    1840            },
    1841            ctx,
    1842        )
    1843        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    1844
    1845        assert json.loads(raw)["success"] is True
    1846        assert task["status"] == "done"
    1847        assert "completion_validation" not in task.get("metadata", {})
    1848    finally:
    1849        db.close()
    1850
    1851
    1852def test_record_tasks_downgrades_done_experiment_without_measurement_evidence(tmp_path):
    1853    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1854    db = AgentDB(tmp_path / "state.db")
    1855    try:
    1856        job_id = db.create_job("Improve a measurable process")
    1857        run_id = db.start_run(job_id, model="fake")
    1858        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1859        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1860
    1861        raw = DEFAULT_REGISTRY.handle(
    1862            "record_tasks",
    1863            {
    1864                "tasks": [{
    1865                    "title": "Run comparison",
    1866                    "status": "done",
    1867                    "output_contract": "experiment",
    1868                    "result": "The comparison improved.",
    1869                }]
    1870            },
    1871            ctx,
    1872        )
    1873        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    1874
    1875        assert json.loads(raw)["success"] is True
    1876        assert task["status"] == "active"
    1877        assert task["metadata"]["completion_validation"] == "missing_experiment_evidence"
    1878    finally:
    1879        db.close()
    1880
    1881
    1882def test_record_tasks_allows_done_experiment_after_measurement_evidence(tmp_path):
    1883    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1884    db = AgentDB(tmp_path / "state.db")
    1885    try:
    1886        job_id = db.create_job("Improve a measurable process")
    1887        run_id = db.start_run(job_id, model="fake")
    1888        experiment_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    1889        db.finish_step(experiment_step, status="completed", summary="experiment measured", output_data={"success": True})
    1890        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1891        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1892
    1893        raw = DEFAULT_REGISTRY.handle(
    1894            "record_tasks",
    1895            {
    1896                "tasks": [{
    1897                    "title": "Run comparison",
    1898                    "status": "done",
    1899                    "output_contract": "experiment",
    1900                    "result": "Experiment ledger records the measured comparison.",
    1901                }]
    1902            },
    1903            ctx,
    1904        )
    1905        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    1906
    1907        assert json.loads(raw)["success"] is True
    1908        assert task["status"] == "done"
    1909        assert "completion_validation" not in task.get("metadata", {})
    1910    finally:
    1911        db.close()
    1912
    1913
    1914def test_record_tasks_downgrades_done_action_after_read_only_shell(tmp_path):
    1915    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1916    db = AgentDB(tmp_path / "state.db")
    1917    try:
    1918        job_id = db.create_job("Change a local workspace")
    1919        run_id = db.start_run(job_id, model="fake")
    1920        shell_step = db.add_step(
    1921            job_id=job_id,
    1922            run_id=run_id,
    1923            kind="tool",
    1924            tool_name="shell_exec",
    1925            input_data={"arguments": {"command": "ls -la"}},
    1926        )
    1927        db.finish_step(shell_step, status="completed", summary="shell_exec rc=0")
    1928        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1929        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1930
    1931        raw = DEFAULT_REGISTRY.handle(
    1932            "record_tasks",
    1933            {
    1934                "tasks": [{
    1935                    "title": "Apply change",
    1936                    "status": "done",
    1937                    "output_contract": "action",
    1938                    "result": "Inspected the workspace.",
    1939                }]
    1940            },
    1941            ctx,
    1942        )
    1943        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    1944
    1945        assert json.loads(raw)["success"] is True
    1946        assert task["status"] == "active"
    1947        assert task["metadata"]["completion_validation"] == "missing_action_evidence"
    1948    finally:
    1949        db.close()
    1950
    1951
    1952def test_record_tasks_allows_done_action_after_action_shell(tmp_path):
    1953    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1954    db = AgentDB(tmp_path / "state.db")
    1955    try:
    1956        job_id = db.create_job("Change a local workspace")
    1957        run_id = db.start_run(job_id, model="fake")
    1958        shell_step = db.add_step(
    1959            job_id=job_id,
    1960            run_id=run_id,
    1961            kind="tool",
    1962            tool_name="shell_exec",
    1963            input_data={"arguments": {"command": "python run_branch.py"}},
    1964        )
    1965        db.finish_step(shell_step, status="completed", summary="shell_exec rc=0")
    1966        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1967        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1968
    1969        raw = DEFAULT_REGISTRY.handle(
    1970            "record_tasks",
    1971            {
    1972                "tasks": [{
    1973                    "title": "Apply change",
    1974                    "status": "done",
    1975                    "output_contract": "action",
    1976                    "result": "Ran the action branch.",
    1977                }]
    1978            },
    1979            ctx,
    1980        )
    1981        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    1982
    1983        assert json.loads(raw)["success"] is True
    1984        assert task["status"] == "done"
    1985        assert "completion_validation" not in task.get("metadata", {})
    1986    finally:
    1987        db.close()
    1988
    1989
    1990def test_record_tasks_downgrades_done_monitor_without_defer_evidence(tmp_path):
    1991    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1992    db = AgentDB(tmp_path / "state.db")
    1993    try:
    1994        job_id = db.create_job("Monitor long-running work")
    1995        run_id = db.start_run(job_id, model="fake")
    1996        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    1997        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    1998
    1999        raw = DEFAULT_REGISTRY.handle(
    2000            "record_tasks",
    2001            {
    2002                "tasks": [{
    2003                    "title": "Wait and check later",
    2004                    "status": "done",
    2005                    "output_contract": "monitor",
    2006                    "result": "Will check later.",
    2007                }]
    2008            },
    2009            ctx,
    2010        )
    2011        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    2012
    2013        assert json.loads(raw)["success"] is True
    2014        assert task["status"] == "active"
    2015        assert task["metadata"]["completion_validation"] == "missing_monitor_evidence"
    2016    finally:
    2017        db.close()
    2018
    2019
    2020def test_record_tasks_allows_done_monitor_after_defer_evidence(tmp_path):
    2021    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2022    db = AgentDB(tmp_path / "state.db")
    2023    try:
    2024        job_id = db.create_job("Monitor long-running work")
    2025        run_id = db.start_run(job_id, model="fake")
    2026        defer_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="defer_job")
    2027        db.finish_step(defer_step, status="completed", summary="deferred")
    2028        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    2029        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    2030
    2031        raw = DEFAULT_REGISTRY.handle(
    2032            "record_tasks",
    2033            {
    2034                "tasks": [{
    2035                    "title": "Wait and check later",
    2036                    "status": "done",
    2037                    "output_contract": "monitor",
    2038                    "result": "A monitor/defer branch is scheduled.",
    2039                }]
    2040            },
    2041            ctx,
    2042        )
    2043        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    2044
    2045        assert json.loads(raw)["success"] is True
    2046        assert task["status"] == "done"
    2047        assert "completion_validation" not in task.get("metadata", {})
    2048    finally:
    2049        db.close()
    2050
    2051
    2052def test_record_tasks_allows_done_artifact_after_delivery_evidence(tmp_path):
    2053    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2054    db = AgentDB(tmp_path / "state.db")
    2055    try:
    2056        job_id = db.create_job("Update a deliverable")
    2057        run_id = db.start_run(job_id, model="fake")
    2058        artifact_step = db.add_step(
    2059            job_id=job_id,
    2060            run_id=run_id,
    2061            kind="tool",
    2062            tool_name="write_artifact",
    2063            input_data={"arguments": {"title": "Final report draft", "summary": "Updated report deliverable"}},
    2064        )
    2065        db.finish_step(artifact_step, status="completed", summary="write_artifact saved art_demo")
    2066        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    2067        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    2068
    2069        raw = DEFAULT_REGISTRY.handle(
    2070            "record_tasks",
    2071            {
    2072                "tasks": [{
    2073                    "title": "Update report draft",
    2074                    "status": "done",
    2075                    "output_contract": "artifact",
    2076                    "result": "Saved final report draft",
    2077                }]
    2078            },
    2079            ctx,
    2080        )
    2081        result = json.loads(raw)
    2082        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    2083
    2084        assert result["success"] is True
    2085        assert task["status"] == "done"
    2086        assert "completion_validation" not in task.get("metadata", {})
    2087    finally:
    2088        db.close()
    2089
    2090
    2091def test_record_tasks_does_not_treat_stderr_redirect_as_delivery_write(tmp_path):
    2092    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2093    db = AgentDB(tmp_path / "state.db")
    2094    try:
    2095        job_id = db.create_job("Update a deliverable")
    2096        run_id = db.start_run(job_id, model="fake")
    2097        shell_step = db.add_step(
    2098            job_id=job_id,
    2099            run_id=run_id,
    2100            kind="tool",
    2101            tool_name="shell_exec",
    2102            input_data={"arguments": {"command": "cat draft.md 2>/dev/null"}},
    2103        )
    2104        db.finish_step(shell_step, status="completed", summary="shell_exec rc=0")
    2105        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    2106        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    2107
    2108        raw = DEFAULT_REGISTRY.handle(
    2109            "record_tasks",
    2110            {
    2111                "tasks": [{
    2112                    "title": "Update report draft",
    2113                    "status": "done",
    2114                    "output_contract": "artifact",
    2115                    "result": "Saved final report draft",
    2116                }]
    2117            },
    2118            ctx,
    2119        )
    2120        result = json.loads(raw)
    2121        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    2122
    2123        assert result["success"] is True
    2124        assert task["status"] == "active"
    2125        assert task["metadata"]["completion_validation"] == "missing_recent_deliverable_evidence"
    2126    finally:
    2127        db.close()
    2128
    2129
    2130def test_record_tasks_rejects_checkpoint_as_delivery_evidence(tmp_path):
    2131    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2132    db = AgentDB(tmp_path / "state.db")
    2133    try:
    2134        job_id = db.create_job("Update a deliverable")
    2135        run_id = db.start_run(job_id, model="fake")
    2136        artifact_step = db.add_step(
    2137            job_id=job_id,
    2138            run_id=run_id,
    2139            kind="tool",
    2140            tool_name="write_artifact",
    2141            input_data={"arguments": {"title": "Compiled report checkpoint", "summary": "Checkpoint before final rewrite"}},
    2142        )
    2143        db.finish_step(artifact_step, status="completed", summary="write_artifact saved art_demo")
    2144        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    2145        ctx = ToolContext(config=config, db=db, artifacts=ArtifactStore(tmp_path, db), job_id=job_id, run_id=run_id, step_id=step_id)
    2146
    2147        raw = DEFAULT_REGISTRY.handle(
    2148            "record_tasks",
    2149            {
    2150                "tasks": [{
    2151                    "title": "Update report draft",
    2152                    "status": "done",
    2153                    "output_contract": "artifact",
    2154                    "result": "Saved final report draft",
    2155                }]
    2156            },
    2157            ctx,
    2158        )
    2159        result = json.loads(raw)
    2160        task = db.get_job(job_id)["metadata"]["task_queue"][0]
    2161
    2162        assert result["success"] is True
    2163        assert task["status"] == "active"
    2164        assert task["metadata"]["completion_validation"] == "missing_recent_deliverable_evidence"
    2165    finally:
    2166        db.close()

tests/nipux_cli/test_uninstall.py 136 lines

       1import subprocess
       2from pathlib import Path
       3
       4from nipux_cli.uninstall import build_uninstall_plan, installed_tool_paths, uninstall_installed_tool, uninstall_runtime
       5
       6
       7def _completed(*_args, **_kwargs):
       8    return subprocess.CompletedProcess(args=[], returncode=0)
       9
      10
      11def test_uninstall_plan_includes_runtime_and_legacy_state(monkeypatch, tmp_path):
      12    home = tmp_path / "user"
      13    profile = tmp_path / "profile"
      14    monkeypatch.setenv("HOME", str(home))
      15    monkeypatch.setenv("NIPUX_HOME", str(profile))
      16
      17    plan = build_uninstall_plan()
      18
      19    assert profile in plan.paths
      20    assert home / ".nipux" in plan.paths
      21    assert home / ".kneepucks" in plan.paths
      22    assert home / "Library" / "LaunchAgents" / "com.nipux.agent.plist" in plan.service_paths
      23    assert home / ".config" / "systemd" / "user" / "nipux.service" in plan.service_paths
      24
      25
      26def test_uninstall_plan_includes_configured_runtime_home(monkeypatch, tmp_path):
      27    home = tmp_path / "user"
      28    profile = tmp_path / "profile"
      29    configured = tmp_path / "configured"
      30    monkeypatch.setenv("HOME", str(home))
      31    monkeypatch.setenv("NIPUX_HOME", str(profile))
      32
      33    plan = build_uninstall_plan(runtime_home=configured)
      34
      35    assert configured in plan.paths
      36    assert profile in plan.paths
      37
      38
      39def test_uninstall_runtime_removes_state_and_service_files(monkeypatch, tmp_path):
      40    home = tmp_path / "user"
      41    profile = tmp_path / "profile"
      42    monkeypatch.setenv("HOME", str(home))
      43    monkeypatch.setenv("NIPUX_HOME", str(profile))
      44
      45    paths = [
      46        profile,
      47        home / ".nipux",
      48        home / ".kneepucks",
      49        home / "Library" / "LaunchAgents",
      50        home / ".config" / "systemd" / "user",
      51    ]
      52    for path in paths:
      53        path.mkdir(parents=True, exist_ok=True)
      54    (home / "Library" / "LaunchAgents" / "com.nipux.agent.plist").write_text("plist", encoding="utf-8")
      55    (home / ".config" / "systemd" / "user" / "nipux.service").write_text("unit", encoding="utf-8")
      56    (profile / "state.db").write_text("state", encoding="utf-8")
      57
      58    lines = uninstall_runtime(runner=_completed)
      59
      60    assert any("removed" in line and str(profile) in line for line in lines)
      61    assert not profile.exists()
      62    assert not (home / ".nipux").exists()
      63    assert not (home / ".kneepucks").exists()
      64    assert not (home / "Library" / "LaunchAgents" / "com.nipux.agent.plist").exists()
      65    assert not (home / ".config" / "systemd" / "user" / "nipux.service").exists()
      66
      67
      68def test_uninstall_runtime_dry_run_keeps_files(monkeypatch, tmp_path):
      69    home = tmp_path / "user"
      70    profile = tmp_path / "profile"
      71    monkeypatch.setenv("HOME", str(home))
      72    monkeypatch.setenv("NIPUX_HOME", str(profile))
      73    profile.mkdir(parents=True)
      74
      75    lines = uninstall_runtime(dry_run=True, runner=_completed)
      76
      77    assert any("would remove" in line and str(profile) in line for line in lines)
      78    assert profile.exists()
      79
      80
      81def test_uninstall_installed_tool_uses_uv_when_available(monkeypatch, tmp_path):
      82    home = tmp_path / "user"
      83    monkeypatch.setenv("HOME", str(home))
      84    monkeypatch.setattr("nipux_cli.uninstall.shutil.which", lambda name: "/usr/bin/uv" if name == "uv" else None)
      85    calls = []
      86
      87    def runner(command):
      88        calls.append(tuple(command))
      89        return subprocess.CompletedProcess(command, 0, stdout="Uninstalled 1 executable: nipux\n")
      90
      91    code, lines = uninstall_installed_tool(runner=runner)
      92
      93    assert code == 0
      94    assert calls == [("/usr/bin/uv", "tool", "uninstall", "nipux")]
      95    assert "Uninstalled 1 executable: nipux" in "\n".join(lines)
      96
      97
      98def test_uninstall_installed_tool_falls_back_to_safe_uv_paths(monkeypatch, tmp_path):
      99    home = tmp_path / "user"
     100    shim = home / ".local" / "bin" / "nipux"
     101    tool_dir = home / ".local" / "share" / "uv" / "tools" / "nipux"
     102    tool_bin = tool_dir / "bin"
     103    tool_bin.mkdir(parents=True)
     104    shim.parent.mkdir(parents=True)
     105    target = tool_bin / "nipux"
     106    target.write_text("script", encoding="utf-8")
     107    shim.symlink_to(target)
     108    monkeypatch.setenv("HOME", str(home))
     109
     110    def which(name):
     111        if name == "nipux":
     112            return str(shim)
     113        return None
     114
     115    monkeypatch.setattr("nipux_cli.uninstall.shutil.which", which)
     116
     117    code, lines = uninstall_installed_tool()
     118
     119    assert code == 0
     120    assert not shim.exists()
     121    assert not tool_dir.exists()
     122    rendered = "\n".join(lines)
     123    assert "uv not found; checking safe local tool paths" in rendered
     124    assert f"removed {shim}" in rendered
     125    assert f"removed {tool_dir}" in rendered
     126
     127
     128def test_installed_tool_paths_ignore_non_user_tool(monkeypatch, tmp_path):
     129    home = tmp_path / "user"
     130    monkeypatch.setenv("HOME", str(home))
     131    monkeypatch.setattr("nipux_cli.uninstall.shutil.which", lambda name: "/usr/local/bin/nipux" if name == "nipux" else None)
     132
     133    paths = installed_tool_paths()
     134
     135    assert Path("/usr/local/bin/nipux") not in paths
     136    assert home / ".local" / "bin" / "nipux" in paths

tests/nipux_cli/test_worker.py 11339 lines

       1import json
       2from pathlib import Path
       3
       4from nipux_cli.artifacts import ArtifactStore
       5from nipux_cli.config import AppConfig, ModelConfig, RuntimeConfig
       6from nipux_cli.db import AgentDB
       7from nipux_cli.llm import LLMResponse, LLMResponseError, ScriptedLLM, ToolCall
       8from nipux_cli.worker import (
       9    MAX_WORKER_PROMPT_CHARS,
      10    SYSTEM_PROMPT,
      11    _concrete_evidence_tokens,
      12    _cited_step_numbers,
      13    _extract_candidate_file_paths,
      14    _file_pattern_tokens_for_grounding,
      15    _rank_candidate_file_paths,
      16    _render_worker_prompt,
      17    build_messages,
      18    run_one_step,
      19)
      20
      21
      22class SnapshotRegistry:
      23    def openai_tools(self):
      24        return []
      25
      26    def handle(self, name, args, ctx):
      27        del name, args, ctx
      28        return json.dumps({"success": True, "data": {"snapshot": "short snapshot"}})
      29
      30
      31class SuccessRegistry:
      32    def openai_tools(self):
      33        return []
      34
      35    def handle(self, name, args, ctx):
      36        del ctx
      37        return json.dumps({"success": True, "tool": name, "args": args, "results": []})
      38
      39
      40class MeasuredShellRegistry:
      41    def openai_tools(self):
      42        return []
      43
      44    def handle(self, name, args, ctx):
      45        del args, ctx
      46        if name == "shell_exec":
      47            return json.dumps({"success": True, "command": "run test", "returncode": 0, "stdout": "score 2.7 units/s", "stderr": ""})
      48        return json.dumps({"success": True, "results": []})
      49
      50
      51class DiagnosticShellRegistry:
      52    def openai_tools(self):
      53        return []
      54
      55    def handle(self, name, args, ctx):
      56        del args, ctx
      57        if name == "shell_exec":
      58            return json.dumps({
      59                "success": True,
      60                "command": "df -h && nproc && free -h",
      61                "returncode": 0,
      62                "stdout": "Filesystem Size Used Avail Use% Mounted on\\n/dev/root 233G 198G 23G 90% /\\nCPU COUNT 24\\nRAM 93Gi",
      63                "stderr": "",
      64            })
      65        return json.dumps({"success": True})
      66
      67
      68class TableBenchmarkShellRegistry:
      69    def openai_tools(self):
      70        return []
      71
      72    def handle(self, name, args, ctx):
      73        del args, ctx
      74        if name == "shell_exec":
      75            return json.dumps({
      76                "success": True,
      77                "command": "run benchmark",
      78                "returncode": 0,
      79                "stdout": (
      80                    "| model | test | t/s |\n"
      81                    "| --- | ---: | ---: |\n"
      82                    "| example | pp32 | 5.48 ± 0.11 |\n"
      83                    "| example | tg128 | 3.44 ± 0.05 |\n"
      84                ),
      85                "stderr": "",
      86            })
      87        return json.dumps({"success": True})
      88
      89
      90class FailedTableBenchmarkShellRegistry:
      91    def openai_tools(self):
      92        return []
      93
      94    def handle(self, name, args, ctx):
      95        del args, ctx
      96        if name == "shell_exec":
      97            return json.dumps({
      98                "success": False,
      99                "error": "command timed out",
     100                "command": "run benchmark",
     101                "returncode": 124,
     102                "stdout": (
     103                    "| model | test | t/s |\n"
     104                    "| --- | ---: | ---: |\n"
     105                    "| example | pp32 | 5.48 ± 0.11 |\n"
     106                ),
     107                "stderr": "",
     108            })
     109        return json.dumps({"success": True})
     110
     111
     112class FailedUrlShellRegistry:
     113    def openai_tools(self):
     114        return []
     115
     116    def handle(self, name, args, ctx):
     117        del ctx
     118        if name == "shell_exec":
     119            return json.dumps({
     120                "success": False,
     121                "command": args.get("command"),
     122                "returncode": 0,
     123                "stdout": "401 Unauthorized",
     124                "stderr": "",
     125                "error": (
     126                    "command output indicates authentication or authorization failure "
     127                    "despite exit status 0: 401 Unauthorized"
     128                ),
     129            })
     130        return json.dumps({"success": True})
     131
     132
     133class HangingLLM:
     134    def next_action(self, *, messages, tools):
     135        del messages, tools
     136        import time
     137
     138        time.sleep(5)
     139        return LLMResponse(tool_calls=[ToolCall(name="report_update", arguments={"message": "late"})])
     140
     141
     142class SlowLLM:
     143    def __init__(self, sleep_seconds: float):
     144        self.sleep_seconds = sleep_seconds
     145
     146    def next_action(self, *, messages, tools):
     147        del messages, tools
     148        import time
     149
     150        time.sleep(self.sleep_seconds)
     151        return LLMResponse(tool_calls=[ToolCall(name="report_update", arguments={"message": "slow but recovered"})])
     152
     153
     154class RepairableLLM:
     155    tool_repair = True
     156
     157    def __init__(self, responses):
     158        self.responses = list(responses)
     159        self.messages = []
     160        self.tools = []
     161
     162    def next_action(self, *, messages, tools):
     163        self.messages.append(messages)
     164        self.tools.append(tools)
     165        if not self.responses:
     166            return LLMResponse(content="No response left.")
     167        return self.responses.pop(0)
     168
     169
     170class SourceCodeShellRegistry:
     171    def openai_tools(self):
     172        return []
     173
     174    def handle(self, name, args, ctx):
     175        del args, ctx
     176        if name == "shell_exec":
     177            return json.dumps({
     178                "success": True,
     179                "command": "git show HEAD:nipux_cli/cli.py",
     180                "returncode": 0,
     181                "stdout": 'for index, task in enumerate(plan["tasks"], start=1):\n    rate(plan["tasks"], start=1)\n',
     182                "stderr": "",
     183            })
     184        return json.dumps({"success": True})
     185
     186
     187class LargeShellEvidenceRegistry:
     188    def openai_tools(self):
     189        return []
     190
     191    def handle(self, name, args, ctx):
     192        del args, ctx
     193        if name == "shell_exec":
     194            return json.dumps({
     195                "success": True,
     196                "command": "find . -type f",
     197                "returncode": 0,
     198                "stdout": "\n".join(f"./file_{index}.py" for index in range(200)),
     199                "stderr": "",
     200            })
     201        return json.dumps({"success": True})
     202
     203
     204class ExtractRegistry:
     205    def openai_tools(self):
     206        return []
     207
     208    def handle(self, name, args, ctx):
     209        del args, ctx
     210        if name == "web_extract":
     211            return json.dumps({
     212                "success": True,
     213                "pages": [
     214                    {"url": "https://source.example/a", "text": "useful source text " * 250},
     215                    {"url": "https://source.example/b", "error": "timeout"},
     216                ],
     217            })
     218        return json.dumps({"success": True})
     219
     220
     221class SearchRegistry:
     222    def openai_tools(self):
     223        return []
     224
     225    def handle(self, name, args, ctx):
     226        del args, ctx
     227        if name == "web_search":
     228            return json.dumps({
     229                "success": True,
     230                "query": "durable progress research",
     231                "results": [
     232                    {"title": "Primary reference", "url": "https://source.example/primary"},
     233                    {"title": "Secondary reference", "url": "https://source.example/secondary"},
     234                ],
     235            })
     236        return json.dumps({"success": True})
     237
     238
     239class BrowserAndWebRegistry:
     240    def openai_tools(self, config=None):
     241        del config
     242        return [
     243            {"type": "function", "function": {"name": "browser_navigate", "parameters": {"type": "object"}}},
     244            {"type": "function", "function": {"name": "web_search", "parameters": {"type": "object"}}},
     245        ]
     246
     247    def handle(self, name, args, ctx):
     248        del args, ctx
     249        return json.dumps({"success": True, "tool": name})
     250
     251
     252class CapturingLLM:
     253    def __init__(self, response):
     254        self.response = response
     255        self.messages = None
     256        self.tools = None
     257
     258    def next_action(self, *, messages, tools):
     259        self.messages = messages
     260        self.tools = tools
     261        return self.response
     262
     263
     264class ExplodingLLM:
     265    def next_action(self, *, messages, tools):
     266        del messages, tools
     267        raise AssertionError("LLM should not be called")
     268
     269
     270class AntiBotBrowserRegistry:
     271    def openai_tools(self):
     272        return []
     273
     274    def handle(self, name, args, ctx):
     275        del args, ctx
     276        if name == "browser_snapshot":
     277            return json.dumps({
     278                "success": True,
     279                "data": {
     280                    "origin": "https://source.example/search",
     281                    "snapshot": 'Iframe "Security CAPTCHA" You have been blocked. You are browsing and clicking at a speed much faster than expected.',
     282                },
     283            })
     284        return json.dumps({"success": True})
     285
     286
     287def test_system_prompt_is_contract_first_not_research_first():
     288    assert "Use a contract-first durable cycle" in SYSTEM_PROMPT
     289    assert "Research is only one possible contract" in SYSTEM_PROMPT
     290    assert "Prefer fresh measured or directly observed evidence over stale summaries" in SYSTEM_PROMPT
     291    assert "available local candidate fall" in SYSTEM_PROMPT
     292    assert "Use this durable cycle: discover one source" not in SYSTEM_PROMPT
     293
     294
     295def test_run_one_step_executes_scripted_tool_call(tmp_path):
     296    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     297    db = AgentDB(tmp_path / "state.db")
     298    try:
     299        job_id = db.create_job("Find 10 durable research findings", title="research", kind="generic")
     300        llm = ScriptedLLM([
     301            LLMResponse(tool_calls=[
     302                ToolCall(
     303                    name="write_artifact",
     304                    arguments={
     305                        "title": "first finding",
     306                        "summary": "smoke finding",
     307                        "content": "Acme Design, https://example.com",
     308                    },
     309                )
     310            ])
     311        ])
     312
     313        result = run_one_step(job_id, config=config, db=db, llm=llm)
     314
     315        assert result.status == "completed"
     316        assert result.tool_name == "write_artifact"
     317        artifacts = db.list_artifacts(job_id)
     318        assert artifacts[0]["title"] == "first finding"
     319        steps = db.list_steps(job_id=job_id)
     320        assert steps[0]["tool_name"] == "write_artifact"
     321        assert steps[0]["status"] == "completed"
     322        memory = db.list_memory(job_id)
     323        assert memory[0]["key"] == "rolling_state"
     324        assert artifacts[0]["id"] in memory[0]["artifact_refs"]
     325    finally:
     326        db.close()
     327
     328
     329def test_run_one_step_records_estimated_usage_for_scripted_model(tmp_path):
     330    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     331    db = AgentDB(tmp_path / "state.db")
     332    try:
     333        job_id = db.create_job("Summarize progress", title="usage", kind="generic")
     334
     335        run_one_step(
     336            job_id,
     337            config=config,
     338            db=db,
     339            llm=ScriptedLLM([LLMResponse(content="No tool this turn.")]),
     340        )
     341
     342        usage = db.job_token_usage(job_id)
     343        assert usage["calls"] == 1
     344        assert usage["prompt_tokens"] > 0
     345        assert usage["completion_tokens"] > 0
     346        assert usage["estimated_calls"] == 1
     347        event = next(
     348            event
     349            for event in db.list_events(job_id=job_id, event_types=["loop"])
     350            if event.get("title") == "message_end"
     351        )
     352        event_usage = event["metadata"]["usage"]
     353        assert event_usage["prompt_chars"] > 0
     354        assert event_usage["context_length"] == config.model.context_length
     355        assert event_usage["context_fraction"] > 0
     356    finally:
     357        db.close()
     358
     359
     360def test_run_one_step_blocks_content_only_worker_turn(tmp_path):
     361    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     362    db = AgentDB(tmp_path / "state.db")
     363    try:
     364        job_id = db.create_job("Keep taking bounded tool actions", title="no tool", kind="generic")
     365
     366        result = run_one_step(
     367            job_id,
     368            config=config,
     369            db=db,
     370            llm=ScriptedLLM([LLMResponse(content="What should I do next?")]),
     371        )
     372
     373        assert result.status == "blocked"
     374        assert result.result["error"] == "worker tool call required"
     375        assert "What should I do next?" in result.result["content"]
     376        step = db.list_steps(job_id=job_id)[0]
     377        assert step["kind"] == "assistant"
     378        assert step["status"] == "blocked"
     379        assert step["error"] == "worker tool call required"
     380        prompt = build_messages(
     381            db.get_job(job_id),
     382            db.list_steps(job_id=job_id),
     383            timeline_events=db.list_timeline_events(job_id, limit=30),
     384        )[-1]["content"]
     385        assert "What should I do next?" not in prompt
     386        assert "worker tool call required" in prompt
     387        job = db.get_job(job_id)
     388        assert job["metadata"]["last_agent_update"]["category"] == "blocked"
     389    finally:
     390        db.close()
     391
     392
     393def test_run_one_step_repairs_content_only_worker_turn_with_tool_retry(tmp_path):
     394    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     395    db = AgentDB(tmp_path / "state.db")
     396    try:
     397        job_id = db.create_job("Keep taking bounded tool actions", title="tool repair", kind="generic")
     398        llm = RepairableLLM([
     399            LLMResponse(content="I should inspect the state next."),
     400            LLMResponse(tool_calls=[ToolCall(name="report_update", arguments={"message": "Continuing with a bounded action."})]),
     401        ])
     402
     403        result = run_one_step(job_id, config=config, db=db, llm=llm)
     404
     405        assert result.status == "completed"
     406        assert result.tool_name == "report_update"
     407        assert len(llm.messages) == 2
     408        assert "did not call a tool" in llm.messages[1][-1]["content"]
     409        steps = db.list_steps(job_id=job_id)
     410        assert len(steps) == 1
     411        assert steps[0]["tool_name"] == "report_update"
     412        usage = db.job_token_usage(job_id)
     413        assert usage["calls"] == 2
     414    finally:
     415        db.close()
     416
     417
     418def test_run_one_step_recovers_repeated_content_only_worker_turns(tmp_path):
     419    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     420    db = AgentDB(tmp_path / "state.db")
     421    try:
     422        job_id = db.create_job("Keep taking bounded tool actions", title="no tool", kind="generic")
     423        llm = ScriptedLLM([
     424            LLMResponse(content="What should I do next?"),
     425            LLMResponse(content="I can continue if you want."),
     426            LLMResponse(content="Please confirm the next step."),
     427        ])
     428
     429        run_one_step(job_id, config=config, db=db, llm=llm)
     430        run_one_step(job_id, config=config, db=db, llm=llm)
     431        run_one_step(job_id, config=config, db=db, llm=llm)
     432        result = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
     433
     434        assert result.status == "completed"
     435        assert result.tool_name == "guard_recovery"
     436        assert result.result["guard_recovery"]["error"] == "worker tool call required"
     437        job = db.get_job(job_id)
     438        assert any(task["title"] == "Resolve guard: worker tool call required" for task in job["metadata"]["task_queue"])
     439    finally:
     440        db.close()
     441
     442
     443def test_run_one_step_records_context_pressure_without_spam(tmp_path):
     444    config = AppConfig(runtime=RuntimeConfig(home=tmp_path), model=ModelConfig(context_length=10_000))
     445    db = AgentDB(tmp_path / "state.db")
     446    try:
     447        job_id = db.create_job("Keep a long-running task stable", title="context pressure", kind="generic")
     448        llm = ScriptedLLM([
     449            LLMResponse(content="first", usage={"prompt_tokens": 7_000, "completion_tokens": 10, "total_tokens": 7_010}),
     450            LLMResponse(content="second", usage={"prompt_tokens": 7_200, "completion_tokens": 10, "total_tokens": 7_210}),
     451            LLMResponse(content="third", usage={"prompt_tokens": 8_600, "completion_tokens": 10, "total_tokens": 8_610}),
     452        ])
     453
     454        run_one_step(job_id, config=config, db=db, llm=llm)
     455        run_one_step(job_id, config=config, db=db, llm=llm)
     456        run_one_step(job_id, config=config, db=db, llm=llm)
     457
     458        pressure_events = [
     459            event
     460            for event in db.list_events(job_id=job_id, event_types=["agent_message"])
     461            if event["metadata"].get("kind") == "context_pressure"
     462        ]
     463        assert len(pressure_events) == 2
     464        assert "Context pressure watch" in pressure_events[0]["body"]
     465        assert "Context pressure high" in pressure_events[1]["body"]
     466        job = db.get_job(job_id)
     467        pressure = job["metadata"]["context_pressure"]
     468        assert pressure["band"] == "high"
     469        assert pressure["prompt_tokens"] == 8_600
     470    finally:
     471        db.close()
     472
     473
     474def test_run_one_step_executes_tool_call_batch_in_order(tmp_path):
     475    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     476    db = AgentDB(tmp_path / "state.db")
     477    try:
     478        job_id = db.create_job("Build a durable report", title="batch", kind="generic")
     479        llm = ScriptedLLM([
     480            LLMResponse(tool_calls=[
     481                ToolCall(
     482                    name="write_artifact",
     483                    arguments={
     484                        "title": "evidence checkpoint",
     485                        "summary": "first useful output",
     486                        "content": "The worker saved evidence before updating the task queue.",
     487                    },
     488                ),
     489                ToolCall(
     490                    name="record_tasks",
     491                    arguments={
     492                        "tasks": [
     493                            {
     494                                "title": "Review saved output",
     495                                "status": "open",
     496                                "priority": 5,
     497                                "output_contract": "report",
     498                                "acceptance_criteria": "Saved evidence has been inspected and summarized.",
     499                                "evidence_needed": "Artifact reference and concrete next action.",
     500                                "stall_behavior": "Record a lesson and pivot if the artifact is not useful.",
     501                            }
     502                        ]
     503                    },
     504                ),
     505            ])
     506        ])
     507
     508        result = run_one_step(job_id, config=config, db=db, llm=llm)
     509
     510        assert result.status == "completed"
     511        assert result.tool_name == "record_tasks"
     512        steps = db.list_steps(job_id=job_id)
     513        assert [step["tool_name"] for step in steps] == ["write_artifact", "record_tasks"]
     514        assert [step["status"] for step in steps] == ["completed", "completed"]
     515        artifacts = db.list_artifacts(job_id)
     516        assert artifacts[0]["title"] == "evidence checkpoint"
     517        job = db.get_job(job_id)
     518        tasks = job["metadata"]["task_queue"]
     519        assert any(task["title"] == "Review saved output" and task["output_contract"] == "report" for task in tasks)
     520        run = db.list_runs(job_id, limit=1)[0]
     521        assert run["status"] == "completed"
     522    finally:
     523        db.close()
     524
     525
     526def test_write_artifact_reconciles_matching_report_task(tmp_path):
     527    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     528    db = AgentDB(tmp_path / "state.db")
     529    try:
     530        job_id = db.create_job(
     531            "Write a durable report",
     532            title="report",
     533            kind="generic",
     534            metadata={
     535                "task_queue": [
     536                    {
     537                        "title": "Draft paper - Methods section",
     538                        "status": "open",
     539                        "priority": 5,
     540                        "output_contract": "report",
     541                        "acceptance_criteria": "Methods section is saved as an output.",
     542                    }
     543                ]
     544            },
     545        )
     546        llm = ScriptedLLM([
     547            LLMResponse(tool_calls=[
     548                ToolCall(
     549                    name="write_artifact",
     550                    arguments={
     551                        "title": "Paper Draft - Section 3: Methods",
     552                        "summary": "Methods section for the report",
     553                        "content": "This methods section explains the approach and evidence.",
     554                    },
     555                )
     556            ])
     557        ])
     558
     559        result = run_one_step(job_id, config=config, db=db, llm=llm)
     560
     561        assert result.status == "completed"
     562        job = db.get_job(job_id)
     563        task = job["metadata"]["task_queue"][0]
     564        assert task["status"] == "done"
     565        assert task["metadata"]["auto_reconciled_from_artifact"]
     566        assert "Saved output" in task["result"]
     567        revision_tasks = [
     568            item
     569            for item in job["metadata"]["task_queue"]
     570            if item["status"] == "open" and item.get("metadata", {}).get("source") == "auto_revision_loop"
     571        ]
     572        assert len(revision_tasks) == 1
     573        assert revision_tasks[0]["output_contract"] == "report"
     574        assert revision_tasks[0]["metadata"]["revision_source_artifact_id"]
     575    finally:
     576        db.close()
     577
     578
     579def test_evidence_artifact_does_not_complete_deliverable_task(tmp_path):
     580    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     581    db = AgentDB(tmp_path / "state.db")
     582    try:
     583        job_id = db.create_job(
     584            "Improve a durable report",
     585            title="report",
     586            kind="generic",
     587            metadata={
     588                "task_queue": [
     589                    {
     590                        "title": "Update report with new citations",
     591                        "status": "open",
     592                        "priority": 5,
     593                        "output_contract": "artifact",
     594                        "acceptance_criteria": "Report text is updated with citations.",
     595                        "evidence_needed": "Updated report draft, not just source notes.",
     596                    }
     597                ]
     598            },
     599        )
     600        llm = ScriptedLLM([
     601            LLMResponse(tool_calls=[
     602                ToolCall(
     603                    name="write_artifact",
     604                    arguments={
     605                        "title": "Evidence: citation sources",
     606                        "summary": "Extracted source notes for citations",
     607                        "content": "These notes describe sources that could later be used in the report.",
     608                    },
     609                )
     610            ])
     611        ])
     612
     613        result = run_one_step(job_id, config=config, db=db, llm=llm)
     614
     615        assert result.status == "completed"
     616        job = db.get_job(job_id)
     617        task = job["metadata"]["task_queue"][0]
     618        assert task["status"] == "open"
     619        assert "auto_reconciled_from_artifact" not in task.get("metadata", {})
     620        assert not [
     621            item
     622            for item in job["metadata"]["task_queue"]
     623            if item.get("metadata", {}).get("source") == "auto_revision_loop"
     624        ]
     625    finally:
     626        db.close()
     627
     628
     629def test_new_deliverable_supersedes_old_auto_revision_task(tmp_path):
     630    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     631    db = AgentDB(tmp_path / "state.db")
     632    try:
     633        job_id = db.create_job(
     634            "Keep improving a durable report",
     635            title="report",
     636            kind="generic",
     637            metadata={
     638                "task_queue": [
     639                    {
     640                        "title": "Review and revise saved output art_old",
     641                        "status": "open",
     642                        "priority": 4,
     643                        "output_contract": "report",
     644                        "metadata": {
     645                            "source": "auto_revision_loop",
     646                            "revision_source_artifact_id": "art_old",
     647                        },
     648                    }
     649                ]
     650            },
     651        )
     652        llm = ScriptedLLM([
     653            LLMResponse(tool_calls=[
     654                ToolCall(
     655                    name="write_artifact",
     656                    arguments={
     657                        "title": "Report Draft Revision",
     658                        "summary": "Updated durable report draft",
     659                        "content": "This revised report draft supersedes the previous saved output.",
     660                    },
     661                )
     662            ])
     663        ])
     664
     665        result = run_one_step(job_id, config=config, db=db, llm=llm)
     666
     667        assert result.status == "completed"
     668        tasks = db.get_job(job_id)["metadata"]["task_queue"]
     669        old = next(task for task in tasks if task["metadata"].get("revision_source_artifact_id") == "art_old")
     670        new = next(task for task in tasks if task["metadata"].get("revision_source_artifact_id") != "art_old")
     671        assert old["status"] == "skipped"
     672        assert old["metadata"]["superseded_by_artifact_id"]
     673        assert new["status"] == "open"
     674        assert new["metadata"]["source"] == "auto_revision_loop"
     675    finally:
     676        db.close()
     677
     678
     679def test_audit_report_draft_counts_as_deliverable_output(tmp_path):
     680    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     681    db = AgentDB(tmp_path / "state.db")
     682    try:
     683        job_id = db.create_job(
     684            "Write a durable audit report",
     685            title="audit report",
     686            kind="generic",
     687            metadata={
     688                "task_queue": [
     689                    {
     690                        "title": "Write audit report draft",
     691                        "status": "open",
     692                        "priority": 5,
     693                        "output_contract": "artifact",
     694                        "acceptance_criteria": "A report draft is saved.",
     695                        "evidence_needed": "Saved report draft, not only notes.",
     696                    }
     697                ]
     698            },
     699        )
     700        llm = ScriptedLLM([
     701            LLMResponse(tool_calls=[
     702                ToolCall(
     703                    name="write_artifact",
     704                    arguments={
     705                        "title": "Market Readiness Audit Report Draft",
     706                        "summary": "Saved audit report draft with current findings and recommendations",
     707                        "content": "This is the current audit report draft.",
     708                    },
     709                )
     710            ])
     711        ])
     712
     713        result = run_one_step(job_id, config=config, db=db, llm=llm)
     714
     715        assert result.status == "completed"
     716        job = db.get_job(job_id)
     717        task = job["metadata"]["task_queue"][0]
     718        assert task["status"] == "done"
     719        assert task["metadata"]["auto_reconciled_from_artifact"]
     720    finally:
     721        db.close()
     722
     723
     724def test_checkpoint_artifact_does_not_complete_deliverable_task(tmp_path):
     725    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     726    db = AgentDB(tmp_path / "state.db")
     727    try:
     728        job_id = db.create_job(
     729            "Compile a durable report",
     730            title="report",
     731            kind="generic",
     732            metadata={
     733                "task_queue": [
     734                    {
     735                        "title": "Compile full report",
     736                        "status": "open",
     737                        "priority": 5,
     738                        "output_contract": "artifact",
     739                        "acceptance_criteria": "Final compiled report is saved.",
     740                    }
     741                ]
     742            },
     743        )
     744        llm = ScriptedLLM([
     745            LLMResponse(tool_calls=[
     746                ToolCall(
     747                    name="write_artifact",
     748                    arguments={
     749                        "title": "Compiled report checkpoint",
     750                        "summary": "Current state checkpoint, not a final compiled report",
     751                        "content": "This checkpoint describes what still needs to be written.",
     752                    },
     753                )
     754            ])
     755        ])
     756
     757        result = run_one_step(job_id, config=config, db=db, llm=llm)
     758
     759        assert result.status == "completed"
     760        job = db.get_job(job_id)
     761        task = job["metadata"]["task_queue"][0]
     762        assert task["status"] == "open"
     763        assert "auto_reconciled_from_artifact" not in task.get("metadata", {})
     764    finally:
     765        db.close()
     766
     767
     768def test_evidence_artifact_can_complete_research_task(tmp_path):
     769    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     770    db = AgentDB(tmp_path / "state.db")
     771    try:
     772        job_id = db.create_job(
     773            "Gather source evidence",
     774            title="research",
     775            kind="generic",
     776            metadata={
     777                "task_queue": [
     778                    {
     779                        "title": "Collect citation source evidence",
     780                        "status": "open",
     781                        "priority": 5,
     782                        "output_contract": "research",
     783                        "acceptance_criteria": "Evidence sources are saved.",
     784                    }
     785                ]
     786            },
     787        )
     788        llm = ScriptedLLM([
     789            LLMResponse(tool_calls=[
     790                ToolCall(
     791                    name="write_artifact",
     792                    arguments={
     793                        "title": "Evidence: citation sources",
     794                        "summary": "Extracted source evidence",
     795                        "content": "Citation source evidence for later report writing.",
     796                    },
     797                )
     798            ])
     799        ])
     800
     801        result = run_one_step(job_id, config=config, db=db, llm=llm)
     802
     803        assert result.status == "completed"
     804        job = db.get_job(job_id)
     805        task = job["metadata"]["task_queue"][0]
     806        assert task["status"] == "done"
     807        assert task["metadata"]["auto_reconciled_from_artifact"]
     808    finally:
     809        db.close()
     810
     811
     812def test_run_one_step_blocks_artifact_churn_until_progress_accounting(tmp_path):
     813    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     814    db = AgentDB(tmp_path / "state.db")
     815    try:
     816        job_id = db.create_job("Keep a durable progress ledger", title="ledger", kind="generic")
     817        for index in range(3):
     818            run_id = db.start_run(job_id, model="test")
     819            step_id = db.add_step(
     820                job_id=job_id,
     821                run_id=run_id,
     822                kind="tool",
     823                tool_name="write_artifact",
     824                input_data={"arguments": {"title": f"Output {index}", "content": "notes"}},
     825            )
     826            db.finish_step(
     827                step_id,
     828                status="completed",
     829                summary=f"write_artifact saved art_{index}",
     830                output_data={"success": True, "artifact_id": f"art_{index}"},
     831            )
     832            db.finish_run(run_id, "completed")
     833
     834        blocked = run_one_step(
     835            job_id,
     836            config=config,
     837            db=db,
     838            llm=ScriptedLLM([
     839                LLMResponse(tool_calls=[
     840                    ToolCall(name="write_artifact", arguments={"title": "Another output", "content": "more notes"})
     841                ])
     842            ]),
     843        )
     844
     845        assert blocked.status == "blocked"
     846        assert blocked.result["error"] == "progress accounting required"
     847        allowed = run_one_step(
     848            job_id,
     849            config=config,
     850            db=db,
     851            llm=ScriptedLLM([
     852                LLMResponse(tool_calls=[
     853                    ToolCall(
     854                        name="record_tasks",
     855                        arguments={"tasks": [{"title": "Review saved outputs", "status": "open", "priority": 2}]},
     856                    )
     857                ])
     858            ]),
     859        )
     860        assert allowed.status == "completed"
     861        assert allowed.tool_name == "record_tasks"
     862    finally:
     863        db.close()
     864
     865
     866def test_activity_checkpoint_streak_blocks_more_churn_until_ledger_update(tmp_path):
     867    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     868    db = AgentDB(tmp_path / "state.db")
     869    try:
     870        job_id = db.create_job("Keep working until durable progress appears", title="stagnation", kind="generic")
     871        db.update_job_metadata(
     872            job_id,
     873            {
     874                "activity_checkpoint_streak": 3,
     875                "last_checkpoint_counts": {
     876                    "findings": 0,
     877                    "sources": 0,
     878                    "tasks": 1,
     879                    "experiments": 0,
     880                    "lessons": 0,
     881                    "milestones": 0,
     882                },
     883            },
     884        )
     885
     886        blocked = run_one_step(
     887            job_id,
     888            config=config,
     889            db=db,
     890            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "more background"})])]),
     891        )
     892
     893        assert blocked.status == "blocked"
     894        assert blocked.result["error"] == "durable progress required"
     895
     896        allowed = run_one_step(
     897            job_id,
     898            config=config,
     899            db=db,
     900            llm=ScriptedLLM([
     901                LLMResponse(tool_calls=[ToolCall(name="record_tasks", arguments={"tasks": [{"title": "Pivot branch", "status": "open"}]})])
     902            ]),
     903        )
     904
     905        assert allowed.status == "completed"
     906        assert allowed.tool_name == "record_tasks"
     907    finally:
     908        db.close()
     909
     910
     911def test_task_only_checkpoint_streak_blocks_new_task_sprawl(tmp_path):
     912    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     913    db = AgentDB(tmp_path / "state.db")
     914    try:
     915        job_id = db.create_job("Keep executing durable work", title="task-sprawl", kind="generic")
     916        db.update_job_metadata(
     917            job_id,
     918            {
     919                "task_planning_checkpoint_streak": 2,
     920                "task_queue": [
     921                    {
     922                        "key": "existing-branch",
     923                        "title": "Existing branch",
     924                        "status": "open",
     925                    }
     926                ],
     927            },
     928        )
     929
     930        blocked = run_one_step(
     931            job_id,
     932            config=config,
     933            db=db,
     934            llm=ScriptedLLM([
     935                LLMResponse(tool_calls=[
     936                    ToolCall(
     937                        name="record_tasks",
     938                        arguments={"tasks": [{"title": "Another open branch", "status": "open"}]},
     939                    )
     940                ])
     941            ]),
     942        )
     943
     944        assert blocked.status == "blocked"
     945        assert blocked.result["error"] == "task execution required"
     946
     947        allowed = run_one_step(
     948            job_id,
     949            config=config,
     950            db=db,
     951            llm=ScriptedLLM([
     952                LLMResponse(tool_calls=[
     953                    ToolCall(
     954                        name="record_tasks",
     955                        arguments={
     956                            "tasks": [
     957                                {
     958                                    "title": "Existing branch",
     959                                    "status": "done",
     960                                    "result": "Executed and checkpointed.",
     961                                }
     962                            ]
     963                        },
     964                    )
     965                ])
     966            ]),
     967        )
     968
     969        assert allowed.status == "completed"
     970        assert allowed.tool_name == "record_tasks"
     971    finally:
     972        db.close()
     973
     974
     975def test_task_only_checkpoint_updates_planning_streak(tmp_path):
     976    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
     977    db = AgentDB(tmp_path / "state.db")
     978    try:
     979        job_id = db.create_job("Track planning-only progress", title="task-streak", kind="generic")
     980        for index in range(9):
     981            run_id = db.start_run(job_id, model="test")
     982            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="web_search")
     983            db.finish_step(step_id, status="completed", summary=f"search {index}", output_data={"success": True})
     984            db.finish_run(run_id, "completed")
     985
     986        result = run_one_step(
     987            job_id,
     988            config=config,
     989            db=db,
     990            llm=ScriptedLLM([
     991                LLMResponse(tool_calls=[
     992                    ToolCall(name="record_tasks", arguments={"tasks": [{"title": "First branch", "status": "open"}]})
     993                ])
     994            ]),
     995        )
     996
     997        assert result.status == "completed"
     998        job = db.get_job(job_id)
     999        assert job["metadata"]["task_planning_checkpoint_streak"] == 1
    1000
    1001        db.append_finding_record(job_id, name="Durable finding")
    1002        for index in range(9):
    1003            run_id = db.start_run(job_id, model="test")
    1004            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="web_search")
    1005            db.finish_step(step_id, status="completed", summary=f"search reset {index}", output_data={"success": True})
    1006            db.finish_run(run_id, "completed")
    1007        result = run_one_step(
    1008            job_id,
    1009            config=config,
    1010            db=db,
    1011            llm=ScriptedLLM([
    1012                LLMResponse(tool_calls=[
    1013                    ToolCall(name="record_tasks", arguments={"tasks": [{"title": "Second branch", "status": "open"}]})
    1014                ])
    1015            ]),
    1016        )
    1017
    1018        assert result.status == "completed"
    1019        job = db.get_job(job_id)
    1020        assert job["metadata"]["task_planning_checkpoint_streak"] == 0
    1021    finally:
    1022        db.close()
    1023
    1024
    1025def test_task_resolution_checkpoint_resets_planning_streak(tmp_path):
    1026    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1027    db = AgentDB(tmp_path / "state.db")
    1028    try:
    1029        job_id = db.create_job("Resolve existing durable branches", title="task-resolution", kind="generic")
    1030        db.append_task_record(job_id, title="Existing branch", status="open", priority=5)
    1031        db.update_job_metadata(
    1032            job_id,
    1033            {
    1034                "last_checkpoint_counts": {
    1035                    "findings": 0,
    1036                    "sources": 0,
    1037                    "tasks": 1,
    1038                    "experiments": 0,
    1039                    "lessons": 0,
    1040                    "milestones": 0,
    1041                },
    1042                "last_checkpoint_at": "2026-01-01T00:00:00+00:00",
    1043                "task_planning_checkpoint_streak": 2,
    1044            },
    1045        )
    1046        for index in range(9):
    1047            run_id = db.start_run(job_id, model="test")
    1048            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="web_search")
    1049            db.finish_step(step_id, status="completed", summary=f"search {index}", output_data={"success": True})
    1050            db.finish_run(run_id, "completed")
    1051
    1052        result = run_one_step(
    1053            job_id,
    1054            config=config,
    1055            db=db,
    1056            llm=ScriptedLLM([
    1057                LLMResponse(tool_calls=[
    1058                    ToolCall(
    1059                        name="record_tasks",
    1060                        arguments={
    1061                            "tasks": [
    1062                                {
    1063                                    "title": "Existing branch",
    1064                                    "status": "done",
    1065                                    "result": "Resolved using the latest evidence.",
    1066                                    "metadata": {"source_url": "file:///tmp/latest-evidence"},
    1067                                }
    1068                            ]
    1069                        },
    1070                    )
    1071                ])
    1072            ]),
    1073        )
    1074
    1075        assert result.status == "completed"
    1076        job = db.get_job(job_id)
    1077        assert job["metadata"]["task_planning_checkpoint_streak"] == 0
    1078        assert job["metadata"]["last_agent_update"]["category"] == "progress"
    1079        assert job["metadata"]["last_agent_update"]["metadata"]["updates"]["tasks"] == 1
    1080        assert job["metadata"]["last_agent_update"]["metadata"]["resolutions"]["tasks"] == 1
    1081    finally:
    1082        db.close()
    1083
    1084
    1085def test_run_one_step_blocks_similar_artifact_search(tmp_path):
    1086    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1087    db = AgentDB(tmp_path / "state.db")
    1088    try:
    1089        job_id = db.create_job("Review saved outputs", title="artifact-search", kind="generic")
    1090        run_id = db.start_run(job_id, model="test")
    1091        step_id = db.add_step(
    1092            job_id=job_id,
    1093            run_id=run_id,
    1094            kind="tool",
    1095            tool_name="search_artifacts",
    1096            input_data={"arguments": {"query": "distillation agentic paper evidence", "limit": 20}},
    1097        )
    1098        db.finish_step(
    1099            step_id,
    1100            status="completed",
    1101            summary="search_artifacts returned 0 results",
    1102            output_data={"success": True, "results": []},
    1103        )
    1104        db.finish_run(run_id, "completed")
    1105
    1106        result = run_one_step(
    1107            job_id,
    1108            config=config,
    1109            db=db,
    1110            llm=ScriptedLLM([
    1111                LLMResponse(tool_calls=[
    1112                    ToolCall(name="search_artifacts", arguments={"query": "paper evidence for agentic distillation", "limit": 20})
    1113                ])
    1114            ]),
    1115        )
    1116
    1117        assert result.status == "blocked"
    1118        assert result.result["error"] == "similar artifact search blocked"
    1119        assert result.result["blocked_tool"] == "search_artifacts"
    1120    finally:
    1121        db.close()
    1122
    1123
    1124def test_run_one_step_blocks_artifact_review_when_tasks_are_exhausted(tmp_path):
    1125    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1126    db = AgentDB(tmp_path / "state.db")
    1127    try:
    1128        job_id = db.create_job(
    1129            "Review saved outputs",
    1130            title="review-exhausted",
    1131            kind="generic",
    1132            metadata={"task_queue": [{"title": "Review first output", "status": "done", "priority": 5}]},
    1133        )
    1134
    1135        result = run_one_step(
    1136            job_id,
    1137            config=config,
    1138            db=db,
    1139            llm=ScriptedLLM([
    1140                LLMResponse(tool_calls=[ToolCall(name="search_artifacts", arguments={"query": "paper evidence"})])
    1141            ]),
    1142        )
    1143
    1144        assert result.status == "blocked"
    1145        assert result.result["error"] == "task branch required before more work"
    1146        assert result.result["blocked_tool"] == "search_artifacts"
    1147    finally:
    1148        db.close()
    1149
    1150
    1151def test_run_one_step_recovers_repeated_guard_blocks_without_llm(tmp_path):
    1152    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1153    db = AgentDB(tmp_path / "state.db")
    1154    try:
    1155        job_id = db.create_job("Recover repeated blocked work", title="guard", kind="generic")
    1156        for index, tool_name in enumerate(["search_artifacts", "shell_exec", "read_artifact"], start=1):
    1157            run_id = db.start_run(job_id, model="test")
    1158            step_id = db.add_step(
    1159                job_id=job_id,
    1160                run_id=run_id,
    1161                kind="tool",
    1162                tool_name=tool_name,
    1163                input_data={"arguments": {"query": f"blocked {index}"}},
    1164            )
    1165            db.finish_step(
    1166                step_id,
    1167                status="blocked",
    1168                summary=f"blocked {tool_name}; progress ledger update required",
    1169                output_data={"success": True, "recoverable": True, "error": "progress ledger update required"},
    1170            )
    1171            db.finish_run(run_id, "completed")
    1172
    1173        result = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    1174
    1175        assert result.status == "completed"
    1176        assert result.tool_name == "guard_recovery"
    1177        assert result.result["guard_recovery"]["error"] == "progress ledger update required"
    1178        job = db.get_job(job_id)
    1179        assert any(task["title"] == "Resolve guard: progress ledger update required" for task in job["metadata"]["task_queue"])
    1180        assert any("Repeated guard block" in lesson["lesson"] for lesson in job["metadata"]["lessons"])
    1181    finally:
    1182        db.close()
    1183
    1184
    1185def test_guard_recovery_does_not_add_task_for_queue_saturation(tmp_path):
    1186    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1187    db = AgentDB(tmp_path / "state.db")
    1188    try:
    1189        job_id = db.create_job(
    1190            "Consolidate a saturated backlog",
    1191            title="guard-saturated-tasks",
    1192            kind="generic",
    1193            metadata={
    1194                "task_queue": [
    1195                    {"title": f"Existing branch {index}", "status": "open", "priority": index}
    1196                    for index in range(40)
    1197                ]
    1198            },
    1199        )
    1200        for index in range(3):
    1201            run_id = db.start_run(job_id, model="test")
    1202            step_id = db.add_step(
    1203                job_id=job_id,
    1204                run_id=run_id,
    1205                kind="tool",
    1206                tool_name="record_tasks",
    1207                input_data={"arguments": {"tasks": [{"title": f"New branch {index}", "status": "open"}]}},
    1208            )
    1209            db.finish_step(
    1210                step_id,
    1211                status="blocked",
    1212                summary="blocked record_tasks; total task queue is too large",
    1213                output_data={
    1214                    "success": False,
    1215                    "recoverable": True,
    1216                    "error": "task queue saturated",
    1217                    "task_queue": {
    1218                        "reason": "total task queue is too large",
    1219                        "open_count": 40,
    1220                        "total_count": 40,
    1221                    },
    1222                },
    1223            )
    1224            db.finish_run(run_id, "completed")
    1225
    1226        result = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    1227
    1228        assert result.status == "completed"
    1229        assert result.tool_name == "guard_recovery"
    1230        assert result.result["task_opened"] is False
    1231        job = db.get_job(job_id)
    1232        tasks = job["metadata"]["task_queue"]
    1233        assert len(tasks) == 40
    1234        assert not any(task["title"].startswith("Resolve guard:") for task in tasks)
    1235        assert job["metadata"]["task_backlog_pressure"]["total_count"] == 40
    1236        assert any("Do not open guard-recovery tasks for saturation" in lesson["lesson"] for lesson in job["metadata"]["lessons"])
    1237    finally:
    1238        db.close()
    1239
    1240
    1241def test_run_one_step_recovers_repeated_evidence_grounding_blocks(tmp_path):
    1242    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1243    db = AgentDB(tmp_path / "state.db")
    1244    try:
    1245        job_id = db.create_job("Recover repeated grounding failures", title="grounding-guard", kind="generic")
    1246        for index in range(3):
    1247            run_id = db.start_run(job_id, model="test")
    1248            step_id = db.add_step(
    1249                job_id=job_id,
    1250                run_id=run_id,
    1251                kind="tool",
    1252                tool_name="record_experiment",
    1253                input_data={"arguments": {"title": f"Unsupported record {index}"}},
    1254            )
    1255            db.finish_step(
    1256                step_id,
    1257                status="blocked",
    1258                summary="blocked record_experiment; evidence grounding required",
    1259                output_data={"success": False, "recoverable": True, "error": "evidence grounding required"},
    1260            )
    1261            db.finish_run(run_id, "completed")
    1262
    1263        result = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    1264
    1265        assert result.status == "completed"
    1266        assert result.tool_name == "guard_recovery"
    1267        assert result.result["guard_recovery"]["error"] == "evidence grounding required"
    1268    finally:
    1269        db.close()
    1270
    1271
    1272def test_run_one_step_recovers_repeated_known_bad_source_blocks(tmp_path):
    1273    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1274    db = AgentDB(tmp_path / "state.db")
    1275    try:
    1276        job_id = db.create_job("Avoid repeatedly blocked sources", title="guard")
    1277        for index in range(3):
    1278            run_id = db.start_run(job_id, model="test")
    1279            step_id = db.add_step(
    1280                job_id=job_id,
    1281                run_id=run_id,
    1282                kind="tool",
    1283                tool_name="web_extract",
    1284                input_data={"arguments": {"urls": ["https://bad.example/source"]}},
    1285            )
    1286            db.finish_step(
    1287                step_id,
    1288                status="blocked",
    1289                summary="blocked web_extract; known bad source https://bad.example/source",
    1290                output_data={"success": False, "error": "known bad source blocked"},
    1291            )
    1292            db.finish_run(run_id, "completed")
    1293
    1294        result = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    1295
    1296        assert result.status == "completed"
    1297        assert result.tool_name == "guard_recovery"
    1298        assert result.result["guard_recovery"]["error"] == "known bad source blocked"
    1299        job = db.get_job(job_id)
    1300        assert any(task["title"] == "Resolve guard: known bad source blocked" for task in job["metadata"]["task_queue"])
    1301    finally:
    1302        db.close()
    1303
    1304
    1305def test_guard_recovery_does_not_repeat_after_recovery_step(tmp_path):
    1306    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1307    db = AgentDB(tmp_path / "state.db")
    1308    try:
    1309        job_id = db.create_job("Recover repeated blocked work once", title="guard-once", kind="generic")
    1310        for index in range(3):
    1311            run_id = db.start_run(job_id, model="test")
    1312            step_id = db.add_step(
    1313                job_id=job_id,
    1314                run_id=run_id,
    1315                kind="tool",
    1316                tool_name="search_artifacts",
    1317                input_data={"arguments": {"query": f"blocked {index}"}},
    1318            )
    1319            db.finish_step(
    1320                step_id,
    1321                status="blocked",
    1322                summary="blocked search_artifacts; progress ledger update required",
    1323                output_data={"success": True, "recoverable": True, "error": "progress ledger update required"},
    1324            )
    1325            db.finish_run(run_id, "completed")
    1326
    1327        first = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    1328        assert first.tool_name == "guard_recovery"
    1329
    1330        second = run_one_step(
    1331            job_id,
    1332            config=config,
    1333            db=db,
    1334            llm=ScriptedLLM([
    1335                LLMResponse(tool_calls=[
    1336                    ToolCall(name="record_lesson", arguments={"lesson": "Recovered guard and chose a new branch", "category": "strategy"})
    1337                ])
    1338            ]),
    1339        )
    1340
    1341        assert second.status == "completed"
    1342        assert second.tool_name == "record_lesson"
    1343        assert [step["tool_name"] for step in db.list_steps(job_id=job_id)[-2:]] == ["guard_recovery", "record_lesson"]
    1344    finally:
    1345        db.close()
    1346
    1347
    1348def test_guard_recovery_does_not_keep_reopening_same_guard(tmp_path):
    1349    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1350    db = AgentDB(tmp_path / "state.db")
    1351    try:
    1352        job_id = db.create_job("Recover repeated blocked work once", title="guard-repeat", kind="generic")
    1353        for batch in range(2):
    1354            for index in range(3):
    1355                run_id = db.start_run(job_id, model="test")
    1356                step_id = db.add_step(
    1357                    job_id=job_id,
    1358                    run_id=run_id,
    1359                    kind="tool",
    1360                    tool_name="search_artifacts",
    1361                    input_data={"arguments": {"query": f"blocked {batch}-{index}"}},
    1362                )
    1363                db.finish_step(
    1364                    step_id,
    1365                    status="blocked",
    1366                    summary="blocked search_artifacts; progress ledger update required",
    1367                    output_data={"success": False, "recoverable": True, "error": "progress ledger update required"},
    1368                )
    1369                db.finish_run(run_id, "completed")
    1370            result = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM() if batch == 0 else ScriptedLLM([
    1371                LLMResponse(tool_calls=[ToolCall(name="record_lesson", arguments={"lesson": "Use a different branch", "category": "strategy"})])
    1372            ]))
    1373
    1374        steps = db.list_steps(job_id=job_id)
    1375        assert sum(1 for step in steps if step["tool_name"] == "guard_recovery") == 1
    1376        assert result.tool_name == "record_lesson"
    1377    finally:
    1378        db.close()
    1379
    1380
    1381def test_guard_recovery_reopens_same_guard_after_progress(tmp_path):
    1382    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1383    db = AgentDB(tmp_path / "state.db")
    1384    try:
    1385        job_id = db.create_job("Recover repeated blocked work after progress", title="guard-progress", kind="generic")
    1386        for index in range(3):
    1387            run_id = db.start_run(job_id, model="test")
    1388            step_id = db.add_step(
    1389                job_id=job_id,
    1390                run_id=run_id,
    1391                kind="tool",
    1392                tool_name="search_artifacts",
    1393                input_data={"arguments": {"query": f"blocked first {index}"}},
    1394            )
    1395            db.finish_step(
    1396                step_id,
    1397                status="blocked",
    1398                summary="blocked search_artifacts; progress ledger update required",
    1399                output_data={"success": False, "recoverable": True, "error": "progress ledger update required"},
    1400            )
    1401            db.finish_run(run_id, "completed")
    1402
    1403        first = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    1404        assert first.tool_name == "guard_recovery"
    1405
    1406        run_id = db.start_run(job_id, model="test")
    1407        progress_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_lesson")
    1408        db.finish_step(progress_step, status="completed", output_data={"success": True, "lesson": "Recovered once."})
    1409        db.finish_run(run_id, "completed")
    1410
    1411        for index in range(3):
    1412            run_id = db.start_run(job_id, model="test")
    1413            step_id = db.add_step(
    1414                job_id=job_id,
    1415                run_id=run_id,
    1416                kind="tool",
    1417                tool_name="read_artifact",
    1418                input_data={"arguments": {"query": f"blocked second {index}"}},
    1419            )
    1420            db.finish_step(
    1421                step_id,
    1422                status="blocked",
    1423                summary="blocked read_artifact; progress ledger update required",
    1424                output_data={"success": False, "recoverable": True, "error": "progress ledger update required"},
    1425            )
    1426            db.finish_run(run_id, "completed")
    1427
    1428        second = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    1429        assert second.tool_name == "guard_recovery"
    1430        assert sum(1 for step in db.list_steps(job_id=job_id) if step["tool_name"] == "guard_recovery") == 2
    1431    finally:
    1432        db.close()
    1433
    1434
    1435def test_guard_recovery_accounts_pending_evidence_checkpoint(tmp_path):
    1436    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1437    db = AgentDB(tmp_path / "state.db")
    1438    try:
    1439        job_id = db.create_job("Recover checkpoint accounting deadlock", title="checkpoint-recovery", kind="generic")
    1440        db.update_job_metadata(
    1441            job_id,
    1442            {
    1443                "pending_evidence_checkpoint": {
    1444                    "artifact_id": "art_checkpoint",
    1445                    "title": "Auto Evidence Checkpoint after step 1",
    1446                    "read_at": "2026-01-01T00:00:00+00:00",
    1447                    "evidence_step_no": 1,
    1448                    "blocked_tool": "shell_exec",
    1449                }
    1450            },
    1451        )
    1452        for index in range(3):
    1453            run_id = db.start_run(job_id, model="test")
    1454            step_id = db.add_step(
    1455                job_id=job_id,
    1456                run_id=run_id,
    1457                kind="tool",
    1458                tool_name="read_artifact",
    1459                input_data={"arguments": {"artifact_id": "art_checkpoint", "retry": index}},
    1460            )
    1461            db.finish_step(
    1462                step_id,
    1463                status="blocked",
    1464                summary="blocked read_artifact; evidence checkpoint accounting required",
    1465                output_data={"success": False, "recoverable": True, "error": "evidence checkpoint accounting required"},
    1466            )
    1467            db.finish_run(run_id, "completed")
    1468
    1469        result = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    1470
    1471        assert result.tool_name == "guard_recovery"
    1472        pending = db.get_job(job_id)["metadata"]["pending_evidence_checkpoint"]
    1473        assert pending["resolved_at"]
    1474        assert pending["resolved_by_tool"] == "guard_recovery"
    1475    finally:
    1476        db.close()
    1477
    1478
    1479def test_guard_recovery_immediately_recovers_already_read_checkpoint_reread(tmp_path):
    1480    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1481    db = AgentDB(tmp_path / "state.db")
    1482    try:
    1483        job_id = db.create_job("Recover checkpoint reread deadlock", title="checkpoint-recovery", kind="generic")
    1484        run_id = db.start_run(job_id, model="test")
    1485        step_id = db.add_step(
    1486            job_id=job_id,
    1487            run_id=run_id,
    1488            kind="tool",
    1489            tool_name="read_artifact",
    1490            input_data={"arguments": {"artifact_id": "art_checkpoint"}},
    1491        )
    1492        db.finish_step(
    1493            step_id,
    1494            status="blocked",
    1495            summary="blocked read_artifact; evidence checkpoint accounting required",
    1496            output_data={
    1497                "success": False,
    1498                "recoverable": True,
    1499                "error": "evidence checkpoint accounting required",
    1500                "blocked_tool": "read_artifact",
    1501                "pending_evidence_checkpoint": {
    1502                    "artifact_id": "art_checkpoint",
    1503                    "checkpoint_read": True,
    1504                    "read_at": "2026-01-01T00:00:00+00:00",
    1505                },
    1506            },
    1507        )
    1508        db.finish_run(run_id, "completed")
    1509
    1510        result = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    1511
    1512        assert result.tool_name == "guard_recovery"
    1513        assert result.result["guard_recovery"]["count"] == 1
    1514        assert result.result["guard_recovery"]["error"] == "evidence checkpoint accounting required"
    1515    finally:
    1516        db.close()
    1517
    1518
    1519def test_prompt_does_not_tell_worker_to_reread_checkpoint_after_it_was_read(tmp_path):
    1520    db = AgentDB(tmp_path / "state.db")
    1521    try:
    1522        job_id = db.create_job("Account for checkpoint", title="checkpoint-prompt", kind="generic")
    1523        db.update_job_metadata(
    1524            job_id,
    1525            {
    1526                "pending_evidence_checkpoint": {
    1527                    "artifact_id": "art_checkpoint",
    1528                    "title": "Auto Evidence Checkpoint after step 1",
    1529                    "read_at": "2026-01-01T00:00:00+00:00",
    1530                    "evidence_step_no": 1,
    1531                    "blocked_tool": "shell_exec",
    1532                }
    1533            },
    1534        )
    1535
    1536        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    1537
    1538        assert "Do not read the checkpoint again" in content
    1539        assert "Next either read that checkpoint artifact" not in content
    1540    finally:
    1541        db.close()
    1542
    1543
    1544def test_checkpoint_reread_block_requires_accounting_not_more_reads(tmp_path):
    1545    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1546    db = AgentDB(tmp_path / "state.db")
    1547    try:
    1548        job_id = db.create_job("Account for checkpoint", title="checkpoint-reread", kind="generic")
    1549        db.update_job_metadata(
    1550            job_id,
    1551            {
    1552                "pending_evidence_checkpoint": {
    1553                    "artifact_id": "art_checkpoint",
    1554                    "title": "Auto Evidence Checkpoint after step 1",
    1555                    "read_at": "2026-01-01T00:00:00+00:00",
    1556                    "evidence_step_no": 1,
    1557                    "blocked_tool": "shell_exec",
    1558                }
    1559            },
    1560        )
    1561
    1562        blocked = run_one_step(
    1563            job_id,
    1564            config=config,
    1565            db=db,
    1566            llm=ScriptedLLM([
    1567                LLMResponse(tool_calls=[ToolCall(name="read_artifact", arguments={"artifact_id": "art_checkpoint"})])
    1568            ]),
    1569        )
    1570
    1571        assert blocked.status == "blocked"
    1572        assert blocked.result["error"] == "evidence checkpoint accounting required"
    1573        assert blocked.result["checkpoint_already_read"] is True
    1574        assert blocked.result["required_next_action"] == "durable_checkpoint_accounting"
    1575        assert "Do not read it again" in blocked.result["guidance"]
    1576
    1577        recovery = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    1578
    1579        assert recovery.tool_name == "guard_recovery"
    1580        task = recovery.result["task"]
    1581        assert task["metadata"]["resolves_evidence_checkpoint"] is True
    1582        assert "Do not read the same checkpoint again" in task["acceptance_criteria"]
    1583    finally:
    1584        db.close()
    1585
    1586
    1587def test_already_read_checkpoint_branch_block_recovers_immediately(tmp_path):
    1588    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1589    db = AgentDB(tmp_path / "state.db")
    1590    try:
    1591        job_id = db.create_job("Recover checkpoint branch deadlock", title="checkpoint-branch", kind="generic")
    1592        db.update_job_metadata(
    1593            job_id,
    1594            {
    1595                "pending_evidence_checkpoint": {
    1596                    "artifact_id": "art_checkpoint",
    1597                    "title": "Auto Evidence Checkpoint after step 1",
    1598                    "read_at": "2026-01-01T00:00:00+00:00",
    1599                    "evidence_step_no": 1,
    1600                    "blocked_tool": "shell_exec",
    1601                }
    1602            },
    1603        )
    1604        run_id = db.start_run(job_id, model="test")
    1605        step_id = db.add_step(
    1606            job_id=job_id,
    1607            run_id=run_id,
    1608            kind="tool",
    1609            tool_name="shell_exec",
    1610            input_data={"arguments": {"command": "echo more branch work"}},
    1611        )
    1612        db.finish_step(
    1613            step_id,
    1614            status="blocked",
    1615            summary="blocked shell_exec; evidence checkpoint accounting required",
    1616            output_data={
    1617                "success": False,
    1618                "recoverable": True,
    1619                "error": "evidence checkpoint accounting required",
    1620                "checkpoint_already_read": True,
    1621                "pending_evidence_checkpoint": {
    1622                    "artifact_id": "art_checkpoint",
    1623                    "checkpoint_read": True,
    1624                },
    1625            },
    1626        )
    1627        db.finish_run(run_id, "completed")
    1628
    1629        result = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    1630
    1631        assert result.tool_name == "guard_recovery"
    1632        assert result.result["guard_recovery"]["count"] == 1
    1633        assert result.result["task"]["metadata"]["resolves_evidence_checkpoint"] is True
    1634        pending = db.get_job(job_id)["metadata"]["pending_evidence_checkpoint"]
    1635        assert pending["resolved_by_tool"] == "guard_recovery"
    1636    finally:
    1637        db.close()
    1638
    1639
    1640def test_evidence_grounding_ignores_format_protocol_tokens():
    1641    tokens = _concrete_evidence_tokens(
    1642        "Parsed JSON from HTTPS REST API URL and saved HTML/YAML/XML CDN SHA256 GGUF excerpts for Model-7B step_123_shell_output. "
    1643        "Download investigation parsed direct API results. Discovery step-2678 located a candidate file after shell_exec_step_1037."
    1644    )
    1645
    1646    assert "JSON" not in tokens
    1647    assert "HTTPS" not in tokens
    1648    assert "REST" not in tokens
    1649    assert "API" not in tokens
    1650    assert "CDN" not in tokens
    1651    assert "SHA256" not in tokens
    1652    assert "GGUF" not in tokens
    1653    assert "URL" not in tokens
    1654    assert "Download" not in tokens
    1655    assert "Discovery" not in tokens
    1656    assert "investigation" not in tokens
    1657    assert "direct" not in tokens
    1658    assert "step_123_shell_output" not in tokens
    1659    assert "step-2678" not in tokens
    1660    assert "shell_exec_step_1037" not in tokens
    1661    assert "Model-7B" in tokens
    1662
    1663
    1664def test_evidence_grounding_ignores_lowercase_command_shorthand_tokens():
    1665    tokens = _concrete_evidence_tokens("Build with cmake --build . -j16 on H100 hardware if observed.")
    1666
    1667    assert "j16" not in tokens
    1668    assert "H100" in tokens
    1669
    1670
    1671def test_record_experiment_allows_not_stub_validation_for_observed_token(tmp_path):
    1672    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1673    db = AgentDB(tmp_path / "state.db")
    1674    try:
    1675        job_id = db.create_job("Validate discovered file", title="grounding", kind="generic")
    1676        run_id = db.start_run(job_id, model="test")
    1677        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    1678        db.finish_step(
    1679            step_id,
    1680            status="completed",
    1681            output_data={
    1682                "success": True,
    1683                "stdout": "-rw-r--r-- 1 user user 12G /srv/models/AlphaModel-99-Q4.foo\n",
    1684                "stderr": "",
    1685            },
    1686        )
    1687        db.finish_run(run_id, "completed")
    1688
    1689        result = run_one_step(
    1690            job_id,
    1691            config=config,
    1692            db=db,
    1693            llm=ScriptedLLM([
    1694                LLMResponse(tool_calls=[
    1695                    ToolCall(
    1696                        name="record_experiment",
    1697                        arguments={
    1698                            "title": "Candidate File Validation",
    1699                            "status": "measured",
    1700                            "metric_name": "usable_files_found",
    1701                            "metric_value": 1,
    1702                            "metric_unit": "files",
    1703                            "result": (
    1704                                "Observed /srv/models/AlphaModel-99-Q4.foo at 12G. "
    1705                                "AlphaModel-99-Q4.foo is not a 29-byte stub."
    1706                            ),
    1707                            "next_action": "Run the next bounded benchmark.",
    1708                        },
    1709                    )
    1710                ])
    1711            ]),
    1712        )
    1713
    1714        assert result.status == "completed"
    1715        assert result.tool_name == "record_experiment"
    1716    finally:
    1717        db.close()
    1718
    1719
    1720def test_record_findings_ignores_generated_step_labels_as_claims(tmp_path):
    1721    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1722    db = AgentDB(tmp_path / "state.db")
    1723    try:
    1724        job_id = db.create_job("Record observed file candidates", title="grounding", kind="generic")
    1725        run_id = db.start_run(job_id, model="test")
    1726        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    1727        db.finish_step(
    1728            step_id,
    1729            status="completed",
    1730            output_data={
    1731                "success": True,
    1732                "stdout": "/srv/models/AlphaModel-99-Q4.foo\n",
    1733                "stderr": "",
    1734            },
    1735        )
    1736        db.finish_run(run_id, "completed")
    1737
    1738        result = run_one_step(
    1739            job_id,
    1740            config=config,
    1741            db=db,
    1742            llm=ScriptedLLM([
    1743                LLMResponse(tool_calls=[
    1744                    ToolCall(
    1745                        name="record_findings",
    1746                        arguments={
    1747                            "findings": [
    1748                                {
    1749                                    "name": "Model file candidate located",
    1750                                    "category": "file_candidate",
    1751                                    "location": "/srv/models/AlphaModel-99-Q4.foo",
    1752                                    "evidence_artifact": "step-2678 shell_exec output",
    1753                                    "reason": "Found via step-2678 shell output.",
    1754                                    "status": "candidate",
    1755                                }
    1756                            ]
    1757                        },
    1758                    )
    1759                ])
    1760            ]),
    1761        )
    1762
    1763        assert result.status == "completed"
    1764        assert result.tool_name == "record_findings"
    1765    finally:
    1766        db.close()
    1767
    1768
    1769def test_write_artifact_allows_plain_prose_headings_without_evidence(tmp_path):
    1770    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1771    db = AgentDB(tmp_path / "state.db")
    1772    try:
    1773        job_id = db.create_job("Summarize observed evidence", title="artifact-grounding", kind="generic")
    1774        run_id = db.start_run(job_id, model="test")
    1775        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    1776        db.finish_step(
    1777            step_id,
    1778            status="completed",
    1779            output_data={
    1780                "success": True,
    1781                "stdout": "Observed status: candidate file exists and benchmark setup is ready for the next measured action.",
    1782                "stderr": "",
    1783            },
    1784        )
    1785        db.finish_run(run_id, "completed")
    1786
    1787        result = run_one_step(
    1788            job_id,
    1789            config=config,
    1790            db=db,
    1791            llm=ScriptedLLM([
    1792                LLMResponse(tool_calls=[
    1793                    ToolCall(
    1794                        name="write_artifact",
    1795                        arguments={
    1796                            "title": "Evidence Consolidation Summary",
    1797                            "content": (
    1798                                "## Discovered\n"
    1799                                "The available observations were consolidated into a concise summary.\n\n"
    1800                                "## Significance\n"
    1801                                "This output records narrative context only and does not introduce a new model, file, or hardware identifier."
    1802                            ),
    1803                        },
    1804                    )
    1805                ])
    1806            ]),
    1807        )
    1808
    1809        assert result.status == "completed"
    1810        assert result.tool_name == "write_artifact"
    1811    finally:
    1812        db.close()
    1813
    1814
    1815def test_write_artifact_blocks_unsupported_high_risk_identifier(tmp_path):
    1816    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1817    db = AgentDB(tmp_path / "state.db")
    1818    try:
    1819        job_id = db.create_job("Summarize observed evidence", title="artifact-grounding", kind="generic")
    1820        run_id = db.start_run(job_id, model="test")
    1821        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    1822        db.finish_step(
    1823            step_id,
    1824            status="completed",
    1825            output_data={
    1826                "success": True,
    1827                "stdout": "Observed model identifier: AlphaModel-99. No other model identifiers were observed.",
    1828                "stderr": "",
    1829            },
    1830        )
    1831        db.finish_run(run_id, "completed")
    1832
    1833        result = run_one_step(
    1834            job_id,
    1835            config=config,
    1836            db=db,
    1837            llm=ScriptedLLM([
    1838                LLMResponse(tool_calls=[
    1839                    ToolCall(
    1840                        name="write_artifact",
    1841                        arguments={
    1842                            "title": "Benchmark Summary",
    1843                            "content": (
    1844                                "The observed candidate was AlphaModel-99.\n"
    1845                                "The final recommendation uses FakeModel-42 for the next benchmark branch."
    1846                            ),
    1847                        },
    1848                    )
    1849                ])
    1850            ]),
    1851        )
    1852
    1853        assert result.status == "blocked"
    1854        assert result.result["error"] == "evidence grounding required"
    1855        assert "FakeModel-42" in result.result["evidence_grounding"]["unsupported_tokens"]
    1856    finally:
    1857        db.close()
    1858
    1859
    1860def test_web_search_auto_records_source_quality(tmp_path):
    1861    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1862    db = AgentDB(tmp_path / "state.db")
    1863    try:
    1864        job_id = db.create_job("Track search sources", title="search-sources", kind="generic")
    1865
    1866        result = run_one_step(
    1867            job_id,
    1868            config=config,
    1869            db=db,
    1870            llm=ScriptedLLM([
    1871                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "durable progress research"})])
    1872            ]),
    1873            registry=SearchRegistry(),
    1874        )
    1875
    1876        assert result.status == "completed"
    1877        sources = db.get_job(job_id)["metadata"]["source_ledger"]
    1878        assert {source["source"] for source in sources} == {
    1879            "https://source.example/primary",
    1880            "https://source.example/secondary",
    1881        }
    1882        assert all(source["source_type"] == "web_search" for source in sources)
    1883    finally:
    1884        db.close()
    1885
    1886
    1887def test_web_extract_auto_records_source_quality(tmp_path):
    1888    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1889    db = AgentDB(tmp_path / "state.db")
    1890    try:
    1891        job_id = db.create_job("Track source quality", title="sources", kind="generic")
    1892
    1893        result = run_one_step(
    1894            job_id,
    1895            config=config,
    1896            db=db,
    1897            llm=ScriptedLLM([
    1898                LLMResponse(tool_calls=[ToolCall(name="web_extract", arguments={"urls": ["https://source.example/a"]})])
    1899            ]),
    1900            registry=ExtractRegistry(),
    1901        )
    1902
    1903        assert result.status == "completed"
    1904        sources = db.get_job(job_id)["metadata"]["source_ledger"]
    1905        assert {source["source"] for source in sources} == {"https://source.example/a", "https://source.example/b"}
    1906        useful = next(source for source in sources if source["source"] == "https://source.example/a")
    1907        failed = next(source for source in sources if source["source"] == "https://source.example/b")
    1908        assert useful["usefulness_score"] >= 0.55
    1909        assert failed["fail_count"] == 1
    1910    finally:
    1911        db.close()
    1912
    1913
    1914def test_worker_cannot_mark_job_completed_by_default(tmp_path):
    1915    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1916    db = AgentDB(tmp_path / "state.db")
    1917    try:
    1918        job_id = db.create_job("Keep improving forever", title="perpetual", kind="generic")
    1919        llm = ScriptedLLM([
    1920            LLMResponse(tool_calls=[
    1921                ToolCall(
    1922                    name="update_job_state",
    1923                    arguments={"status": "completed", "note": "best result saved"},
    1924                )
    1925            ])
    1926        ])
    1927
    1928        result = run_one_step(job_id, config=config, db=db, llm=llm)
    1929        job = db.get_job(job_id)
    1930
    1931        assert result.status == "completed"
    1932        assert result.result["kept_running"] is True
    1933        assert job["status"] == "running"
    1934        assert job["metadata"]["agent_updates"][-1]["metadata"]["requested_status"] == "completed"
    1935    finally:
    1936        db.close()
    1937
    1938
    1939def test_report_update_completion_claim_is_rewritten_as_checkpoint(tmp_path):
    1940    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1941    db = AgentDB(tmp_path / "state.db")
    1942    try:
    1943        job_id = db.create_job("Keep improving forever", title="perpetual", kind="generic")
    1944        result = run_one_step(
    1945            job_id,
    1946            config=config,
    1947            db=db,
    1948            llm=ScriptedLLM([
    1949                LLMResponse(tool_calls=[
    1950                    ToolCall(
    1951                        name="report_update",
    1952                        arguments={"message": "Job completed. Best result saved.", "category": "progress"},
    1953                    )
    1954                ])
    1955            ]),
    1956        )
    1957
    1958        assert result.status == "completed"
    1959        update = db.get_job(job_id)["metadata"]["last_agent_update"]
    1960        assert update["message"] == "Checkpoint reported; continuing work. Best result saved."
    1961        assert update["metadata"]["rewritten_completion_claim"] is True
    1962        assert update["metadata"]["original_message"] == "Job completed. Best result saved."
    1963        assert update["metadata"]["follow_up_task"]
    1964        tasks = db.get_job(job_id)["metadata"]["task_queue"]
    1965        follow_up = next(task for task in tasks if task["key"] == update["metadata"]["follow_up_task"])
    1966        assert follow_up["title"] == "Audit latest checkpoint against objective"
    1967        assert follow_up["status"] == "open"
    1968        assert follow_up["output_contract"] == "decision"
    1969        assert follow_up["metadata"]["completion_audit_required"] is True
    1970    finally:
    1971        db.close()
    1972
    1973
    1974def test_run_one_step_claims_one_message_but_keeps_all_steering_in_prompt(tmp_path):
    1975    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    1976    db = AgentDB(tmp_path / "state.db")
    1977    try:
    1978        job_id = db.create_job("Find durable research findings", title="research", kind="generic")
    1979        db.append_operator_message(job_id, "first instruction", source="chat")
    1980        db.append_operator_message(job_id, "second instruction", source="chat")
    1981        llm = CapturingLLM(LLMResponse(content="No tool this turn."))
    1982
    1983        result = run_one_step(job_id, config=config, db=db, llm=llm)
    1984
    1985        assert result.status == "blocked"
    1986        assert result.result["error"] == "worker tool call required"
    1987        prompt = llm.messages[-1]["content"]
    1988        job = db.get_job(job_id)
    1989        events = db.list_timeline_events(job_id, limit=30)
    1990        assert "first instruction" in prompt
    1991        assert "second instruction" in prompt
    1992        assert job["metadata"]["operator_messages"][0]["claimed_at"]
    1993        assert not job["metadata"]["operator_messages"][1].get("claimed_at")
    1994        assert any(event["event_type"] == "loop" and event["title"] == "agent_start" for event in events)
    1995        assert any(event["event_type"] == "loop" and event["title"] == "turn_end" for event in events)
    1996    finally:
    1997        db.close()
    1998
    1999
    2000class FailingLLM:
    2001    def next_action(self, *, messages, tools):
    2002        del messages, tools
    2003        raise RuntimeError("provider returned no choices")
    2004
    2005
    2006class HardProviderFailingLLM:
    2007    def next_action(self, *, messages, tools):
    2008        del messages, tools
    2009        raise LLMResponseError(
    2010            "Key limit exceeded (total limit)",
    2011            payload={"error": {"message": "Key limit exceeded (total limit)", "code": 403}},
    2012        )
    2013
    2014
    2015def test_run_one_step_records_model_failures_instead_of_raising(tmp_path):
    2016    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2017    db = AgentDB(tmp_path / "state.db")
    2018    try:
    2019        job_id = db.create_job("Keep running despite provider failures", title="provider")
    2020
    2021        result = run_one_step(job_id, config=config, db=db, llm=FailingLLM())
    2022
    2023        assert result.status == "failed"
    2024        assert result.result["error"] == "provider returned no choices"
    2025        assert result.result["duration_seconds"] >= 0
    2026        steps = db.list_steps(job_id=job_id)
    2027        assert steps[0]["kind"] == "llm"
    2028        assert steps[0]["status"] == "failed"
    2029        assert steps[0]["error"] == "provider returned no choices"
    2030        assert steps[0]["input"]["duration_seconds"] >= 0
    2031        assert db.list_runs(job_id)[0]["status"] == "failed"
    2032    finally:
    2033        db.close()
    2034
    2035
    2036def test_run_one_step_blocks_missing_tool_arguments_as_recoverable(tmp_path):
    2037    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2038    db = AgentDB(tmp_path / "state.db")
    2039    try:
    2040        job_id = db.create_job("Keep running despite malformed tool calls", title="tool args")
    2041        llm = ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={})])])
    2042
    2043        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2044
    2045        assert result.status == "blocked"
    2046        assert result.result["recoverable"] is True
    2047        assert result.result["missing_arguments"] == ["command"]
    2048        step = db.list_steps(job_id=job_id)[0]
    2049        assert step["status"] == "blocked"
    2050        assert "missing required arguments" in step["summary"]
    2051        assert not step["error"]
    2052    finally:
    2053        db.close()
    2054
    2055
    2056def test_run_one_step_continues_after_malformed_tool_arguments_when_batch_has_more_work(tmp_path):
    2057    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2058    db = AgentDB(tmp_path / "state.db")
    2059    try:
    2060        job_id = db.create_job("Keep running through recoverable malformed tool calls", title="tool batch")
    2061        llm = ScriptedLLM([
    2062            LLMResponse(tool_calls=[
    2063                ToolCall(name="shell_exec", arguments={}),
    2064                ToolCall(name="record_lesson", arguments={"lesson": "continue with the remaining valid tool call"}),
    2065            ])
    2066        ])
    2067
    2068        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2069
    2070        assert result.tool_name == "record_lesson"
    2071        assert result.status == "completed"
    2072        tool_steps = [step for step in db.list_steps(job_id=job_id) if step["kind"] == "tool"]
    2073        assert [step["tool_name"] for step in tool_steps] == ["shell_exec", "record_lesson"]
    2074        assert tool_steps[0]["status"] == "blocked"
    2075        assert tool_steps[0]["output"]["error"] == "missing required tool arguments"
    2076        assert tool_steps[1]["status"] == "completed"
    2077        assert db.list_runs(job_id)[0]["status"] == "completed"
    2078    finally:
    2079        db.close()
    2080
    2081
    2082def test_run_one_step_continues_after_missing_artifact_when_batch_has_more_work(tmp_path):
    2083    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2084    db = AgentDB(tmp_path / "state.db")
    2085    try:
    2086        job_id = db.create_job("Recover from invented artifact references", title="artifact batch")
    2087        llm = ScriptedLLM([
    2088            LLMResponse(tool_calls=[
    2089                ToolCall(name="read_artifact", arguments={"artifact_id": "art_missing"}),
    2090                ToolCall(name="record_lesson", arguments={"lesson": "search artifacts before reading unknown artifact ids"}),
    2091            ])
    2092        ])
    2093
    2094        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2095
    2096        assert result.tool_name == "record_lesson"
    2097        assert result.status == "completed"
    2098        tool_steps = [step for step in db.list_steps(job_id=job_id) if step["kind"] == "tool"]
    2099        assert [step["tool_name"] for step in tool_steps] == ["read_artifact", "record_lesson"]
    2100        assert tool_steps[0]["status"] == "blocked"
    2101        assert tool_steps[0]["output"]["recoverable"] is True
    2102        assert tool_steps[0]["output"]["error"] == "artifact not found: art_missing"
    2103        assert tool_steps[1]["status"] == "completed"
    2104        assert db.list_runs(job_id)[0]["status"] == "completed"
    2105    finally:
    2106        db.close()
    2107
    2108
    2109def test_run_one_step_continues_after_empty_operator_ack_when_batch_has_more_work(tmp_path):
    2110    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2111    db = AgentDB(tmp_path / "state.db")
    2112    try:
    2113        job_id = db.create_job("Recover from harmless no-op acknowledgements", title="ack batch")
    2114        llm = ScriptedLLM([
    2115            LLMResponse(tool_calls=[
    2116                ToolCall(name="acknowledge_operator_context", arguments={"summary": "already handled"}),
    2117                ToolCall(name="record_lesson", arguments={"lesson": "continue with ordinary progress when no operator ack is needed"}),
    2118            ])
    2119        ])
    2120
    2121        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2122
    2123        assert result.tool_name == "record_lesson"
    2124        assert result.status == "completed"
    2125        tool_steps = [step for step in db.list_steps(job_id=job_id) if step["kind"] == "tool"]
    2126        assert [step["tool_name"] for step in tool_steps] == ["acknowledge_operator_context", "record_lesson"]
    2127        assert tool_steps[0]["status"] == "blocked"
    2128        assert tool_steps[0]["output"]["recoverable"] is True
    2129        assert tool_steps[0]["output"]["error"] == "no active operator context to acknowledge"
    2130        assert tool_steps[1]["status"] == "completed"
    2131        assert db.list_runs(job_id)[0]["status"] == "completed"
    2132    finally:
    2133        db.close()
    2134
    2135
    2136def test_run_one_step_blocks_placeholder_tool_arguments_as_recoverable(tmp_path):
    2137    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2138    db = AgentDB(tmp_path / "state.db")
    2139    try:
    2140        job_id = db.create_job("Keep running despite placeholder tool calls", title="placeholder args")
    2141        llm = ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="read_artifact", arguments={"artifact_id": "..."})])])
    2142
    2143        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2144
    2145        assert result.status == "blocked"
    2146        assert result.result["recoverable"] is True
    2147        assert result.result["error"] == "missing required tool arguments"
    2148        assert result.result["missing_arguments"] == ["artifact reference"]
    2149        step = db.list_steps(job_id=job_id)[0]
    2150        assert step["status"] == "blocked"
    2151        assert "missing required arguments" in step["summary"]
    2152    finally:
    2153        db.close()
    2154
    2155
    2156def test_run_one_step_blocks_truncated_optional_reference_arguments(tmp_path):
    2157    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2158    db = AgentDB(tmp_path / "state.db")
    2159    try:
    2160        job_id = db.create_job("Resolve concrete optional references before recording", title="truncated optional")
    2161        llm = ScriptedLLM([
    2162            LLMResponse(tool_calls=[
    2163                ToolCall(
    2164                    name="record_experiment",
    2165                    arguments={
    2166                        "title": "Validate artifact",
    2167                        "evidence_artifact": "art_123...",
    2168                        "next_action": "read the concrete artifact id",
    2169                    },
    2170                )
    2171            ])
    2172        ])
    2173
    2174        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2175
    2176        assert result.status == "blocked"
    2177        assert result.result["recoverable"] is True
    2178        assert result.result["error"] == "placeholder tool arguments"
    2179        assert result.result["placeholder_arguments"] == ["evidence_artifact"]
    2180        step = db.list_steps(job_id=job_id)[0]
    2181        assert step["status"] == "blocked"
    2182        assert "placeholder tool arguments" in step["summary"]
    2183    finally:
    2184        db.close()
    2185
    2186
    2187def test_run_one_step_blocks_placeholder_shell_command_before_execution(tmp_path):
    2188    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2189    db = AgentDB(tmp_path / "state.db")
    2190    try:
    2191        job_id = db.create_job("Resolve concrete shell inputs before execution", title="placeholder shell")
    2192        llm = ScriptedLLM([
    2193            LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "wget http://output/"})])
    2194        ])
    2195
    2196        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2197
    2198        assert result.status == "blocked"
    2199        assert result.result["error"] == "unresolved placeholder in shell command"
    2200        assert result.result["placeholder"]["value"] == "http://output/"
    2201        assert result.result["recoverable"] is True
    2202        step = db.list_steps(job_id=job_id)[0]
    2203        assert step["status"] == "blocked"
    2204        assert "unresolved placeholder" in step["summary"]
    2205    finally:
    2206        db.close()
    2207
    2208
    2209def test_run_one_step_blocks_tool_markup_shell_command_before_execution(tmp_path):
    2210    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2211    db = AgentDB(tmp_path / "state.db")
    2212    try:
    2213        job_id = db.create_job("Reject malformed tool markup before shell execution", title="tool markup shell")
    2214        llm = ScriptedLLM([
    2215            LLMResponse(tool_calls=[
    2216                ToolCall(
    2217                    name="shell_exec",
    2218                    arguments={"command": "echo ok\n</parameter> }, {"},
    2219                )
    2220            ])
    2221        ])
    2222
    2223        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2224
    2225        assert result.status == "blocked"
    2226        assert result.result["error"] == "unresolved placeholder in shell command"
    2227        assert result.result["placeholder"]["value"] == "</parameter>"
    2228        step = db.list_steps(job_id=job_id)[0]
    2229        assert step["status"] == "blocked"
    2230    finally:
    2231        db.close()
    2232
    2233
    2234def test_run_one_step_blocks_unbalanced_shell_quotes_before_execution(tmp_path):
    2235    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2236    db = AgentDB(tmp_path / "state.db")
    2237    try:
    2238        job_id = db.create_job("Reject partial shell before execution", title="bad shell syntax")
    2239        llm = ScriptedLLM([
    2240            LLMResponse(tool_calls=[
    2241                ToolCall(
    2242                    name="shell_exec",
    2243                    arguments={"command": "echo 'start && ls /tmp"},
    2244                )
    2245            ])
    2246        ])
    2247
    2248        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2249
    2250        assert result.status == "blocked"
    2251        assert result.result["error"] == "malformed shell command"
    2252        assert result.result["recoverable"] is True
    2253        assert result.result["syntax"]["kind"] == "shell_syntax"
    2254        step = db.list_steps(job_id=job_id)[0]
    2255        assert step["status"] == "blocked"
    2256        assert "malformed command syntax" in step["summary"]
    2257    finally:
    2258        db.close()
    2259
    2260
    2261def test_run_one_step_blocks_markdown_fenced_shell_command_before_execution(tmp_path):
    2262    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2263    db = AgentDB(tmp_path / "state.db")
    2264    try:
    2265        job_id = db.create_job("Reject markdown prose before shell execution", title="markdown shell")
    2266        llm = ScriptedLLM([
    2267            LLMResponse(tool_calls=[
    2268                ToolCall(
    2269                    name="shell_exec",
    2270                    arguments={
    2271                        "command": (
    2272                            "ls -la /srv/models/model.bin\n\n"
    2273                            "--- Chapter 2\n\n"
    2274                            "1. ```shell\n"
    2275                            "   chmod +x /tmp/example\n"
    2276                            "```"
    2277                        )
    2278                    },
    2279                )
    2280            ])
    2281        ])
    2282
    2283        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2284
    2285        assert result.status == "blocked"
    2286        assert result.result["error"] == "unresolved placeholder in shell command"
    2287        assert result.result["placeholder"]["kind"] == "markdown_code_fence"
    2288        step = db.list_steps(job_id=job_id)[0]
    2289        assert step["status"] == "blocked"
    2290        assert "unresolved placeholder" in step["summary"]
    2291    finally:
    2292        db.close()
    2293
    2294
    2295def test_run_one_step_times_out_stalled_model_call(tmp_path):
    2296    config = AppConfig(
    2297        runtime=RuntimeConfig(home=tmp_path),
    2298        model=ModelConfig(request_timeout_seconds=0.05),
    2299    )
    2300    db = AgentDB(tmp_path / "state.db")
    2301    try:
    2302        job_id = db.create_job("Keep daemon moving through stalled model calls", title="provider")
    2303
    2304        result = run_one_step(job_id, config=config, db=db, llm=HangingLLM())
    2305
    2306        assert result.status == "failed"
    2307        assert "model call timed out" in result.result["error"]
    2308        assert result.result["duration_seconds"] >= 0.04
    2309        step = db.list_steps(job_id=job_id)[0]
    2310        assert step["kind"] == "llm"
    2311        assert step["status"] == "failed"
    2312        assert step["input"]["duration_seconds"] >= 0.04
    2313    finally:
    2314        db.close()
    2315
    2316
    2317def test_repeated_model_failures_do_not_create_automatic_defer(tmp_path):
    2318    config = AppConfig(
    2319        runtime=RuntimeConfig(home=tmp_path),
    2320        model=ModelConfig(request_timeout_seconds=120),
    2321    )
    2322    db = AgentDB(tmp_path / "state.db")
    2323    try:
    2324        job_id = db.create_job("Keep running through provider instability", title="provider failures")
    2325        for _index in range(2):
    2326            run_id = db.start_run(job_id, model="test")
    2327            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="llm", status="failed")
    2328            db.finish_step(
    2329                step_id,
    2330                status="failed",
    2331                summary="model call failed: APITimeoutError",
    2332                output_data={"success": False, "error": "Request timed out.", "error_type": "APITimeoutError"},
    2333                error="Request timed out.",
    2334            )
    2335            db.finish_run(run_id, "failed", error="Request timed out.")
    2336
    2337        result = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    2338
    2339        assert result.status == "failed"
    2340        assert result.tool_name is None
    2341        job = db.get_job(job_id)
    2342        assert not job["metadata"].get("defer_until")
    2343        assert all(step.get("tool_name") != "defer_job" for step in db.list_steps(job_id=job_id))
    2344    finally:
    2345        db.close()
    2346
    2347
    2348def test_legacy_model_cooldown_metadata_is_ignored(tmp_path):
    2349    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2350    db = AgentDB(tmp_path / "state.db")
    2351    try:
    2352        job_id = db.create_job("Continue after provider instability", title="provider recovered")
    2353        db.update_job_metadata(job_id, {"transient_model_cooldown_streak": 3})
    2354        llm = ScriptedLLM([
    2355            LLMResponse(tool_calls=[ToolCall(name="report_update", arguments={"message": "provider recovered"})])
    2356        ])
    2357
    2358        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2359
    2360        assert result.status == "completed"
    2361        assert result.tool_name == "report_update"
    2362        job = db.get_job(job_id)
    2363        assert job["metadata"]["transient_model_cooldown_streak"] == 3
    2364        assert "transient_model_recovered_at" not in job["metadata"]
    2365        message_end = next(event for event in db.list_events(job_id=job_id, limit=10) if event["event_type"] == "loop" and event["title"] == "message_end")
    2366        assert message_end["metadata"]["duration_seconds"] >= 0
    2367    finally:
    2368        db.close()
    2369
    2370
    2371def test_run_one_step_pauses_job_on_hard_provider_failure(tmp_path):
    2372    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2373    db = AgentDB(tmp_path / "state.db")
    2374    try:
    2375        job_id = db.create_job("Keep running when provider is configured", title="provider")
    2376
    2377        result = run_one_step(job_id, config=config, db=db, llm=HardProviderFailingLLM())
    2378
    2379        assert result.status == "failed"
    2380        assert result.result["provider_action_required"] is True
    2381        assert result.result["pause_reason"] == "llm_provider_blocked"
    2382        job = db.get_job(job_id)
    2383        assert job["status"] == "paused"
    2384        assert "operator action" in job["metadata"]["last_note"]
    2385        assert job["metadata"]["provider_blocked_at"]
    2386        events = db.list_events(job_id=job_id, limit=10)
    2387        assert any(event["event_type"] == "agent_message" and event["title"] == "error" for event in events)
    2388    finally:
    2389        db.close()
    2390
    2391
    2392def test_prompt_includes_recent_tool_arguments_and_observations():
    2393    job = {"title": "research", "kind": "generic", "objective": "find research"}
    2394    steps = [{
    2395        "step_no": 7,
    2396        "kind": "tool",
    2397        "status": "completed",
    2398        "tool_name": "web_search",
    2399        "summary": "web_search query='target model docs' returned 1 results",
    2400        "input": {"arguments": {"query": "target model docs", "limit": 5}},
    2401        "output": {"query": "target model docs", "results": [{"title": "Target Docs", "url": "https://example.com"}]},
    2402    }]
    2403
    2404    messages = build_messages(job, steps)
    2405
    2406    content = messages[-1]["content"]
    2407    assert "target model docs" in content
    2408    assert "Target Docs <https://example.com>" in content
    2409    assert "do not search the same query again" in content
    2410    assert "shell_exec runs on the machine hosting this Nipux worker" in content
    2411    assert str(Path.cwd()) not in content
    2412    assert "read_artifact is only for those saved outputs" in content
    2413
    2414
    2415def test_prompt_recovers_from_missing_artifact_reference():
    2416    job = {"title": "artifact recovery", "kind": "generic", "objective": "use saved evidence"}
    2417    steps = [{
    2418        "step_no": 12,
    2419        "kind": "tool",
    2420        "status": "failed",
    2421        "tool_name": "read_artifact",
    2422        "summary": "read_artifact failed: artifact not found: art_missing",
    2423        "input": {"arguments": {"artifact_id": "art_missing"}},
    2424        "output": {
    2425            "success": False,
    2426            "error": "artifact not found: art_missing",
    2427            "guidance": "Use one of the recent_artifacts refs, call search_artifacts, or continue from evidence.",
    2428            "recent_artifacts": [{"number": "1", "id": "art_real", "title": "Real Evidence"}],
    2429        },
    2430    }]
    2431
    2432    messages = build_messages(job, steps)
    2433    content = messages[-1]["content"]
    2434
    2435    assert "valid_recent_artifacts=art_real=Real Evidence" in content
    2436    assert "Do not invent or retry artifact ids" in content
    2437    assert "search_artifacts" in content
    2438
    2439
    2440def test_prompt_does_not_inject_local_ssh_alias_context(monkeypatch, tmp_path):
    2441    monkeypatch.setenv("HOME", str(tmp_path))
    2442    ssh_dir = tmp_path / ".ssh"
    2443    ssh_dir.mkdir()
    2444    (ssh_dir / "config").write_text("Host remote-box\n  HostName 100.64.0.1\n  User operator\n", encoding="utf-8")
    2445    job = {"title": "remote work", "kind": "generic", "objective": "benchmark remote target"}
    2446
    2447    messages = build_messages(job, [])
    2448
    2449    content = messages[-1]["content"]
    2450    assert "Local CLI context:" not in content
    2451    assert "100.64.0.1" not in content
    2452    assert "remote-box ->" not in content
    2453
    2454
    2455def test_prompt_includes_operator_steering_messages():
    2456    job = {
    2457        "title": "research",
    2458        "kind": "generic",
    2459        "objective": "find research",
    2460        "metadata": {
    2461            "operator_messages": [{
    2462                "at": "2026-04-24T20:40:00+00:00",
    2463                "source": "shell",
    2464                "message": "Focus on actual strong evidence sources, not competing irrelevant sources.",
    2465            }],
    2466        },
    2467    }
    2468
    2469    messages = build_messages(job, [])
    2470
    2471    assert "Operator context:" in messages[-1]["content"]
    2472    assert "Focus on actual strong evidence sources" in messages[-1]["content"]
    2473
    2474
    2475def test_prompt_keeps_claimed_operator_context_until_acknowledged(tmp_path):
    2476    db = AgentDB(tmp_path / "state.db")
    2477    try:
    2478        job_id = db.create_job("Find durable research findings", title="research", kind="generic")
    2479        entry = db.append_operator_message(job_id, "use the corrected target from chat", source="chat")
    2480        claimed = db.claim_operator_messages(job_id, modes=("steer",), limit=1)
    2481        assert claimed[0]["event_id"] == entry["event_id"]
    2482
    2483        job = db.get_job(job_id)
    2484        messages = build_messages(job, [], include_unclaimed_operator_messages=False)
    2485        content = messages[-1]["content"]
    2486
    2487        assert "Operator context:" in content
    2488        assert "use the corrected target from chat" in content
    2489        assert "delivered" in content
    2490
    2491        db.acknowledge_operator_messages(job_id, message_ids=[entry["event_id"]], summary="incorporated correction")
    2492        job = db.get_job(job_id)
    2493        messages = build_messages(job, [], include_unclaimed_operator_messages=False)
    2494
    2495        assert "use the corrected target from chat" not in messages[-1]["content"]
    2496    finally:
    2497        db.close()
    2498
    2499
    2500def test_prompt_keeps_unclaimed_steering_but_not_followup_until_claimed(tmp_path):
    2501    db = AgentDB(tmp_path / "state.db")
    2502    try:
    2503        job_id = db.create_job("Find durable research findings", title="research", kind="generic")
    2504        db.append_operator_message(job_id, "use the corrected target from chat", source="chat", mode="steer")
    2505        db.append_operator_message(job_id, "after this branch settles, write a recap", source="chat", mode="follow_up")
    2506
    2507        job = db.get_job(job_id)
    2508        content = build_messages(job, [], include_unclaimed_operator_messages=True)[-1]["content"]
    2509
    2510        assert "use the corrected target from chat" in content
    2511        assert "after this branch settles" not in content
    2512    finally:
    2513        db.close()
    2514
    2515
    2516def test_prompt_includes_context_pressure_constraint():
    2517    job = {
    2518        "title": "context pressure",
    2519        "kind": "generic",
    2520        "objective": "keep a long-running job stable",
    2521        "metadata": {
    2522            "context_pressure": {
    2523                "band": "high",
    2524                "prompt_tokens": 8_600,
    2525                "context_length": 10_000,
    2526                "fraction": 0.86,
    2527            }
    2528        },
    2529    }
    2530
    2531    content = build_messages(job, [])[-1]["content"]
    2532
    2533    assert "Context pressure:" in content
    2534    assert "Context pressure is high" in content
    2535    assert "8.6K/10.0K" in content
    2536    assert "artifact references" in content
    2537
    2538
    2539def test_prompt_includes_cumulative_usage_pressure():
    2540    job = {
    2541        "title": "usage pressure",
    2542        "kind": "generic",
    2543        "objective": "keep a long-running job useful",
    2544        "metadata": {
    2545            "finding_ledger": [{"name": "durable fact"}],
    2546            "source_ledger": [{"source": "local evidence"}],
    2547            "experiment_ledger": [{"title": "trial", "metric_value": 1}],
    2548            "task_queue": [{"title": "done branch", "status": "done", "result": "validated"}],
    2549        },
    2550    }
    2551
    2552    content = build_messages(
    2553        job,
    2554        [],
    2555        token_usage={
    2556            "calls": 2_100,
    2557            "prompt_tokens": 21_000_000,
    2558            "completion_tokens": 1_000_000,
    2559            "total_tokens": 22_000_000,
    2560            "latest_prompt_tokens": 10_000,
    2561            "latest_context_length": 262_144,
    2562            "cost": 10.25,
    2563            "has_cost": True,
    2564        },
    2565    )[-1]["content"]
    2566
    2567    assert "Usage pressure:" in content
    2568    assert "Cumulative model usage pressure is critical" in content
    2569    assert "calls=2100" in content
    2570    assert "tokens=22.0M" in content
    2571    assert "cost=$10.2500" in content
    2572    assert "high leverage" in content
    2573
    2574
    2575def test_prompt_renders_task_contract_from_metadata_for_existing_tasks():
    2576    job = {
    2577        "title": "contract fallback",
    2578        "kind": "generic",
    2579        "objective": "keep existing task contracts visible",
    2580        "metadata": {
    2581            "task_queue": [
    2582                {
    2583                    "title": "Validate concrete candidate",
    2584                    "status": "active",
    2585                    "priority": 9,
    2586                    "metadata": {"output_contract": "action"},
    2587                    "acceptance_criteria": "candidate tested",
    2588                }
    2589            ],
    2590        },
    2591    }
    2592
    2593    content = build_messages(job, [])[-1]["content"]
    2594
    2595    assert "Task queue:" in content
    2596    assert "Validate concrete candidate" in content
    2597    assert "contract=action" in content
    2598
    2599
    2600def test_prompt_keeps_persistent_task_backlog_pressure_visible():
    2601    job = {
    2602        "title": "persistent backlog pressure",
    2603        "kind": "generic",
    2604        "objective": "keep a long-running job focused",
    2605        "metadata": {
    2606            "task_backlog_pressure": {
    2607                "reason": "total task queue is too large",
    2608                "open_count": 42,
    2609                "total_count": 81,
    2610                "guard_recovery": {
    2611                    "latest_step_no": 123,
    2612                    "task_queue": {"open_titles": ["Existing branch"]},
    2613                },
    2614            },
    2615            "task_queue": [
    2616                {"title": f"Existing branch {index}", "status": "open", "priority": 9}
    2617                for index in range(81)
    2618            ],
    2619        },
    2620    }
    2621
    2622    content = build_messages(job, [])[-1]["content"]
    2623
    2624    assert "Task queue saturation:" in content
    2625    assert "Task backlog pressure remains active from guard recovery #123" in content
    2626    assert "open_tasks=81" in content
    2627    assert "total_tasks=81" in content
    2628    assert "Do not create new task branches" in content
    2629
    2630
    2631def test_prompt_shows_current_task_backlog_pressure_without_prior_block():
    2632    job = {
    2633        "title": "large backlog",
    2634        "kind": "generic",
    2635        "objective": "execute a broad long-running job",
    2636        "metadata": {
    2637            "task_queue": [
    2638                {"title": f"Existing branch {index}", "status": "done", "priority": index}
    2639                for index in range(81)
    2640            ],
    2641        },
    2642    }
    2643
    2644    content = build_messages(job, [])[-1]["content"]
    2645
    2646    assert "Task queue saturation:" in content
    2647    assert "Task backlog pressure remains active from current queue #current" in content
    2648    assert "total_tasks=81" in content
    2649    assert "Do not create new task branches" in content
    2650
    2651
    2652def test_prompt_ignores_stale_task_backlog_pressure_after_queue_is_cleaned_up():
    2653    job = {
    2654        "title": "clean backlog",
    2655        "kind": "generic",
    2656        "objective": "execute a focused long-running job",
    2657        "metadata": {
    2658            "task_backlog_pressure": {
    2659                "reason": "total task queue is too large",
    2660                "open_count": 42,
    2661                "total_count": 80,
    2662                "latest_step_no": 123,
    2663                "source": "blocked_record_tasks",
    2664            },
    2665            "task_queue": [
    2666                {"title": "Focused branch", "status": "active", "priority": 9},
    2667            ],
    2668        },
    2669    }
    2670
    2671    content = build_messages(job, [])[-1]["content"]
    2672
    2673    assert "Task backlog pressure remains active" not in content
    2674    assert "Task queue saturation:\nNone." in content
    2675
    2676
    2677def test_run_one_step_clears_stale_task_backlog_pressure(tmp_path):
    2678    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2679    db = AgentDB(tmp_path / "state.db")
    2680    try:
    2681        job_id = db.create_job(
    2682            "Continue after backlog cleanup",
    2683            title="cleaned-backlog",
    2684            kind="generic",
    2685            metadata={
    2686                "task_backlog_pressure": {
    2687                    "reason": "total task queue is too large",
    2688                    "open_count": 42,
    2689                    "total_count": 80,
    2690                    "source": "blocked_record_tasks",
    2691                },
    2692                "task_queue": [
    2693                    {"title": "Focused branch", "status": "active", "priority": 9},
    2694                ],
    2695            },
    2696        )
    2697        llm = CapturingLLM(LLMResponse(tool_calls=[ToolCall(name="record_lesson", arguments={"lesson": "continue focused work"})]))
    2698
    2699        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2700
    2701        assert result.status == "completed"
    2702        job = db.get_job(job_id)
    2703        assert job["metadata"]["task_backlog_pressure"] == {}
    2704        assert "Task queue saturation:\nNone." in llm.messages[-1]["content"]
    2705        assert any(
    2706            event["event_type"] == "agent_message"
    2707            and event["title"] == "progress"
    2708            and "Task backlog pressure cleared" in event["body"]
    2709            for event in db.list_events(job_id=job_id, limit=20)
    2710        )
    2711    finally:
    2712        db.close()
    2713
    2714
    2715def test_run_one_step_records_usage_pressure_without_spam(tmp_path):
    2716    config = AppConfig(runtime=RuntimeConfig(home=tmp_path), model=ModelConfig(context_length=10_000_000))
    2717    db = AgentDB(tmp_path / "state.db")
    2718    try:
    2719        job_id = db.create_job("Keep a long-running task efficient", title="usage pressure", kind="generic")
    2720        llm = ScriptedLLM([
    2721            LLMResponse(
    2722                tool_calls=[ToolCall(name="record_lesson", arguments={"lesson": "consolidate before spending more", "category": "strategy"})],
    2723                usage={"prompt_tokens": 1_100_000, "completion_tokens": 100, "total_tokens": 1_100_100, "cost": 1.1},
    2724            ),
    2725            LLMResponse(
    2726                tool_calls=[ToolCall(name="record_lesson", arguments={"lesson": "second consolidation", "category": "strategy"})],
    2727                usage={"prompt_tokens": 300_000, "completion_tokens": 100, "total_tokens": 300_100, "cost": 0.3},
    2728            ),
    2729        ])
    2730
    2731        run_one_step(job_id, config=config, db=db, llm=llm)
    2732        run_one_step(job_id, config=config, db=db, llm=llm)
    2733
    2734        pressure_events = [
    2735            event
    2736            for event in db.list_events(job_id=job_id, event_types=["agent_message"])
    2737            if event["metadata"].get("kind") == "usage_pressure"
    2738        ]
    2739        assert len(pressure_events) == 1
    2740        assert "Usage pressure watch" in pressure_events[0]["body"]
    2741        job = db.get_job(job_id)
    2742        pressure = job["metadata"]["usage_pressure"]
    2743        assert pressure["band"] == "watch"
    2744        assert pressure["calls"] == 2
    2745        assert pressure["total_tokens"] == 1_400_200
    2746    finally:
    2747        db.close()
    2748
    2749
    2750def test_critical_usage_does_not_create_automatic_defer(tmp_path):
    2751    config = AppConfig(runtime=RuntimeConfig(home=tmp_path), model=ModelConfig(context_length=262_144))
    2752    db = AgentDB(tmp_path / "state.db")
    2753    try:
    2754        job_id = db.create_job("Keep a long-running task efficient", title="usage pressure", kind="generic")
    2755        db.append_event(
    2756            job_id,
    2757            event_type="loop",
    2758            title="message_end",
    2759            metadata={"usage": {"prompt_tokens": 21_000_000, "completion_tokens": 10_000, "total_tokens": 21_010_000, "cost": 11.0}},
    2760        )
    2761        llm = ScriptedLLM([
    2762            LLMResponse(tool_calls=[ToolCall(name="record_lesson", arguments={"lesson": "keep useful work moving", "category": "strategy"})])
    2763        ])
    2764
    2765        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2766
    2767        assert result.status == "completed"
    2768        assert result.tool_name == "record_lesson"
    2769        job = db.get_job(job_id)
    2770        assert not job["metadata"].get("defer_until")
    2771        assert "usage_pressure_circuit_breaker" not in job["metadata"]
    2772    finally:
    2773        db.close()
    2774
    2775
    2776def test_prompt_ignores_legacy_usage_pressure_recovery_metadata():
    2777    job = {
    2778        "title": "usage recovery",
    2779        "kind": "generic",
    2780        "objective": "Keep long-running work efficient.",
    2781        "metadata": {
    2782            "usage_pressure_circuit_breaker": {
    2783                "latest_step_no": 12,
    2784                "streak": 2,
    2785                "calls": 2200,
    2786                "total_tokens": 25_000_000,
    2787                "cost": 12.5,
    2788                "has_cost": True,
    2789            },
    2790            "task_queue": [{"title": "Focused task", "status": "active", "priority": 9}],
    2791        },
    2792    }
    2793    steps = [
    2794        {"step_no": 13, "kind": "recovery", "status": "completed", "tool_name": "defer_job", "summary": "legacy cooldown"},
    2795        {"step_no": 14, "kind": "tool", "status": "blocked", "tool_name": "web_search", "summary": "blocked search"},
    2796    ]
    2797
    2798    content = build_messages(job, steps)[-1]["content"]
    2799
    2800    assert "Usage pressure:" in content
    2801    assert "Usage pressure recovery" not in content
    2802    assert "cooldown is still unresolved" not in content
    2803
    2804
    2805def test_run_one_step_pauses_when_configured_cost_limit_is_reached(tmp_path):
    2806    config = AppConfig(
    2807        runtime=RuntimeConfig(home=tmp_path, max_job_cost_usd=5.0),
    2808        model=ModelConfig(context_length=262_144),
    2809    )
    2810    db = AgentDB(tmp_path / "state.db")
    2811    try:
    2812        job_id = db.create_job("Keep a long-running task inside budget", title="budget limit", kind="generic")
    2813        db.append_event(
    2814            job_id,
    2815            event_type="loop",
    2816            title="message_end",
    2817            metadata={"usage": {"prompt_tokens": 1_000_000, "completion_tokens": 10_000, "total_tokens": 1_010_000, "cost": 5.25}},
    2818        )
    2819
    2820        result = run_one_step(job_id, config=config, db=db, llm=ExplodingLLM())
    2821
    2822        assert result.status == "completed"
    2823        assert result.tool_name == "budget_limit"
    2824        assert result.result["paused"] is True
    2825        assert result.result["cost"] == 5.25
    2826        job = db.get_job(job_id)
    2827        assert job["status"] == "paused"
    2828        assert job["metadata"]["usage_budget_limit"]["limit"] == 5.0
    2829        assert "configured model cost limit" in job["metadata"]["last_note"]
    2830    finally:
    2831        db.close()
    2832
    2833
    2834def test_run_one_step_ignores_cost_limit_without_provider_cost_metadata(tmp_path):
    2835    config = AppConfig(
    2836        runtime=RuntimeConfig(home=tmp_path, max_job_cost_usd=5.0),
    2837        model=ModelConfig(context_length=262_144),
    2838    )
    2839    db = AgentDB(tmp_path / "state.db")
    2840    try:
    2841        job_id = db.create_job("Keep a long-running task inside budget", title="budget estimate", kind="generic")
    2842        db.append_event(
    2843            job_id,
    2844            event_type="loop",
    2845            title="message_end",
    2846            metadata={"usage": {"prompt_tokens": 1_000_000, "completion_tokens": 10_000, "total_tokens": 1_010_000}},
    2847        )
    2848        llm = ScriptedLLM([
    2849            LLMResponse(tool_calls=[ToolCall(name="report_update", arguments={"message": "cost not provider reported"})])
    2850        ])
    2851
    2852        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2853
    2854        assert result.status == "completed"
    2855        assert result.tool_name == "report_update"
    2856        assert db.get_job(job_id)["status"] == "running"
    2857    finally:
    2858        db.close()
    2859
    2860
    2861def test_run_one_step_does_not_defer_critical_usage_after_progress(tmp_path):
    2862    config = AppConfig(runtime=RuntimeConfig(home=tmp_path), model=ModelConfig(context_length=262_144))
    2863    db = AgentDB(tmp_path / "state.db")
    2864    try:
    2865        job_id = db.create_job("Keep a long-running task efficient", title="usage progress", kind="generic")
    2866        db.append_event(
    2867            job_id,
    2868            event_type="loop",
    2869            title="message_end",
    2870            metadata={"usage": {"prompt_tokens": 21_000_000, "completion_tokens": 10_000, "total_tokens": 21_010_000, "cost": 11.0}},
    2871        )
    2872        for error_type in ["APITimeoutError", "APITimeoutError"]:
    2873            run_id = db.start_run(job_id, model="test")
    2874            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="llm", status="failed")
    2875            db.finish_step(
    2876                step_id,
    2877                status="failed",
    2878                summary=f"model call failed: {error_type}",
    2879                output_data={"success": False, "error": "timeout", "error_type": error_type},
    2880                error="timeout",
    2881            )
    2882            db.finish_run(run_id, "failed", error="timeout")
    2883        run_id = db.start_run(job_id, model="test")
    2884        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    2885        db.finish_step(
    2886            step_id,
    2887            status="completed",
    2888            summary="recorded measured result",
    2889            output_data={
    2890                "success": True,
    2891                "experiment": {
    2892                    "title": "measured result",
    2893                    "status": "measured",
    2894                    "metric_name": "score",
    2895                    "metric_value": 1.0,
    2896                },
    2897            },
    2898        )
    2899        db.finish_run(run_id, "completed")
    2900        llm = ScriptedLLM([
    2901            LLMResponse(tool_calls=[ToolCall(name="report_update", arguments={"message": "continuing from measured progress"})])
    2902        ])
    2903
    2904        result = run_one_step(job_id, config=config, db=db, llm=llm)
    2905
    2906        assert result.status == "completed"
    2907        assert result.tool_name == "report_update"
    2908    finally:
    2909        db.close()
    2910
    2911
    2912def test_run_one_step_drops_conversation_only_chat_from_worker_prompt(tmp_path):
    2913    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    2914    db = AgentDB(tmp_path / "state.db")
    2915    try:
    2916        job_id = db.create_job("Keep improving a generic task", title="context", kind="generic")
    2917        chat = db.append_operator_message(job_id, "hello", source="chat")
    2918        correction = db.append_operator_message(job_id, "use the corrected target from chat", source="chat")
    2919        llm = CapturingLLM(
    2920            LLMResponse(tool_calls=[ToolCall(name="report_update", arguments={"message": "noted", "category": "progress"})])
    2921        )
    2922
    2923        run_one_step(job_id, config=config, db=db, llm=llm)
    2924
    2925        content = llm.messages[-1]["content"]
    2926        assert "hello" not in content
    2927        assert "use the corrected target from chat" in content
    2928        job = db.get_job(job_id)
    2929        messages = {entry["event_id"]: entry for entry in job["metadata"]["operator_messages"]}
    2930        assert messages[chat["event_id"]]["acknowledged_at"]
    2931        assert messages[correction["event_id"]]["claimed_at"]
    2932        assert not messages[correction["event_id"]].get("acknowledged_at")
    2933    finally:
    2934        db.close()
    2935
    2936
    2937def test_build_messages_keeps_generic_context_under_budget():
    2938    job = {
    2939        "title": "large context",
    2940        "kind": "generic",
    2941        "objective": "Improve a measurable process without looping.",
    2942        "metadata": {
    2943            "operator_messages": [
    2944                {"event_id": "chat", "mode": "steer", "message": "how is it going?"},
    2945                {"event_id": "use", "mode": "steer", "message": "use the corrected target from chat"},
    2946            ],
    2947            "lessons": [{"category": "memory", "lesson": "lesson " + "x" * 700} for _ in range(30)],
    2948            "task_queue": [
    2949                {
    2950                    "title": f"Task {index}",
    2951                    "status": "open" if index % 3 else "done",
    2952                    "priority": index,
    2953                    "output_contract": "experiment",
    2954                    "acceptance_criteria": "accept " + "x" * 500,
    2955                    "evidence_needed": "evidence " + "x" * 500,
    2956                    "stall_behavior": "stall " + "x" * 500,
    2957                }
    2958                for index in range(40)
    2959            ],
    2960            "finding_ledger": [{"name": f"Finding {index}", "category": "generic", "score": index} for index in range(200)],
    2961            "source_ledger": [
    2962                {
    2963                    "source": f"https://source{index}.example",
    2964                    "source_type": "web",
    2965                    "usefulness_score": index / 100,
    2966                    "yield_count": index % 4,
    2967                    "fail_count": index % 3,
    2968                    "last_outcome": "outcome " + "x" * 500,
    2969                }
    2970                for index in range(90)
    2971            ],
    2972            "experiment_ledger": [
    2973                {
    2974                    "title": f"Experiment {index}",
    2975                    "status": "measured",
    2976                    "metric_name": "score",
    2977                    "metric_value": index,
    2978                    "metric_unit": "units",
    2979                    "best_observed": index in {38, 39},
    2980                    "result": "result " + "x" * 600,
    2981                    "next_action": "next " + "x" * 600,
    2982                }
    2983                for index in range(40)
    2984            ],
    2985            "reflections": [{"summary": "summary " + "x" * 800, "strategy": "strategy " + "x" * 800} for _ in range(20)],
    2986        },
    2987    }
    2988    steps = [
    2989        {
    2990            "step_no": index,
    2991            "kind": "tool",
    2992            "status": "completed",
    2993            "tool_name": "shell_exec",
    2994            "summary": "summary " + "x" * 800,
    2995            "input": {"arguments": {"command": "command " + "x" * 800}},
    2996            "output": {"success": True, "command": "command", "returncode": 0, "stdout": "stdout " + "x" * 3000},
    2997        }
    2998        for index in range(30)
    2999    ]
    3000    memory_entries = [{"key": "rolling_state", "summary": "memory " + "x" * 20000, "artifact_refs": [f"art_{i}" for i in range(40)]}]
    3001    timeline = [{"event_type": "tool_result", "title": "event", "body": "body " + "x" * 900} for _ in range(40)]
    3002
    3003    messages = build_messages(job, steps, memory_entries=memory_entries, timeline_events=timeline)
    3004    content = messages[-1]["content"]
    3005
    3006    assert "use the corrected target from chat" in content
    3007    assert "how is it going" not in content
    3008    assert len(content) < MAX_WORKER_PROMPT_CHARS
    3009    assert "Next-action constraint:" in content
    3010
    3011
    3012def test_prompt_timeline_filters_low_signal_tool_noise():
    3013    job = {
    3014        "title": "timeline",
    3015        "kind": "generic",
    3016        "objective": "keep useful context visible",
    3017        "metadata": {},
    3018    }
    3019    timeline = [
    3020        {
    3021            "event_type": "tool_result",
    3022            "title": "web_search",
    3023            "body": f"search noise {index}",
    3024            "metadata": {"status": "completed"},
    3025            "created_at": f"2026-05-01T12:{index:02d}:00+00:00",
    3026        }
    3027        for index in range(20)
    3028    ]
    3029    timeline.extend([
    3030        {
    3031            "event_type": "artifact",
    3032            "title": "Saved durable report",
    3033            "body": "operator-visible output",
    3034            "metadata": {},
    3035            "created_at": "2026-05-01T13:00:00+00:00",
    3036        },
    3037        {
    3038            "event_type": "finding",
    3039            "title": "Useful durable finding",
    3040            "body": "result worth keeping",
    3041            "metadata": {},
    3042            "created_at": "2026-05-01T13:01:00+00:00",
    3043        },
    3044        {
    3045            "event_type": "tool_result",
    3046            "title": "shell_exec",
    3047            "body": "command failed with actionable blocker",
    3048            "metadata": {"status": "failed"},
    3049            "created_at": "2026-05-01T13:02:00+00:00",
    3050        },
    3051    ])
    3052
    3053    content = build_messages(job, [], timeline_events=timeline)[-1]["content"]
    3054
    3055    assert "Recent visible timeline:" in content
    3056    assert "High-signal timeline counts:" in content
    3057    assert "Saved durable report" in content
    3058    assert "Useful durable finding" in content
    3059    assert "command failed with actionable blocker" in content
    3060    assert "search noise" not in content
    3061
    3062
    3063def test_prompt_includes_durable_outcome_summary():
    3064    job = {
    3065        "title": "outcomes",
    3066        "kind": "generic",
    3067        "objective": "keep useful durable progress visible",
    3068        "metadata": {},
    3069    }
    3070    events = [
    3071        {
    3072            "event_type": "artifact",
    3073            "title": "Draft checkpoint",
    3074            "body": "",
    3075            "metadata": {},
    3076        },
    3077        {
    3078            "event_type": "finding",
    3079            "title": "Reusable finding",
    3080            "body": "",
    3081            "metadata": {},
    3082        },
    3083        {
    3084            "event_type": "experiment",
    3085            "title": "Quality check",
    3086            "body": "",
    3087            "metadata": {"metric_name": "score", "metric_value": 0.82, "metric_unit": ""},
    3088        },
    3089        {
    3090            "event_type": "tool_result",
    3091            "title": "web_search",
    3092            "body": "web_search query='background' returned 5 results",
    3093            "metadata": {"status": "completed"},
    3094        },
    3095    ]
    3096
    3097    content = build_messages(job, [], timeline_events=events)[-1]["content"]
    3098    outcome_section = content.split("Durable outcomes:", 1)[1].split("Ledgers:", 1)[0]
    3099
    3100    assert "Outcome counts: 1 outputs 1 findings 1 measurements." in outcome_section
    3101    assert "save: Draft checkpoint" in outcome_section
    3102    assert "find: Reusable finding" in outcome_section
    3103    assert "test: Quality check" in outcome_section
    3104    assert "background" not in outcome_section
    3105
    3106
    3107def test_emergency_prompt_clipping_repeats_operator_and_next_action():
    3108    job = {"title": "clip", "kind": "generic", "objective": "keep context safe"}
    3109    sections = [(f"Noise {index}", "noise " * 2000) for index in range(90)]
    3110    sections.insert(45, ("Operator context", "Still-active durable operator context: use the corrected target."))
    3111    sections.append(("Next-action constraint", "Next use the validated branch."))
    3112
    3113    content = _render_worker_prompt(job, sections=sections)
    3114
    3115    assert len(content) <= MAX_WORKER_PROMPT_CHARS
    3116    assert "middle context clipped" in content
    3117    suffix = content.split("middle context clipped", 1)[1]
    3118    assert "Operator context:" in suffix
    3119    assert "use the corrected target" in suffix
    3120    assert "Next-action constraint:" in suffix
    3121    assert "Next use the validated branch" in suffix
    3122
    3123
    3124def test_build_messages_keeps_rolling_memory_when_not_first():
    3125    job = {"title": "memory order", "kind": "generic", "objective": "keep long-running context stable"}
    3126    memory_entries = [
    3127        {"key": "newer_note", "summary": "newer side note"},
    3128        {"key": "other_note", "summary": "less important side note"},
    3129        {"key": "rolling_state", "summary": "durable rolling state with usage and task progress"},
    3130    ]
    3131
    3132    content = build_messages(job, [], memory_entries=memory_entries)[-1]["content"]
    3133
    3134    assert "durable rolling state with usage and task progress" in content
    3135    assert "newer side note" in content
    3136    assert "less important side note" not in content
    3137
    3138
    3139def test_build_messages_surfaces_recent_measurement_evidence_outside_state_window():
    3140    job = {"title": "measure", "kind": "generic", "objective": "improve a measurable process", "metadata": {}}
    3141    recent_steps = [
    3142        {
    3143            "step_no": 1,
    3144            "kind": "tool",
    3145            "status": "completed",
    3146            "tool_name": "shell_exec",
    3147            "input": {"arguments": {"command": "run benchmark"}},
    3148            "output": {
    3149                "success": True,
    3150                "stdout": (
    3151                    "| model | test | t/s |\n"
    3152                    "| --- | ---: | ---: |\n"
    3153                    "| example | pp32 | 5.48 ± 0.11 |\n"
    3154                    "| example | tg128 | 3.44 ± 0.05 |\n"
    3155                ),
    3156            },
    3157        },
    3158        *[
    3159            {
    3160                "step_no": index,
    3161                "kind": "tool",
    3162                "status": "completed",
    3163                "tool_name": "record_lesson",
    3164                "summary": f"later step {index}",
    3165                "input": {},
    3166                "output": {},
    3167            }
    3168            for index in range(2, 12)
    3169        ],
    3170    ]
    3171
    3172    content = build_messages(job, recent_steps)[-1]["content"]
    3173
    3174    assert "Recent measurement evidence:" in content
    3175    assert "step #1 completed" in content
    3176    assert "pp32 5.48 ± 0.11 t/s" in content
    3177    assert "tg128 3.44 ± 0.05 t/s" in content
    3178
    3179
    3180def test_measurement_obligation_blocks_research_until_recorded(tmp_path):
    3181    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3182    db = AgentDB(tmp_path / "state.db")
    3183    try:
    3184        job_id = db.create_job("Improve a measurable process", title="measure", kind="generic")
    3185
    3186        first = run_one_step(
    3187            job_id,
    3188            config=config,
    3189            db=db,
    3190            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "run test"})])]),
    3191            registry=MeasuredShellRegistry(),
    3192        )
    3193        job = db.get_job(job_id)
    3194        assert first.tool_name == "shell_exec"
    3195        assert job["metadata"]["pending_measurement_obligation"]["metric_candidates"]
    3196
    3197        second = run_one_step(
    3198            job_id,
    3199            config=config,
    3200            db=db,
    3201            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "more notes"})])]),
    3202            registry=MeasuredShellRegistry(),
    3203        )
    3204        assert second.status == "blocked"
    3205        assert second.result["error"] == "measurement obligation pending"
    3206
    3207        third = run_one_step(
    3208            job_id,
    3209            config=config,
    3210            db=db,
    3211            llm=ScriptedLLM([
    3212                LLMResponse(tool_calls=[
    3213                    ToolCall(
    3214                        name="record_experiment",
    3215                            arguments={
    3216                                "title": "measured trial",
    3217                                "status": "measured",
    3218                                "metric_name": "score",
    3219                                "metric_value": 2.7,
    3220                                "metric_unit": "units/s",
    3221                                "next_action": "compare the next concrete variant",
    3222                            },
    3223                        )
    3224                    ])
    3225                ]),
    3226        )
    3227        job = db.get_job(job_id)
    3228        assert third.tool_name == "record_experiment"
    3229        assert job["metadata"].get("pending_measurement_obligation") == {}
    3230        assert job["metadata"]["experiment_ledger"][0]["metric_value"] == 2.7
    3231    finally:
    3232        db.close()
    3233
    3234
    3235def test_measurement_obligation_preserves_table_metric_candidates(tmp_path):
    3236    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3237    db = AgentDB(tmp_path / "state.db")
    3238    try:
    3239        job_id = db.create_job("Improve a measurable process", title="measure-table", kind="generic")
    3240
    3241        step = run_one_step(
    3242            job_id,
    3243            config=config,
    3244            db=db,
    3245            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "run benchmark"})])]),
    3246            registry=TableBenchmarkShellRegistry(),
    3247        )
    3248
    3249        job = db.get_job(job_id)
    3250        candidates = job["metadata"]["pending_measurement_obligation"]["metric_candidates"]
    3251        assert step.tool_name == "shell_exec"
    3252        assert "pp32 5.48 ± 0.11 t/s" in candidates
    3253        assert "tg128 3.44 ± 0.05 t/s" in candidates
    3254    finally:
    3255        db.close()
    3256
    3257
    3258def test_failed_shell_measurement_output_still_requires_accounting(tmp_path):
    3259    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3260    db = AgentDB(tmp_path / "state.db")
    3261    try:
    3262        job_id = db.create_job("Improve a measurable process", title="measure-failed-table", kind="generic")
    3263
    3264        step = run_one_step(
    3265            job_id,
    3266            config=config,
    3267            db=db,
    3268            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "run benchmark"})])]),
    3269            registry=FailedTableBenchmarkShellRegistry(),
    3270        )
    3271
    3272        job = db.get_job(job_id)
    3273        candidates = job["metadata"]["pending_measurement_obligation"]["metric_candidates"]
    3274        assert step.status == "failed"
    3275        assert "pp32 5.48 ± 0.11 t/s" in candidates
    3276    finally:
    3277        db.close()
    3278
    3279
    3280def test_measurement_obligation_blocks_operator_acknowledgement_churn(tmp_path):
    3281    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3282    db = AgentDB(tmp_path / "state.db")
    3283    try:
    3284        job_id = db.create_job("Improve a measurable process", title="measure-ack", kind="generic")
    3285        db.update_job_metadata(
    3286            job_id,
    3287            {
    3288                "pending_measurement_obligation": {
    3289                    "source_step_no": 12,
    3290                    "tool": "shell_exec",
    3291                    "metric_candidates": ["2.7 tok/s"],
    3292                    "command": "run benchmark",
    3293                }
    3294            },
    3295        )
    3296
    3297        result = run_one_step(
    3298            job_id,
    3299            config=config,
    3300            db=db,
    3301            llm=ScriptedLLM([
    3302                LLMResponse(tool_calls=[
    3303                    ToolCall(
    3304                        name="acknowledge_operator_context",
    3305                        arguments={"message_ids": [], "summary": "acknowledged"},
    3306                    )
    3307                ])
    3308            ]),
    3309        )
    3310
    3311        assert result.status == "blocked"
    3312        assert result.tool_name == "acknowledge_operator_context"
    3313        assert result.result["error"] == "measurement obligation pending"
    3314    finally:
    3315        db.close()
    3316
    3317
    3318def test_pending_measurement_narrows_available_tools(tmp_path):
    3319    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3320    db = AgentDB(tmp_path / "state.db")
    3321    try:
    3322        job_id = db.create_job("Improve a measurable process", title="measure-tools", kind="generic")
    3323        db.update_job_metadata(
    3324            job_id,
    3325            {
    3326                "pending_measurement_obligation": {
    3327                    "source_step_no": 12,
    3328                    "tool": "shell_exec",
    3329                    "metric_candidates": ["2.7 tok/s"],
    3330                    "command": "run benchmark",
    3331                }
    3332            },
    3333        )
    3334        llm = CapturingLLM(
    3335            LLMResponse(tool_calls=[
    3336                ToolCall(
    3337                    name="record_experiment",
    3338                    arguments={
    3339                        "title": "measured trial",
    3340                        "status": "measured",
    3341                        "metric_name": "speed",
    3342                        "metric_value": 2.7,
    3343                        "metric_unit": "tok/s",
    3344                        "next_action": "try the next measured branch",
    3345                    },
    3346                )
    3347            ])
    3348        )
    3349
    3350        run_one_step(job_id, config=config, db=db, llm=llm)
    3351
    3352        tool_names = {tool["function"]["name"] for tool in llm.tools}
    3353        assert {"record_experiment", "record_lesson", "record_tasks"}.issubset(tool_names)
    3354        assert "shell_exec" not in tool_names
    3355        assert "web_search" not in tool_names
    3356        assert "acknowledge_operator_context" not in tool_names
    3357    finally:
    3358        db.close()
    3359
    3360
    3361def test_resolution_tools_survive_task_saturation_suppression(tmp_path):
    3362    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3363    db = AgentDB(tmp_path / "state.db")
    3364    try:
    3365        job_id = db.create_job(
    3366            "Improve a measurable process",
    3367            title="measure-tools-after-saturation",
    3368            kind="generic",
    3369            metadata={
    3370                "pending_measurement_obligation": {
    3371                    "source_step_no": 12,
    3372                    "tool": "shell_exec",
    3373                    "metric_candidates": ["2.7 tok/s"],
    3374                    "command": "run benchmark",
    3375                }
    3376            },
    3377        )
    3378        run_id = db.start_run(job_id, model="fake")
    3379        for step_no in range(2):
    3380            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    3381            db.finish_step(
    3382                step_id,
    3383                status="blocked",
    3384                summary="blocked record_tasks; task queue saturated",
    3385                output_data={
    3386                    "success": False,
    3387                    "error": "task queue saturated",
    3388                    "task_queue": {"reason": "total task queue is too large", "total_count": 80 + step_no},
    3389                },
    3390            )
    3391        llm = CapturingLLM(
    3392            LLMResponse(tool_calls=[
    3393                ToolCall(
    3394                    name="record_lesson",
    3395                    arguments={"lesson": "Measurement is blocked until the current branch is reconciled."},
    3396                )
    3397            ])
    3398        )
    3399
    3400        run_one_step(job_id, config=config, db=db, llm=llm)
    3401
    3402        tool_names = {tool["function"]["name"] for tool in llm.tools}
    3403        assert {"record_experiment", "record_lesson", "record_tasks"}.issubset(tool_names)
    3404        assert "web_search" not in tool_names
    3405    finally:
    3406        db.close()
    3407
    3408
    3409def test_pending_evidence_checkpoint_narrows_available_tools(tmp_path):
    3410    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3411    db = AgentDB(tmp_path / "state.db")
    3412    try:
    3413        job_id = db.create_job("Account for checkpointed evidence", title="checkpoint-tools", kind="generic")
    3414        db.update_job_metadata(
    3415            job_id,
    3416            {
    3417                "pending_evidence_checkpoint": {
    3418                    "artifact_id": "art_checkpoint",
    3419                    "title": "Checkpoint",
    3420                    "evidence_step_no": 12,
    3421                    "blocked_tool": "shell_exec",
    3422                    "created_at": "2026-01-01T00:00:00+00:00",
    3423                }
    3424            },
    3425        )
    3426        llm = CapturingLLM(
    3427            LLMResponse(tool_calls=[
    3428                ToolCall(name="record_lesson", arguments={"lesson": "checkpoint accounted for", "category": "memory"})
    3429            ])
    3430        )
    3431
    3432        run_one_step(job_id, config=config, db=db, llm=llm)
    3433
    3434        tool_names = {tool["function"]["name"] for tool in llm.tools}
    3435        assert "read_artifact" in tool_names
    3436        assert {"record_findings", "record_source", "record_lesson", "record_experiment"}.issubset(tool_names)
    3437        assert "record_tasks" not in tool_names
    3438        assert "shell_exec" not in tool_names
    3439        assert "web_search" not in tool_names
    3440        assert "acknowledge_operator_context" not in tool_names
    3441    finally:
    3442        db.close()
    3443
    3444
    3445def test_acknowledge_operator_context_hidden_without_active_operator_context(tmp_path):
    3446    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3447    db = AgentDB(tmp_path / "state.db")
    3448    try:
    3449        job_id = db.create_job("Run ordinary autonomous work", title="no-operator", kind="generic")
    3450        llm = CapturingLLM(LLMResponse(tool_calls=[ToolCall(name="report_update", arguments={"message": "working"})]))
    3451
    3452        run_one_step(job_id, config=config, db=db, llm=llm)
    3453
    3454        tool_names = {tool["function"]["name"] for tool in llm.tools}
    3455        assert "acknowledge_operator_context" not in tool_names
    3456    finally:
    3457        db.close()
    3458
    3459
    3460def test_acknowledge_operator_context_visible_with_active_operator_context(tmp_path):
    3461    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3462    db = AgentDB(tmp_path / "state.db")
    3463    try:
    3464        job_id = db.create_job("Run with operator steering", title="operator", kind="generic")
    3465        db.append_operator_message(job_id, "use the corrected target", source="chat")
    3466        llm = CapturingLLM(LLMResponse(tool_calls=[ToolCall(name="report_update", arguments={"message": "working"})]))
    3467
    3468        run_one_step(job_id, config=config, db=db, llm=llm)
    3469
    3470        tool_names = {tool["function"]["name"] for tool in llm.tools}
    3471        assert "acknowledge_operator_context" in tool_names
    3472    finally:
    3473        db.close()
    3474
    3475
    3476def test_diagnostic_shell_output_does_not_create_measurement_obligation(tmp_path):
    3477    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3478    db = AgentDB(tmp_path / "state.db")
    3479    try:
    3480        job_id = db.create_job("Improve a measurable process", title="measure", kind="generic")
    3481
    3482        result = run_one_step(
    3483            job_id,
    3484            config=config,
    3485            db=db,
    3486            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "df -h && nproc && free -h"})])]),
    3487            registry=DiagnosticShellRegistry(),
    3488        )
    3489
    3490        job = db.get_job(job_id)
    3491        assert result.tool_name == "shell_exec"
    3492        assert job["metadata"].get("pending_measurement_obligation") in (None, {})
    3493    finally:
    3494        db.close()
    3495
    3496
    3497def test_source_code_shell_output_does_not_create_measurement_obligation(tmp_path):
    3498    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3499    db = AgentDB(tmp_path / "state.db")
    3500    try:
    3501        job_id = db.create_job("Improve a measurable process", title="measure", kind="generic")
    3502
    3503        result = run_one_step(
    3504            job_id,
    3505            config=config,
    3506            db=db,
    3507            llm=ScriptedLLM([
    3508                LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "git show HEAD:nipux_cli/cli.py"})])
    3509            ]),
    3510            registry=SourceCodeShellRegistry(),
    3511        )
    3512
    3513        job = db.get_job(job_id)
    3514        assert result.tool_name == "shell_exec"
    3515        assert job["metadata"].get("pending_measurement_obligation") in (None, {})
    3516    finally:
    3517        db.close()
    3518
    3519
    3520def test_prose_from_timed_command_does_not_create_measurement_obligation(tmp_path):
    3521    class ProseShellRegistry:
    3522        def openai_tools(self):
    3523            return []
    3524
    3525        def handle(self, name, args, ctx):
    3526            del args, ctx
    3527            if name == "shell_exec":
    3528                return json.dumps({
    3529                    "success": True,
    3530                    "command": "time cat draft.txt",
    3531                    "returncode": 0,
    3532                    "stdout": 'This draft says "time". 2 examples are listed. It asks readers to rate a story.',
    3533                    "stderr": "",
    3534                })
    3535            return json.dumps({"success": True})
    3536
    3537    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3538    db = AgentDB(tmp_path / "state.db")
    3539    try:
    3540        job_id = db.create_job("Improve a measurable process", title="measure", kind="generic")
    3541
    3542        result = run_one_step(
    3543            job_id,
    3544            config=config,
    3545            db=db,
    3546            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "time cat draft.txt"})])]),
    3547            registry=ProseShellRegistry(),
    3548        )
    3549
    3550        job = db.get_job(job_id)
    3551        assert result.tool_name == "shell_exec"
    3552        assert job["metadata"].get("pending_measurement_obligation") in (None, {})
    3553    finally:
    3554        db.close()
    3555
    3556
    3557def test_large_shell_output_must_be_saved_before_more_shell_churn(tmp_path):
    3558    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3559    db = AgentDB(tmp_path / "state.db")
    3560    try:
    3561        job_id = db.create_job("Audit a repository", title="audit", kind="generic")
    3562
    3563        first = run_one_step(
    3564            job_id,
    3565            config=config,
    3566            db=db,
    3567            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "find . -type f"})])]),
    3568            registry=LargeShellEvidenceRegistry(),
    3569        )
    3570        second = run_one_step(
    3571            job_id,
    3572            config=config,
    3573            db=db,
    3574            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "find . -name '*.md'"})])]),
    3575            registry=LargeShellEvidenceRegistry(),
    3576        )
    3577
    3578        assert first.tool_name == "shell_exec"
    3579        assert second.status == "blocked"
    3580        assert second.result["error"] == "artifact required before more research"
    3581    finally:
    3582        db.close()
    3583
    3584
    3585def test_stale_diagnostic_measurement_obligation_is_cleared(tmp_path):
    3586    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3587    db = AgentDB(tmp_path / "state.db")
    3588    try:
    3589        job_id = db.create_job(
    3590            "Improve a measurable process",
    3591            title="measure",
    3592            kind="generic",
    3593            metadata={
    3594                "pending_measurement_obligation": {
    3595                    "source_step_no": 1,
    3596                    "command": "df -h && nproc && free -h",
    3597                    "metric_candidates": ["CPU COUNT 24", "RAM 93"],
    3598                }
    3599            },
    3600        )
    3601
    3602        result = run_one_step(
    3603            job_id,
    3604            config=config,
    3605            db=db,
    3606            llm=ScriptedLLM([
    3607                LLMResponse(tool_calls=[
    3608                    ToolCall(
    3609                        name="record_lesson",
    3610                        arguments={
    3611                            "lesson": "The stale output is diagnostic context, not a valid measurement; rerun with a metric.",
    3612                            "category": "memory",
    3613                        },
    3614                    )
    3615                ])
    3616            ]),
    3617        )
    3618
    3619        job = db.get_job(job_id)
    3620        assert result.tool_name == "record_lesson"
    3621        assert job["metadata"].get("pending_measurement_obligation") == {}
    3622        assert "diagnostic context" in job["metadata"]["last_agent_update"]["message"]
    3623    finally:
    3624        db.close()
    3625
    3626
    3627def test_measurable_objective_blocks_research_after_budget_but_allows_action(tmp_path):
    3628    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3629    db = AgentDB(tmp_path / "state.db")
    3630    try:
    3631        job_id = db.create_job("Optimize a measurable process", title="measured", kind="generic")
    3632        for index in range(19):
    3633            run_id = db.start_run(job_id)
    3634            step_id = db.add_step(
    3635                job_id=job_id,
    3636                run_id=run_id,
    3637                kind="tool",
    3638                tool_name="web_search" if index % 2 == 0 else "web_extract",
    3639                input_data={"arguments": {"query": f"research branch {index}"}},
    3640            )
    3641            db.finish_step(step_id, status="completed", output_data={"success": True})
    3642            db.finish_run(run_id, "completed")
    3643
    3644        blocked = run_one_step(
    3645            job_id,
    3646            config=config,
    3647            db=db,
    3648            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "more research"})])]),
    3649            registry=MeasuredShellRegistry(),
    3650        )
    3651        assert blocked.status == "blocked"
    3652        assert blocked.result["error"] == "measured progress required"
    3653
    3654        action = run_one_step(
    3655            job_id,
    3656            config=config,
    3657            db=db,
    3658            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "run test"})])]),
    3659            registry=MeasuredShellRegistry(),
    3660        )
    3661        job = db.get_job(job_id)
    3662        assert action.status == "completed"
    3663        assert action.tool_name == "shell_exec"
    3664        assert job["metadata"]["pending_measurement_obligation"]["metric_candidates"]
    3665    finally:
    3666        db.close()
    3667
    3668
    3669def test_measurable_objective_blocks_shell_churn_without_experiment_accounting(tmp_path):
    3670    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3671    db = AgentDB(tmp_path / "state.db")
    3672    try:
    3673        job_id = db.create_job("Optimize a measurable process", title="measured", kind="generic")
    3674        for index in range(4):
    3675            run_id = db.start_run(job_id)
    3676            step_id = db.add_step(
    3677                job_id=job_id,
    3678                run_id=run_id,
    3679                kind="tool",
    3680                tool_name="shell_exec",
    3681                input_data={"arguments": {"command": f"probe {index}"}},
    3682            )
    3683            db.finish_step(step_id, status="completed", output_data={"success": True, "stdout": "no metric"})
    3684            db.finish_run(run_id, "completed")
    3685
    3686        blocked = run_one_step(
    3687            job_id,
    3688            config=config,
    3689            db=db,
    3690            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "probe again"})])]),
    3691            registry=MeasuredShellRegistry(),
    3692        )
    3693
    3694        assert blocked.status == "blocked"
    3695        assert blocked.result["error"] == "measured progress required"
    3696    finally:
    3697        db.close()
    3698
    3699
    3700def test_measured_progress_guard_narrows_available_tools_after_shell_budget(tmp_path):
    3701    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3702    db = AgentDB(tmp_path / "state.db")
    3703    try:
    3704        job_id = db.create_job("Optimize a measurable process", title="measured-tools", kind="generic")
    3705        for index in range(4):
    3706            run_id = db.start_run(job_id)
    3707            step_id = db.add_step(
    3708                job_id=job_id,
    3709                run_id=run_id,
    3710                kind="tool",
    3711                tool_name="shell_exec",
    3712                input_data={"arguments": {"command": f"probe {index}"}},
    3713            )
    3714            db.finish_step(step_id, status="completed", output_data={"success": True, "stdout": "no metric"})
    3715            db.finish_run(run_id, "completed")
    3716        llm = CapturingLLM(
    3717            LLMResponse(tool_calls=[
    3718                ToolCall(
    3719                    name="record_lesson",
    3720                    arguments={"lesson": "measurement blocked after probes", "category": "blocker"},
    3721                )
    3722            ])
    3723        )
    3724
    3725        run_one_step(job_id, config=config, db=db, llm=llm)
    3726
    3727        tool_names = {tool["function"]["name"] for tool in llm.tools}
    3728        assert {"record_experiment", "record_lesson", "record_tasks"}.issubset(tool_names)
    3729        assert "shell_exec" not in tool_names
    3730        assert "write_artifact" not in tool_names
    3731        assert "web_search" not in tool_names
    3732    finally:
    3733        db.close()
    3734
    3735
    3736def test_measured_progress_guard_keeps_shell_available_before_shell_budget(tmp_path):
    3737    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3738    db = AgentDB(tmp_path / "state.db")
    3739    try:
    3740        job_id = db.create_job("Optimize a measurable process", title="measured-tools", kind="generic")
    3741        for index in range(19):
    3742            run_id = db.start_run(job_id)
    3743            step_id = db.add_step(
    3744                job_id=job_id,
    3745                run_id=run_id,
    3746                kind="tool",
    3747                tool_name="web_search" if index % 2 == 0 else "web_extract",
    3748                input_data={"arguments": {"query": f"research branch {index}"}},
    3749            )
    3750            db.finish_step(step_id, status="completed", output_data={"success": True})
    3751            db.finish_run(run_id, "completed")
    3752        llm = CapturingLLM(
    3753            LLMResponse(tool_calls=[
    3754                ToolCall(
    3755                    name="record_lesson",
    3756                    arguments={"lesson": "convert research budget into a measured trial", "category": "strategy"},
    3757                )
    3758            ])
    3759        )
    3760
    3761        run_one_step(job_id, config=config, db=db, llm=llm)
    3762
    3763        tool_names = {tool["function"]["name"] for tool in llm.tools}
    3764        assert "shell_exec" in tool_names
    3765        assert {"record_experiment", "record_lesson", "record_tasks"}.issubset(tool_names)
    3766        assert "write_artifact" not in tool_names
    3767        assert "web_search" not in tool_names
    3768    finally:
    3769        db.close()
    3770
    3771
    3772def test_measured_progress_guard_ignores_non_measurement_task_updates(tmp_path):
    3773    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3774    db = AgentDB(tmp_path / "state.db")
    3775    try:
    3776        job_id = db.create_job("Optimize a measurable process", title="measured", kind="generic")
    3777        for index in range(18):
    3778            run_id = db.start_run(job_id)
    3779            step_id = db.add_step(
    3780                job_id=job_id,
    3781                run_id=run_id,
    3782                kind="tool",
    3783                tool_name="web_search",
    3784                input_data={"arguments": {"query": f"research branch {index}"}},
    3785            )
    3786            db.finish_step(step_id, status="completed", output_data={"success": True})
    3787            db.finish_run(run_id, "completed")
    3788        run_id = db.start_run(job_id)
    3789        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    3790        db.finish_step(
    3791            step_id,
    3792            status="completed",
    3793            output_data={
    3794                "success": True,
    3795                "tasks": [{"title": "Write notes", "status": "open", "output_contract": "report"}],
    3796            },
    3797        )
    3798        db.finish_run(run_id, "completed")
    3799
    3800        blocked = run_one_step(
    3801            job_id,
    3802            config=config,
    3803            db=db,
    3804            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "more notes"})])]),
    3805            registry=MeasuredShellRegistry(),
    3806        )
    3807
    3808        assert blocked.status == "blocked"
    3809        assert blocked.result["error"] == "measured progress required"
    3810    finally:
    3811        db.close()
    3812
    3813
    3814def test_measured_progress_guard_accepts_measurement_task_update(tmp_path):
    3815    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3816    db = AgentDB(tmp_path / "state.db")
    3817    try:
    3818        job_id = db.create_job("Optimize a measurable process", title="measured", kind="generic")
    3819        for index in range(18):
    3820            run_id = db.start_run(job_id)
    3821            step_id = db.add_step(
    3822                job_id=job_id,
    3823                run_id=run_id,
    3824                kind="tool",
    3825                tool_name="web_search",
    3826                input_data={"arguments": {"query": f"research branch {index}"}},
    3827            )
    3828            db.finish_step(step_id, status="completed", output_data={"success": True})
    3829            db.finish_run(run_id, "completed")
    3830        run_id = db.start_run(job_id)
    3831        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_tasks")
    3832        db.finish_step(
    3833            step_id,
    3834            status="completed",
    3835            output_data={
    3836                "success": True,
    3837                "tasks": [{"title": "Run measured variant", "status": "open", "output_contract": "experiment"}],
    3838            },
    3839        )
    3840        db.finish_run(run_id, "completed")
    3841
    3842        allowed = run_one_step(
    3843            job_id,
    3844            config=config,
    3845            db=db,
    3846            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "run measured variant"})])]),
    3847            registry=MeasuredShellRegistry(),
    3848        )
    3849
    3850        assert allowed.status == "completed"
    3851        assert allowed.tool_name == "shell_exec"
    3852    finally:
    3853        db.close()
    3854
    3855
    3856def test_measurable_objective_allows_candidate_file_validation_shell_after_budget(tmp_path):
    3857    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    3858    db = AgentDB(tmp_path / "state.db")
    3859    try:
    3860        job_id = db.create_job("Optimize a measurable file-backed process", title="measured-file", kind="generic")
    3861        db.update_job_metadata(
    3862            job_id,
    3863            {
    3864                "task_queue": [
    3865                    {
    3866                        "title": "Validate candidate file and benchmark",
    3867                        "status": "active",
    3868                        "acceptance_criteria": "Exact candidate path is validated before benchmarking.",
    3869                        "evidence_needed": "Shell output showing file size for /srv/models/AlphaModel-99-Q4.foo.",
    3870                        "output_contract": "experiment",
    3871                    }
    3872                ]
    3873            },
    3874        )
    3875        for index in range(4):
    3876            run_id = db.start_run(job_id)
    3877            stdout = "no metric"
    3878            if index == 0:
    3879                stdout = "/srv/models/AlphaModel-99-Q4.foo\n"
    3880            step_id = db.add_step(
    3881                job_id=job_id,
    3882                run_id=run_id,
    3883                kind="tool",
    3884                tool_name="shell_exec",
    3885                input_data={"arguments": {"command": f"probe {index}"}},
    3886            )
    3887            db.finish_step(step_id, status="completed", output_data={"success": True, "stdout": stdout})
    3888            db.finish_run(run_id, "completed")
    3889
    3890        result = run_one_step(
    3891            job_id,
    3892            config=config,
    3893            db=db,
    3894            llm=ScriptedLLM([
    3895                LLMResponse(tool_calls=[
    3896                    ToolCall(
    3897                        name="shell_exec",
    3898                        arguments={"command": "ls -lh /srv/models/AlphaModel-99-Q4.foo && file /srv/models/AlphaModel-99-Q4.foo"},
    3899                    )
    3900                ])
    3901            ]),
    3902            registry=MeasuredShellRegistry(),
    3903        )
    3904
    3905        assert result.status == "completed"
    3906        assert result.tool_name == "shell_exec"
    3907    finally:
    3908        db.close()
    3909
    3910
    3911def test_prompt_includes_durable_lessons():
    3912    job = {
    3913        "title": "research",
    3914        "kind": "generic",
    3915        "objective": "find research",
    3916        "metadata": {
    3917            "lessons": [{
    3918                "category": "source_quality",
    3919                "lesson": "Low-evidence pages are background noise, not durable findings.",
    3920            }],
    3921        },
    3922    }
    3923
    3924    messages = build_messages(job, [])
    3925
    3926    content = messages[-1]["content"]
    3927    assert "Lessons learned:" in content
    3928    assert "Low-evidence pages are background noise" in content
    3929
    3930
    3931def test_prompt_suppresses_stale_negative_lessons_when_positive_durable_evidence_exists():
    3932    job = {
    3933        "title": "research",
    3934        "kind": "generic",
    3935        "objective": "keep facts current",
    3936        "metadata": {
    3937            "finding_ledger": [
    3938                {
    3939                    "name": "Observed local model",
    3940                    "reason": "ModelX-99 appears in the local model list with size 17 GB.",
    3941                }
    3942            ],
    3943            "lessons": [
    3944                {
    3945                    "category": "strategy",
    3946                    "lesson": (
    3947                        "No ModelX-99 model has been successfully downloaded, so keep the download "
    3948                        "branch as the primary blocker before benchmark work."
    3949                    ),
    3950                }
    3951            ],
    3952        },
    3953    }
    3954
    3955    content = build_messages(job, [])[-1]["content"]
    3956
    3957    assert "Potentially stale negative lesson suppressed for ModelX-99" in content
    3958    assert "No ModelX-99 model has been successfully downloaded" not in content
    3959
    3960
    3961def test_prompt_keeps_negative_lessons_when_durable_evidence_is_negative():
    3962    job = {
    3963        "title": "research",
    3964        "kind": "generic",
    3965        "objective": "keep facts current",
    3966        "metadata": {
    3967            "finding_ledger": [
    3968                {
    3969                    "name": "Missing local model",
    3970                    "reason": "ls cannot access ModelX-99: no such file or directory.",
    3971                }
    3972            ],
    3973            "lessons": [
    3974                {
    3975                    "category": "strategy",
    3976                    "lesson": "No ModelX-99 file exists in the checked path; use a different observed source.",
    3977                }
    3978            ],
    3979        },
    3980    }
    3981
    3982    content = build_messages(job, [])[-1]["content"]
    3983
    3984    assert "No ModelX-99 file exists" in content
    3985    assert "Potentially stale negative lesson suppressed" not in content
    3986
    3987
    3988def test_prompt_includes_memory_graph_slice():
    3989    job = {
    3990        "title": "research",
    3991        "kind": "generic",
    3992        "objective": "keep improving the output",
    3993        "metadata": {
    3994            "memory_graph": {
    3995                "nodes": [
    3996                    {
    3997                        "key": "validated-checkpoints",
    3998                        "title": "Validated checkpoints compound progress",
    3999                        "kind": "strategy",
    4000                        "status": "active",
    4001                        "summary": "Save evidence, validate it, then branch from the gap.",
    4002                        "salience": 0.95,
    4003                        "tags": ["progress"],
    4004                        "evidence_refs": ["art_1"],
    4005                    },
    4006                    {
    4007                        "key": "weak-source",
    4008                        "title": "Weak source path",
    4009                        "kind": "source",
    4010                        "status": "deprecated",
    4011                        "summary": "This path produced low-yield repeats.",
    4012                        "salience": 0.1,
    4013                    },
    4014                ],
    4015                "edges": [
    4016                    {
    4017                        "from_key": "validated-checkpoints",
    4018                        "to_key": "weak-source",
    4019                        "relation": "replaces",
    4020                    }
    4021                ],
    4022            }
    4023        },
    4024    }
    4025
    4026    content = build_messages(job, [])[-1]["content"]
    4027
    4028    assert "Memory graph:" in content
    4029    assert "Validated checkpoints compound progress" in content
    4030    assert "strategy" in content
    4031    assert "replaces -> weak-source" in content
    4032    assert "art_1" in content
    4033
    4034
    4035def test_prompt_suppresses_memory_graph_nodes_matching_stale_claim_tokens(tmp_path):
    4036    db = AgentDB(tmp_path / "state.db")
    4037    try:
    4038        job_id = db.create_job("Prefer current durable evidence", title="stale-graph", kind="generic")
    4039        db.append_lesson(
    4040            job_id,
    4041            "Evidence grounding rejected unsupported concrete tokens for record_experiment: E5-2690, v3. Treat matching prior ledger claims as stale.",
    4042            category="mistake",
    4043        )
    4044        db.append_memory_graph_records(
    4045            job_id,
    4046            nodes=[
    4047                {
    4048                    "key": "old-hardware",
    4049                    "title": "Intel Xeon E5-2690 v3 baseline",
    4050                    "kind": "fact",
    4051                    "status": "stable",
    4052                    "summary": "Old baseline that should not enter the prompt after contradiction.",
    4053                },
    4054                {
    4055                    "key": "current-evidence",
    4056                    "title": "Current observed hardware needs verification",
    4057                    "kind": "fact",
    4058                    "status": "active",
    4059                    "summary": "Continue from fresh shell evidence only.",
    4060                },
    4061            ],
    4062        )
    4063
    4064        job = db.get_job(job_id)
    4065        content = build_messages(job, db.list_steps(job_id=job_id))[-1]["content"]
    4066
    4067        assert "Suppressed 1 stale memory node" in content
    4068        assert "Current observed hardware needs verification" in content
    4069        assert "Intel Xeon E5-2690 v3 baseline" not in content
    4070    finally:
    4071        db.close()
    4072
    4073
    4074def test_prompt_suppresses_negative_memory_graph_nodes_matching_stale_file_type(tmp_path):
    4075    db = AgentDB(tmp_path / "state.db")
    4076    try:
    4077        job_id = db.create_job("Prefer current file evidence", title="stale-file-type", kind="generic")
    4078        db.update_job_metadata(
    4079            job_id,
    4080            {
    4081                "stale_negative_records": [
    4082                    {
    4083                        "kind": "memory_node",
    4084                        "record_id": "old-absence",
    4085                        "token": ".foo",
    4086                        "evidence": "/srv/models/AlphaModel.foo",
    4087                    }
    4088                ]
    4089            },
    4090        )
    4091        db.append_memory_graph_records(
    4092            job_id,
    4093            nodes=[
    4094                {
    4095                    "key": "download-blocker",
    4096                    "title": "Model file download critical blocker",
    4097                    "kind": "constraint",
    4098                    "status": "active",
    4099                    "summary": "FOO model download attempts return 0 files. All downstream work is blocked until a model file exists locally.",
    4100                },
    4101                {
    4102                    "key": "format-skill",
    4103                    "title": "FOO format tuning",
    4104                    "kind": "skill",
    4105                    "status": "active",
    4106                    "summary": "Use the FOO runtime flags after a valid file is selected.",
    4107                },
    4108            ],
    4109        )
    4110
    4111        job = db.get_job(job_id)
    4112        content = build_messages(job, db.list_steps(job_id=job_id))[-1]["content"]
    4113
    4114        assert "Suppressed 1 stale memory node" in content
    4115        assert "Model file download critical blocker" not in content
    4116        assert "FOO format tuning" in content
    4117    finally:
    4118        db.close()
    4119
    4120
    4121def test_prompt_pushes_memory_graph_consolidation_when_ledgers_exist_without_nodes():
    4122    job = {
    4123        "title": "research",
    4124        "kind": "generic",
    4125        "objective": "keep improving the output",
    4126        "metadata": {
    4127            "lessons": [{"lesson": "Prefer validated checkpoints.", "category": "strategy"}],
    4128            "experiment_ledger": [{"title": "Trial", "status": "measured"}],
    4129            "task_queue": [{"title": "Next branch", "status": "open"}],
    4130        },
    4131    }
    4132
    4133    content = build_messages(job, [])[-1]["content"]
    4134
    4135    assert "No memory graph yet" in content
    4136    assert "Durable ledgers already contain 3 reusable item" in content
    4137    assert "record_memory_graph" in content
    4138
    4139
    4140def test_prompt_adds_memory_consolidation_guard_when_graph_lags_ledgers():
    4141    job = {
    4142        "title": "research",
    4143        "kind": "generic",
    4144        "objective": "keep improving the output",
    4145        "metadata": {
    4146            "lessons": [
    4147                {"lesson": "Use validated checkpoints.", "category": "strategy"},
    4148                {"lesson": "Reject low-yield branches.", "category": "strategy"},
    4149            ],
    4150            "experiment_ledger": [{"title": "Trial", "status": "measured"}],
    4151            "finding_ledger": [{"name": "Finding A"}, {"name": "Finding B"}],
    4152            "source_ledger": [{"source": "source:a"}],
    4153        },
    4154    }
    4155
    4156    content = build_messages(job, [])[-1]["content"]
    4157
    4158    assert "Memory consolidation guard:" in content
    4159    assert "durable_records=6" in content
    4160    assert "record_memory_graph" in content
    4161
    4162
    4163def test_prompt_adds_research_balance_guard_for_execution_without_sources():
    4164    job = {
    4165        "title": "workflow builder",
    4166        "kind": "generic",
    4167        "objective": "build a durable workflow and keep improving it",
    4168        "metadata": {
    4169            "experiment_ledger": [{"title": "Validation check", "status": "measured"}],
    4170        },
    4171    }
    4172    steps = [
    4173        {
    4174            "step_no": index,
    4175            "kind": "tool",
    4176            "tool_name": "shell_exec",
    4177            "status": "completed",
    4178            "input": {"arguments": {"command": f"echo branch-{index}"}},
    4179        }
    4180        for index in range(1, 7)
    4181    ]
    4182
    4183    content = build_messages(job, steps)[-1]["content"]
    4184
    4185    assert "Research balance guard:" in content
    4186    assert "execution-heavy" in content
    4187    assert "sources=0" in content
    4188    assert "record_source" in content
    4189
    4190
    4191def test_prompt_research_balance_guard_clears_when_sources_exist():
    4192    job = {
    4193        "title": "workflow builder",
    4194        "kind": "generic",
    4195        "objective": "build a durable workflow and keep improving it",
    4196        "metadata": {
    4197            "source_ledger": [{"source": "project docs"}],
    4198            "experiment_ledger": [{"title": "Validation check", "status": "measured"}],
    4199        },
    4200    }
    4201    steps = [
    4202        {
    4203            "step_no": index,
    4204            "kind": "tool",
    4205            "tool_name": "shell_exec",
    4206            "status": "completed",
    4207            "input": {"arguments": {"command": f"echo branch-{index}"}},
    4208        }
    4209        for index in range(1, 7)
    4210    ]
    4211
    4212    content = build_messages(job, steps)[-1]["content"]
    4213
    4214    assert "Recent work is execution-heavy" not in content
    4215
    4216
    4217def _source_yield_metadata(source_count: int = 16, finding_count: int = 1, *, include_memory_graph: bool = True) -> dict:
    4218    metadata = {
    4219        "source_ledger": [
    4220            {
    4221                "source": f"https://source.example/{index}",
    4222                "source_type": "web_extract",
    4223                "usefulness_score": 0.55,
    4224                "yield_count": 0,
    4225                "last_outcome": "extracted source text for possible use",
    4226            }
    4227            for index in range(source_count)
    4228        ],
    4229        "finding_ledger": [
    4230            {
    4231                "name": f"Finding {index}",
    4232                "source_url": f"https://source.example/{index}",
    4233            }
    4234            for index in range(finding_count)
    4235        ],
    4236    }
    4237    if include_memory_graph:
    4238        metadata["memory_graph"] = {
    4239            "nodes": [
    4240                {"key": f"source-node-{index}", "kind": "source", "title": f"Source set {index}"}
    4241                for index in range(4)
    4242            ],
    4243            "edges": [
    4244                {"from": "source-node-0", "to": "source-node-1", "kind": "supports"},
    4245                {"from": "source-node-1", "to": "source-node-2", "kind": "supports"},
    4246                {"from": "source-node-2", "to": "source-node-3", "kind": "supports"},
    4247            ],
    4248        }
    4249    return metadata
    4250
    4251
    4252def _source_gathering_steps(count: int = 6) -> list[dict]:
    4253    return [
    4254        {
    4255            "step_no": index,
    4256            "kind": "tool",
    4257            "tool_name": "web_extract" if index % 2 else "web_search",
    4258            "status": "completed",
    4259            "input": {"arguments": {"query": f"source branch {index}"}},
    4260        }
    4261        for index in range(1, count + 1)
    4262    ]
    4263
    4264
    4265def test_prompt_adds_source_yield_guard_when_sources_are_not_synthesized():
    4266    job = {
    4267        "title": "source-heavy job",
    4268        "kind": "generic",
    4269        "objective": "research and produce durable conclusions",
    4270        "metadata": _source_yield_metadata(),
    4271    }
    4272
    4273    content = build_messages(job, _source_gathering_steps())[-1]["content"]
    4274
    4275    assert "Source yield guard:" in content
    4276    assert "Many sources have been gathered" in content
    4277    assert "sources=16" in content
    4278    assert "findings=1" in content
    4279    assert "record_findings" in content
    4280
    4281
    4282def test_prompt_source_yield_guard_clears_when_findings_cover_sources():
    4283    job = {
    4284        "title": "source-heavy job",
    4285        "kind": "generic",
    4286        "objective": "research and produce durable conclusions",
    4287        "metadata": _source_yield_metadata(finding_count=2),
    4288    }
    4289
    4290    content = build_messages(job, _source_gathering_steps())[-1]["content"]
    4291
    4292    assert "Source yield guard:" in content
    4293    assert "Many sources have been gathered" not in content
    4294
    4295
    4296def test_run_one_step_blocks_more_source_gathering_when_source_yield_is_missing(tmp_path):
    4297    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4298    db = AgentDB(tmp_path / "state.db")
    4299    try:
    4300        job_id = db.create_job(
    4301            "Research and produce durable conclusions",
    4302            title="source-yield",
    4303            kind="generic",
    4304            metadata=_source_yield_metadata(),
    4305        )
    4306        run_id = db.start_run(job_id, model="test")
    4307        for step in _source_gathering_steps():
    4308            step_id = db.add_step(
    4309                job_id=job_id,
    4310                run_id=run_id,
    4311                kind="tool",
    4312                tool_name=step["tool_name"],
    4313                input_data=step["input"],
    4314            )
    4315            db.finish_step(step_id, status="completed", output_data={"success": True})
    4316        db.finish_run(run_id, "completed")
    4317
    4318        result = run_one_step(
    4319            job_id,
    4320            config=config,
    4321            db=db,
    4322            llm=ScriptedLLM([
    4323                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "more sources"})])
    4324            ]),
    4325        )
    4326
    4327        assert result.status == "blocked"
    4328        assert result.result["error"] == "source yield accounting required"
    4329        assert result.result["source_yield"]["sources"] == 16
    4330    finally:
    4331        db.close()
    4332
    4333
    4334def test_source_yield_guard_takes_priority_over_memory_consolidation(tmp_path):
    4335    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4336    db = AgentDB(tmp_path / "state.db")
    4337    try:
    4338        job_id = db.create_job(
    4339            "Research and produce durable conclusions",
    4340            title="source-yield-priority",
    4341            kind="generic",
    4342            metadata=_source_yield_metadata(include_memory_graph=False),
    4343        )
    4344        run_id = db.start_run(job_id, model="test")
    4345        for step in _source_gathering_steps():
    4346            step_id = db.add_step(
    4347                job_id=job_id,
    4348                run_id=run_id,
    4349                kind="tool",
    4350                tool_name=step["tool_name"],
    4351                input_data=step["input"],
    4352            )
    4353            db.finish_step(step_id, status="completed", output_data={"success": True})
    4354        db.finish_run(run_id, "completed")
    4355
    4356        result = run_one_step(
    4357            job_id,
    4358            config=config,
    4359            db=db,
    4360            llm=ScriptedLLM([
    4361                LLMResponse(tool_calls=[ToolCall(name="web_extract", arguments={"urls": ["https://source.example/new"]})])
    4362            ]),
    4363        )
    4364
    4365        assert result.status == "blocked"
    4366        assert result.result["error"] == "source yield accounting required"
    4367    finally:
    4368        db.close()
    4369
    4370
    4371def test_run_one_step_allows_source_yield_accounting(tmp_path):
    4372    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4373    db = AgentDB(tmp_path / "state.db")
    4374    try:
    4375        job_id = db.create_job(
    4376            "Research and produce durable conclusions",
    4377            title="source-yield",
    4378            kind="generic",
    4379            metadata=_source_yield_metadata(),
    4380        )
    4381        run_id = db.start_run(job_id, model="test")
    4382        for step in _source_gathering_steps():
    4383            step_id = db.add_step(
    4384                job_id=job_id,
    4385                run_id=run_id,
    4386                kind="tool",
    4387                tool_name=step["tool_name"],
    4388                input_data=step["input"],
    4389            )
    4390            db.finish_step(step_id, status="completed", output_data={"success": True})
    4391        db.finish_run(run_id, "completed")
    4392
    4393        result = run_one_step(
    4394            job_id,
    4395            config=config,
    4396            db=db,
    4397            llm=ScriptedLLM([
    4398                LLMResponse(tool_calls=[
    4399                    ToolCall(
    4400                        name="record_source",
    4401                        arguments={
    4402                            "source": "https://source.example/0",
    4403                            "source_type": "web_extract",
    4404                            "yield_count": 1,
    4405                            "outcome": "Source produced a durable conclusion for the active branch.",
    4406                        },
    4407                    )
    4408                ])
    4409            ]),
    4410        )
    4411
    4412        assert result.status == "completed"
    4413        assert result.tool_name == "record_source"
    4414    finally:
    4415        db.close()
    4416
    4417
    4418def test_run_one_step_blocks_execution_when_research_balance_is_missing(tmp_path):
    4419    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4420    db = AgentDB(tmp_path / "state.db")
    4421    try:
    4422        job_id = db.create_job(
    4423            "Build a durable workflow and keep improving it",
    4424            title="research-balance",
    4425            kind="generic",
    4426            metadata={"experiment_ledger": [{"title": "Validation check", "status": "measured"}]},
    4427        )
    4428        run_id = db.start_run(job_id, model="test")
    4429        for index in range(6):
    4430            step_id = db.add_step(
    4431                job_id=job_id,
    4432                run_id=run_id,
    4433                kind="tool",
    4434                tool_name="shell_exec",
    4435                input_data={"arguments": {"command": f"python branch_{index}.py"}},
    4436            )
    4437            db.finish_step(step_id, status="completed", output_data={"success": True, "stdout": "ok"})
    4438        db.finish_run(run_id, "completed")
    4439
    4440        result = run_one_step(
    4441            job_id,
    4442            config=config,
    4443            db=db,
    4444            llm=ScriptedLLM([
    4445                LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "python continue_branch.py"})])
    4446            ]),
    4447        )
    4448
    4449        assert result.status == "blocked"
    4450        assert result.result["error"] == "research balance required"
    4451        assert result.result["blocked_tool"] == "shell_exec"
    4452        assert "record_source" in result.result["guidance"]
    4453    finally:
    4454        db.close()
    4455
    4456
    4457def test_run_one_step_blocks_lesson_churn_when_research_balance_is_missing(tmp_path):
    4458    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4459    db = AgentDB(tmp_path / "state.db")
    4460    try:
    4461        job_id = db.create_job(
    4462            "Build a durable workflow and keep improving it",
    4463            title="research-balance-lessons",
    4464            kind="generic",
    4465            metadata={"experiment_ledger": [{"title": "Validation check", "status": "measured"}]},
    4466        )
    4467        run_id = db.start_run(job_id, model="test")
    4468        for index in range(6):
    4469            step_id = db.add_step(
    4470                job_id=job_id,
    4471                run_id=run_id,
    4472                kind="tool",
    4473                tool_name="shell_exec",
    4474                input_data={"arguments": {"command": f"python branch_{index}.py"}},
    4475            )
    4476            db.finish_step(step_id, status="completed", output_data={"success": True, "stdout": "ok"})
    4477        db.finish_run(run_id, "completed")
    4478
    4479        result = run_one_step(
    4480            job_id,
    4481            config=config,
    4482            db=db,
    4483            llm=ScriptedLLM([
    4484                LLMResponse(tool_calls=[
    4485                    ToolCall(
    4486                        name="record_lesson",
    4487                        arguments={
    4488                            "lesson": "The latest execution branch worked; continue similar attempts.",
    4489                            "category": "strategy",
    4490                        },
    4491                    )
    4492                ])
    4493            ]),
    4494        )
    4495
    4496        assert result.status == "blocked"
    4497        assert result.result["error"] == "research balance required"
    4498        assert result.result["blocked_tool"] == "record_lesson"
    4499        assert "raw lesson accumulation" in result.result["guidance"]
    4500    finally:
    4501        db.close()
    4502
    4503
    4504def test_run_one_step_blocks_durable_records_with_unsupported_concrete_claims(tmp_path):
    4505    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4506    db = AgentDB(tmp_path / "state.db")
    4507    try:
    4508        job_id = db.create_job("Optimize a measurable process on observed hardware", title="grounding", kind="generic")
    4509        run_id = db.start_run(job_id, model="test")
    4510        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4511        db.finish_step(
    4512            step_id,
    4513            status="completed",
    4514            output_data={
    4515                "success": True,
    4516                "stdout": "GPU: AMD Device 7590\nCPU: AMD Ryzen 9 7900X\nMemory: 93Gi\n",
    4517                "stderr": "",
    4518            },
    4519        )
    4520        db.finish_run(run_id, "completed")
    4521
    4522        result = run_one_step(
    4523            job_id,
    4524            config=config,
    4525            db=db,
    4526            llm=ScriptedLLM([
    4527                LLMResponse(tool_calls=[
    4528                    ToolCall(
    4529                        name="record_roadmap",
    4530                        arguments={
    4531                            "title": "Performance roadmap",
    4532                            "status": "active",
    4533                            "current_milestone": "Environment",
    4534                            "metadata": {
    4535                                "hardware": "NVIDIA GTX 970 with CUDA and i5-8400 CPU",
    4536                                "claim": "Use CUDA-first optimization.",
    4537                            },
    4538                            "milestones": [],
    4539                        },
    4540                    )
    4541                ])
    4542            ]),
    4543        )
    4544
    4545        assert result.status == "blocked"
    4546        assert result.result["error"] == "evidence grounding required"
    4547        assert result.result["blocked_tool"] == "record_roadmap"
    4548        assert "GTX" in result.result["evidence_grounding"]["unsupported_tokens"]
    4549        lessons = db.get_job(job_id)["metadata"]["lessons"]
    4550        assert any("GTX" in lesson["lesson"] and "stale" in lesson["lesson"] for lesson in lessons)
    4551    finally:
    4552        db.close()
    4553
    4554
    4555def test_record_experiment_blocks_unsupported_proper_noun_hardware_claims(tmp_path):
    4556    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4557    db = AgentDB(tmp_path / "state.db")
    4558    try:
    4559        job_id = db.create_job("Record exact observed environment", title="grounding", kind="generic")
    4560        run_id = db.start_run(job_id, model="test")
    4561        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4562        db.finish_step(
    4563            step_id,
    4564            status="completed",
    4565            output_data={
    4566                "success": True,
    4567                "stdout": (
    4568                    "NO_NVIDIA_GPU\n"
    4569                    "GPU: Advanced Micro Devices Device 7590\n"
    4570                    "Threads: 24\n"
    4571                    "CPU: AMD Ryzen 9 7900X 12-Core Processor\n"
    4572                ),
    4573            },
    4574        )
    4575        db.finish_run(run_id, "completed")
    4576
    4577        blocked = run_one_step(
    4578            job_id,
    4579            config=config,
    4580            db=db,
    4581            llm=ScriptedLLM([
    4582                LLMResponse(tool_calls=[
    4583                    ToolCall(
    4584                        name="record_experiment",
    4585                        arguments={
    4586                            "title": "Environment Baseline - Hardware Runtime Facts",
    4587                            "status": "measured",
    4588                            "metric_name": "cpu_threads",
    4589                            "metric_value": 16,
    4590                            "metric_unit": "threads",
    4591                            "result": "Environment baseline captured. Hardware: Dual Intel Xeon CPUs, 16 threads total.",
    4592                            "next_action": "Continue from exact observed hardware facts.",
    4593                        },
    4594                    )
    4595                ])
    4596            ]),
    4597        )
    4598
    4599        assert blocked.status == "blocked"
    4600        assert blocked.result["error"] == "evidence grounding required"
    4601        assert {"Dual", "Intel", "Xeon"} <= set(blocked.result["evidence_grounding"]["unsupported_tokens"])
    4602    finally:
    4603        db.close()
    4604
    4605
    4606def test_record_experiment_allows_supported_proper_noun_hardware_claims(tmp_path):
    4607    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4608    db = AgentDB(tmp_path / "state.db")
    4609    try:
    4610        job_id = db.create_job("Record exact observed environment", title="grounding", kind="generic")
    4611        run_id = db.start_run(job_id, model="test")
    4612        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4613        db.finish_step(
    4614            step_id,
    4615            status="completed",
    4616            output_data={
    4617                "success": True,
    4618                "stdout": (
    4619                    "NO_NVIDIA_GPU\n"
    4620                    "GPU: Advanced Micro Devices Device 7590\n"
    4621                    "Threads: 24\n"
    4622                    "CPU: AMD Ryzen 9 7900X 12-Core Processor\n"
    4623                ),
    4624            },
    4625        )
    4626        db.finish_run(run_id, "completed")
    4627
    4628        result = run_one_step(
    4629            job_id,
    4630            config=config,
    4631            db=db,
    4632            llm=ScriptedLLM([
    4633                LLMResponse(tool_calls=[
    4634                    ToolCall(
    4635                        name="record_experiment",
    4636                        arguments={
    4637                            "title": "Environment Baseline - Hardware Runtime Facts",
    4638                            "status": "measured",
    4639                            "metric_name": "cpu_threads",
    4640                            "metric_value": 24,
    4641                            "metric_unit": "threads",
    4642                            "result": "Environment baseline captured. Hardware: AMD Ryzen 9 7900X, 24 threads total.",
    4643                            "next_action": "Continue from exact observed hardware facts.",
    4644                        },
    4645                    )
    4646                ])
    4647            ]),
    4648        )
    4649
    4650        assert result.status == "completed"
    4651        experiment = db.get_job(job_id)["metadata"]["experiment_ledger"][0]
    4652        assert "AMD Ryzen 9 7900X" in experiment["result"]
    4653    finally:
    4654        db.close()
    4655
    4656
    4657def test_record_lesson_blocks_negative_claim_that_conflicts_with_positive_evidence(tmp_path):
    4658    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4659    db = AgentDB(tmp_path / "state.db")
    4660    try:
    4661        job_id = db.create_job("Keep exact observed facts durable", title="lesson-grounding", kind="generic")
    4662        run_id = db.start_run(job_id, model="test")
    4663        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4664        db.finish_step(
    4665            step_id,
    4666            status="completed",
    4667            output_data={
    4668                "success": True,
    4669                "stdout": (
    4670                    "NAME ID SIZE MODIFIED\n"
    4671                    "ModelX-99 a50eda8ed977 17 GB 2 weeks ago\n"
    4672                    "OtherModel 69492d6584c5 14 GB 2 months ago\n"
    4673                ),
    4674            },
    4675        )
    4676        db.finish_run(run_id, "completed")
    4677
    4678        blocked = run_one_step(
    4679            job_id,
    4680            config=config,
    4681            db=db,
    4682            llm=ScriptedLLM([
    4683                LLMResponse(tool_calls=[
    4684                    ToolCall(
    4685                        name="record_lesson",
    4686                        arguments={
    4687                            "category": "strategy",
    4688                            "lesson": (
    4689                                "No ModelX-99 model has been successfully downloaded, so keep the download branch "
    4690                                "as the primary blocker before any benchmark work."
    4691                            ),
    4692                        },
    4693                    )
    4694                ])
    4695            ]),
    4696        )
    4697
    4698        assert blocked.status == "blocked"
    4699        assert blocked.result["error"] == "evidence grounding required"
    4700        grounding = blocked.result["evidence_grounding"]
    4701        assert "ModelX-99" in grounding["unsupported_tokens"]
    4702        assert grounding["negative_claim_conflicts"][0]["token"] == "ModelX-99"
    4703    finally:
    4704        db.close()
    4705
    4706
    4707def test_record_lesson_ignores_plain_titlecase_negative_conflict_tokens(tmp_path):
    4708    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4709    db = AgentDB(tmp_path / "state.db")
    4710    try:
    4711        job_id = db.create_job("Keep exact observed facts durable", title="lesson-grounding", kind="generic")
    4712        run_id = db.start_run(job_id, model="test")
    4713        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4714        db.finish_step(
    4715            step_id,
    4716            status="completed",
    4717            output_data={
    4718                "success": True,
    4719                "stdout": "/srv/vendor/lmstudio-community/Model.foo\n",
    4720            },
    4721        )
    4722        db.finish_run(run_id, "completed")
    4723
    4724        result = run_one_step(
    4725            job_id,
    4726            config=config,
    4727            db=db,
    4728            llm=ScriptedLLM([
    4729                LLMResponse(tool_calls=[
    4730                    ToolCall(
    4731                        name="record_lesson",
    4732                        arguments={
    4733                            "category": "strategy",
    4734                            "lesson": "No Studio-specific conclusion should be drawn from this branch yet.",
    4735                        },
    4736                    )
    4737                ])
    4738            ]),
    4739        )
    4740
    4741        assert result.status == "completed"
    4742    finally:
    4743        db.close()
    4744
    4745
    4746def test_record_lesson_allows_negative_claim_when_evidence_is_also_negative(tmp_path):
    4747    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4748    db = AgentDB(tmp_path / "state.db")
    4749    try:
    4750        job_id = db.create_job("Keep exact observed facts durable", title="lesson-grounding", kind="generic")
    4751        run_id = db.start_run(job_id, model="test")
    4752        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4753        db.finish_step(
    4754            step_id,
    4755            status="completed",
    4756            output_data={
    4757                "success": True,
    4758                "stdout": "ls: cannot access '/tmp/ModelX-99.gguf': No such file or directory\n",
    4759            },
    4760        )
    4761        db.finish_run(run_id, "completed")
    4762
    4763        result = run_one_step(
    4764            job_id,
    4765            config=config,
    4766            db=db,
    4767            llm=ScriptedLLM([
    4768                LLMResponse(tool_calls=[
    4769                    ToolCall(
    4770                        name="record_lesson",
    4771                        arguments={
    4772                            "category": "strategy",
    4773                            "lesson": (
    4774                                "No ModelX-99 file exists in the checked path, so the next branch must use a "
    4775                                "different observed source or record the missing file as blocked."
    4776                            ),
    4777                        },
    4778                    )
    4779                ])
    4780            ]),
    4781        )
    4782
    4783        assert result.status == "completed"
    4784        lesson = db.get_job(job_id)["metadata"]["lessons"][0]
    4785        assert "ModelX-99" in lesson["lesson"]
    4786    finally:
    4787        db.close()
    4788
    4789
    4790def test_shell_path_recovery_prompt_shows_missing_executable(tmp_path):
    4791    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4792    db = AgentDB(tmp_path / "state.db")
    4793    try:
    4794        job_id = db.create_job("Run a measured tool after validating paths", title="missing-executable", kind="generic")
    4795        run_id = db.start_run(job_id, model="test")
    4796        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4797        db.finish_step(
    4798            step_id,
    4799            status="completed",
    4800            output_data={
    4801                "success": True,
    4802                "command": "/opt/tools/runner --measure",
    4803                "stdout": "/bin/sh: 1: /opt/tools/runner: not found\n",
    4804                "stderr": "",
    4805            },
    4806        )
    4807        db.finish_run(run_id, "completed")
    4808        llm = CapturingLLM(LLMResponse(tool_calls=[
    4809            ToolCall(
    4810                name="record_lesson",
    4811                arguments={
    4812                    "category": "strategy",
    4813                    "lesson": "The /opt/tools/runner executable was missing, so validate a real executable path before measuring.",
    4814                },
    4815            )
    4816        ]))
    4817
    4818        result = run_one_step(job_id, config=config, db=db, llm=llm)
    4819
    4820        assert result.status == "completed"
    4821        prompt = llm.messages[-1]["content"]
    4822        assert "Shell path recovery" in prompt
    4823        assert "/opt/tools/runner" in prompt
    4824        assert "Do not treat this output as a successful measurement" in prompt
    4825        assert "locate or verify the real executable/file path" in prompt
    4826    finally:
    4827        db.close()
    4828
    4829
    4830def test_shell_path_recovery_prompt_prefers_observed_candidate_executable(tmp_path):
    4831    db = AgentDB(tmp_path / "state.db")
    4832    try:
    4833        job_id = db.create_job("Build and benchmark a generic project", title="candidate-executable", kind="generic")
    4834        run_id = db.start_run(job_id, model="test")
    4835        observed_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4836        db.finish_step(
    4837            observed_step,
    4838            status="completed",
    4839            output_data={
    4840                "success": True,
    4841                "command": "ls /tmp/tools/build-tool",
    4842                "stdout": "/tmp/tools/build-tool\n---\nbuild-tool\nhelper\n",
    4843                "stderr": "",
    4844            },
    4845        )
    4846        failed_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4847        db.finish_step(
    4848            failed_step,
    4849            status="failed",
    4850            output_data={
    4851                "success": False,
    4852                "command": "cd /tmp/project && build-tool ..",
    4853                "stdout": "/bin/sh: 1: build-tool: not found\n",
    4854                "stderr": "",
    4855                "error": "command output indicates missing command despite exit status 0",
    4856            },
    4857        )
    4858        db.finish_run(run_id, "completed")
    4859
    4860        messages = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))
    4861        prompt = messages[-1]["content"]
    4862
    4863        assert "Shell path recovery" in prompt
    4864        assert "Missing commands: build-tool" in prompt
    4865        assert "Observed candidate executable for build-tool: /tmp/tools/build-tool" in prompt
    4866        assert "try the exact candidate path or add its directory to PATH" in prompt
    4867    finally:
    4868        db.close()
    4869
    4870
    4871def test_shell_path_recovery_prompt_preserves_partial_success_paths(tmp_path):
    4872    db = AgentDB(tmp_path / "state.db")
    4873    try:
    4874        job_id = db.create_job("Recover from mixed shell output", title="partial-shell-paths", kind="generic")
    4875        run_id = db.start_run(job_id, model="test")
    4876        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4877        db.finish_step(
    4878            step_id,
    4879            status="failed",
    4880            output_data={
    4881                "success": False,
    4882                "command": "ls /tmp/bin/build-tool /tmp/bin/compiler; which compiler",
    4883                "returncode": 1,
    4884                "stdout": (
    4885                    "ls: cannot access '/tmp/bin/compiler': No such file or directory\n"
    4886                    "lrwxrwxrwx 1 user user 30 Jan 1 00:00 /tmp/bin/build-tool -> /tmp/runtime/build-tool\n"
    4887                    "/usr/bin/compiler\n"
    4888                ),
    4889                "stderr": "",
    4890                "error": "command exited with status 1",
    4891            },
    4892        )
    4893        db.finish_run(run_id, "completed")
    4894
    4895        messages = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))
    4896        prompt = messages[-1]["content"]
    4897
    4898        assert "Shell path recovery" in prompt
    4899        assert "Missing paths: /tmp/bin/compiler" in prompt
    4900        assert "Observed executable paths in partial shell output" in prompt
    4901        assert "/tmp/bin/build-tool" in prompt
    4902        assert "/tmp/runtime/build-tool" in prompt
    4903        assert "/usr/bin/compiler" in prompt
    4904        assert "Observed executable paths in partial shell output: /tmp/bin/compiler" not in prompt
    4905    finally:
    4906        db.close()
    4907
    4908
    4909def test_shell_exec_blocks_bare_retry_when_candidate_executable_observed(tmp_path):
    4910    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4911    db = AgentDB(tmp_path / "state.db")
    4912    try:
    4913        job_id = db.create_job("Recover with observed executable", title="candidate-retry", kind="generic")
    4914        run_id = db.start_run(job_id, model="test")
    4915        observed_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4916        db.finish_step(
    4917            observed_step,
    4918            status="completed",
    4919            output_data={
    4920                "success": True,
    4921                "command": "ls /tmp/tools/build-tool",
    4922                "stdout": "/tmp/tools/build-tool\n",
    4923                "stderr": "",
    4924            },
    4925        )
    4926        failed_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    4927        db.finish_step(
    4928            failed_step,
    4929            status="failed",
    4930            output_data={
    4931                "success": False,
    4932                "command": "build-tool --version",
    4933                "stdout": "/bin/sh: 1: build-tool: not found\n",
    4934                "stderr": "",
    4935                "error": "command output indicates missing command despite exit status 0",
    4936            },
    4937        )
    4938        db.finish_run(run_id, "completed")
    4939
    4940        result = run_one_step(
    4941            job_id,
    4942            config=config,
    4943            db=db,
    4944            llm=ScriptedLLM([
    4945                LLMResponse(tool_calls=[
    4946                    ToolCall(name="shell_exec", arguments={"command": "build-tool --version"})
    4947                ])
    4948            ]),
    4949            registry=SuccessRegistry(),
    4950        )
    4951
    4952        assert result.status == "blocked"
    4953        assert result.result["error"] == "observed executable recovery required"
    4954        assert result.result["candidate_recovery"]["missing_command"] == "build-tool"
    4955        assert result.result["candidate_recovery"]["candidate_executables"] == ["/tmp/tools/build-tool"]
    4956    finally:
    4957        db.close()
    4958
    4959
    4960def test_permission_failure_prompt_blocks_package_manager_retry(tmp_path):
    4961    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    4962    db = AgentDB(tmp_path / "state.db")
    4963    try:
    4964        job_id = db.create_job("Recover from generic build prerequisites", title="permission-recovery", kind="generic")
    4965        run_id = db.start_run(job_id, model="test")
    4966        step_id = db.add_step(
    4967            job_id=job_id,
    4968            run_id=run_id,
    4969            kind="tool",
    4970            tool_name="shell_exec",
    4971            input_data={"arguments": {"command": "apt-get install -y build-tool"}},
    4972        )
    4973        db.finish_step(
    4974            step_id,
    4975            status="failed",
    4976            output_data={
    4977                "success": False,
    4978                "command": "apt-get install -y build-tool",
    4979                "returncode": 0,
    4980                "stdout": (
    4981                    "E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)\n"
    4982                    "E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?\n"
    4983                ),
    4984                "stderr": "",
    4985                "error": "command output indicates authentication or authorization failure despite exit status 0",
    4986            },
    4987        )
    4988        db.finish_run(run_id, "completed")
    4989        llm = ScriptedLLM([LLMResponse(tool_calls=[
    4990            ToolCall(name="shell_exec", arguments={"command": "apt-get install -y another-tool"})
    4991        ])])
    4992
    4993        result = run_one_step(job_id, config=config, db=db, llm=llm, registry=SuccessRegistry())
    4994
    4995        assert result.status == "blocked"
    4996        assert result.result["error"] == "privileged command recovery required"
    4997        assert result.result["privileged_failure"]["step_no"] == 1
    4998        assert "non-privileged recovery" in result.result["guidance"]
    4999    finally:
    5000        db.close()
    5001
    5002
    5003def test_permission_failure_prompt_mentions_non_privileged_recovery(tmp_path):
    5004    db = AgentDB(tmp_path / "state.db")
    5005    try:
    5006        job_id = db.create_job("Recover from generic build prerequisites", title="permission-prompt", kind="generic")
    5007        run_id = db.start_run(job_id, model="test")
    5008        step_id = db.add_step(
    5009            job_id=job_id,
    5010            run_id=run_id,
    5011            kind="tool",
    5012            tool_name="shell_exec",
    5013            input_data={"arguments": {"command": "sudo package-manager install build-tool"}},
    5014        )
    5015        db.finish_step(
    5016            step_id,
    5017            status="failed",
    5018            output_data={
    5019                "success": False,
    5020                "command": "sudo package-manager install build-tool",
    5021                "stdout": "sudo: a password is required\n",
    5022                "stderr": "",
    5023                "error": "authentication or authorization failure",
    5024            },
    5025        )
    5026        db.finish_run(run_id, "completed")
    5027
    5028        messages = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))
    5029        prompt = messages[-1]["content"]
    5030
    5031        assert "Shell permission recovery" in prompt
    5032        assert "failed because a privileged/package-manager command lacked permission" in prompt
    5033        assert "non-privileged alternatives" in prompt
    5034        assert "operator credentials" in prompt
    5035    finally:
    5036        db.close()
    5037
    5038
    5039def test_record_findings_blocks_negative_file_pattern_that_conflicts_with_positive_evidence(tmp_path):
    5040    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5041    db = AgentDB(tmp_path / "state.db")
    5042    try:
    5043        job_id = db.create_job("Keep file discovery evidence exact", title="file-grounding", kind="generic")
    5044        run_id = db.start_run(job_id, model="test")
    5045        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5046        db.finish_step(
    5047            step_id,
    5048            status="completed",
    5049            output_data={
    5050                "success": True,
    5051                "stdout": (
    5052                    "/srv/data/WidgetModel-99-Q4.foo\n"
    5053                    "/tmp/results/report.alpha\n"
    5054                    "/var/cache/other-file.foo\n"
    5055                ),
    5056            },
    5057        )
    5058        db.finish_run(run_id, "completed")
    5059
    5060        blocked = run_one_step(
    5061            job_id,
    5062            config=config,
    5063            db=db,
    5064            llm=ScriptedLLM([
    5065                LLMResponse(tool_calls=[
    5066                    ToolCall(
    5067                        name="record_findings",
    5068                        arguments={
    5069                            "findings": [
    5070                                {
    5071                                    "name": "No .foo files found on filesystem",
    5072                                    "category": "environment_baseline",
    5073                                    "status": "confirmed",
    5074                                    "reason": "Shell search found zero .foo files larger than 1MB anywhere on the system.",
    5075                                }
    5076                            ]
    5077                        },
    5078                    )
    5079                ])
    5080            ]),
    5081        )
    5082
    5083        assert blocked.status == "blocked"
    5084        assert blocked.result["error"] == "evidence grounding required"
    5085        grounding = blocked.result["evidence_grounding"]
    5086        assert ".foo" in grounding["unsupported_tokens"]
    5087        assert grounding["negative_claim_conflicts"][0]["token"] == ".foo"
    5088    finally:
    5089        db.close()
    5090
    5091
    5092def test_file_pattern_grounding_ignores_hidden_path_components():
    5093    text = (
    5094        "No compiled binary exists yet. Valid data is at "
    5095        "/srv/cache/.lmstudio/models/ModelX.gguf and /tmp/.cache/item.bin. "
    5096        "No *.foo files were found."
    5097    )
    5098
    5099    tokens = _file_pattern_tokens_for_grounding(text)
    5100
    5101    assert ".lmstudio" not in tokens
    5102    assert ".cache" not in tokens
    5103    assert ".foo" in tokens
    5104
    5105
    5106def test_record_experiment_allows_classifying_observed_files_as_non_primary(tmp_path):
    5107    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5108    db = AgentDB(tmp_path / "state.db")
    5109    try:
    5110        job_id = db.create_job("Validate observed files before primary artifact work", title="file-classification", kind="generic")
    5111        run_id = db.start_run(job_id, model="test")
    5112        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5113        db.finish_step(
    5114            step_id,
    5115            status="completed",
    5116            output_data={
    5117                "success": True,
    5118                "stdout": (
    5119                    "/srv/data/support-alpha-v2.foo\n"
    5120                    "/srv/data/support-beta-v2.foo\n"
    5121                ),
    5122            },
    5123        )
    5124        db.finish_run(run_id, "completed")
    5125
    5126        result = run_one_step(
    5127            job_id,
    5128            config=config,
    5129            db=db,
    5130            llm=ScriptedLLM([
    5131                LLMResponse(tool_calls=[
    5132                    ToolCall(
    5133                        name="record_experiment",
    5134                        arguments={
    5135                            "title": "primary artifact scan",
    5136                            "status": "measured",
    5137                            "metric_name": "primary_artifacts_found",
    5138                            "metric_value": 0,
    5139                            "metric_unit": "files",
    5140                            "config": {
    5141                                "files_found": [
    5142                                    "/srv/data/support-alpha-v2.foo",
    5143                                    "/srv/data/support-beta-v2.foo",
    5144                                ],
    5145                            },
    5146                            "result": (
    5147                                "scan found only support files: /srv/data/support-alpha-v2.foo and "
    5148                                "/srv/data/support-beta-v2.foo. observed files are not the required "
    5149                                "primary artifact, so the primary artifact remains missing."
    5150                            ),
    5151                            "next_action": "select a different observed source for the primary artifact.",
    5152                        },
    5153                    )
    5154                ])
    5155            ]),
    5156        )
    5157
    5158        assert result.status == "completed"
    5159        assert result.tool_name == "record_experiment"
    5160    finally:
    5161        db.close()
    5162
    5163
    5164def test_record_findings_requires_exact_paths_when_file_candidates_exist(tmp_path):
    5165    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5166    db = AgentDB(tmp_path / "state.db")
    5167    try:
    5168        job_id = db.create_job("Keep file candidate evidence exact", title="path-grounding", kind="generic")
    5169        run_id = db.start_run(job_id, model="test")
    5170        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5171        db.finish_step(
    5172            step_id,
    5173            status="completed",
    5174            output_data={
    5175                "success": True,
    5176                "stdout": (
    5177                    "/srv/models/AlphaModel-Q4.foo\n"
    5178                    "/srv/models/BetaModel-Q8.foo\n"
    5179                    "/tmp/results/summary.json\n"
    5180                ),
    5181            },
    5182        )
    5183        db.finish_run(run_id, "completed")
    5184
    5185        blocked = run_one_step(
    5186            job_id,
    5187            config=config,
    5188            db=db,
    5189            llm=ScriptedLLM([
    5190                LLMResponse(tool_calls=[
    5191                    ToolCall(
    5192                        name="record_findings",
    5193                        arguments={
    5194                            "findings": [
    5195                                {
    5196                                    "name": "Model files found on disk",
    5197                                    "category": "environment",
    5198                                    "status": "new",
    5199                                    "reason": "Shell search found candidate files, so the next branch can validate them.",
    5200                                }
    5201                            ]
    5202                        },
    5203                    )
    5204                ])
    5205            ]),
    5206        )
    5207
    5208        assert blocked.status == "blocked"
    5209        assert blocked.result["error"] == "evidence grounding required"
    5210        grounding = blocked.result["evidence_grounding"]
    5211        assert "/srv/models/AlphaModel-Q4.foo" in grounding["missing_candidate_paths"]
    5212        assert "exact observed candidate paths" in grounding["guidance"]
    5213    finally:
    5214        db.close()
    5215
    5216
    5217def test_missing_candidate_paths_are_ranked_before_grounding_guidance(tmp_path):
    5218    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5219    db = AgentDB(tmp_path / "state.db")
    5220    try:
    5221        job_id = db.create_job("Validate OmegaModel file before benchmarking", title="omega benchmark", kind="generic")
    5222        run_id = db.start_run(job_id, model="test")
    5223        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5224        db.finish_step(
    5225            step_id,
    5226            status="completed",
    5227            output_data={
    5228                "success": True,
    5229                "stdout": "\n".join(
    5230                    [f"/srv/models/ggml-vocab-{index}.foo" for index in range(20)]
    5231                    + ["/srv/models/OmegaModel-primary.foo"]
    5232                ),
    5233            },
    5234        )
    5235        db.finish_run(run_id, "completed")
    5236
    5237        blocked = run_one_step(
    5238            job_id,
    5239            config=config,
    5240            db=db,
    5241            llm=ScriptedLLM([
    5242                LLMResponse(tool_calls=[
    5243                    ToolCall(
    5244                        name="record_findings",
    5245                        arguments={
    5246                            "findings": [
    5247                                {
    5248                                    "name": "Candidate files found",
    5249                                    "category": "environment",
    5250                                    "status": "new",
    5251                                    "reason": "A file search found candidate files to validate.",
    5252                                }
    5253                            ]
    5254                        },
    5255                    )
    5256                ])
    5257            ]),
    5258        )
    5259
    5260        assert blocked.status == "blocked"
    5261        grounding = blocked.result["evidence_grounding"]
    5262        assert grounding["missing_candidate_paths"][0] == "/srv/models/OmegaModel-primary.foo"
    5263    finally:
    5264        db.close()
    5265
    5266
    5267def test_record_findings_allows_exact_candidate_path_summary(tmp_path):
    5268    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5269    db = AgentDB(tmp_path / "state.db")
    5270    try:
    5271        job_id = db.create_job("Keep file candidate evidence exact", title="path-grounding", kind="generic")
    5272        run_id = db.start_run(job_id, model="test")
    5273        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5274        db.finish_step(
    5275            step_id,
    5276            status="completed",
    5277            output_data={
    5278                "success": True,
    5279                "stdout": "/srv/models/AlphaModel-Q4.foo\n/tmp/results/summary.json\n",
    5280            },
    5281        )
    5282        db.finish_run(run_id, "completed")
    5283
    5284        result = run_one_step(
    5285            job_id,
    5286            config=config,
    5287            db=db,
    5288            llm=ScriptedLLM([
    5289                LLMResponse(tool_calls=[
    5290                    ToolCall(
    5291                        name="record_findings",
    5292                        arguments={
    5293                            "findings": [
    5294                                {
    5295                                    "name": "Model file candidate",
    5296                                    "category": "environment",
    5297                                    "status": "new",
    5298                                    "reason": "Candidate path /srv/models/AlphaModel-Q4.foo should be validated next.",
    5299                                }
    5300                            ]
    5301                        },
    5302                    )
    5303                ])
    5304            ]),
    5305        )
    5306
    5307        assert result.status == "completed"
    5308    finally:
    5309        db.close()
    5310
    5311
    5312def test_evidence_grounding_blocks_positive_claim_for_missing_path(tmp_path):
    5313    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5314    db = AgentDB(tmp_path / "state.db")
    5315    try:
    5316        job_id = db.create_job("Verify a generic executable path", title="path polarity", kind="generic")
    5317        run_id = db.start_run(job_id, model="test")
    5318        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5319        db.finish_step(
    5320            step_id,
    5321            status="completed",
    5322            output_data={
    5323                "success": True,
    5324                "command": "ls /tmp/tools/build-tool /usr/bin/make",
    5325                "stdout": (
    5326                    "ls: cannot access '/tmp/tools/build-tool': No such file or directory\n"
    5327                    "/usr/bin/make\n"
    5328                    "This shell probe checked candidate executable paths before the build step.\n"
    5329                ),
    5330                "stderr": "",
    5331            },
    5332        )
    5333        db.finish_run(run_id, "completed")
    5334
    5335        blocked = run_one_step(
    5336            job_id,
    5337            config=config,
    5338            db=db,
    5339            llm=ScriptedLLM([
    5340                LLMResponse(tool_calls=[
    5341                    ToolCall(
    5342                        name="record_experiment",
    5343                        arguments={
    5344                            "title": "Build tool path verification",
    5345                            "status": "measured",
    5346                            "metric_name": "build_prerequisites",
    5347                            "metric_value": 2,
    5348                            "metric_unit": "items",
    5349                            "result": "Found build tool at /tmp/tools/build-tool and make at /usr/bin/make. Build prerequisites are verified.",
    5350                        },
    5351                    )
    5352                ])
    5353            ]),
    5354        )
    5355
    5356        assert blocked.status == "blocked"
    5357        assert blocked.result["error"] == "evidence grounding required"
    5358        grounding = blocked.result["evidence_grounding"]
    5359        assert "/tmp/tools/build-tool" in grounding["unsupported_tokens"]
    5360        assert grounding["negative_path_conflicts"][0]["path"] == "/tmp/tools/build-tool"
    5361        assert "claims a path or executable is present" in grounding["guidance"]
    5362    finally:
    5363        db.close()
    5364
    5365
    5366def test_evidence_grounding_checks_later_positive_path_mentions(tmp_path):
    5367    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5368    db = AgentDB(tmp_path / "state.db")
    5369    try:
    5370        job_id = db.create_job("Verify executable path polarity", title="path later mention", kind="generic")
    5371        run_id = db.start_run(job_id, model="test")
    5372        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5373        db.finish_step(
    5374            step_id,
    5375            status="completed",
    5376            output_data={
    5377                "success": True,
    5378                "stdout": (
    5379                    "ls: cannot access '/tmp/tools/build-tool': No such file or directory\n"
    5380                    "The probe also checked unrelated files and returned partial output for review.\n"
    5381                ),
    5382                "stderr": "",
    5383            },
    5384        )
    5385        db.finish_run(run_id, "completed")
    5386
    5387        blocked = run_one_step(
    5388            job_id,
    5389            config=config,
    5390            db=db,
    5391            llm=ScriptedLLM([
    5392                LLMResponse(tool_calls=[
    5393                    ToolCall(
    5394                        name="record_lesson",
    5395                        arguments={
    5396                            "category": "constraint",
    5397                            "lesson": (
    5398                                "candidate path /tmp/tools/build-tool was examined. "
    5399                                "The executable is at /tmp/tools/build-tool and should be used for the next build."
    5400                            ),
    5401                        },
    5402                    )
    5403                ])
    5404            ]),
    5405        )
    5406
    5407        assert blocked.status == "blocked"
    5408        assert blocked.result["error"] == "evidence grounding required"
    5409        grounding = blocked.result["evidence_grounding"]
    5410        assert grounding["negative_path_conflicts"][0]["path"] == "/tmp/tools/build-tool"
    5411    finally:
    5412        db.close()
    5413
    5414
    5415def test_record_findings_allows_negative_file_pattern_when_evidence_is_negative(tmp_path):
    5416    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5417    db = AgentDB(tmp_path / "state.db")
    5418    try:
    5419        job_id = db.create_job("Keep file discovery evidence exact", title="file-grounding", kind="generic")
    5420        run_id = db.start_run(job_id, model="test")
    5421        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5422        db.finish_step(
    5423            step_id,
    5424            status="completed",
    5425            output_data={
    5426                "success": True,
    5427                "stdout": "find: '/tmp/WidgetModel-99.foo': No such file or directory\n",
    5428            },
    5429        )
    5430        db.finish_run(run_id, "completed")
    5431
    5432        result = run_one_step(
    5433            job_id,
    5434            config=config,
    5435            db=db,
    5436            llm=ScriptedLLM([
    5437                LLMResponse(tool_calls=[
    5438                    ToolCall(
    5439                        name="record_findings",
    5440                        arguments={
    5441                            "findings": [
    5442                                {
    5443                                    "name": "No .foo file exists in the checked path",
    5444                                    "category": "environment_baseline",
    5445                                    "status": "confirmed",
    5446                                    "reason": "The shell output says the checked .foo path does not exist.",
    5447                                }
    5448                            ]
    5449                        },
    5450                    )
    5451                ])
    5452            ]),
    5453        )
    5454
    5455        assert result.status == "completed"
    5456        findings = db.get_job(job_id)["metadata"]["finding_ledger"]
    5457        assert findings[0]["name"] == "No .foo file exists in the checked path"
    5458    finally:
    5459        db.close()
    5460
    5461
    5462def test_run_one_step_marks_contradicted_negative_finding_stale(tmp_path):
    5463    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5464    db = AgentDB(tmp_path / "state.db")
    5465    try:
    5466        job_id = db.create_job("Keep durable findings aligned with fresh evidence", title="stale-finding", kind="generic")
    5467        db.append_finding_record(
    5468            job_id,
    5469            name="No .foo files found",
    5470            category="environment_baseline",
    5471            reason="Shell search found zero .foo files anywhere in the checked filesystem.",
    5472            status="confirmed",
    5473        )
    5474        run_id = db.start_run(job_id, model="test")
    5475        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5476        db.finish_step(
    5477            step_id,
    5478            status="completed",
    5479            output_data={
    5480                "success": True,
    5481                "stdout": (
    5482                    "discovery results from current filesystem scan:\n"
    5483                    "/srv/data/WidgetModel-99-Q4.foo\n"
    5484                    "/var/cache/other-file.foo\n"
    5485                ),
    5486            },
    5487        )
    5488        db.finish_run(run_id, "completed")
    5489
    5490        result = run_one_step(
    5491            job_id,
    5492            config=config,
    5493            db=db,
    5494            llm=ScriptedLLM([
    5495                LLMResponse(tool_calls=[
    5496                    ToolCall(
    5497                        name="record_lesson",
    5498                        arguments={
    5499                            "category": "strategy",
    5500                            "lesson": "Fresh file-discovery evidence should override older absence claims.",
    5501                        },
    5502                    )
    5503                ])
    5504            ]),
    5505        )
    5506
    5507        assert result.status == "completed"
    5508        job = db.get_job(job_id)
    5509        stale_records = job["metadata"].get("stale_negative_records")
    5510        assert isinstance(stale_records, list)
    5511        assert stale_records[0]["kind"] == "finding"
    5512        assert stale_records[0]["token"] == ".foo"
    5513
    5514        from nipux_cli.worker_prompt_context import _ledgers_for_prompt
    5515
    5516        ledgers = _ledgers_for_prompt(job)
    5517        assert "Contradicted negative findings suppressed" in ledgers
    5518        assert "Suppressed 1 stale finding" in ledgers
    5519    finally:
    5520        db.close()
    5521
    5522
    5523def test_run_one_step_marks_contradicted_negative_memory_node_stale(tmp_path):
    5524    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5525    db = AgentDB(tmp_path / "state.db")
    5526    try:
    5527        job_id = db.create_job("Keep memory aligned with fresh evidence", title="stale-memory", kind="generic")
    5528        db.append_memory_graph_records(
    5529            job_id,
    5530            nodes=[
    5531                {
    5532                    "key": "fact-no-local-foo",
    5533                    "title": "No local foo files",
    5534                    "kind": "fact",
    5535                    "status": "active",
    5536                    "summary": "Filesystem searches for *.foo files return 0 results.",
    5537                },
    5538                {
    5539                    "key": "current-branch",
    5540                    "title": "Current branch",
    5541                    "kind": "strategy",
    5542                    "status": "active",
    5543                    "summary": "Use fresh shell evidence before recording durable claims.",
    5544                },
    5545            ],
    5546        )
    5547        run_id = db.start_run(job_id, model="test")
    5548        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5549        db.finish_step(
    5550            step_id,
    5551            status="completed",
    5552            output_data={
    5553                "success": True,
    5554                "stdout": (
    5555                    "Fresh filesystem discovery found an exact candidate path with enough surrounding context "
    5556                    "to count as evidence: /srv/data/WidgetModel-99-Q4.foo\n"
    5557                ),
    5558            },
    5559        )
    5560        db.finish_run(run_id, "completed")
    5561
    5562        result = run_one_step(
    5563            job_id,
    5564            config=config,
    5565            db=db,
    5566            llm=ScriptedLLM([
    5567                LLMResponse(tool_calls=[
    5568                    ToolCall(
    5569                        name="record_lesson",
    5570                        arguments={
    5571                            "category": "strategy",
    5572                            "lesson": "Fresh file evidence overrides stale absence memory.",
    5573                        },
    5574                    )
    5575                ])
    5576            ]),
    5577        )
    5578
    5579        assert result.status == "completed"
    5580        job = db.get_job(job_id)
    5581        stale_records = job["metadata"].get("stale_negative_records")
    5582        assert any(record["kind"] == "memory_node" and record["record_id"] == "fact-no-local-foo" for record in stale_records)
    5583
    5584        content = build_messages(job, db.list_steps(job_id=job_id))[-1]["content"]
    5585        assert "Suppressed 1 stale memory node" in content
    5586        assert "No local foo files" not in content
    5587        assert "Current branch" in content
    5588    finally:
    5589        db.close()
    5590
    5591
    5592def test_record_lesson_allows_generic_strategy_without_concrete_facts(tmp_path):
    5593    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5594    db = AgentDB(tmp_path / "state.db")
    5595    try:
    5596        job_id = db.create_job("Improve a workflow", title="lesson-grounding", kind="generic")
    5597        run_id = db.start_run(job_id, model="test")
    5598        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5599        db.finish_step(step_id, status="completed", output_data={"success": True, "stdout": "branch stalled\n"})
    5600        db.finish_run(run_id, "completed")
    5601
    5602        result = run_one_step(
    5603            job_id,
    5604            config=config,
    5605            db=db,
    5606            llm=ScriptedLLM([
    5607                LLMResponse(tool_calls=[
    5608                    ToolCall(
    5609                        name="record_lesson",
    5610                        arguments={
    5611                            "category": "strategy",
    5612                            "lesson": "When a branch stalls, pivot to the next measurable action instead of adding more notes.",
    5613                        },
    5614                    )
    5615                ])
    5616            ]),
    5617        )
    5618
    5619        assert result.status == "completed"
    5620    finally:
    5621        db.close()
    5622
    5623
    5624def test_record_lesson_allows_positive_checkpoint_summary_with_new_concrete_terms(tmp_path):
    5625    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5626    db = AgentDB(tmp_path / "state.db")
    5627    try:
    5628        job_id = db.create_job("Summarize a broad checkpoint", title="lesson-grounding", kind="generic")
    5629        run_id = db.start_run(job_id, model="test")
    5630        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5631        db.finish_step(
    5632            step_id,
    5633            status="completed",
    5634            output_data={"success": True, "stdout": "checkpoint read and accounting required\n"},
    5635        )
    5636        db.finish_run(run_id, "completed")
    5637
    5638        result = run_one_step(
    5639            job_id,
    5640            config=config,
    5641            db=db,
    5642            llm=ScriptedLLM([
    5643                LLMResponse(tool_calls=[
    5644                    ToolCall(
    5645                        name="record_lesson",
    5646                        arguments={
    5647                            "category": "memory",
    5648                            "lesson": (
    5649                                "Recording checkpoint context says PackageManager-42 and RuntimeProbe-7 should stay "
    5650                                "available for the next branch, but no final benchmark decision has been made."
    5651                            ),
    5652                        },
    5653                    )
    5654                ])
    5655            ]),
    5656        )
    5657
    5658        assert result.status == "completed"
    5659    finally:
    5660        db.close()
    5661
    5662
    5663def test_record_findings_blocks_single_unsupported_identifier(tmp_path):
    5664    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5665    db = AgentDB(tmp_path / "state.db")
    5666    try:
    5667        job_id = db.create_job("Record only observed identifiers", title="grounding", kind="generic")
    5668        run_id = db.start_run(job_id, model="test")
    5669        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5670        db.finish_step(
    5671            step_id,
    5672            status="completed",
    5673            output_data={
    5674                "success": True,
    5675                "stdout": (
    5676                    "Observed candidate list from tool output. "
    5677                    "The source contains AlphaCandidate and BetaCandidate with ordinary text evidence. "
    5678                    "No generated opaque identifiers are present in this evidence."
    5679                )
    5680                * 12,
    5681            },
    5682        )
    5683        db.finish_run(run_id, "completed")
    5684
    5685        result = run_one_step(
    5686            job_id,
    5687            config=config,
    5688            db=db,
    5689            llm=ScriptedLLM([
    5690                LLMResponse(tool_calls=[ToolCall(name="record_findings", arguments={
    5691                    "findings": [{
    5692                        "name": "WWHHH5 generated candidate",
    5693                        "category": "test",
    5694                        "reason": "Observed candidate list needs follow-up, but this identifier was not in evidence.",
    5695                        "status": "new",
    5696                    }]
    5697                })])
    5698            ]),
    5699        )
    5700
    5701        assert result.status == "blocked"
    5702        assert result.result["error"] == "evidence grounding required"
    5703        assert result.result["evidence_grounding"]["unsupported_tokens"] == ["WWHHH5"]
    5704    finally:
    5705        db.close()
    5706
    5707
    5708def test_evidence_grounding_ignores_job_context_labels(tmp_path):
    5709    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5710    db = AgentDB(tmp_path / "state.db")
    5711    try:
    5712        job_id = db.create_job(
    5713            "Benchmark AlphaModel throughput",
    5714            title="alphamodel throughput fixed",
    5715            kind="generic",
    5716        )
    5717        run_id = db.start_run(job_id, model="test")
    5718        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5719        db.finish_step(
    5720            step_id,
    5721            status="completed",
    5722            output_data={
    5723                "success": True,
    5724                "stdout": (
    5725                    "Observed benchmark setup is ready. Runtime exists, candidate file exists, "
    5726                    "and the next action is a planned baseline measurement. "
    5727                )
    5728                * 6,
    5729            },
    5730        )
    5731        db.finish_run(run_id, "completed")
    5732
    5733        result = run_one_step(
    5734            job_id,
    5735            config=config,
    5736            db=db,
    5737            llm=ScriptedLLM([
    5738                LLMResponse(tool_calls=[ToolCall(name="record_experiment", arguments={
    5739                    "title": "Baseline Throughput - AlphaModel",
    5740                    "status": "planned",
    5741                    "higher_is_better": True,
    5742                    "metadata": {"project": "alphamodel-throughput"},
    5743                    "next_action": "Run the baseline measurement and record the observed metric.",
    5744                })])
    5745            ]),
    5746        )
    5747
    5748        assert result.status == "completed"
    5749    finally:
    5750        db.close()
    5751
    5752
    5753def test_evidence_grounding_blocks_unsupported_numeric_measurements(tmp_path):
    5754    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5755    db = AgentDB(tmp_path / "state.db")
    5756    try:
    5757        job_id = db.create_job("Validate candidate file size", title="size-grounding", kind="generic")
    5758        run_id = db.start_run(job_id, model="test")
    5759        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5760        db.finish_step(
    5761            step_id,
    5762            status="completed",
    5763            output_data={
    5764                "success": True,
    5765                "stdout": "-rw-r--r-- 1 user user 12G May 14 /srv/models/AlphaModel-Q4.foo\n",
    5766            },
    5767        )
    5768        db.finish_run(run_id, "completed")
    5769
    5770        result = run_one_step(
    5771            job_id,
    5772            config=config,
    5773            db=db,
    5774            llm=ScriptedLLM([
    5775                LLMResponse(tool_calls=[ToolCall(name="record_findings", arguments={
    5776                    "findings": [{
    5777                        "name": "Candidate file",
    5778                        "category": "environment",
    5779                        "location": "/srv/models/AlphaModel-Q4.foo",
    5780                        "metadata": {"file_size": "16G"},
    5781                        "status": "verified",
    5782                    }]
    5783                })])
    5784            ]),
    5785        )
    5786
    5787        assert result.status == "blocked"
    5788        assert result.result["error"] == "evidence grounding required"
    5789        assert "16G" in result.result["evidence_grounding"]["unsupported_tokens"]
    5790    finally:
    5791        db.close()
    5792
    5793
    5794def test_evidence_grounding_ignores_record_schema_keys(tmp_path):
    5795    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5796    db = AgentDB(tmp_path / "state.db")
    5797    try:
    5798        job_id = db.create_job("Record observed setup status", title="grounding", kind="generic")
    5799        run_id = db.start_run(job_id, model="test")
    5800        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5801        db.finish_step(
    5802            step_id,
    5803            status="completed",
    5804            output_data={"success": True, "stdout": "Python 3 is installed. curl is available. No token file was found."},
    5805        )
    5806        db.finish_run(run_id, "completed")
    5807
    5808        result = run_one_step(
    5809            job_id,
    5810            config=config,
    5811            db=db,
    5812            llm=ScriptedLLM([
    5813                LLMResponse(tool_calls=[
    5814                    ToolCall(
    5815                        name="record_experiment",
    5816                        arguments={
    5817                            "title": "Setup status",
    5818                            "status": "measured",
    5819                            "metric_name": "ready_components",
    5820                                "metric_value": 1,
    5821                                "config": {"python_3_installed": True, "curl_available": True},
    5822                                "result": "Python 3 is installed and curl is available.",
    5823                                "next_action": "record remaining setup gaps or proceed to the next validation",
    5824                            },
    5825                        )
    5826                    ])
    5827                ]),
    5828        )
    5829
    5830        assert result.status == "completed"
    5831        experiment = db.get_job(job_id)["metadata"]["experiment_ledger"][0]
    5832        assert experiment["config"]["python_3_installed"] is True
    5833    finally:
    5834        db.close()
    5835
    5836
    5837def test_evidence_grounding_uses_durable_finding_location_and_metadata(tmp_path):
    5838    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5839    db = AgentDB(tmp_path / "state.db")
    5840    try:
    5841        job_id = db.create_job("Record known candidate from durable state", title="durable-grounding", kind="generic")
    5842        db.append_finding_record(
    5843            job_id,
    5844            name="Candidate runtime model",
    5845            category="environment",
    5846            location="/srv/models/AlphaModel-99-Q4.gguf",
    5847            reason="Observed candidate model path is ready for later measurement.",
    5848            status="available",
    5849            metadata={"quantization": "Q4"},
    5850        )
    5851
    5852        result = run_one_step(
    5853            job_id,
    5854            config=config,
    5855            db=db,
    5856            llm=ScriptedLLM([
    5857                LLMResponse(tool_calls=[
    5858                    ToolCall(
    5859                        name="record_experiment",
    5860                        arguments={
    5861                            "title": "Candidate runtime model readiness",
    5862                            "status": "measured",
    5863                            "metric_name": "candidate_files",
    5864                            "metric_value": 1,
    5865                            "config": {"model": "/srv/models/AlphaModel-99-Q4.gguf"},
    5866                            "result": "Durable finding shows /srv/models/AlphaModel-99-Q4.gguf is available.",
    5867                            "next_action": "measure throughput with the durable candidate model",
    5868                        },
    5869                    )
    5870                ])
    5871            ]),
    5872        )
    5873
    5874        assert result.status == "completed"
    5875        assert result.tool_name == "record_experiment"
    5876    finally:
    5877        db.close()
    5878
    5879
    5880def test_evidence_grounding_ignores_json_literals_even_when_stale_tokens_exist(tmp_path):
    5881    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5882    db = AgentDB(tmp_path / "state.db")
    5883    try:
    5884        job_id = db.create_job("Record observed benchmark plan", title="literal-grounding", kind="generic")
    5885        db.update_job_metadata(job_id, {"unsupported_claim_tokens": ["true"]})
    5886        run_id = db.start_run(job_id, model="test")
    5887        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5888        db.finish_step(
    5889            step_id,
    5890            status="completed",
    5891            output_data={
    5892                "success": True,
    5893                "stdout": "Observed benchmark harness is ready and next action is to measure throughput. " * 4,
    5894            },
    5895        )
    5896        db.finish_run(run_id, "completed")
    5897
    5898        result = run_one_step(
    5899            job_id,
    5900            config=config,
    5901            db=db,
    5902            llm=ScriptedLLM([
    5903                LLMResponse(tool_calls=[ToolCall(name="record_experiment", arguments={
    5904                    "title": "Baseline benchmark plan",
    5905                    "status": "planned",
    5906                    "higher_is_better": True,
    5907                    "metric_name": "throughput",
    5908                    "metric_unit": "tokens/sec",
    5909                    "next_action": "Run the benchmark and record the observed metric.",
    5910                })])
    5911            ]),
    5912        )
    5913
    5914        assert result.status == "completed"
    5915    finally:
    5916        db.close()
    5917
    5918
    5919def test_evidence_grounding_ignores_planning_and_status_labels(tmp_path):
    5920    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5921    db = AgentDB(tmp_path / "state.db")
    5922    try:
    5923        job_id = db.create_job("Record observed build validation", title="status-grounding", kind="generic")
    5924        run_id = db.start_run(job_id, model="test")
    5925        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5926        db.finish_step(
    5927            step_id,
    5928            status="completed",
    5929            output_data={
    5930                "success": True,
    5931                "stdout": (
    5932                    "Observed file /srv/models/AlphaModel-Q4.foo exists. "
    5933                    "The tool output showed rc=0 and the benchmark branch can continue. "
    5934                )
    5935                * 4,
    5936            },
    5937        )
    5938        db.finish_run(run_id, "completed")
    5939
    5940        result = run_one_step(
    5941            job_id,
    5942            config=config,
    5943            db=db,
    5944            llm=ScriptedLLM([
    5945                LLMResponse(tool_calls=[ToolCall(name="record_roadmap", arguments={
    5946                    "title": "Build validation roadmap",
    5947                    "scope": "Checking the observed candidate before ongoing benchmark work.",
    5948                    "milestones": [
    5949                        {"title": "P1 validate observed candidate", "status": "active"},
    5950                        {"title": "P2 proceed to benchmark", "status": "planned"},
    5951                    ],
    5952                })])
    5953            ]),
    5954        )
    5955
    5956        assert result.status == "completed"
    5957    finally:
    5958        db.close()
    5959
    5960
    5961def test_run_one_step_blocks_memory_graph_with_unsupported_claims(tmp_path):
    5962    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    5963    db = AgentDB(tmp_path / "state.db")
    5964    try:
    5965        job_id = db.create_job("Consolidate observed facts", title="memory-grounding", kind="generic")
    5966        run_id = db.start_run(job_id, model="test")
    5967        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    5968        db.finish_step(
    5969            step_id,
    5970            status="completed",
    5971            output_data={
    5972                "success": True,
    5973                "stdout": "GPU: AMD Device 7590\nCPU: AMD Ryzen 9 7900X\nMemory: 93Gi\n",
    5974            },
    5975        )
    5976        db.finish_run(run_id, "completed")
    5977
    5978        result = run_one_step(
    5979            job_id,
    5980            config=config,
    5981            db=db,
    5982            llm=ScriptedLLM([
    5983                LLMResponse(tool_calls=[
    5984                    ToolCall(
    5985                        name="record_memory_graph",
    5986                        arguments={
    5987                            "nodes": [
    5988                                {
    5989                                    "key": "hardware",
    5990                                    "kind": "fact",
    5991                                    "title": "NVIDIA GTX 970 CUDA hardware",
    5992                                    "summary": "The machine has NVIDIA GTX 970 CUDA hardware.",
    5993                                }
    5994                            ]
    5995                        },
    5996                    )
    5997                ])
    5998            ]),
    5999        )
    6000
    6001        assert result.status == "blocked"
    6002        assert result.result["error"] == "evidence grounding required"
    6003        assert result.result["blocked_tool"] == "record_memory_graph"
    6004    finally:
    6005        db.close()
    6006
    6007
    6008def test_run_one_step_allows_memory_graph_identifier_labels_without_evidence(tmp_path):
    6009    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    6010    db = AgentDB(tmp_path / "state.db")
    6011    try:
    6012        job_id = db.create_job("Consolidate abstract graph labels", title="memory-grounding", kind="generic")
    6013        run_id = db.start_run(job_id, model="test")
    6014        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6015        db.finish_step(
    6016            step_id,
    6017            status="completed",
    6018            output_data={
    6019                "success": True,
    6020                "stdout": (
    6021                    "Current observation: AMD Ryzen 9 7900X host with fresh API discovery evidence. "
    6022                    "The next branch is to convert existing source evidence into a download decision."
    6023                ),
    6024            },
    6025        )
    6026        db.finish_run(run_id, "completed")
    6027
    6028        result = run_one_step(
    6029            job_id,
    6030            config=config,
    6031            db=db,
    6032            llm=ScriptedLLM([
    6033                LLMResponse(tool_calls=[
    6034                    ToolCall(
    6035                        name="record_memory_graph",
    6036                        arguments={
    6037                            "edges": [
    6038                                {
    6039                                    "from_key": "decision-q4-km-primary",
    6040                                    "relation": "informs",
    6041                                    "to_key": "question-download-q4-km-url",
    6042                                },
    6043                                {
    6044                                    "from_key": "skill-api-download-pattern",
    6045                                    "relation": "supports",
    6046                                    "to_key": "milestone-direct-url-download",
    6047                                },
    6048                            ]
    6049                        },
    6050                    )
    6051                ])
    6052            ]),
    6053        )
    6054
    6055        assert result.status == "completed"
    6056        assert result.tool_name == "record_memory_graph"
    6057    finally:
    6058        db.close()
    6059
    6060
    6061def test_run_one_step_still_blocks_stale_memory_graph_key_claims(tmp_path):
    6062    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    6063    db = AgentDB(tmp_path / "state.db")
    6064    try:
    6065        job_id = db.create_job(
    6066            "Do not reintroduce stale graph labels",
    6067            title="memory-grounding",
    6068            kind="generic",
    6069            metadata={"unsupported_claim_tokens": ["XeonE5-2690"]},
    6070        )
    6071        run_id = db.start_run(job_id, model="test")
    6072        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6073        db.finish_step(
    6074            step_id,
    6075            status="completed",
    6076            output_data={
    6077                "success": True,
    6078                "stdout": (
    6079                    "Current observation: AMD Ryzen 9 7900X host with no legacy CPU marker in fresh evidence. "
    6080                    "Durable memory must not reuse unsupported old hardware claims."
    6081                ),
    6082            },
    6083        )
    6084        db.finish_run(run_id, "completed")
    6085
    6086        result = run_one_step(
    6087            job_id,
    6088            config=config,
    6089            db=db,
    6090            llm=ScriptedLLM([
    6091                LLMResponse(tool_calls=[
    6092                    ToolCall(
    6093                        name="record_memory_graph",
    6094                        arguments={
    6095                            "edges": [
    6096                                {
    6097                                    "from_key": "XeonE5-2690",
    6098                                    "relation": "constrains",
    6099                                    "to_key": "current-plan",
    6100                                }
    6101                            ]
    6102                        },
    6103                    )
    6104                ])
    6105            ]),
    6106        )
    6107
    6108        assert result.status == "blocked"
    6109        assert result.result["error"] == "evidence grounding required"
    6110        assert "XeonE5-2690" in result.result["evidence_grounding"]["unsupported_tokens"]
    6111    finally:
    6112        db.close()
    6113
    6114
    6115def test_run_one_step_allows_memory_graph_grounded_in_durable_records(tmp_path):
    6116    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    6117    db = AgentDB(tmp_path / "state.db")
    6118    try:
    6119        job_id = db.create_job("Consolidate durable facts", title="memory-grounded-ledger", kind="generic")
    6120        db.append_finding_record(
    6121            job_id,
    6122            name="Artifact cache includes Package_A-2.7.1 and backend XYZ123",
    6123            category="environment_fact",
    6124            reason="A saved checkpoint established Package_A-2.7.1 and backend XYZ123 as available options.",
    6125            metadata={"evidence_artifact": "art_env"},
    6126        )
    6127
    6128        result = run_one_step(
    6129            job_id,
    6130            config=config,
    6131            db=db,
    6132            llm=ScriptedLLM([
    6133                LLMResponse(tool_calls=[
    6134                    ToolCall(
    6135                        name="record_memory_graph",
    6136                        arguments={
    6137                            "nodes": [
    6138                                {
    6139                                    "key": "package-a",
    6140                                    "kind": "fact",
    6141                                    "title": "Package_A-2.7.1 via backend XYZ123",
    6142                                    "summary": "Durable finding says Package_A-2.7.1 is available through backend XYZ123.",
    6143                                }
    6144                            ]
    6145                        },
    6146                    )
    6147                ])
    6148            ]),
    6149        )
    6150
    6151        assert result.status == "completed"
    6152        assert result.tool_name == "record_memory_graph"
    6153    finally:
    6154        db.close()
    6155
    6156
    6157def test_run_one_step_blocks_memory_graph_grounded_only_in_stale_records(tmp_path):
    6158    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    6159    db = AgentDB(tmp_path / "state.db")
    6160    try:
    6161        job_id = db.create_job(
    6162            "Consolidate durable facts",
    6163            title="memory-stale-ledger",
    6164            kind="generic",
    6165            metadata={"unsupported_claim_tokens": ["XeonE5-2690"]},
    6166        )
    6167        db.append_finding_record(
    6168            job_id,
    6169            name="Artifact cache includes XeonE5-2690",
    6170            category="environment_fact",
    6171            reason="Older ledger record mentioned XeonE5-2690.",
    6172            metadata={"evidence_artifact": "art_old"},
    6173        )
    6174
    6175        result = run_one_step(
    6176            job_id,
    6177            config=config,
    6178            db=db,
    6179            llm=ScriptedLLM([
    6180                LLMResponse(tool_calls=[
    6181                    ToolCall(
    6182                        name="record_memory_graph",
    6183                        arguments={
    6184                            "nodes": [
    6185                                {
    6186                                    "key": "package-a",
    6187                                    "kind": "fact",
    6188                                    "title": "XeonE5-2690",
    6189                                    "summary": "XeonE5-2690 is still valid.",
    6190                                }
    6191                            ]
    6192                        },
    6193                    )
    6194                ])
    6195            ]),
    6196        )
    6197
    6198        assert result.status == "blocked"
    6199        assert result.result["error"] == "evidence grounding required"
    6200        assert "XeonE5-2690" in result.result["evidence_grounding"]["unsupported_tokens"]
    6201    finally:
    6202        db.close()
    6203
    6204
    6205def test_run_one_step_allows_stale_token_when_fresh_evidence_revalidates_it(tmp_path):
    6206    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    6207    db = AgentDB(tmp_path / "state.db")
    6208    try:
    6209        job_id = db.create_job(
    6210            "Revalidate durable facts",
    6211            title="memory-stale-revalidated",
    6212            kind="generic",
    6213            metadata={"unsupported_claim_tokens": ["XeonE5-2690"]},
    6214        )
    6215        run_id = db.start_run(job_id, model="test")
    6216        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6217        db.finish_step(
    6218            step_id,
    6219            status="completed",
    6220            output_data={"success": True, "stdout": "Fresh probe: CPU marker XeonE5-2690 is visible in this environment."},
    6221        )
    6222        db.finish_run(run_id, "completed")
    6223
    6224        result = run_one_step(
    6225            job_id,
    6226            config=config,
    6227            db=db,
    6228            llm=ScriptedLLM([
    6229                LLMResponse(tool_calls=[
    6230                    ToolCall(
    6231                        name="record_memory_graph",
    6232                        arguments={
    6233                            "nodes": [
    6234                                {
    6235                                    "key": "fresh-cpu",
    6236                                    "kind": "fact",
    6237                                    "title": "XeonE5-2690",
    6238                                    "summary": "Fresh shell evidence revalidated XeonE5-2690.",
    6239                                }
    6240                            ]
    6241                        },
    6242                    )
    6243                ])
    6244            ]),
    6245        )
    6246
    6247        assert result.status == "completed"
    6248        assert result.tool_name == "record_memory_graph"
    6249    finally:
    6250        db.close()
    6251
    6252
    6253def test_run_one_step_allows_durable_records_grounded_in_read_artifact(tmp_path):
    6254    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    6255    db = AgentDB(tmp_path / "state.db")
    6256    try:
    6257        job_id = db.create_job("Record facts from saved evidence", title="grounded-read", kind="generic")
    6258        run_id = db.start_run(job_id, model="test")
    6259        step_id = db.add_step(
    6260            job_id=job_id,
    6261            run_id=run_id,
    6262            kind="tool",
    6263            tool_name="read_artifact",
    6264            input_data={"arguments": {"artifact_id": "art_checkpoint"}},
    6265        )
    6266        db.finish_step(
    6267            step_id,
    6268            status="completed",
    6269            output_data={
    6270                "success": True,
    6271                "content": (
    6272                    "Environment evidence: CPU Intel Xeon E5-2690 v3, architecture x86_64, "
    6273                    "memory 62.8G total, no NVIDIA GPU visible from nvidia-smi. "
    6274                    "This content is the source for durable records."
    6275                ),
    6276            },
    6277        )
    6278        db.finish_run(run_id, "completed")
    6279
    6280        result = run_one_step(
    6281            job_id,
    6282            config=config,
    6283            db=db,
    6284            llm=ScriptedLLM([
    6285                LLMResponse(tool_calls=[
    6286                    ToolCall(
    6287                        name="record_findings",
    6288                        arguments={
    6289                            "findings": [
    6290                                {
    6291                                    "name": "Intel Xeon E5-2690 v3 x86_64 environment",
    6292                                    "category": "hardware_fact",
    6293                                    "reason": "Saved checkpoint states CPU Intel Xeon E5-2690 v3, x86_64, memory 62.8G total, and no NVIDIA GPU visible.",
    6294                                    "evidence_artifact": "art_checkpoint",
    6295                                }
    6296                            ]
    6297                        },
    6298                    )
    6299                ])
    6300            ]),
    6301        )
    6302
    6303        assert result.status == "completed"
    6304        findings = db.get_job(job_id)["metadata"]["finding_ledger"]
    6305        assert findings[0]["name"] == "Intel Xeon E5-2690 v3 x86_64 environment"
    6306    finally:
    6307        db.close()
    6308
    6309
    6310def test_run_one_step_scopes_grounding_to_cited_step(tmp_path):
    6311    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    6312    db = AgentDB(tmp_path / "state.db")
    6313    try:
    6314        job_id = db.create_job("Record facts from cited evidence", title="cited-grounding", kind="generic")
    6315        run_id = db.start_run(job_id, model="test")
    6316        old_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6317        db.finish_step(
    6318            old_step,
    6319            status="completed",
    6320            output_data={
    6321                "success": True,
    6322                "stdout": (
    6323                    "Old evidence: Intel Xeon E5-2690 v3 with 62.8G memory. "
    6324                    "This is intentionally stale evidence from an earlier step and should not validate step #2."
    6325                ),
    6326            },
    6327        )
    6328        new_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6329        db.finish_step(
    6330            new_step,
    6331            status="completed",
    6332            output_data={
    6333                "success": True,
    6334                "stdout": (
    6335                    "Current evidence: AMD Ryzen 9 7900X with 93Gi memory and AMD GPU. "
    6336                    "This newer cited step is the only source that should ground claims citing step #2."
    6337                ),
    6338            },
    6339        )
    6340        db.finish_run(run_id, "completed")
    6341
    6342        result = run_one_step(
    6343            job_id,
    6344            config=config,
    6345            db=db,
    6346            llm=ScriptedLLM([
    6347                LLMResponse(tool_calls=[
    6348                    ToolCall(
    6349                        name="write_artifact",
    6350                        arguments={
    6351                            "title": "Cited baseline",
    6352                            "summary": "Baseline from step #2.",
    6353                            "content": "From step #2: Intel Xeon E5-2690 v3 with 62.8G memory.",
    6354                            "artifact_type": "text",
    6355                        },
    6356                    )
    6357                ])
    6358            ]),
    6359        )
    6360
    6361        assert result.status == "blocked"
    6362        assert result.result["error"] == "evidence grounding required"
    6363        assert "E5-2690" in result.result["evidence_grounding"]["unsupported_tokens"]
    6364        assert result.result["evidence_grounding"]["evidence_steps"] == [2]
    6365        assert "E5-2690" in db.get_job(job_id)["metadata"]["unsupported_claim_tokens"]
    6366    finally:
    6367        db.close()
    6368
    6369
    6370def test_cited_step_numbers_ignore_ordinal_hash_labels():
    6371    text = (
    6372        "llama.cpp Build Attempt #3 should not cite old evidence. "
    6373        "Use step #42 and shell_exec_step_1037 if explicit evidence is needed. "
    6374        "The older step-2678 reference is also explicit."
    6375    )
    6376
    6377    assert _cited_step_numbers(text) == {42, 1037, 2678}
    6378
    6379
    6380def test_prompt_shows_evidence_grounding_tokens_after_block(tmp_path):
    6381    db = AgentDB(tmp_path / "state.db")
    6382    try:
    6383        job_id = db.create_job("Use only observed evidence", title="grounding-prompt", kind="generic")
    6384        run_id = db.start_run(job_id, model="test")
    6385        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
    6386        db.finish_step(
    6387            step_id,
    6388            status="blocked",
    6389            output_data={
    6390                "success": True,
    6391                "recoverable": True,
    6392                "error": "evidence grounding required",
    6393                "evidence_grounding": {"unsupported_tokens": ["NVIDIA", "Xeon", "AVX-512"]},
    6394            },
    6395        )
    6396        job = db.get_job(job_id)
    6397        content = build_messages(job, db.list_steps(job_id=job_id))[-1]["content"]
    6398
    6399        assert "unsupported=NVIDIA, Xeon, AVX-512" in content
    6400        assert "use only tokens present in recent observed evidence" in content
    6401    finally:
    6402        db.close()
    6403
    6404
    6405def test_prompt_shows_missing_candidate_paths_after_grounding_block(tmp_path):
    6406    db = AgentDB(tmp_path / "state.db")
    6407    try:
    6408        job_id = db.create_job("Optimize benchmark speed with exact file evidence", title="grounding-paths", kind="generic")
    6409        run_id = db.start_run(job_id, model="test")
    6410        for index in range(18):
    6411            shell_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6412            db.finish_step(shell_step, status="completed", output_data={"success": True, "stdout": f"probe {index}"})
    6413        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    6414        db.finish_step(
    6415            step_id,
    6416            status="blocked",
    6417            summary="blocked record_experiment; evidence grounding required",
    6418            output_data={
    6419                "success": True,
    6420                "recoverable": True,
    6421                "error": "evidence grounding required",
    6422                "evidence_grounding": {
    6423                    "missing_candidate_paths": [
    6424                        "/srv/models/AlphaModel-Q4.foo",
    6425                        "/srv/models/BetaModel-Q8.foo",
    6426                    ],
    6427                    "unsupported_tokens": [
    6428                        "/srv/models/AlphaModel-Q4.foo",
    6429                        "/srv/models/BetaModel-Q8.foo",
    6430                    ],
    6431                },
    6432            },
    6433        )
    6434        job = db.get_job(job_id)
    6435        content = build_messages(job, db.list_steps(job_id=job_id))[-1]["content"]
    6436
    6437        assert "Recent evidence grounding blocked a durable record" in content
    6438        assert "This job needs measured progress" not in content
    6439        assert "/srv/models/AlphaModel-Q4.foo" in content
    6440        assert "rewrite the durable record with exact observed paths" in content
    6441    finally:
    6442        db.close()
    6443
    6444
    6445def test_prompt_adds_ranked_current_candidates_to_stale_grounding_block(tmp_path):
    6446    db = AgentDB(tmp_path / "state.db")
    6447    try:
    6448        job_id = db.create_job("Benchmark OmegaModel throughput", title="grounding-current-candidates", kind="generic")
    6449        db.update_job_metadata(
    6450            job_id,
    6451            {
    6452                "task_queue": [
    6453                    {
    6454                        "title": "Validate OmegaModel file path",
    6455                        "status": "open",
    6456                        "contract": "experiment",
    6457                        "acceptance_criteria": "Use a validated candidate path.",
    6458                        "evidence_needed": "Shell output with file size and benchmark result.",
    6459                    }
    6460                ]
    6461            },
    6462        )
    6463        run_id = db.start_run(job_id, model="test")
    6464        shell_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6465        db.finish_step(
    6466            shell_step,
    6467            status="completed",
    6468            output_data={
    6469                "success": True,
    6470                "stdout": (
    6471                    "/tmp/aux/ggml-vocab-alpha.foo\n"
    6472                    "/srv/models/OmegaModel-primary.foo\n"
    6473                ),
    6474            },
    6475        )
    6476        block_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    6477        db.finish_step(
    6478            block_step,
    6479            status="blocked",
    6480            summary="blocked record_experiment; evidence grounding required",
    6481            output_data={
    6482                "success": True,
    6483                "recoverable": True,
    6484                "error": "evidence grounding required",
    6485                "evidence_grounding": {
    6486                    "missing_candidate_paths": ["/tmp/aux/ggml-vocab-alpha.foo"],
    6487                    "unsupported_tokens": ["/tmp/aux/ggml-vocab-alpha.foo"],
    6488                },
    6489            },
    6490        )
    6491
    6492        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    6493        idx = content.index("Next-action constraint:")
    6494        next_constraint = content[idx: idx + 1200]
    6495
    6496        assert "current ranked candidate paths are available" in next_constraint
    6497        ranked_text = next_constraint[next_constraint.index("Candidate paths:"):]
    6498        assert ranked_text.index("/srv/models/OmegaModel-primary.foo") < ranked_text.index("/tmp/aux/ggml-vocab-alpha.foo")
    6499    finally:
    6500        db.close()
    6501
    6502
    6503def test_prompt_does_not_resurface_grounding_block_after_durable_resolution(tmp_path):
    6504    db = AgentDB(tmp_path / "state.db")
    6505    try:
    6506        job_id = db.create_job("Use exact file evidence", title="grounding-resolved", kind="generic")
    6507        run_id = db.start_run(job_id, model="test")
    6508        block_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_findings")
    6509        db.finish_step(
    6510            block_step,
    6511            status="blocked",
    6512            summary="blocked record_findings; evidence grounding required",
    6513            output_data={
    6514                "success": True,
    6515                "recoverable": True,
    6516                "error": "evidence grounding required",
    6517                "evidence_grounding": {
    6518                    "missing_candidate_paths": ["/srv/models/AlphaModel-Q4.foo"],
    6519                    "unsupported_tokens": ["/srv/models/AlphaModel-Q4.foo"],
    6520                },
    6521            },
    6522        )
    6523        resolved_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_findings")
    6524        db.finish_step(
    6525            resolved_step,
    6526            status="completed",
    6527            output_data={
    6528                "success": True,
    6529                "findings": [{"name": "Exact path accounted", "reason": "/srv/models/AlphaModel-Q4.foo was validated."}],
    6530            },
    6531        )
    6532        db.finish_run(run_id, "completed")
    6533
    6534        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    6535        next_action = content.split("Next-action constraint:", 1)[1].split("\n\n", 1)[0]
    6536
    6537        assert "Recent evidence grounding blocked a durable record" not in content
    6538        assert "/srv/models/AlphaModel-Q4.foo" not in next_action
    6539    finally:
    6540        db.close()
    6541
    6542
    6543def test_prompt_suppresses_findings_matching_stale_claim_tokens(tmp_path):
    6544    db = AgentDB(tmp_path / "state.db")
    6545    try:
    6546        job_id = db.create_job("Prefer current durable evidence", title="stale-ledger", kind="generic")
    6547        db.append_finding_record(job_id, name="Intel Xeon E5-2690 v3 baseline", category="hardware")
    6548        db.append_finding_record(job_id, name="AMD Ryzen 9 7900X baseline", category="hardware")
    6549        db.append_lesson(
    6550            job_id,
    6551            "Evidence grounding rejected unsupported concrete tokens for record_experiment: E5-2690, v3, RAM. Treat matching prior ledger claims as stale.",
    6552            category="mistake",
    6553        )
    6554
    6555        job = db.get_job(job_id)
    6556        content = build_messages(job, db.list_steps(job_id=job_id))[-1]["content"]
    6557
    6558        assert "Unsupported/stale claim tokens to avoid until re-verified: [unsupported-stale-claim]" in content
    6559        assert "Suppressed 1 stale finding" in content
    6560        assert "AMD Ryzen 9 7900X baseline" in content
    6561        assert "Intel Xeon E5-2690 v3 baseline" not in content
    6562    finally:
    6563        db.close()
    6564
    6565
    6566def test_prompt_prioritizes_validation_for_recent_candidate_file_paths(tmp_path):
    6567    db = AgentDB(tmp_path / "state.db")
    6568    try:
    6569        job_id = db.create_job("Validate a discovered runtime file", title="candidate-file", kind="generic")
    6570        db.update_job_metadata(
    6571            job_id,
    6572            {
    6573                "task_queue": [
    6574                    {
    6575                        "title": "Run baseline benchmark with the discovered file",
    6576                        "status": "open",
    6577                        "contract": "experiment",
    6578                        "acceptance_criteria": "Benchmark command uses a validated file path.",
    6579                        "evidence_needed": "Shell output showing file size and benchmark result.",
    6580                    }
    6581                ]
    6582            },
    6583        )
    6584        run_id = db.start_run(job_id, model="test")
    6585        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6586        db.finish_step(
    6587            step_id,
    6588            status="completed",
    6589            output_data={
    6590                "success": True,
    6591                "stdout": (
    6592                    "candidate files:\n"
    6593                    "/srv/models/ExampleModel-Q4.foo\n"
    6594                    "/srv/models/sidecar.txt\n"
    6595                ),
    6596            },
    6597        )
    6598        db.finish_run(run_id, "completed")
    6599
    6600        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    6601
    6602        assert "Candidate file discovery:" in content
    6603        assert "/srv/models/ExampleModel-Q4.foo" in content
    6604        assert "Validate likely candidates with shell_exec" in content
    6605        assert "Do not reject a non-empty candidate binary from `file` output alone" in content
    6606    finally:
    6607        db.close()
    6608
    6609
    6610def test_prompt_deprioritizes_recent_stub_candidate_file_paths(tmp_path):
    6611    db = AgentDB(tmp_path / "state.db")
    6612    try:
    6613        job_id = db.create_job("Benchmark AlphaModel throughput", title="alpha benchmark", kind="generic")
    6614        db.update_job_metadata(
    6615            job_id,
    6616            {
    6617                "task_queue": [
    6618                    {
    6619                        "title": "Validate candidate model file before benchmark",
    6620                        "status": "open",
    6621                        "contract": "experiment",
    6622                        "acceptance_criteria": "Benchmark uses a validated model file.",
    6623                        "evidence_needed": "Shell output showing file size and parser/header status.",
    6624                    }
    6625                ]
    6626            },
    6627        )
    6628        run_id = db.start_run(job_id, model="test")
    6629        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6630        db.finish_step(
    6631            step_id,
    6632            status="completed",
    6633            output_data={
    6634                "success": True,
    6635                "stdout": (
    6636                    "-rw-r--r-- 1 user user 29 May 15 10:00 /tmp/models/AlphaModel-Q4.foo\n"
    6637                    "/tmp/models/AlphaModel-Q4.foo: ASCII text, with no line terminators\n"
    6638                    "-rw-r--r-- 1 user user 12G May 15 10:01 /srv/models/AlphaModel-IQ3.foo\n"
    6639                ),
    6640            },
    6641        )
    6642        db.finish_run(run_id, "completed")
    6643
    6644        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    6645
    6646        idx = content.index("Next-action constraint:")
    6647        next_constraint = content[idx: idx + 1400]
    6648        assert "/srv/models/AlphaModel-IQ3.foo" in next_constraint
    6649        assert "/tmp/models/AlphaModel-Q4.foo" not in next_constraint
    6650        assert "Recently invalid or stub-like candidates" in content
    6651        assert "/tmp/models/AlphaModel-Q4.foo" in content
    6652    finally:
    6653        db.close()
    6654
    6655
    6656def test_prompt_isolates_current_execution_focus_for_candidate_validation(tmp_path):
    6657    db = AgentDB(tmp_path / "state.db")
    6658    try:
    6659        job_id = db.create_job("Benchmark AlphaModel throughput", title="alpha benchmark", kind="generic")
    6660        db.update_job_metadata(
    6661            job_id,
    6662            {
    6663                "task_queue": [
    6664                    {
    6665                        "title": f"Old branch {index}",
    6666                        "status": "open",
    6667                        "priority": index,
    6668                    }
    6669                    for index in range(82)
    6670                ] + [
    6671                    {
    6672                        "title": "Validate AlphaModel candidate file before benchmark",
    6673                        "status": "active",
    6674                        "priority": 100,
    6675                        "contract": "experiment",
    6676                        "acceptance_criteria": "Validated candidate file is used in a measurement.",
    6677                        "evidence_needed": "Shell output with candidate file size and benchmark result.",
    6678                    }
    6679                ]
    6680            },
    6681        )
    6682        run_id = db.start_run(job_id, model="test")
    6683        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6684        db.finish_step(
    6685            step_id,
    6686            status="failed",
    6687            output_data={
    6688                "success": False,
    6689                "stdout": "-rw-r--r-- 1 user user 12G May 15 10:01 /srv/models/AlphaModel-IQ3.foo\n",
    6690                "stderr": "ls: cannot access '/tmp/models/AlphaModel-Q4.foo': No such file or directory\n",
    6691            },
    6692        )
    6693        db.finish_run(run_id, "failed")
    6694
    6695        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    6696
    6697        focus = content[content.index("Current execution focus:"): content.index("Pending measurement obligation:")]
    6698        assert "phase=execute_with_validated_candidate" in focus
    6699        assert "Use the recently validated candidate path: /srv/models/AlphaModel-IQ3.foo" in focus
    6700        assert "backlog=83 tasks" in focus
    6701        assert "Treat it as advisory" in focus
    6702        next_constraint = content[content.index("Next-action constraint:"):]
    6703        assert "/srv/models/AlphaModel-IQ3.foo" in next_constraint
    6704        assert "/tmp/models/AlphaModel-Q4.foo" not in next_constraint
    6705    finally:
    6706        db.close()
    6707
    6708
    6709def test_prompt_moves_from_candidate_validation_to_candidate_use_after_positive_evidence(tmp_path):
    6710    db = AgentDB(tmp_path / "state.db")
    6711    try:
    6712        job_id = db.create_job("Benchmark AlphaModel throughput", title="alpha benchmark", kind="generic")
    6713        db.update_job_metadata(
    6714            job_id,
    6715            {
    6716                "task_queue": [
    6717                    {
    6718                        "title": "Run benchmark with validated AlphaModel file",
    6719                        "status": "active",
    6720                        "priority": 100,
    6721                        "contract": "experiment",
    6722                        "acceptance_criteria": "Benchmark command uses a validated file path.",
    6723                        "evidence_needed": "Shell output showing file size and benchmark result.",
    6724                    }
    6725                ]
    6726            },
    6727        )
    6728        run_id = db.start_run(job_id, model="test")
    6729        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6730        db.finish_step(
    6731            step_id,
    6732            status="completed",
    6733            output_data={
    6734                "success": True,
    6735                "stdout": "-rw-r--r-- 1 user user 12G May 15 10:01 /srv/models/AlphaModel-IQ3.foo\n",
    6736            },
    6737        )
    6738        db.finish_run(run_id, "completed")
    6739
    6740        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    6741
    6742        focus = content[content.index("Current execution focus:"): content.index("Pending measurement obligation:")]
    6743        assert "phase=execute_with_validated_candidate" in focus
    6744        assert "Use the recently validated candidate path: /srv/models/AlphaModel-IQ3.foo" in focus
    6745        next_constraint = content[content.index("Next-action constraint:"):]
    6746        assert "Use it in the next bounded action or measurement" in next_constraint
    6747        assert "repeating existence checks" in next_constraint
    6748    finally:
    6749        db.close()
    6750
    6751
    6752def test_prompt_ranks_context_matching_candidate_paths_before_auxiliary_files(tmp_path):
    6753    db = AgentDB(tmp_path / "state.db")
    6754    try:
    6755        job_id = db.create_job("Benchmark AlphaModel throughput", title="alpha benchmark", kind="generic")
    6756        db.update_job_metadata(
    6757            job_id,
    6758            {
    6759                "task_queue": [
    6760                    {
    6761                        "title": "Validate candidate file path before benchmark",
    6762                        "status": "open",
    6763                        "contract": "experiment",
    6764                        "acceptance_criteria": "Validated primary file is used in a measurement.",
    6765                        "evidence_needed": "Shell output with file size and benchmark result.",
    6766                    }
    6767                ]
    6768            },
    6769        )
    6770        run_id = db.start_run(job_id, model="test")
    6771        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6772        db.finish_step(
    6773            step_id,
    6774            status="completed",
    6775            output_data={
    6776                "success": True,
    6777                "stdout": (
    6778                    "/srv/models/ggml-vocab-alpha.foo\n"
    6779                    "/srv/models/sidecar-mmproj-alpha.foo\n"
    6780                    "/srv/models/AlphaModel-Q4.foo\n"
    6781                ),
    6782            },
    6783        )
    6784        db.finish_run(run_id, "completed")
    6785
    6786        job = db.get_job(job_id)
    6787        content = build_messages(job, db.list_steps(job_id=job_id))[-1]["content"]
    6788        ranked = _rank_candidate_file_paths(
    6789            job,
    6790            "Validate candidate file path before benchmark",
    6791            [
    6792                "/srv/models/ggml-vocab-alpha.foo",
    6793                "/srv/models/sidecar-mmproj-alpha.foo",
    6794                "/srv/models/AlphaModel-Q4.foo",
    6795            ],
    6796        )
    6797
    6798        section = content[content.index("Candidate file discovery:"): content.index("Measured progress guard:")]
    6799        assert "Candidate paths:" in section
    6800        assert ranked[0] == "/srv/models/AlphaModel-Q4.foo"
    6801        assert "/srv/models/Alp" in section
    6802        assert "This supersedes stale no-candidate/no-file memory" in section
    6803        assert "header/signature bytes" in section
    6804    finally:
    6805        db.close()
    6806
    6807
    6808def test_next_action_prioritizes_candidate_file_validation_over_download_retry(tmp_path):
    6809    db = AgentDB(tmp_path / "state.db")
    6810    try:
    6811        job_id = db.create_job("Benchmark AlphaModel throughput", title="alpha benchmark", kind="generic")
    6812        db.update_job_metadata(
    6813            job_id,
    6814            {
    6815                "task_queue": [
    6816                    {
    6817                        "title": "Run baseline benchmark with the discovered file",
    6818                        "status": "open",
    6819                        "contract": "experiment",
    6820                        "acceptance_criteria": "Benchmark command uses a validated file path.",
    6821                        "evidence_needed": "Shell output showing file size and benchmark result.",
    6822                    }
    6823                ],
    6824                "experiment_ledger": [
    6825                    {
    6826                        "title": "Remote download failed",
    6827                        "status": "failed",
    6828                        "metric_name": "downloaded_files",
    6829                        "metric_value": 0,
    6830                        "next_action": "Record tasks to explore alternative download methods and remote sources.",
    6831                    }
    6832                ],
    6833            },
    6834        )
    6835        run_id = db.start_run(job_id, model="test")
    6836        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6837        db.finish_step(
    6838            step_id,
    6839            status="completed",
    6840            output_data={
    6841                "success": True,
    6842                "stdout": "/srv/models/AlphaModel-Q4.foo\n",
    6843            },
    6844        )
    6845        db.finish_run(run_id, "completed")
    6846
    6847        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    6848
    6849        idx = content.index("Next-action constraint:")
    6850        next_constraint = content[idx: idx + 900]
    6851        assert "Concrete candidate file paths are available" in next_constraint
    6852        assert "/srv/models/AlphaModel-Q4.foo" in next_constraint
    6853        assert "alternative download methods" not in next_constraint
    6854    finally:
    6855        db.close()
    6856
    6857
    6858def test_prompt_ranks_late_candidate_paths_from_large_shell_listing(tmp_path):
    6859    db = AgentDB(tmp_path / "state.db")
    6860    try:
    6861        job_id = db.create_job("Benchmark OmegaModel throughput", title="omega benchmark", kind="generic")
    6862        db.update_job_metadata(
    6863            job_id,
    6864            {
    6865                "task_queue": [
    6866                    {
    6867                        "title": "Validate OmegaModel candidate file before measurement",
    6868                        "status": "open",
    6869                        "contract": "experiment",
    6870                        "acceptance_criteria": "Validated candidate file is used in a measurement.",
    6871                        "evidence_needed": "Shell output with candidate file size and benchmark result.",
    6872                    }
    6873                ]
    6874            },
    6875        )
    6876        run_id = db.start_run(job_id, model="test")
    6877        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6878        db.finish_step(
    6879            step_id,
    6880            status="completed",
    6881            output_data={
    6882                "success": True,
    6883                "stdout": "\n".join(
    6884                    [f"/srv/models/ggml-vocab-{index}.foo" for index in range(30)]
    6885                    + ["/srv/models/OmegaModel-primary.foo"]
    6886                ),
    6887            },
    6888        )
    6889        db.finish_run(run_id, "completed")
    6890
    6891        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    6892        section = content[content.index("Candidate file discovery:"): content.index("Measured progress guard:")]
    6893
    6894        assert "/srv/models/Ome" in section
    6895    finally:
    6896        db.close()
    6897
    6898
    6899def test_prompt_prioritizes_structured_candidate_file_paths(tmp_path):
    6900    db = AgentDB(tmp_path / "state.db")
    6901    try:
    6902        job_id = db.create_job("Validate a discovered remote file", title="candidate-file", kind="generic")
    6903        db.update_job_metadata(
    6904            job_id,
    6905            {
    6906                "task_queue": [
    6907                    {
    6908                        "title": "Download and validate a candidate file",
    6909                        "status": "open",
    6910                        "contract": "action",
    6911                        "acceptance_criteria": "A candidate file path is selected and validated before use.",
    6912                        "evidence_needed": "Shell output with size, hash, or validation metadata.",
    6913                    }
    6914                ]
    6915            },
    6916        )
    6917        run_id = db.start_run(job_id, model="test")
    6918        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    6919        db.finish_step(
    6920            step_id,
    6921            status="completed",
    6922            output_data={
    6923                "success": True,
    6924                "stdout": (
    6925                    '[{"type":"file","size":123456789,"path":"ExampleModel-Q4.foo"},'
    6926                    '{"type":"file","size":42,"path":".gitattributes"}]'
    6927                ),
    6928            },
    6929        )
    6930        db.finish_run(run_id, "completed")
    6931
    6932        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    6933
    6934        assert "Candidate file discovery:" in content
    6935        assert "ExampleModel-Q4.foo" in content
    6936        assert "Validate likely candidates with shell_exec" in content
    6937    finally:
    6938        db.close()
    6939
    6940
    6941def test_prompt_filters_truncated_and_url_like_candidate_file_paths(tmp_path):
    6942    db = AgentDB(tmp_path / "state.db")
    6943    try:
    6944        job_id = db.create_job("Validate concrete local candidates", title="candidate-file", kind="generic")
    6945        db.update_job_metadata(
    6946            job_id,
    6947            {
    6948                "task_queue": [
    6949                    {
    6950                        "title": "Validate remembered file path",
    6951                        "status": "open",
    6952                        "contract": "action",
    6953                        "acceptance_criteria": "A candidate file path is validated before use.",
    6954                        "evidence_needed": "Shell output with file size or hash.",
    6955                    }
    6956                ],
    6957                "experiment_ledger": [
    6958                    {
    6959                        "title": "Prior candidate discovery",
    6960                        "result": "Avoid pseudo-paths like //example.com and truncated paths like /tmp/...",
    6961                        "next_action": "Validate /opt/models/ConcreteModel-Q4.foo before declaring no usable file.",
    6962                    }
    6963                ],
    6964            },
    6965        )
    6966
    6967        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    6968
    6969        assert "Candidate file discovery:" in content
    6970        assert "/opt/models/ConcreteModel-Q4.foo" in content
    6971        assert "//example.com" not in content
    6972        assert "/tmp/..." not in content
    6973    finally:
    6974        db.close()
    6975
    6976
    6977def test_candidate_path_extraction_stops_at_escaped_newline_metadata():
    6978    text = (
    6979        'output="/srv/models/AlphaModel-Q4.foo\\n-rw-rw-r-- owner size" '
    6980        '{"path": "/srv/models/BetaModel-Q8.foo\\n-rw-rw-r--"}'
    6981    )
    6982
    6983    paths = _extract_candidate_file_paths(text)
    6984
    6985    assert "/srv/models/AlphaModel-Q4.foo" in paths
    6986    assert "/srv/models/BetaModel-Q8.foo" in paths
    6987    assert all("\\n-rw-rw-r--" not in path for path in paths)
    6988
    6989
    6990def test_candidate_path_extraction_skips_globs_and_truncated_fragments():
    6991    text = (
    6992        "/srv/models/*.foo\n"
    6993        "/srv/models/AlphaModel-Q4.foo\n"
    6994        "/srv/models/AlphaModel-Q4\n"
    6995        "/srv/models/AlphaModel-v1.2-UnfinishedFrag\n"
    6996        "/srv/models/BetaModel-v1.2-Q8.foo\n"
    6997    )
    6998
    6999    paths = _extract_candidate_file_paths(text)
    7000
    7001    assert "/srv/models/AlphaModel-Q4.foo" in paths
    7002    assert "/srv/models/BetaModel-v1.2-Q8.foo" in paths
    7003    assert "/srv/models/*.foo" not in paths
    7004    assert "/srv/models/AlphaModel-Q4" not in paths
    7005    assert "/srv/models/AlphaModel-v1.2-UnfinishedFrag" not in paths
    7006
    7007
    7008def test_prompt_resurfaces_durable_candidate_file_paths(tmp_path):
    7009    db = AgentDB(tmp_path / "state.db")
    7010    try:
    7011        job_id = db.create_job("Validate a remembered file candidate", title="durable-candidate", kind="generic")
    7012        db.update_job_metadata(
    7013            job_id,
    7014            {
    7015                "task_queue": [
    7016                    {
    7017                        "title": "Validate remembered file path",
    7018                        "status": "open",
    7019                        "contract": "action",
    7020                        "acceptance_criteria": "A candidate file path is validated before use.",
    7021                        "evidence_needed": "Shell output with file size or hash.",
    7022                    }
    7023                ],
    7024                "experiment_ledger": [
    7025                    {
    7026                        "title": "Prior candidate discovery",
    7027                        "result": "A previous branch listed /opt/models/Remembered-Model-Q4.foo as a candidate.",
    7028                        "next_action": "Validate /opt/models/Remembered-Model-Q4.foo before declaring no usable file.",
    7029                    }
    7030                ],
    7031            },
    7032        )
    7033
    7034        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    7035
    7036        assert "Candidate file discovery:" in content
    7037        assert "Durable records mention candidate file paths" in content
    7038        assert "/opt/models/Remembered-Model-Q4.foo" in content
    7039        assert "Treat durable-record candidates as candidates until revalidated" in content
    7040    finally:
    7041        db.close()
    7042
    7043
    7044def test_prompt_resurfaces_candidate_paths_from_recent_grounding_block(tmp_path):
    7045    db = AgentDB(tmp_path / "state.db")
    7046    try:
    7047        job_id = db.create_job("Validate a candidate file", title="candidate-file", kind="generic")
    7048        db.update_job_metadata(
    7049            job_id,
    7050            {
    7051                "task_queue": [
    7052                    {
    7053                        "title": "Validate candidate file path",
    7054                        "status": "open",
    7055                        "contract": "action",
    7056                        "acceptance_criteria": "A candidate file path is validated before use.",
    7057                        "evidence_needed": "Shell output with file size or hash.",
    7058                    }
    7059                ]
    7060            },
    7061        )
    7062        run_id = db.start_run(job_id, model="test")
    7063        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_findings")
    7064        db.finish_step(
    7065            step_id,
    7066            status="blocked",
    7067            output_data={
    7068                "success": True,
    7069                "error": "evidence grounding required",
    7070                "evidence_grounding": {
    7071                    "missing_candidate_paths": [
    7072                        "/srv/models/ExactModel-Q4.foo",
    7073                        "/srv/models/*.foo",
    7074                        "/srv/models/Fragment-v1.2-Unfinished",
    7075                    ]
    7076                },
    7077            },
    7078            summary="blocked record_findings; evidence grounding required",
    7079        )
    7080        db.finish_run(run_id, "blocked")
    7081
    7082        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    7083
    7084        assert "Candidate file discovery:" in content
    7085        assert "/srv/models/ExactModel-Q4.foo" in content
    7086        assert "/srv/models/*.foo" not in content
    7087        assert "/srv/models/Fragment-v1.2-Unfinished" not in content
    7088    finally:
    7089        db.close()
    7090
    7091
    7092def test_grounding_uses_recent_missing_candidate_paths_after_raw_evidence_ages(tmp_path):
    7093    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7094    db = AgentDB(tmp_path / "state.db")
    7095    try:
    7096        job_id = db.create_job("Validate a candidate file", title="candidate-file", kind="generic")
    7097        db.update_job_metadata(
    7098            job_id,
    7099            {
    7100                "task_queue": [
    7101                    {
    7102                        "title": "Validate candidate file path",
    7103                        "status": "open",
    7104                        "contract": "experiment",
    7105                        "acceptance_criteria": "A candidate file path is validated before use.",
    7106                        "evidence_needed": "Shell output with file size or hash.",
    7107                    }
    7108                ]
    7109            },
    7110        )
    7111        run_id = db.start_run(job_id, model="test")
    7112        for index in range(10):
    7113            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_lesson")
    7114            db.finish_step(
    7115                step_id,
    7116                status="completed",
    7117                output_data={"success": True, "lesson": {"lesson": f"filler {index}"}},
    7118                summary=f"filler {index}",
    7119            )
    7120        blocked_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_findings")
    7121        db.finish_step(
    7122            blocked_id,
    7123            status="blocked",
    7124            output_data={
    7125                "success": True,
    7126                "error": "evidence grounding required",
    7127                "evidence_grounding": {
    7128                    "missing_candidate_paths": ["/srv/models/ExactModel-Q4.foo"]
    7129                },
    7130            },
    7131            summary="blocked record_findings; evidence grounding required",
    7132        )
    7133        db.finish_run(run_id, "blocked")
    7134
    7135        result = run_one_step(
    7136            job_id,
    7137            config=config,
    7138            db=db,
    7139            llm=ScriptedLLM([
    7140                LLMResponse(tool_calls=[
    7141                    ToolCall(
    7142                        name="record_experiment",
    7143                        arguments={
    7144                            "title": "Candidate file validation",
    7145                            "hypothesis": "A candidate model file may be available.",
    7146                            "metric_name": "validated_files",
    7147                            "metric_value": 0,
    7148                            "metric_unit": "files",
    7149                            "result": "Candidate files were summarized but not named.",
    7150                        },
    7151                    )
    7152                ])
    7153            ]),
    7154        )
    7155
    7156        assert result.status == "blocked"
    7157        grounding = result.result["evidence_grounding"]
    7158        assert "/srv/models/ExactModel-Q4.foo" in grounding["missing_candidate_paths"]
    7159    finally:
    7160        db.close()
    7161
    7162
    7163def test_prompt_filters_stale_generated_and_objective_tokens(tmp_path):
    7164    db = AgentDB(tmp_path / "state.db")
    7165    try:
    7166        job_id = db.create_job("Optimize Qwen3.6-27B GGUF throughput", title="qwen job", kind="generic")
    7167        db.append_lesson(
    7168            job_id,
    7169            (
    7170                "Evidence grounding rejected unsupported concrete tokens for record_experiment: "
    7171                "Qwen3.6-27B-GGUF, JSON, shell_exec_step_1037, timeout_after_300s, E5-2690. "
    7172                "Treat matching prior ledger claims as stale."
    7173            ),
    7174            category="mistake",
    7175        )
    7176        db.append_finding_record(job_id, name="Qwen3.6-27B-GGUF source", category="source")
    7177        db.append_finding_record(job_id, name="Intel Xeon E5-2690 baseline", category="hardware")
    7178
    7179        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    7180
    7181        assert "Qwen3.6-27B-GGUF" in content
    7182        assert "JSON" not in content
    7183        assert "shell_exec_step_1037" not in content
    7184        assert "timeout_after_300s" not in content
    7185        assert "Unsupported/stale claim tokens to avoid until re-verified: [unsupported-stale-claim]" in content
    7186        assert "Intel Xeon E5-2690 baseline" not in content
    7187    finally:
    7188        db.close()
    7189
    7190
    7191def test_prompt_redacts_stale_tokens_from_recent_state(tmp_path):
    7192    db = AgentDB(tmp_path / "state.db")
    7193    try:
    7194        job_id = db.create_job(
    7195            "Prefer current durable evidence",
    7196            title="stale-recent-state",
    7197            kind="generic",
    7198            metadata={"unsupported_claim_tokens": ["E5-2690", "v3"]},
    7199        )
    7200        run_id = db.start_run(job_id, model="test")
    7201        step_id = db.add_step(
    7202            job_id=job_id,
    7203            run_id=run_id,
    7204            kind="tool",
    7205            tool_name="record_findings",
    7206            input_data={"arguments": {"finding": "Old CPU claim: Intel Xeon E5-2690 v3"}},
    7207        )
    7208        db.finish_step(
    7209            step_id,
    7210            status="blocked",
    7211            output_data={"success": False, "error": "evidence grounding required"},
    7212            summary="blocked record_findings; Intel Xeon E5-2690 v3 unsupported",
    7213        )
    7214        db.finish_run(run_id, "blocked")
    7215
    7216        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    7217
    7218        assert "E5-2690" not in content
    7219        assert "[unsupported-stale-claim]" in content
    7220    finally:
    7221        db.close()
    7222
    7223
    7224def test_prompt_does_not_redact_stale_tokens_inside_exact_paths(tmp_path):
    7225    db = AgentDB(tmp_path / "state.db")
    7226    try:
    7227        job_id = db.create_job(
    7228            "Validate exact candidate paths",
    7229            title="path-redaction",
    7230            kind="generic",
    7231            metadata={"unsupported_claim_tokens": ["AlphaModel-99"]},
    7232        )
    7233        db.update_job_metadata(
    7234            job_id,
    7235            {
    7236                "task_queue": [
    7237                    {
    7238                        "title": "Validate candidate file path",
    7239                        "status": "open",
    7240                        "contract": "experiment",
    7241                        "acceptance_criteria": "Exact path is validated.",
    7242                        "evidence_needed": "Shell output with file size.",
    7243                    }
    7244                ]
    7245            },
    7246        )
    7247        run_id = db.start_run(job_id, model="test")
    7248        step_id = db.add_step(
    7249            job_id=job_id,
    7250            run_id=run_id,
    7251            kind="tool",
    7252            tool_name="record_findings",
    7253            input_data={"arguments": {"finding": "Old unsupported AlphaModel-99 claim"}},
    7254        )
    7255        db.finish_step(
    7256            step_id,
    7257            status="blocked",
    7258            output_data={
    7259                "success": False,
    7260                "error": "evidence grounding required",
    7261                "evidence_grounding": {
    7262                    "missing_candidate_paths": ["/srv/models/AlphaModel-99-Q4.foo"],
    7263                    "unsupported_tokens": ["/srv/models/AlphaModel-99-Q4.foo"],
    7264                },
    7265            },
    7266            summary="blocked record_findings; AlphaModel-99 unsupported",
    7267        )
    7268        db.finish_run(run_id, "blocked")
    7269
    7270        content = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))[-1]["content"]
    7271
    7272        assert "/srv/models/AlphaModel-99-Q4.foo" in content
    7273        assert "unsupported [unsupported-stale-claim] claim" in content
    7274    finally:
    7275        db.close()
    7276
    7277
    7278def test_prompt_redacts_older_stale_tokens_from_task_queue(tmp_path):
    7279    stale_tail = [f"GPU{i}X" for i in range(60)]
    7280    job = {
    7281        "title": "stale task cleanup",
    7282        "kind": "generic",
    7283        "objective": "use current evidence",
    7284        "metadata": {
    7285            "unsupported_claim_tokens": ["E5-2690", *stale_tail],
    7286            "task_queue": [
    7287                {
    7288                    "title": "Record old baseline",
    7289                    "status": "active",
    7290                    "priority": 10,
    7291                    "goal": "Record CPU: Dual Intel Xeon E5-2690 v3 from old evidence.",
    7292                    "output_contract": "experiment",
    7293                }
    7294            ],
    7295        },
    7296    }
    7297
    7298    content = build_messages(job, [])[-1]["content"]
    7299
    7300    assert "E5-2690" not in content
    7301    assert "[unsupported-stale-claim]" in content
    7302
    7303
    7304def test_run_one_step_requires_accounting_after_auto_checkpoint_read(tmp_path):
    7305    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7306    db = AgentDB(tmp_path / "state.db")
    7307    try:
    7308        job_id = db.create_job("Convert evidence checkpoints into progress", title="checkpoint", kind="generic")
    7309        run_id = db.start_run(job_id, model="test")
    7310        checkpoint_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    7311        db.finish_step(
    7312            checkpoint_step,
    7313            status="blocked",
    7314            output_data={
    7315                "success": True,
    7316                "error": "artifact required before more research",
    7317                "auto_checkpoint": {
    7318                    "artifact_id": "art_checkpoint",
    7319                    "path": str(tmp_path / "checkpoint.md"),
    7320                    "title": "Auto Evidence Checkpoint after step 1",
    7321                    "evidence_step": "step_evidence",
    7322                    "blocked_tool": "shell_exec",
    7323                },
    7324            },
    7325        )
    7326        read_step = db.add_step(
    7327            job_id=job_id,
    7328            run_id=run_id,
    7329            kind="tool",
    7330            tool_name="read_artifact",
    7331            input_data={"arguments": {"artifact_id": "art_checkpoint"}},
    7332        )
    7333        db.finish_step(read_step, status="completed", output_data={"success": True, "content": "evidence"})
    7334        db.finish_run(run_id, "completed")
    7335
    7336        result = run_one_step(
    7337            job_id,
    7338            config=config,
    7339            db=db,
    7340            llm=ScriptedLLM([
    7341                LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "echo more discovery"})])
    7342            ]),
    7343        )
    7344
    7345        assert result.status == "blocked"
    7346        assert result.result["error"] == "evidence checkpoint accounting required"
    7347        assert result.result["blocked_tool"] == "shell_exec"
    7348    finally:
    7349        db.close()
    7350
    7351
    7352def test_run_one_step_reads_checkpoint_before_batched_branch_work(tmp_path):
    7353    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7354    db = AgentDB(tmp_path / "state.db")
    7355    try:
    7356        job_id = db.create_job("Convert evidence checkpoints into progress", title="checkpoint", kind="generic")
    7357        db.update_job_metadata(
    7358            job_id,
    7359            {
    7360                "pending_evidence_checkpoint": {
    7361                    "artifact_id": "art_checkpoint",
    7362                    "title": "Auto Evidence Checkpoint after step 1",
    7363                    "evidence_step_no": 1,
    7364                    "blocked_tool": "shell_exec",
    7365                }
    7366            },
    7367        )
    7368
    7369        result = run_one_step(
    7370            job_id,
    7371            config=config,
    7372            db=db,
    7373            llm=ScriptedLLM([
    7374                LLMResponse(tool_calls=[
    7375                    ToolCall(name="shell_exec", arguments={"command": "echo more discovery"}),
    7376                    ToolCall(name="read_artifact", arguments={"artifact_id": "art_checkpoint"}),
    7377                ])
    7378            ]),
    7379            registry=SuccessRegistry(),
    7380        )
    7381
    7382        tool_steps = [step for step in db.list_steps(job_id=job_id) if step.get("kind") == "tool"]
    7383        assert [step["tool_name"] for step in tool_steps[-2:]] == ["read_artifact", "shell_exec"]
    7384        assert result.status == "blocked"
    7385        assert result.result["error"] == "evidence checkpoint accounting required"
    7386        assert result.result["blocked_tool"] == "shell_exec"
    7387        assert result.result["checkpoint_already_read"] is True
    7388        pending = db.get_job(job_id)["metadata"]["pending_evidence_checkpoint"]
    7389        assert pending["read_at"]
    7390    finally:
    7391        db.close()
    7392
    7393
    7394def test_run_one_step_allows_checkpoint_read_when_deliverable_guard_is_active(tmp_path):
    7395    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7396    db = AgentDB(tmp_path / "state.db")
    7397    try:
    7398        job_id = db.create_job("Write a report from long research", title="report checkpoint", kind="generic")
    7399        run_id = db.start_run(job_id, model="test")
    7400        for index in range(18):
    7401            step_id = db.add_step(
    7402                job_id=job_id,
    7403                run_id=run_id,
    7404                kind="tool",
    7405                tool_name="shell_exec",
    7406                input_data={"arguments": {"command": f"ls item-{index}"}},
    7407            )
    7408            db.finish_step(step_id, status="completed", output_data={"success": True, "stdout": "evidence"})
    7409        db.finish_run(run_id, "completed")
    7410        db.update_job_metadata(
    7411            job_id,
    7412            {
    7413                "pending_evidence_checkpoint": {
    7414                    "artifact_id": "art_checkpoint",
    7415                    "title": "Auto Evidence Checkpoint after step 18",
    7416                    "evidence_step_no": 18,
    7417                    "blocked_tool": "shell_exec",
    7418                }
    7419            },
    7420        )
    7421
    7422        result = run_one_step(
    7423            job_id,
    7424            config=config,
    7425            db=db,
    7426            llm=ScriptedLLM([
    7427                LLMResponse(tool_calls=[ToolCall(name="read_artifact", arguments={"artifact_id": "art_checkpoint"})])
    7428            ]),
    7429            registry=SuccessRegistry(),
    7430        )
    7431
    7432        assert result.status == "completed"
    7433        assert result.tool_name == "read_artifact"
    7434        pending = db.get_job(job_id)["metadata"]["pending_evidence_checkpoint"]
    7435        assert pending["read_at"]
    7436        assert pending["read_step_no"] == 19
    7437    finally:
    7438        db.close()
    7439
    7440
    7441def test_run_one_step_accounts_checkpoint_before_batched_branch_work(tmp_path):
    7442    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7443    db = AgentDB(tmp_path / "state.db")
    7444    try:
    7445        job_id = db.create_job("Convert evidence checkpoints into progress", title="checkpoint", kind="generic")
    7446        db.update_job_metadata(
    7447            job_id,
    7448            {
    7449                "pending_evidence_checkpoint": {
    7450                    "artifact_id": "art_checkpoint",
    7451                    "title": "Auto Evidence Checkpoint after step 1",
    7452                    "read_at": "2026-01-01T00:00:00+00:00",
    7453                    "evidence_step_no": 1,
    7454                    "blocked_tool": "shell_exec",
    7455                }
    7456            },
    7457        )
    7458
    7459        result = run_one_step(
    7460            job_id,
    7461            config=config,
    7462            db=db,
    7463            llm=ScriptedLLM([
    7464                LLMResponse(tool_calls=[
    7465                    ToolCall(name="shell_exec", arguments={"command": "echo more discovery"}),
    7466                    ToolCall(name="record_lesson", arguments={"lesson": "checkpoint accounted", "category": "strategy"}),
    7467                ])
    7468            ]),
    7469            registry=SuccessRegistry(),
    7470        )
    7471
    7472        tool_steps = [step for step in db.list_steps(job_id=job_id) if step.get("kind") == "tool"]
    7473        assert [step["tool_name"] for step in tool_steps[-2:]] == ["record_lesson", "shell_exec"]
    7474        assert result.status == "completed"
    7475        assert result.tool_name == "shell_exec"
    7476        pending = db.get_job(job_id)["metadata"]["pending_evidence_checkpoint"]
    7477        assert pending["resolved_at"]
    7478        assert pending["resolved_by_tool"] == "record_lesson"
    7479    finally:
    7480        db.close()
    7481
    7482
    7483def test_run_one_step_treats_guard_recovery_as_checkpoint_accounting(tmp_path):
    7484    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7485    db = AgentDB(tmp_path / "state.db")
    7486    try:
    7487        job_id = db.create_job("Convert evidence checkpoints into progress", title="checkpoint", kind="generic")
    7488        run_id = db.start_run(job_id, model="test")
    7489        checkpoint_step = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    7490        db.finish_step(
    7491            checkpoint_step,
    7492            status="blocked",
    7493            output_data={
    7494                "success": True,
    7495                "error": "artifact required before more research",
    7496                "auto_checkpoint": {
    7497                    "artifact_id": "art_checkpoint",
    7498                    "path": str(tmp_path / "checkpoint.md"),
    7499                    "title": "Auto Evidence Checkpoint after step 1",
    7500                    "evidence_step": "step_evidence",
    7501                    "blocked_tool": "shell_exec",
    7502                },
    7503            },
    7504        )
    7505        read_step = db.add_step(
    7506            job_id=job_id,
    7507            run_id=run_id,
    7508            kind="tool",
    7509            tool_name="read_artifact",
    7510            input_data={"arguments": {"artifact_id": "art_checkpoint"}},
    7511        )
    7512        db.finish_step(read_step, status="completed", output_data={"success": True, "content": "evidence"})
    7513        guard_step = db.add_step(
    7514            job_id=job_id,
    7515            run_id=run_id,
    7516            kind="tool",
    7517            tool_name="read_artifact",
    7518            input_data={"arguments": {"artifact_id": "art_checkpoint"}},
    7519        )
    7520        db.finish_step(
    7521            guard_step,
    7522            status="blocked",
    7523            output_data={
    7524                "success": True,
    7525                "recoverable": True,
    7526                "error": "evidence checkpoint accounting required",
    7527                "pending_evidence_checkpoint": {
    7528                    "artifact_id": "art_checkpoint",
    7529                    "title": "Auto Evidence Checkpoint after step 1",
    7530                    "checkpoint_read": True,
    7531                },
    7532            },
    7533        )
    7534        recovery_step = db.add_step(job_id=job_id, run_id=run_id, kind="recovery", tool_name="guard_recovery")
    7535        db.finish_step(
    7536            recovery_step,
    7537            status="completed",
    7538            output_data={
    7539                "success": True,
    7540                "lesson": {"lesson": "Open a task after repeated guard blocks."},
    7541                "task": {"title": "Resolve guard"},
    7542            },
    7543        )
    7544        db.finish_run(run_id, "completed")
    7545
    7546        result = run_one_step(
    7547            job_id,
    7548            config=config,
    7549            db=db,
    7550            llm=ScriptedLLM([
    7551                LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "echo more discovery"})])
    7552            ]),
    7553            registry=SuccessRegistry(),
    7554        )
    7555
    7556        assert result.status == "completed"
    7557        assert result.tool_name == "shell_exec"
    7558    finally:
    7559        db.close()
    7560
    7561
    7562def test_checkpoint_resolution_tool_bypasses_measured_progress_guard(tmp_path):
    7563    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7564    db = AgentDB(tmp_path / "state.db")
    7565    try:
    7566        job_id = db.create_job("Optimize benchmark speed", title="checkpoint-measure", kind="generic")
    7567        db.update_job_metadata(
    7568            job_id,
    7569            {
    7570                "pending_evidence_checkpoint": {
    7571                    "artifact_id": "art_checkpoint",
    7572                    "title": "Auto Evidence Checkpoint after step 1",
    7573                    "read_at": "2026-01-01T00:00:00+00:00",
    7574                    "evidence_step_no": 1,
    7575                    "blocked_tool": "shell_exec",
    7576                }
    7577            },
    7578        )
    7579        run_id = db.start_run(job_id, model="fake")
    7580        for index in range(18):
    7581            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    7582            db.finish_step(step_id, status="completed", output_data={"success": True, "stdout": f"probe {index}"})
    7583        db.finish_run(run_id, "completed")
    7584
    7585        result = run_one_step(
    7586            job_id,
    7587            config=config,
    7588            db=db,
    7589            llm=ScriptedLLM([
    7590                LLMResponse(tool_calls=[
    7591                    ToolCall(
    7592                        name="record_source",
    7593                        arguments={
    7594                            "source": "file:///tmp/checkpoint",
    7595                            "source_type": "checkpoint",
    7596                            "outcome": "checkpoint accounted before more benchmark work",
    7597                        },
    7598                    )
    7599                ])
    7600            ]),
    7601        )
    7602
    7603        assert result.status == "completed"
    7604        assert result.tool_name == "record_source"
    7605        pending = db.get_job(job_id)["metadata"]["pending_evidence_checkpoint"]
    7606        assert pending["resolved_by_tool"] == "record_source"
    7607    finally:
    7608        db.close()
    7609
    7610
    7611def test_run_one_step_persists_checkpoint_obligation_until_accounted(tmp_path):
    7612    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7613    db = AgentDB(tmp_path / "state.db")
    7614    try:
    7615        job_id = db.create_job("Convert evidence checkpoints into progress", title="checkpoint", kind="generic")
    7616        db.update_job_metadata(
    7617            job_id,
    7618            {
    7619                "pending_evidence_checkpoint": {
    7620                    "artifact_id": "art_checkpoint",
    7621                    "title": "Auto Evidence Checkpoint after step 1",
    7622                    "path": str(tmp_path / "checkpoint.md"),
    7623                    "created_at": "2026-01-01T00:00:00+00:00",
    7624                    "evidence_step": "step_evidence",
    7625                    "evidence_step_no": 1,
    7626                    "blocked_tool": "shell_exec",
    7627                }
    7628            },
    7629        )
    7630
    7631        blocked = run_one_step(
    7632            job_id,
    7633            config=config,
    7634            db=db,
    7635            llm=ScriptedLLM([
    7636                LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "echo more discovery"})])
    7637            ]),
    7638        )
    7639        assert blocked.status == "blocked"
    7640        assert blocked.result["error"] == "evidence checkpoint accounting required"
    7641
    7642        accounted = run_one_step(
    7643            job_id,
    7644            config=config,
    7645            db=db,
    7646            llm=ScriptedLLM([
    7647                LLMResponse(tool_calls=[
    7648                    ToolCall(
    7649                        name="record_lesson",
    7650                        arguments={
    7651                            "lesson": "The checkpoint contains only diagnostic setup evidence; record it and move to the next concrete branch.",
    7652                            "category": "strategy",
    7653                        },
    7654                    )
    7655                ])
    7656            ]),
    7657        )
    7658        assert accounted.status == "completed"
    7659        pending = db.get_job(job_id)["metadata"]["pending_evidence_checkpoint"]
    7660        assert pending["resolved_at"]
    7661        assert pending["resolved_by_tool"] == "record_lesson"
    7662    finally:
    7663        db.close()
    7664
    7665
    7666def test_run_one_step_blocks_branch_work_when_memory_graph_needs_consolidation(tmp_path):
    7667    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7668    db = AgentDB(tmp_path / "state.db")
    7669    try:
    7670        job_id = db.create_job("Keep improving durable work", title="memory")
    7671        db.append_lesson(job_id, "Use validated checkpoints.", category="strategy")
    7672        db.append_lesson(job_id, "Reject low-yield branches.", category="strategy")
    7673        db.append_finding_record(job_id, name="Finding A")
    7674        db.append_finding_record(job_id, name="Finding B")
    7675        db.append_source_record(job_id, "source:a")
    7676        db.append_experiment_record(job_id, title="Trial", status="measured")
    7677        llm = ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "more research"})])])
    7678
    7679        result = run_one_step(job_id, config=config, db=db, llm=llm)
    7680
    7681        assert result.status == "blocked"
    7682        assert result.result["error"] == "memory graph consolidation required"
    7683        assert result.result["blocked_tool"] == "web_search"
    7684    finally:
    7685        db.close()
    7686
    7687
    7688def test_run_one_step_allows_memory_graph_consolidation_when_guard_is_active(tmp_path):
    7689    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7690    db = AgentDB(tmp_path / "state.db")
    7691    try:
    7692        job_id = db.create_job("Keep improving durable work", title="memory")
    7693        db.append_lesson(job_id, "Use validated checkpoints.", category="strategy")
    7694        db.append_lesson(job_id, "Reject low-yield branches.", category="strategy")
    7695        db.append_finding_record(job_id, name="Finding A")
    7696        db.append_finding_record(job_id, name="Finding B")
    7697        db.append_source_record(job_id, "source:a")
    7698        db.append_experiment_record(job_id, title="Trial", status="measured")
    7699        llm = ScriptedLLM([
    7700            LLMResponse(
    7701                tool_calls=[
    7702                    ToolCall(
    7703                        name="record_memory_graph",
    7704                        arguments={
    7705                            "nodes": [
    7706                                {
    7707                                    "key": "validated-checkpoints",
    7708                                    "kind": "strategy",
    7709                                    "title": "Validated checkpoints",
    7710                                    "summary": "Use measured checkpoints to decide the next branch.",
    7711                                    "salience": 0.9,
    7712                                }
    7713                            ]
    7714                        },
    7715                    )
    7716                ]
    7717            )
    7718        ])
    7719
    7720        result = run_one_step(job_id, config=config, db=db, llm=llm)
    7721
    7722        assert result.status == "completed"
    7723        assert result.tool_name == "record_memory_graph"
    7724        graph = db.get_job(job_id)["metadata"]["memory_graph"]
    7725        assert graph["nodes"][0]["key"] == "validated-checkpoints"
    7726    finally:
    7727        db.close()
    7728
    7729
    7730def test_prompt_adds_lesson_consolidation_guard_when_raw_lessons_sprawl():
    7731    job = {
    7732        "title": "lesson sprawl",
    7733        "kind": "generic",
    7734        "objective": "keep improving a long-running job",
    7735        "metadata": {
    7736            "lessons": [
    7737                {"lesson": f"Reusable lesson {index}", "category": "strategy"}
    7738                for index in range(30)
    7739            ],
    7740        },
    7741    }
    7742    steps = [
    7743        {
    7744            "step_no": index,
    7745            "kind": "tool",
    7746            "tool_name": "record_lesson",
    7747            "status": "completed",
    7748            "summary": f"lesson {index}",
    7749        }
    7750        for index in range(1, 4)
    7751    ]
    7752
    7753    content = build_messages(job, steps)[-1]["content"]
    7754
    7755    assert "Lesson consolidation guard:" in content
    7756    assert "Raw lessons are accumulating faster than consolidated memory" in content
    7757    assert "lessons=30" in content
    7758    assert "record_memory_graph" in content
    7759
    7760
    7761def test_run_one_step_blocks_more_lessons_when_lesson_sprawl_needs_graph(tmp_path):
    7762    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7763    db = AgentDB(tmp_path / "state.db")
    7764    try:
    7765        job_id = db.create_job("Keep improving durable work", title="lesson-sprawl")
    7766        for index in range(30):
    7767            db.append_lesson(job_id, f"Reusable lesson {index}", category="strategy")
    7768        run_id = db.start_run(job_id, model="test")
    7769        for index in range(3):
    7770            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_lesson")
    7771            db.finish_step(step_id, status="completed", output_data={"success": True, "lesson": {"lesson": f"recent {index}"}})
    7772        db.finish_run(run_id, "completed")
    7773
    7774        result = run_one_step(
    7775            job_id,
    7776            config=config,
    7777            db=db,
    7778            llm=ScriptedLLM([
    7779                LLMResponse(tool_calls=[ToolCall(name="record_lesson", arguments={"lesson": "one more lesson"})])
    7780            ]),
    7781        )
    7782
    7783        assert result.status == "blocked"
    7784        assert result.result["error"] == "lesson consolidation required"
    7785        assert result.result["lesson_consolidation"]["lessons"] == 30
    7786        assert result.result["blocked_tool"] == "record_lesson"
    7787    finally:
    7788        db.close()
    7789
    7790
    7791def test_run_one_step_allows_memory_graph_when_lesson_sprawl_is_active(tmp_path):
    7792    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    7793    db = AgentDB(tmp_path / "state.db")
    7794    try:
    7795        job_id = db.create_job("Keep improving durable work", title="lesson-sprawl")
    7796        for index in range(30):
    7797            db.append_lesson(job_id, f"Reusable lesson {index}", category="strategy")
    7798        run_id = db.start_run(job_id, model="test")
    7799        for index in range(3):
    7800            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_lesson")
    7801            db.finish_step(step_id, status="completed", output_data={"success": True, "lesson": {"lesson": f"recent {index}"}})
    7802        db.finish_run(run_id, "completed")
    7803
    7804        result = run_one_step(
    7805            job_id,
    7806            config=config,
    7807            db=db,
    7808            llm=ScriptedLLM([
    7809                LLMResponse(
    7810                    tool_calls=[
    7811                        ToolCall(
    7812                            name="record_memory_graph",
    7813                            arguments={
    7814                                "nodes": [
    7815                                    {
    7816                                        "key": "lesson-sprawl-strategy",
    7817                                        "kind": "strategy",
    7818                                        "title": "Consolidate repeated lessons",
    7819                                        "summary": "Compress repeated lessons into graph memory before adding more.",
    7820                                    }
    7821                                ]
    7822                            },
    7823                        )
    7824                    ]
    7825                )
    7826            ]),
    7827        )
    7828
    7829        assert result.status == "completed"
    7830        assert result.tool_name == "record_memory_graph"
    7831    finally:
    7832        db.close()
    7833
    7834
    7835def test_prompt_includes_activity_stagnation_context():
    7836    job = {
    7837        "title": "research",
    7838        "kind": "generic",
    7839        "objective": "keep making durable progress",
    7840        "metadata": {
    7841            "activity_checkpoint_streak": 3,
    7842            "last_checkpoint_counts": {
    7843                "findings": 1,
    7844                "sources": 2,
    7845                "tasks": 4,
    7846                "experiments": 0,
    7847                "lessons": 1,
    7848                "milestones": 0,
    7849            },
    7850        },
    7851    }
    7852
    7853    content = build_messages(job, [])[-1]["content"]
    7854
    7855    assert "Activity stagnation" in content
    7856    assert "activity_checkpoint_streak=3" in content
    7857    assert "Recent checkpoints show activity without durable progress" in content
    7858
    7859
    7860def test_prompt_includes_task_planning_guard_context():
    7861    job = {
    7862        "title": "research",
    7863        "kind": "generic",
    7864        "objective": "keep making durable progress",
    7865        "metadata": {
    7866            "task_planning_checkpoint_streak": 2,
    7867            "task_queue": [
    7868                {"title": "Plan branch", "status": "open"},
    7869                {"title": "Executed branch", "status": "done"},
    7870            ],
    7871        },
    7872    }
    7873
    7874    content = build_messages(job, [])[-1]["content"]
    7875
    7876    assert "Task planning guard" in content
    7877    assert "task_only_checkpoints=2" in content
    7878    assert "Do not create more new open tasks next" in content
    7879
    7880
    7881def test_prompt_includes_durable_yield_pressure():
    7882    job = {
    7883        "title": "research",
    7884        "kind": "generic",
    7885        "objective": "keep making durable progress",
    7886        "metadata": {},
    7887    }
    7888    steps = [
    7889        {
    7890            "step_no": index,
    7891            "kind": "tool",
    7892            "status": "completed",
    7893            "tool_name": "web_search",
    7894            "summary": f"search {index}",
    7895        }
    7896        for index in range(1, 31)
    7897    ]
    7898
    7899    content = build_messages(job, steps)[-1]["content"]
    7900
    7901    assert "Durable progress yield" in content
    7902    assert "No durable progress records after 30 completed actions" in content
    7903    assert "record findings/source/experiment/lesson/roadmap progress" in content
    7904
    7905
    7906def test_prompt_includes_finding_source_ledgers_and_reflections():
    7907    job = {
    7908        "title": "research",
    7909        "kind": "generic",
    7910        "objective": "find research",
    7911        "metadata": {
    7912            "finding_ledger": [{"name": "Acme Finding", "category": "example category", "location": "Toronto", "score": 0.8}],
    7913            "task_queue": [{"title": "Explore primary sources", "status": "open", "priority": 5, "goal": "Find evidence"}],
    7914            "source_ledger": [{"source": "https://example.com", "source_type": "web_source", "usefulness_score": 0.9, "yield_count": 3}],
    7915            "reflections": [{"summary": "Primary source map is working", "strategy": "Try archival sources next"}],
    7916        },
    7917    }
    7918
    7919    messages = build_messages(job, [])
    7920
    7921    content = messages[-1]["content"]
    7922    assert "Finding ledger: 1 unique candidates." in content
    7923    assert "Acme Finding" in content
    7924    assert "Explore primary sources" in content
    7925    assert "https://example.com" in content
    7926    assert "Primary source map is working" in content
    7927
    7928
    7929def test_prompt_includes_experiment_ledger_and_best_result():
    7930    job = {
    7931        "title": "improve process",
    7932        "kind": "generic",
    7933        "objective": "make a measurable process better",
    7934        "metadata": {
    7935            "experiment_ledger": [
    7936                {
    7937                    "title": "variant a",
    7938                    "status": "measured",
    7939                    "metric_name": "score",
    7940                    "metric_value": 2.0,
    7941                    "metric_unit": "units",
    7942                    "higher_is_better": True,
    7943                    "result": "baseline",
    7944                    "best_observed": False,
    7945                },
    7946                {
    7947                    "title": "variant b",
    7948                    "status": "measured",
    7949                    "metric_name": "score",
    7950                    "metric_value": 3.5,
    7951                    "metric_unit": "units",
    7952                    "higher_is_better": True,
    7953                    "result": "better",
    7954                    "next_action": "try another independent variant",
    7955                    "best_observed": True,
    7956                },
    7957            ],
    7958        },
    7959    }
    7960
    7961    messages = build_messages(job, [])
    7962
    7963    content = messages[-1]["content"]
    7964    assert "Experiment ledger:" in content
    7965    assert "Best observed results:" in content
    7966    assert "variant b" in content
    7967    assert "score=3.5 units" in content
    7968    assert "Next-action constraint:" in content
    7969    assert "latest measured experiment selected a concrete next action" in content
    7970    assert "try another independent variant" in content
    7971
    7972
    7973def _stagnant_experiments():
    7974    return [
    7975        {
    7976            "title": "best variant",
    7977            "status": "measured",
    7978            "metric_name": "score",
    7979            "metric_value": 10.0,
    7980            "metric_unit": "units",
    7981            "higher_is_better": True,
    7982            "best_observed": True,
    7983            "next_action": "try a materially different branch",
    7984        },
    7985        *[
    7986            {
    7987                "title": f"flat variant {index}",
    7988                "status": "measured",
    7989                "metric_name": "score",
    7990                "metric_value": 8.0 + index * 0.1,
    7991                "metric_unit": "units",
    7992                "higher_is_better": True,
    7993                "best_observed": False,
    7994                "next_action": "try another small variant",
    7995            }
    7996            for index in range(1, 6)
    7997        ],
    7998    ]
    7999
    8000
    8001def _stagnant_experiment_metadata():
    8002    return {
    8003        "experiment_ledger": _stagnant_experiments(),
    8004        "memory_graph": {
    8005            "nodes": [
    8006                {"key": "best-variant", "kind": "decision", "title": "Best measured variant"},
    8007                {"key": "stagnant-branch", "kind": "strategy", "title": "Stagnant branch should pivot"},
    8008            ]
    8009        },
    8010    }
    8011
    8012
    8013def test_prompt_includes_experiment_stagnation_guard():
    8014    job = {
    8015        "title": "improve measured process",
    8016        "kind": "generic",
    8017        "objective": "optimize throughput and keep improving",
    8018        "metadata": _stagnant_experiment_metadata(),
    8019    }
    8020
    8021    content = build_messages(job, [])[-1]["content"]
    8022
    8023    assert "Experiment stagnation guard:" in content
    8024    assert "Recent measured trials have not improved" in content
    8025    assert "best=10.0" in content
    8026    assert "non_improving=5" in content
    8027
    8028
    8029def test_prompt_infers_experiment_stagnation_from_metric_direction():
    8030    job = {
    8031        "title": "improve measured process",
    8032        "kind": "generic",
    8033        "objective": "reduce latency and keep improving",
    8034        "metadata": {
    8035            "experiment_ledger": [
    8036                {
    8037                    "title": "best latency",
    8038                    "status": "measured",
    8039                    "metric_name": "latency",
    8040                    "metric_value": 1.0,
    8041                    "metric_unit": "s",
    8042                    "higher_is_better": False,
    8043                },
    8044                *[
    8045                    {
    8046                        "title": f"slower variant {index}",
    8047                        "status": "measured",
    8048                        "metric_name": "latency",
    8049                        "metric_value": 1.0 + index * 0.1,
    8050                        "metric_unit": "s",
    8051                        "higher_is_better": False,
    8052                    }
    8053                    for index in range(1, 6)
    8054                ],
    8055            ],
    8056            "memory_graph": {
    8057                "nodes": [
    8058                    {"key": "latency-best", "kind": "decision", "title": "Best latency"},
    8059                    {"key": "latency-pivot", "kind": "strategy", "title": "Pivot stagnant latency branch"},
    8060                ]
    8061            },
    8062        },
    8063    }
    8064
    8065    content = build_messages(job, [])[-1]["content"]
    8066
    8067    assert "Experiment stagnation guard:" in content
    8068    assert "best=1.0" in content
    8069    assert "latest=1.5" in content
    8070    assert "Recent measured trials have not improved" in content
    8071
    8072
    8073def test_prompt_does_not_treat_unmarked_improvements_as_stagnation():
    8074    job = {
    8075        "title": "improve measured process",
    8076        "kind": "generic",
    8077        "objective": "increase score and keep improving",
    8078        "metadata": {
    8079            "experiment_ledger": [
    8080                {
    8081                    "title": f"better variant {index}",
    8082                    "status": "measured",
    8083                    "metric_name": "score",
    8084                    "metric_value": float(index),
    8085                    "metric_unit": "points",
    8086                    "higher_is_better": True,
    8087                    "best_observed": False,
    8088                }
    8089                for index in range(1, 7)
    8090            ],
    8091            "memory_graph": {
    8092                "nodes": [
    8093                    {"key": "score-progress", "kind": "decision", "title": "Score is improving"},
    8094                    {"key": "score-next", "kind": "strategy", "title": "Continue measured branch"},
    8095                ]
    8096            },
    8097        },
    8098    }
    8099
    8100    content = build_messages(job, [])[-1]["content"]
    8101
    8102    assert "Experiment stagnation guard:" in content
    8103    assert "Recent measured trials have not improved" not in content
    8104
    8105
    8106def test_run_one_step_blocks_branch_work_after_experiment_stagnation(tmp_path):
    8107    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8108    db = AgentDB(tmp_path / "state.db")
    8109    try:
    8110        job_id = db.create_job(
    8111            "Optimize a measurable process and keep improving",
    8112            title="experiment-stagnation",
    8113            kind="generic",
    8114            metadata=_stagnant_experiment_metadata(),
    8115        )
    8116        run_id = db.start_run(job_id, model="test")
    8117        for index in range(6):
    8118            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    8119            db.finish_step(step_id, status="completed", output_data={"success": True, "experiment": {"title": f"trial {index}"}})
    8120        db.finish_run(run_id, "completed")
    8121
    8122        result = run_one_step(
    8123            job_id,
    8124            config=config,
    8125            db=db,
    8126            llm=ScriptedLLM([
    8127                LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "python run_next_trial.py"})])
    8128            ]),
    8129        )
    8130
    8131        assert result.status == "blocked"
    8132        assert result.result["error"] == "experiment stagnation decision required"
    8133        assert result.result["blocked_tool"] == "shell_exec"
    8134        assert result.result["experiment_stagnation"]["non_improving_count"] == 5
    8135    finally:
    8136        db.close()
    8137
    8138
    8139def test_run_one_step_allows_branch_decision_after_experiment_stagnation(tmp_path):
    8140    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8141    db = AgentDB(tmp_path / "state.db")
    8142    try:
    8143        job_id = db.create_job(
    8144            "Optimize a measurable process and keep improving",
    8145            title="experiment-stagnation",
    8146            kind="generic",
    8147            metadata=_stagnant_experiment_metadata(),
    8148        )
    8149        run_id = db.start_run(job_id, model="test")
    8150        for index in range(6):
    8151            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    8152            db.finish_step(step_id, status="completed", output_data={"success": True, "experiment": {"title": f"trial {index}"}})
    8153        db.finish_run(run_id, "completed")
    8154
    8155        result = run_one_step(
    8156            job_id,
    8157            config=config,
    8158            db=db,
    8159            llm=ScriptedLLM([
    8160                LLMResponse(tool_calls=[
    8161                    ToolCall(
    8162                        name="record_tasks",
    8163                        arguments={
    8164                            "tasks": [
    8165                                {
    8166                                    "title": "Pivot away from stagnant measured branch",
    8167                                    "status": "open",
    8168                                    "output_contract": "decision",
    8169                                    "acceptance_criteria": "A materially different branch is selected.",
    8170                                }
    8171                            ]
    8172                        },
    8173                    )
    8174                ])
    8175            ]),
    8176        )
    8177
    8178        assert result.status == "completed"
    8179        assert result.tool_name == "record_tasks"
    8180    finally:
    8181        db.close()
    8182
    8183
    8184def test_run_one_step_allows_blocked_experiment_after_experiment_stagnation(tmp_path):
    8185    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8186    db = AgentDB(tmp_path / "state.db")
    8187    try:
    8188        job_id = db.create_job(
    8189            "Optimize a measurable process and keep improving",
    8190            title="experiment-stagnation",
    8191            kind="generic",
    8192            metadata=_stagnant_experiment_metadata(),
    8193        )
    8194        run_id = db.start_run(job_id, model="test")
    8195        for index in range(6):
    8196            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_experiment")
    8197            db.finish_step(step_id, status="completed", output_data={"success": True, "experiment": {"title": f"trial {index}"}})
    8198        db.finish_run(run_id, "completed")
    8199
    8200        result = run_one_step(
    8201            job_id,
    8202            config=config,
    8203            db=db,
    8204            llm=ScriptedLLM([
    8205                LLMResponse(tool_calls=[
    8206                    ToolCall(
    8207                        name="record_experiment",
    8208                        arguments={
    8209                            "title": "Stagnant branch decision",
    8210                            "status": "blocked",
    8211                            "metric_name": "score",
    8212                            "metric_unit": "units",
    8213                            "result": "recent trials did not improve the objective",
    8214                            "next_action": "pivot to a materially different branch",
    8215                        },
    8216                    )
    8217                ])
    8218            ]),
    8219        )
    8220
    8221        assert result.status == "completed"
    8222        assert result.tool_name == "record_experiment"
    8223    finally:
    8224        db.close()
    8225
    8226
    8227def test_delivery_experiment_next_action_blocks_unrelated_research(tmp_path):
    8228    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8229    db = AgentDB(tmp_path / "state.db")
    8230    try:
    8231        job_id = db.create_job("Improve a generic deliverable", title="deliverable", kind="generic")
    8232        db.update_job_metadata(job_id, {
    8233            "experiment_ledger": [{
    8234                "title": "deliverable gap",
    8235                "status": "measured",
    8236                "metric_name": "coverage",
    8237                "metric_value": 0.25,
    8238                "metric_unit": "ratio",
    8239                "next_action": "merge the measured output into the deliverable file",
    8240            }],
    8241        })
    8242
    8243        result = run_one_step(
    8244            job_id,
    8245            config=config,
    8246            db=db,
    8247            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "more background"})])]),
    8248            registry=SuccessRegistry(),
    8249        )
    8250
    8251        assert result.status == "blocked"
    8252        assert result.result["error"] == "experiment next action pending"
    8253        assert "merge the measured output" in result.result["experiment_next_action"]["next_action"]
    8254    finally:
    8255        db.close()
    8256
    8257
    8258def test_research_experiment_next_action_allows_research(tmp_path):
    8259    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8260    db = AgentDB(tmp_path / "state.db")
    8261    try:
    8262        job_id = db.create_job("Improve a generic deliverable", title="deliverable", kind="generic")
    8263        db.update_job_metadata(job_id, {
    8264            "experiment_ledger": [{
    8265                "title": "source gap",
    8266                "status": "measured",
    8267                "metric_name": "coverage",
    8268                "metric_value": 0.25,
    8269                "metric_unit": "ratio",
    8270                "next_action": "search for additional independent sources",
    8271            }],
    8272        })
    8273
    8274        result = run_one_step(
    8275            job_id,
    8276            config=config,
    8277            db=db,
    8278            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "more background"})])]),
    8279            registry=SuccessRegistry(),
    8280        )
    8281
    8282        assert result.status == "completed"
    8283        assert result.tool_name == "web_search"
    8284    finally:
    8285        db.close()
    8286
    8287
    8288def test_delivery_experiment_next_action_blocks_read_only_shell(tmp_path):
    8289    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8290    db = AgentDB(tmp_path / "state.db")
    8291    try:
    8292        job_id = db.create_job("Improve a generic deliverable", title="deliverable", kind="generic")
    8293        db.update_job_metadata(job_id, {
    8294            "experiment_ledger": [{
    8295                "title": "deliverable gap",
    8296                "status": "measured",
    8297                "metric_name": "coverage",
    8298                "metric_value": 0.25,
    8299                "metric_unit": "ratio",
    8300                "next_action": "merge the measured output into the deliverable file",
    8301            }],
    8302        })
    8303
    8304        result = run_one_step(
    8305            job_id,
    8306            config=config,
    8307            db=db,
    8308            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "cat output.txt 2>/dev/null"})])]),
    8309            registry=SuccessRegistry(),
    8310        )
    8311
    8312        assert result.status == "blocked"
    8313        assert result.result["error"] == "experiment next action pending"
    8314    finally:
    8315        db.close()
    8316
    8317
    8318def test_delivery_experiment_next_action_allows_bounded_verification_shell(tmp_path):
    8319    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8320    db = AgentDB(tmp_path / "state.db")
    8321    try:
    8322        job_id = db.create_job("Improve a generic runtime", title="runtime", kind="generic")
    8323        db.update_job_metadata(job_id, {
    8324            "experiment_ledger": [{
    8325                "title": "runtime gap",
    8326                "status": "measured",
    8327                "metric_name": "valid_files",
    8328                "metric_value": 1,
    8329                "metric_unit": "files",
    8330                "next_action": "build runner binary then run benchmark with validated file",
    8331            }],
    8332        })
    8333
    8334        result = run_one_step(
    8335            job_id,
    8336            config=config,
    8337            db=db,
    8338            llm=ScriptedLLM([LLMResponse(tool_calls=[
    8339                ToolCall(name="shell_exec", arguments={"command": "ls build/bin/runner 2>/dev/null || command -v runner"})
    8340            ])]),
    8341            registry=SuccessRegistry(),
    8342        )
    8343
    8344        assert result.status == "completed"
    8345        assert result.tool_name == "shell_exec"
    8346    finally:
    8347        db.close()
    8348
    8349
    8350def test_failed_next_action_requires_accounting_before_more_shell(tmp_path):
    8351    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8352    db = AgentDB(tmp_path / "state.db")
    8353    try:
    8354        job_id = db.create_job("Improve a generic runtime", title="runtime", kind="generic")
    8355        db.update_job_metadata(job_id, {
    8356            "experiment_ledger": [{
    8357                "title": "runtime gap",
    8358                "status": "measured",
    8359                "metric_name": "valid_files",
    8360                "metric_value": 1,
    8361                "metric_unit": "files",
    8362                "next_action": "build runner binary then run benchmark with validated file",
    8363            }],
    8364        })
    8365        run_id = db.start_run(job_id, model="test")
    8366        step_id = db.add_step(
    8367            job_id=job_id,
    8368            run_id=run_id,
    8369            kind="tool",
    8370            tool_name="shell_exec",
    8371            input_data={"arguments": {"command": "cd /tmp/runtime && mkdir -p build && build-tool .."}},
    8372        )
    8373        db.finish_step(
    8374            step_id,
    8375            status="completed",
    8376            output_data={
    8377                "success": True,
    8378                "returncode": 0,
    8379                "stdout": "/bin/sh: 1: build-tool: not found\n",
    8380                "stderr": "",
    8381            },
    8382        )
    8383        db.finish_run(run_id, "completed")
    8384
    8385        result = run_one_step(
    8386            job_id,
    8387            config=config,
    8388            db=db,
    8389            llm=ScriptedLLM([LLMResponse(tool_calls=[
    8390                ToolCall(name="shell_exec", arguments={"command": "ls /tmp/runtime/build/bin/runner 2>&1"})
    8391            ])]),
    8392            registry=SuccessRegistry(),
    8393        )
    8394
    8395        assert result.status == "blocked"
    8396        assert result.result["error"] == "action result accounting required"
    8397        assert result.result["action_failure"]["step_no"] == 1
    8398        assert result.result["action_failure"]["missing_commands"] == ["build-tool"]
    8399        assert "build-tool: not found" in result.result["action_failure"]["excerpt"]
    8400    finally:
    8401        db.close()
    8402
    8403
    8404def test_failed_next_action_prompt_prioritizes_accounting(tmp_path):
    8405    db = AgentDB(tmp_path / "state.db")
    8406    try:
    8407        job_id = db.create_job("Improve a generic runtime", title="runtime", kind="generic")
    8408        db.update_job_metadata(job_id, {
    8409            "experiment_ledger": [{
    8410                "title": "runtime gap",
    8411                "status": "measured",
    8412                "metric_name": "valid_files",
    8413                "metric_value": 1,
    8414                "metric_unit": "files",
    8415                "next_action": "build runner binary then run benchmark with validated file",
    8416            }],
    8417        })
    8418        run_id = db.start_run(job_id, model="test")
    8419        step_id = db.add_step(
    8420            job_id=job_id,
    8421            run_id=run_id,
    8422            kind="tool",
    8423            tool_name="shell_exec",
    8424            input_data={"arguments": {"command": "cd /tmp/runtime && build-tool .."}},
    8425        )
    8426        db.finish_step(
    8427            step_id,
    8428            status="failed",
    8429            output_data={
    8430                "success": False,
    8431                "returncode": 0,
    8432                "stdout": "/bin/sh: 1: build-tool: not found\n",
    8433                "error": "command output indicates missing command despite exit status 0",
    8434            },
    8435        )
    8436        db.finish_run(run_id, "completed")
    8437
    8438        messages = build_messages(db.get_job(job_id), db.list_steps(job_id=job_id))
    8439        prompt = messages[-1]["content"]
    8440
    8441        assert "latest experiment next action was attempted" in prompt
    8442        assert "Missing commands: build-tool" in prompt
    8443        assert "record_experiment" in prompt
    8444        assert "build-tool: not found" in prompt
    8445    finally:
    8446        db.close()
    8447
    8448
    8449def test_failed_next_action_narrows_available_tools_to_accounting(tmp_path):
    8450    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8451    db = AgentDB(tmp_path / "state.db")
    8452    try:
    8453        job_id = db.create_job("Improve a generic runtime", title="runtime", kind="generic")
    8454        db.update_job_metadata(job_id, {
    8455            "experiment_ledger": [{
    8456                "title": "runtime gap",
    8457                "status": "measured",
    8458                "metric_name": "valid_files",
    8459                "metric_value": 1,
    8460                "metric_unit": "files",
    8461                "next_action": "build runner binary then run benchmark with validated file",
    8462            }],
    8463        })
    8464        run_id = db.start_run(job_id, model="test")
    8465        step_id = db.add_step(
    8466            job_id=job_id,
    8467            run_id=run_id,
    8468            kind="tool",
    8469            tool_name="shell_exec",
    8470            input_data={"arguments": {"command": "cd /tmp/runtime && build-tool .."}},
    8471        )
    8472        db.finish_step(
    8473            step_id,
    8474            status="completed",
    8475            output_data={"success": True, "returncode": 0, "stdout": "/bin/sh: 1: build-tool: not found\n"},
    8476        )
    8477        db.finish_run(run_id, "completed")
    8478        llm = CapturingLLM(
    8479            LLMResponse(tool_calls=[ToolCall(name="record_lesson", arguments={"lesson": "missing build tool"})])
    8480        )
    8481
    8482        run_one_step(job_id, config=config, db=db, llm=llm)
    8483
    8484        tool_names = {tool["function"]["name"] for tool in llm.tools}
    8485        assert {"record_experiment", "record_lesson", "record_tasks"}.issubset(tool_names)
    8486        assert "shell_exec" not in tool_names
    8487        assert "web_search" not in tool_names
    8488        assert "write_artifact" not in tool_names
    8489    finally:
    8490        db.close()
    8491
    8492
    8493def test_accounted_next_action_failure_does_not_keep_blocking(tmp_path):
    8494    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8495    db = AgentDB(tmp_path / "state.db")
    8496    try:
    8497        job_id = db.create_job("Improve a generic runtime", title="runtime", kind="generic")
    8498        db.update_job_metadata(job_id, {
    8499            "experiment_ledger": [{
    8500                "title": "runtime gap",
    8501                "status": "measured",
    8502                "metric_name": "valid_files",
    8503                "metric_value": 1,
    8504                "metric_unit": "files",
    8505                "next_action": "build runner binary then run benchmark with validated file",
    8506            }],
    8507        })
    8508        run_id = db.start_run(job_id, model="test")
    8509        failed_step_id = db.add_step(
    8510            job_id=job_id,
    8511            run_id=run_id,
    8512            kind="tool",
    8513            tool_name="shell_exec",
    8514            input_data={"arguments": {"command": "cd /tmp/runtime && build-tool .."}},
    8515        )
    8516        db.finish_step(
    8517            failed_step_id,
    8518            status="completed",
    8519            output_data={"success": True, "returncode": 0, "stdout": "/bin/sh: 1: build-tool: not found\n"},
    8520        )
    8521        accounted_step_id = db.add_step(
    8522            job_id=job_id,
    8523            run_id=run_id,
    8524            kind="tool",
    8525            tool_name="record_experiment",
    8526            input_data={"arguments": {"title": "build failed"}},
    8527        )
    8528        db.finish_step(
    8529            accounted_step_id,
    8530            status="completed",
    8531            output_data={"success": True, "experiment": {"title": "build failed", "status": "failed"}},
    8532        )
    8533        db.finish_run(run_id, "completed")
    8534
    8535        result = run_one_step(
    8536            job_id,
    8537            config=config,
    8538            db=db,
    8539            llm=ScriptedLLM([LLMResponse(tool_calls=[
    8540                ToolCall(name="shell_exec", arguments={"command": "printf updated > /tmp/runtime/recovery-plan.txt"})
    8541            ])]),
    8542            registry=SuccessRegistry(),
    8543        )
    8544
    8545        assert result.status == "completed"
    8546        assert result.tool_name == "shell_exec"
    8547    finally:
    8548        db.close()
    8549
    8550
    8551def test_delivery_experiment_next_action_allows_write_shell(tmp_path):
    8552    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8553    db = AgentDB(tmp_path / "state.db")
    8554    try:
    8555        job_id = db.create_job("Improve a generic deliverable", title="deliverable", kind="generic")
    8556        db.update_job_metadata(job_id, {
    8557            "experiment_ledger": [{
    8558                "title": "deliverable gap",
    8559                "status": "measured",
    8560                "metric_name": "coverage",
    8561                "metric_value": 0.25,
    8562                "metric_unit": "ratio",
    8563                "next_action": "merge the measured output into the deliverable file",
    8564            }],
    8565        })
    8566
    8567        result = run_one_step(
    8568            job_id,
    8569            config=config,
    8570            db=db,
    8571            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "printf updated > output.txt"})])]),
    8572            registry=SuccessRegistry(),
    8573        )
    8574
    8575        assert result.status == "completed"
    8576        assert result.tool_name == "shell_exec"
    8577    finally:
    8578        db.close()
    8579
    8580
    8581def test_write_file_can_consume_recent_shell_evidence(tmp_path):
    8582    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8583    db = AgentDB(tmp_path / "state.db")
    8584    try:
    8585        job_id = db.create_job("Create a concrete output", title="output", kind="generic")
    8586
    8587        first = run_one_step(
    8588            job_id,
    8589            config=config,
    8590            db=db,
    8591            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "find . -type f"})])]),
    8592            registry=LargeShellEvidenceRegistry(),
    8593        )
    8594        second = run_one_step(
    8595            job_id,
    8596            config=config,
    8597            db=db,
    8598            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="write_file", arguments={"path": "out.txt", "content": "done"})])]),
    8599            registry=SuccessRegistry(),
    8600        )
    8601
    8602        assert first.tool_name == "shell_exec"
    8603        assert second.status == "completed"
    8604        assert second.tool_name == "write_file"
    8605    finally:
    8606        db.close()
    8607
    8608
    8609def test_write_file_creates_validation_obligation_for_code_outputs(tmp_path):
    8610    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8611    db = AgentDB(tmp_path / "state.db")
    8612    try:
    8613        job_id = db.create_job("Create a validated script", title="validate-file", kind="generic")
    8614        path = tmp_path / "generated.py"
    8615
    8616        first = run_one_step(
    8617            job_id,
    8618            config=config,
    8619            db=db,
    8620            llm=ScriptedLLM([
    8621                LLMResponse(tool_calls=[ToolCall(
    8622                    name="write_file",
    8623                    arguments={"path": str(path), "content": "print('ok')\n"},
    8624                )])
    8625            ]),
    8626        )
    8627        job = db.get_job(job_id)
    8628        obligation = job["metadata"]["pending_file_validation_obligation"]
    8629
    8630        assert first.status == "completed"
    8631        assert obligation["path"] == str(path)
    8632        assert "py_compile" in obligation["suggested_validation"]
    8633    finally:
    8634        db.close()
    8635
    8636
    8637def test_file_validation_obligation_blocks_research_until_validated(tmp_path):
    8638    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8639    db = AgentDB(tmp_path / "state.db")
    8640    try:
    8641        job_id = db.create_job("Create a validated script", title="validate-file", kind="generic")
    8642        path = tmp_path / "generated.py"
    8643        run_one_step(
    8644            job_id,
    8645            config=config,
    8646            db=db,
    8647            llm=ScriptedLLM([
    8648                LLMResponse(tool_calls=[ToolCall(
    8649                    name="write_file",
    8650                    arguments={"path": str(path), "content": "print('ok')\n"},
    8651                )])
    8652            ]),
    8653        )
    8654
    8655        blocked = run_one_step(
    8656            job_id,
    8657            config=config,
    8658            db=db,
    8659            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "more context"})])]),
    8660            registry=SuccessRegistry(),
    8661        )
    8662        validated = run_one_step(
    8663            job_id,
    8664            config=config,
    8665            db=db,
    8666            llm=ScriptedLLM([
    8667                LLMResponse(tool_calls=[ToolCall(
    8668                    name="shell_exec",
    8669                    arguments={"command": f"python3 -m py_compile {path}"},
    8670                )])
    8671            ]),
    8672        )
    8673        job = db.get_job(job_id)
    8674
    8675        assert blocked.status == "blocked"
    8676        assert blocked.result["error"] == "file validation pending"
    8677        assert validated.status == "completed"
    8678        assert job["metadata"].get("pending_file_validation_obligation") == {}
    8679        assert job["metadata"]["last_file_validation_obligation"]["resolution_status"] == "validated"
    8680    finally:
    8681        db.close()
    8682
    8683
    8684def test_delivery_experiment_next_action_allows_internal_artifact_review(tmp_path):
    8685    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8686    db = AgentDB(tmp_path / "state.db")
    8687    try:
    8688        job_id = db.create_job("Improve a generic deliverable", title="deliverable", kind="generic")
    8689        db.update_job_metadata(job_id, {
    8690            "experiment_ledger": [{
    8691                "title": "deliverable gap",
    8692                "status": "measured",
    8693                "metric_name": "coverage",
    8694                "metric_value": 0.25,
    8695                "metric_unit": "ratio",
    8696                "next_action": "merge the measured output into the deliverable file",
    8697            }],
    8698        })
    8699
    8700        result = run_one_step(
    8701            job_id,
    8702            config=config,
    8703            db=db,
    8704            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="search_artifacts", arguments={"query": "saved evidence"})])]),
    8705            registry=SuccessRegistry(),
    8706        )
    8707
    8708        assert result.status == "completed"
    8709        assert result.tool_name == "search_artifacts"
    8710    finally:
    8711        db.close()
    8712
    8713
    8714def test_prompt_marks_recent_anti_bot_browser_source():
    8715    job = {"title": "research", "kind": "generic", "objective": "find research"}
    8716    steps = [{
    8717        "step_no": 8,
    8718        "kind": "tool",
    8719        "status": "completed",
    8720        "tool_name": "browser_navigate",
    8721        "summary": "browser_navigate opened Just a moment... <https://clutch.co/example>",
    8722        "input": {"arguments": {"url": "https://clutch.co/example"}},
    8723        "output": {
    8724            "data": {"title": "Just a moment...", "url": "https://clutch.co/example"},
    8725            "snapshot": "Performing security verification. Cloudflare security challenge.",
    8726        },
    8727    }]
    8728
    8729    messages = build_messages(job, steps)
    8730
    8731    assert "source_warning=cloudflare anti-bot challenge" in messages[-1]["content"]
    8732
    8733
    8734def test_prompt_marks_recent_captcha_browser_block():
    8735    job = {"title": "research", "kind": "generic", "objective": "find research"}
    8736    steps = [{
    8737        "step_no": 8,
    8738        "kind": "tool",
    8739        "status": "completed",
    8740        "tool_name": "browser_snapshot",
    8741        "summary": "browser_snapshot returned 1250 chars",
    8742        "input": {"arguments": {"full": True}},
    8743        "output": {
    8744            "data": {
    8745                "origin": "https://source.example/search",
    8746                "snapshot": 'Iframe "Security CAPTCHA" You have been blocked. You are browsing and clicking at a speed much faster than expected.',
    8747            },
    8748        },
    8749    }]
    8750
    8751    messages = build_messages(job, steps)
    8752
    8753    assert "source_warning=captcha/anti-bot block" in messages[-1]["content"]
    8754
    8755
    8756def test_prompt_includes_browser_candidate_names():
    8757    job = {"title": "research", "kind": "generic", "objective": "find research"}
    8758    steps = [{
    8759        "step_no": 9,
    8760        "kind": "tool",
    8761        "status": "completed",
    8762        "tool_name": "browser_snapshot",
    8763        "summary": "browser_snapshot returned 2000 chars",
    8764        "input": {"arguments": {"full": False}},
    8765        "output": {
    8766            "data": {
    8767                "snapshot": "source page",
    8768                "refs": {
    8769                    "e1": {"name": "Contact", "role": "link"},
    8770                    "e2": {"name": "Drytech Interiors", "role": "link"},
    8771                    "e3": {"name": "Flavour Chaser", "role": "link"},
    8772                },
    8773            },
    8774        },
    8775    }]
    8776
    8777    messages = build_messages(job, steps)
    8778
    8779    assert "Drytech Interiors (@e2)" in messages[-1]["content"]
    8780    assert "Flavour Chaser (@e3)" in messages[-1]["content"]
    8781    assert "Contact (@e1)" not in messages[-1]["content"]
    8782
    8783
    8784def test_prompt_includes_candidate_names_from_table_cells():
    8785    job = {"title": "research", "kind": "generic", "objective": "find research"}
    8786    steps = [{
    8787        "step_no": 10,
    8788        "kind": "tool",
    8789        "status": "completed",
    8790        "tool_name": "browser_navigate",
    8791        "summary": "browser_navigate opened list",
    8792        "input": {"arguments": {"url": "https://example.com/list"}},
    8793        "output": {
    8794            "data": {"title": "list", "url": "https://example.com/list"},
    8795            "snapshot": "table",
    8796                "refs": {
    8797                    "e100": {"name": "Organization Name", "role": "cell"},
    8798                "e101": {"name": "Services", "role": "cell"},
    8799                "e102": {
    8800                    "name": "Custom integration, workflow automation, reliability testing, reporting",
    8801                    "role": "cell",
    8802                },
    8803                "e103": {"name": "4.8", "role": "cell"},
    8804                "e104": {"name": "Major Tom", "role": "cell"},
    8805                "e105": {"name": "Kffein", "role": "cell"},
    8806            },
    8807        },
    8808    }]
    8809
    8810    messages = build_messages(job, steps)
    8811
    8812    content = messages[-1]["content"]
    8813    assert "Major Tom (@e104)" in content
    8814    assert "Kffein (@e105)" in content
    8815    assert "Organization Name (@e100)" not in content
    8816    assert "Custom ecommerce" not in content
    8817    assert "4.8 (@e103)" not in content
    8818
    8819
    8820def test_prompt_includes_recovery_candidates_after_stale_ref():
    8821    job = {"title": "research", "kind": "generic", "objective": "find research"}
    8822    steps = [{
    8823        "step_no": 10,
    8824        "kind": "tool",
    8825        "status": "failed",
    8826        "tool_name": "browser_click",
    8827        "summary": "browser_click failed: Unknown ref: e102",
    8828        "input": {"arguments": {"ref": "@e102"}},
    8829        "error": "Unknown ref: e102",
    8830        "output": {
    8831            "success": False,
    8832            "error": "Unknown ref: e102",
    8833            "recovery_guidance": "The ref was stale or missing.",
    8834            "recovery_snapshot": {
    8835                "data": {
    8836                    "refs": {
    8837                        "e4": {"name": "Clearset Vac Truck Services", "role": "link"},
    8838                    },
    8839                },
    8840            },
    8841        },
    8842    }]
    8843
    8844    messages = build_messages(job, steps)
    8845
    8846    content = messages[-1]["content"]
    8847    assert "Unknown ref: e102" in content
    8848    assert "Clearset Vac Truck Services (@e4)" in content
    8849
    8850
    8851def test_run_one_step_blocks_exact_duplicate_tool_call(tmp_path):
    8852    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8853    db = AgentDB(tmp_path / "state.db")
    8854    call = ToolCall(
    8855        name="write_artifact",
    8856        arguments={"title": "same", "content": "same content"},
    8857    )
    8858    try:
    8859        job_id = db.create_job("Do not repeat exact tools", title="dedupe")
    8860        first = run_one_step(job_id, config=config, db=db, llm=ScriptedLLM([LLMResponse(tool_calls=[call])]))
    8861        second = run_one_step(job_id, config=config, db=db, llm=ScriptedLLM([LLMResponse(tool_calls=[call])]))
    8862
    8863        assert first.status == "completed"
    8864        assert second.status == "blocked"
    8865        assert second.result["error"] == "duplicate tool call blocked"
    8866        assert second.result["recoverable"] is True
    8867        assert "previous_step" in second.result
    8868    finally:
    8869        db.close()
    8870
    8871
    8872def test_duplicate_artifact_read_guidance_pushes_follow_up_work(tmp_path):
    8873    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8874    db = AgentDB(tmp_path / "state.db")
    8875    try:
    8876        job_id = db.create_job("Use artifact once", title="artifact")
    8877        run_id = db.start_run(job_id)
    8878        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
    8879        artifacts = ArtifactStore(tmp_path, db)
    8880        stored = artifacts.write_text(job_id=job_id, run_id=run_id, step_id=step_id, title="Evidence", content="saved")
    8881        db.finish_step(step_id, status="completed", output_data={"success": True, "artifact_id": stored.id, "path": str(stored.path)})
    8882        db.finish_run(run_id, "completed")
    8883        call = ToolCall(name="read_artifact", arguments={"artifact_id": stored.id})
    8884
    8885        first = run_one_step(job_id, config=config, db=db, llm=ScriptedLLM([LLMResponse(tool_calls=[call])]))
    8886        second = run_one_step(job_id, config=config, db=db, llm=ScriptedLLM([LLMResponse(tool_calls=[call])]))
    8887
    8888        assert first.status == "completed"
    8889        assert second.status == "blocked"
    8890        assert "Do not read it again" in second.result["guidance"]
    8891    finally:
    8892        db.close()
    8893
    8894
    8895def test_fresh_evidence_guard_takes_priority_over_duplicate_read(tmp_path):
    8896    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8897    db = AgentDB(tmp_path / "state.db")
    8898    try:
    8899        job_id = db.create_job("Save fresh evidence before reviewing old artifacts", title="fresh-evidence")
    8900        run_id = db.start_run(job_id)
    8901        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
    8902        artifacts = ArtifactStore(tmp_path, db)
    8903        stored = artifacts.write_text(job_id=job_id, run_id=run_id, step_id=step_id, title="Old Evidence", content="saved")
    8904        db.finish_step(step_id, status="completed", output_data={"success": True, "artifact_id": stored.id, "path": str(stored.path)})
    8905        db.finish_run(run_id, "completed")
    8906
    8907        read = ToolCall(name="read_artifact", arguments={"artifact_id": stored.id})
    8908        first_read = run_one_step(job_id, config=config, db=db, llm=ScriptedLLM([LLMResponse(tool_calls=[read])]))
    8909        assert first_read.status == "completed"
    8910
    8911        shell = ToolCall(name="shell_exec", arguments={"command": "find . -type f"})
    8912        evidence = run_one_step(
    8913            job_id,
    8914            config=config,
    8915            db=db,
    8916            llm=ScriptedLLM([LLMResponse(tool_calls=[shell])]),
    8917            registry=LargeShellEvidenceRegistry(),
    8918        )
    8919        assert evidence.status == "completed"
    8920
    8921        blocked = run_one_step(job_id, config=config, db=db, llm=ScriptedLLM([LLMResponse(tool_calls=[read])]))
    8922
    8923        assert blocked.status == "blocked"
    8924        assert blocked.result["error"] == "artifact required before more research"
    8925        assert blocked.result["blocked_tool"] == "read_artifact"
    8926    finally:
    8927        db.close()
    8928
    8929
    8930def test_run_one_step_allows_repeated_browser_snapshot(tmp_path):
    8931    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8932    db = AgentDB(tmp_path / "state.db")
    8933    try:
    8934        job_id = db.create_job("Snapshots are stateful", title="snap")
    8935        first = run_one_step(
    8936            job_id,
    8937            config=config,
    8938            db=db,
    8939            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="browser_snapshot", arguments={"full": False})])]),
    8940            registry=SnapshotRegistry(),
    8941        )
    8942        second = run_one_step(
    8943            job_id,
    8944            config=config,
    8945            db=db,
    8946            llm=ScriptedLLM([LLMResponse(tool_calls=[ToolCall(name="browser_snapshot", arguments={"full": False})])]),
    8947            registry=SnapshotRegistry(),
    8948        )
    8949
    8950        assert first.status == "completed"
    8951        assert second.status == "completed"
    8952    finally:
    8953        db.close()
    8954
    8955
    8956def test_run_one_step_blocks_browser_tools_after_runtime_missing(tmp_path):
    8957    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    8958    db = AgentDB(tmp_path / "state.db")
    8959    try:
    8960        job_id = db.create_job("Browser runtime can be unavailable", title="browser-runtime")
    8961        run_id = db.start_run(job_id)
    8962        step_id = db.add_step(
    8963            job_id=job_id,
    8964            run_id=run_id,
    8965            kind="tool",
    8966            tool_name="browser_navigate",
    8967            input_data={"arguments": {"url": "https://example.test"}},
    8968        )
    8969        db.finish_step(
    8970            step_id,
    8971            status="failed",
    8972            output_data={
    8973                "success": False,
    8974                "error": "Chrome not found. Checked: Playwright browser cache and Puppeteer browser cache.",
    8975            },
    8976            summary="browser_navigate failed: Chrome not found",
    8977        )
    8978        db.finish_run(run_id, "failed")
    8979
    8980        result = run_one_step(
    8981            job_id,
    8982            config=config,
    8983            db=db,
    8984            llm=ScriptedLLM([
    8985                LLMResponse(tool_calls=[ToolCall(name="browser_snapshot", arguments={"full": False})])
    8986            ]),
    8987            registry=SnapshotRegistry(),
    8988        )
    8989
    8990        assert result.status == "blocked"
    8991        assert result.result["error"] == "browser runtime unavailable"
    8992        assert result.result["browser_runtime"]["tool"] == "browser_navigate"
    8993        assert "Use web_search" in result.result["guidance"]
    8994    finally:
    8995        db.close()
    8996
    8997
    8998def test_run_one_step_allows_non_browser_work_after_runtime_missing(tmp_path):
    8999    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9000    db = AgentDB(tmp_path / "state.db")
    9001    try:
    9002        job_id = db.create_job("Use fallback tools when browser is missing", title="browser-runtime")
    9003        run_id = db.start_run(job_id)
    9004        step_id = db.add_step(
    9005            job_id=job_id,
    9006            run_id=run_id,
    9007            kind="tool",
    9008            tool_name="browser_navigate",
    9009            input_data={"arguments": {"url": "https://example.test"}},
    9010        )
    9011        db.finish_step(
    9012            step_id,
    9013            status="failed",
    9014            output_data={"success": False, "error": "Browser executable doesn't exist on this host."},
    9015            summary="browser_navigate failed: browser executable missing",
    9016        )
    9017        db.finish_run(run_id, "failed")
    9018
    9019        result = run_one_step(
    9020            job_id,
    9021            config=config,
    9022            db=db,
    9023            llm=ScriptedLLM([
    9024                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "public docs", "limit": 5})])
    9025            ]),
    9026            registry=SuccessRegistry(),
    9027        )
    9028
    9029        assert result.status == "completed"
    9030        assert result.tool_name == "web_search"
    9031    finally:
    9032        db.close()
    9033
    9034
    9035def test_run_one_step_skips_batched_browser_call_when_runtime_missing_and_fallback_present(tmp_path):
    9036    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9037    db = AgentDB(tmp_path / "state.db")
    9038    try:
    9039        job_id = db.create_job("Use fallback tools when browser is missing", title="browser-runtime")
    9040        run_id = db.start_run(job_id)
    9041        step_id = db.add_step(
    9042            job_id=job_id,
    9043            run_id=run_id,
    9044            kind="tool",
    9045            tool_name="browser_navigate",
    9046            input_data={"arguments": {"url": "https://example.test"}},
    9047        )
    9048        db.finish_step(
    9049            step_id,
    9050            status="failed",
    9051            output_data={"success": False, "error": "Chrome not found. Checked: Playwright browser cache."},
    9052            summary="browser_navigate failed: Chrome not found",
    9053        )
    9054        db.finish_run(run_id, "failed")
    9055
    9056        result = run_one_step(
    9057            job_id,
    9058            config=config,
    9059            db=db,
    9060            llm=ScriptedLLM([
    9061                LLMResponse(tool_calls=[
    9062                    ToolCall(name="browser_navigate", arguments={"url": "https://example.test/next"}),
    9063                    ToolCall(name="web_search", arguments={"query": "public docs", "limit": 5}),
    9064                ])
    9065            ]),
    9066            registry=SuccessRegistry(),
    9067        )
    9068
    9069        tool_steps = [step for step in db.list_steps(job_id=job_id) if step.get("kind") == "tool"]
    9070        assert result.status == "completed"
    9071        assert result.tool_name == "web_search"
    9072        assert tool_steps[-1]["tool_name"] == "web_search"
    9073        assert all(
    9074            step["input"].get("arguments", {}).get("url") != "https://example.test/next"
    9075            for step in tool_steps
    9076            if step.get("tool_name") == "browser_navigate"
    9077        )
    9078    finally:
    9079        db.close()
    9080
    9081
    9082def test_run_one_step_removes_browser_tools_from_schema_after_runtime_missing(tmp_path):
    9083    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9084    db = AgentDB(tmp_path / "state.db")
    9085    try:
    9086        job_id = db.create_job("Use fallback tools when browser is missing", title="browser-runtime")
    9087        run_id = db.start_run(job_id)
    9088        step_id = db.add_step(
    9089            job_id=job_id,
    9090            run_id=run_id,
    9091            kind="tool",
    9092            tool_name="browser_navigate",
    9093            input_data={"arguments": {"url": "https://example.test"}},
    9094        )
    9095        db.finish_step(
    9096            step_id,
    9097            status="failed",
    9098            output_data={"success": False, "error": "Chrome not found. Checked: Playwright browser cache."},
    9099            summary="browser_navigate failed: Chrome not found",
    9100        )
    9101        db.finish_run(run_id, "failed")
    9102        llm = CapturingLLM(LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "fallback"})]))
    9103
    9104        run_one_step(
    9105            job_id,
    9106            config=config,
    9107            db=db,
    9108            llm=llm,
    9109            registry=BrowserAndWebRegistry(),
    9110        )
    9111
    9112        tool_names = [tool["function"]["name"] for tool in llm.tools]
    9113        assert tool_names == ["web_search"]
    9114    finally:
    9115        db.close()
    9116
    9117
    9118def test_run_one_step_removes_browser_tools_after_older_runtime_missing(tmp_path):
    9119    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9120    db = AgentDB(tmp_path / "state.db")
    9121    try:
    9122        job_id = db.create_job("Use fallback tools when browser is missing", title="browser-runtime")
    9123        run_id = db.start_run(job_id)
    9124        step_id = db.add_step(
    9125            job_id=job_id,
    9126            run_id=run_id,
    9127            kind="tool",
    9128            tool_name="browser_navigate",
    9129            input_data={"arguments": {"url": "https://example.test"}},
    9130        )
    9131        db.finish_step(
    9132            step_id,
    9133            status="failed",
    9134            output_data={"success": False, "error": "Chrome not found. Checked: Playwright browser cache."},
    9135            summary="browser_navigate failed: Chrome not found",
    9136        )
    9137        for index in range(80):
    9138            filler_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="web_search")
    9139            db.finish_step(
    9140                filler_id,
    9141                status="completed",
    9142                output_data={"success": True, "query": f"query {index}", "results": []},
    9143                summary=f"web_search query {index}",
    9144            )
    9145        db.finish_run(run_id, "completed")
    9146        llm = CapturingLLM(LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "fallback"})]))
    9147
    9148        run_one_step(
    9149            job_id,
    9150            config=config,
    9151            db=db,
    9152            llm=llm,
    9153            registry=BrowserAndWebRegistry(),
    9154        )
    9155
    9156        tool_names = [tool["function"]["name"] for tool in llm.tools]
    9157        assert tool_names == ["web_search"]
    9158    finally:
    9159        db.close()
    9160
    9161
    9162def test_run_one_step_allows_repeated_defer_for_monitor_intervals(tmp_path):
    9163    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9164    db = AgentDB(tmp_path / "state.db")
    9165    call = ToolCall(name="defer_job", arguments={"seconds": 60, "reason": "wait for monitor interval"})
    9166    try:
    9167        job_id = db.create_job("Check a long-running process later", title="defer")
    9168        first = run_one_step(job_id, config=config, db=db, llm=ScriptedLLM([LLMResponse(tool_calls=[call])]))
    9169        second = run_one_step(job_id, config=config, db=db, llm=ScriptedLLM([LLMResponse(tool_calls=[call])]))
    9170
    9171        assert first.status == "completed"
    9172        assert second.status == "completed"
    9173        assert first.tool_name == "defer_job"
    9174        assert second.tool_name == "defer_job"
    9175    finally:
    9176        db.close()
    9177
    9178
    9179def test_run_one_step_blocks_self_defer_for_next_worker_turn(tmp_path):
    9180    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9181    db = AgentDB(tmp_path / "state.db")
    9182    try:
    9183        job_id = db.create_job("Keep making progress", title="self-defer")
    9184
    9185        result = run_one_step(
    9186            job_id,
    9187            config=config,
    9188            db=db,
    9189            llm=ScriptedLLM([
    9190                LLMResponse(tool_calls=[
    9191                    ToolCall(
    9192                        name="defer_job",
    9193                        arguments={
    9194                            "seconds": 300,
    9195                            "reason": "waiting for tasks to be picked up by next worker turn",
    9196                            "next_action": "continue in the next worker step",
    9197                        },
    9198                    )
    9199                ])
    9200            ]),
    9201        )
    9202
    9203        assert result.status == "blocked"
    9204        assert result.tool_name == "defer_job"
    9205        assert result.result["error"] == "self-defer blocked"
    9206        assert result.result["self_defer"]["matched"] == "next worker turn"
    9207    finally:
    9208        db.close()
    9209
    9210
    9211def test_run_one_step_blocks_defer_without_wait_reason(tmp_path):
    9212    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9213    db = AgentDB(tmp_path / "state.db")
    9214    try:
    9215        job_id = db.create_job("Keep making progress", title="self-defer")
    9216
    9217        result = run_one_step(
    9218            job_id,
    9219            config=config,
    9220            db=db,
    9221            llm=ScriptedLLM([
    9222                LLMResponse(tool_calls=[
    9223                    ToolCall(
    9224                        name="defer_job",
    9225                        arguments={
    9226                            "seconds": 300,
    9227                            "next_action": "build the project and run the measurement",
    9228                        },
    9229                    )
    9230                ])
    9231            ]),
    9232        )
    9233
    9234        assert result.status == "blocked"
    9235        assert result.tool_name == "defer_job"
    9236        assert result.result["error"] == "self-defer blocked"
    9237        assert result.result["self_defer"]["matched"] == "missing wait reason"
    9238    finally:
    9239        db.close()
    9240
    9241
    9242def test_run_one_step_blocks_search_after_unpersisted_extract(tmp_path):
    9243    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9244    db = AgentDB(tmp_path / "state.db")
    9245    try:
    9246        job_id = db.create_job("Save extracted evidence before more search", title="guard")
    9247        run_id = db.start_run(job_id)
    9248        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="web_extract")
    9249        db.finish_step(
    9250            step_id,
    9251            status="completed",
    9252            output_data={"success": True, "pages": [{"url": "https://example.com", "text": "useful evidence"}]},
    9253        )
    9254        db.finish_run(run_id, "completed")
    9255
    9256        result = run_one_step(
    9257            job_id,
    9258            config=config,
    9259            db=db,
    9260            llm=ScriptedLLM([
    9261                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "more findings", "limit": 5})])
    9262            ]),
    9263        )
    9264
    9265        assert result.status == "blocked"
    9266        assert result.result["error"] == "artifact required before more research"
    9267        assert result.result["blocked_tool"] == "web_search"
    9268        assert "auto_checkpoint" in result.result
    9269        artifacts = db.list_artifacts(job_id)
    9270        assert artifacts[0]["title"].startswith("Auto Evidence Checkpoint")
    9271
    9272        next_result = run_one_step(
    9273            job_id,
    9274            config=config,
    9275            db=db,
    9276            llm=ScriptedLLM([
    9277                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "different findings", "limit": 5})])
    9278            ]),
    9279            registry=SuccessRegistry(),
    9280        )
    9281        assert next_result.status == "blocked"
    9282        assert next_result.result["error"] == "evidence checkpoint accounting required"
    9283    finally:
    9284        db.close()
    9285
    9286
    9287def test_prompt_tells_model_to_save_unpersisted_evidence_before_more_research(tmp_path):
    9288    db = AgentDB(tmp_path / "state.db")
    9289    try:
    9290        job_id = db.create_job("Save evidence before searching", title="guard")
    9291        run_id = db.start_run(job_id)
    9292        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="web_extract")
    9293        db.finish_step(
    9294            step_id,
    9295            status="completed",
    9296            output_data={"success": True, "pages": [{"url": "https://example.com", "text": "useful evidence"}]},
    9297        )
    9298        job = db.get_job(job_id)
    9299        steps = db.list_steps(job_id=job_id)
    9300
    9301        messages = build_messages(job, steps)
    9302
    9303        assert "Next-action constraint:" in messages[-1]["content"]
    9304        assert "Your next tool call should usually be write_artifact" in messages[-1]["content"]
    9305    finally:
    9306        db.close()
    9307
    9308
    9309def test_run_one_step_blocks_research_after_unpersisted_browser_snapshot(tmp_path):
    9310    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9311    db = AgentDB(tmp_path / "state.db")
    9312    try:
    9313        job_id = db.create_job("Save browser evidence before more browsing", title="guard")
    9314        run_id = db.start_run(job_id)
    9315        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="browser_snapshot")
    9316        db.finish_step(
    9317            step_id,
    9318            status="completed",
    9319            output_data={
    9320                "success": True,
    9321                "data": {"origin": "https://example.com"},
    9322                "snapshot": "Useful finding evidence. " * 40,
    9323            },
    9324        )
    9325        db.finish_run(run_id, "completed")
    9326
    9327        result = run_one_step(
    9328            job_id,
    9329            config=config,
    9330            db=db,
    9331            llm=ScriptedLLM([
    9332                LLMResponse(tool_calls=[ToolCall(name="browser_scroll", arguments={"direction": "down"})])
    9333            ]),
    9334        )
    9335
    9336        assert result.status == "blocked"
    9337        assert result.result["error"] == "artifact required before more research"
    9338        assert result.result["blocked_tool"] == "browser_scroll"
    9339        assert "auto_checkpoint" in result.result
    9340    finally:
    9341        db.close()
    9342
    9343
    9344def test_prompt_tells_model_to_open_new_branch_when_tasks_are_exhausted():
    9345    job = {
    9346        "title": "research",
    9347        "kind": "generic",
    9348        "objective": "keep improving",
    9349        "metadata": {
    9350            "task_queue": [
    9351                {"title": "Initial branch", "status": "done", "priority": 5, "result": "Checkpoint saved"},
    9352                {"title": "Blocked branch", "status": "blocked", "priority": 4, "result": "Source unavailable"},
    9353            ],
    9354        },
    9355    }
    9356
    9357    messages = build_messages(job, [])
    9358
    9359    content = messages[-1]["content"]
    9360    assert "All durable task branches are done" in content
    9361    assert "use record_tasks to open the next concrete branch" in content
    9362
    9363
    9364def test_prompt_pushes_deliverable_checkpoint_after_long_research():
    9365    job = {
    9366        "title": "paper",
    9367        "kind": "generic",
    9368        "objective": "write a complete research paper from evidence",
    9369        "metadata": {
    9370            "task_queue": [
    9371                {
    9372                    "title": "Save the first durable draft",
    9373                    "status": "open",
    9374                    "priority": 8,
    9375                    "output_contract": "report",
    9376                }
    9377            ],
    9378        },
    9379    }
    9380    steps = [
    9381        {
    9382            "step_no": index + 1,
    9383            "status": "completed",
    9384            "kind": "tool",
    9385            "tool_name": "shell_exec",
    9386            "input": {"arguments": {"command": f"cat source_{index}.txt"}},
    9387        }
    9388        for index in range(18)
    9389    ]
    9390
    9391    content = build_messages(job, steps)[-1]["content"]
    9392
    9393    assert "Deliverable progress guard:" in content
    9394    assert "durable deliverable checkpoint" in content
    9395    assert "write_file or write_artifact" in content
    9396
    9397
    9398def test_low_priority_report_task_does_not_block_execution_task_prompt():
    9399    job = {
    9400        "title": "execution",
    9401        "kind": "generic",
    9402        "objective": "keep useful work moving",
    9403        "metadata": {
    9404            "task_queue": [
    9405                {
    9406                    "title": "Review saved output later",
    9407                    "status": "open",
    9408                    "priority": 4,
    9409                    "output_contract": "report",
    9410                },
    9411                {
    9412                    "title": "Run current experiment",
    9413                    "status": "active",
    9414                    "priority": 9,
    9415                    "output_contract": "experiment",
    9416                },
    9417            ],
    9418        },
    9419    }
    9420    steps = [
    9421        {
    9422            "step_no": index + 1,
    9423            "status": "completed",
    9424            "kind": "tool",
    9425            "tool_name": "shell_exec",
    9426            "input": {"arguments": {"command": f"probe_{index}"}},
    9427        }
    9428        for index in range(18)
    9429    ]
    9430
    9431    content = build_messages(job, steps)[-1]["content"]
    9432
    9433    assert "Deliverable progress guard:\nNone." in content
    9434    assert "durable deliverable checkpoint" not in content
    9435
    9436
    9437def test_run_one_step_blocks_more_research_when_deliverable_needs_checkpoint(tmp_path):
    9438    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9439    db = AgentDB(tmp_path / "state.db")
    9440    try:
    9441        job_id = db.create_job(
    9442            "Write a complete report from collected evidence",
    9443            title="deliverable",
    9444            metadata={
    9445                "task_queue": [
    9446                    {
    9447                        "title": "Save the first durable report checkpoint",
    9448                        "status": "open",
    9449                        "priority": 8,
    9450                        "output_contract": "report",
    9451                    }
    9452                ]
    9453            },
    9454        )
    9455        run_id = db.start_run(job_id, model="fake")
    9456        for index in range(15):
    9457            step_id = db.add_step(
    9458                job_id=job_id,
    9459                run_id=run_id,
    9460                kind="tool",
    9461                tool_name="shell_exec",
    9462                input_data={"arguments": {"command": f"cat source_{index}.txt"}},
    9463            )
    9464            db.finish_step(step_id, status="completed", output_data={"success": True, "stdout": "note"})
    9465        ledger_step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_findings")
    9466        db.finish_step(ledger_step_id, status="completed", output_data={"success": True})
    9467        for index in range(15, 18):
    9468            step_id = db.add_step(
    9469                job_id=job_id,
    9470                run_id=run_id,
    9471                kind="tool",
    9472                tool_name="shell_exec",
    9473                input_data={"arguments": {"command": f"cat source_{index}.txt"}},
    9474            )
    9475            db.finish_step(step_id, status="completed", output_data={"success": True, "stdout": "note"})
    9476        db.finish_run(run_id, "completed")
    9477
    9478        result = run_one_step(
    9479            job_id,
    9480            config=config,
    9481            db=db,
    9482            llm=ScriptedLLM([
    9483                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "more background sources"})])
    9484            ]),
    9485        )
    9486
    9487        assert result.status == "blocked"
    9488        assert result.result["error"] == "deliverable checkpoint required"
    9489        assert result.result["blocked_tool"] == "web_search"
    9490        assert result.result["recoverable"] is True
    9491    finally:
    9492        db.close()
    9493
    9494
    9495def test_prompt_includes_roadmap_and_validation_constraints():
    9496    job = {
    9497        "title": "broad work",
    9498        "kind": "generic",
    9499        "objective": "build a broad durable outcome",
    9500        "metadata": {
    9501            "roadmap": {
    9502                "title": "Broad Roadmap",
    9503                "status": "active",
    9504                "current_milestone": "Foundation",
    9505                "validation_contract": "check observable evidence",
    9506                "milestones": [{
    9507                    "title": "Foundation",
    9508                    "status": "validating",
    9509                    "validation_status": "pending",
    9510                    "acceptance_criteria": "evidence exists",
    9511                    "evidence_needed": "saved output",
    9512                    "features": [{"title": "First feature", "status": "done"}],
    9513                }],
    9514            },
    9515        },
    9516    }
    9517
    9518    messages = build_messages(job, [])
    9519    content = messages[-1]["content"]
    9520
    9521    assert "Roadmap:" in content
    9522    assert "Broad Roadmap" in content
    9523    assert "validation=pending" in content
    9524    assert "Use record_milestone_validation" in content
    9525
    9526
    9527def test_prompt_suggests_roadmap_for_broad_jobs_without_one():
    9528    job = {
    9529        "title": "broad work",
    9530        "kind": "generic",
    9531        "objective": "research and implement a broad multi phase system with validation and durable output",
    9532        "metadata": {},
    9533    }
    9534
    9535    messages = build_messages(job, [])
    9536    content = messages[-1]["content"]
    9537
    9538    assert "No roadmap yet" in content
    9539    assert "use record_roadmap" in content
    9540
    9541
    9542def test_run_one_step_blocks_branch_work_when_milestone_needs_validation(tmp_path):
    9543    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9544    db = AgentDB(tmp_path / "state.db")
    9545    try:
    9546        job_id = db.create_job(
    9547            "Keep broad work gated by validation",
    9548            title="roadmap-gate",
    9549            metadata={
    9550                "roadmap": {
    9551                    "title": "Generic Roadmap",
    9552                    "status": "active",
    9553                    "milestones": [{
    9554                        "title": "Foundation",
    9555                        "status": "validating",
    9556                        "validation_status": "pending",
    9557                        "acceptance_criteria": "evidence exists",
    9558                        "evidence_needed": "saved artifact",
    9559                    }],
    9560                },
    9561            },
    9562        )
    9563
    9564        result = run_one_step(
    9565            job_id,
    9566            config=config,
    9567            db=db,
    9568            llm=ScriptedLLM([
    9569                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "new branch", "limit": 5})])
    9570            ]),
    9571        )
    9572
    9573        assert result.status == "blocked"
    9574        assert result.result["error"] == "milestone validation required"
    9575        assert result.result["blocked_tool"] == "web_search"
    9576    finally:
    9577        db.close()
    9578
    9579
    9580def test_run_one_step_allows_milestone_validation_when_gate_is_active(tmp_path):
    9581    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9582    db = AgentDB(tmp_path / "state.db")
    9583    try:
    9584        job_id = db.create_job(
    9585            "Validate a gated milestone",
    9586            title="roadmap-validate",
    9587            metadata={
    9588                "roadmap": {
    9589                    "title": "Generic Roadmap",
    9590                    "status": "active",
    9591                    "milestones": [{
    9592                        "title": "Foundation",
    9593                        "status": "validating",
    9594                        "validation_status": "pending",
    9595                    }],
    9596                },
    9597            },
    9598        )
    9599
    9600        result = run_one_step(
    9601            job_id,
    9602            config=config,
    9603            db=db,
    9604            llm=ScriptedLLM([
    9605                LLMResponse(tool_calls=[ToolCall(name="record_milestone_validation", arguments={
    9606                    "milestone": "Foundation",
    9607                    "validation_status": "passed",
    9608                    "result": "Acceptance criteria met.",
    9609                    "evidence": "artifact",
    9610                })])
    9611            ]),
    9612        )
    9613
    9614        assert result.status == "completed"
    9615        assert result.tool_name == "record_milestone_validation"
    9616        roadmap = db.get_job(job_id)["metadata"]["roadmap"]
    9617        assert roadmap["milestones"][0]["validation_status"] == "passed"
    9618    finally:
    9619        db.close()
    9620
    9621
    9622def test_run_one_step_allows_matching_pending_milestone_evidence_action(tmp_path):
    9623    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9624    db = AgentDB(tmp_path / "state.db")
    9625    try:
    9626        job_id = db.create_job(
    9627            "Validate a pending milestone",
    9628            title="roadmap-pending-shell",
    9629            metadata={
    9630                "roadmap": {
    9631                    "title": "Generic Roadmap",
    9632                    "status": "validating",
    9633                    "milestones": [{
    9634                        "title": "Environment baseline",
    9635                        "status": "validating",
    9636                        "validation_status": "pending",
    9637                        "next_action": "Validate candidate files with a shell probe.",
    9638                        "evidence_needed": "Shell output showing candidate file status.",
    9639                    }],
    9640                },
    9641            },
    9642        )
    9643
    9644        result = run_one_step(
    9645            job_id,
    9646            config=config,
    9647            db=db,
    9648            llm=ScriptedLLM([
    9649                LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={
    9650                    "command": "printf 'candidate file ok\\n'",
    9651                    "timeout_seconds": 5,
    9652                })])
    9653            ]),
    9654        )
    9655
    9656        assert result.status == "completed"
    9657        assert result.tool_name == "shell_exec"
    9658    finally:
    9659        db.close()
    9660
    9661
    9662def test_run_one_step_allows_matching_pending_milestone_validation_evidence_action(tmp_path):
    9663    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9664    db = AgentDB(tmp_path / "state.db")
    9665    try:
    9666        job_id = db.create_job(
    9667            "Validate a pending milestone",
    9668            title="roadmap-pending-validation-evidence",
    9669            metadata={
    9670                "roadmap": {
    9671                    "title": "Generic Roadmap",
    9672                    "status": "validating",
    9673                    "milestones": [{
    9674                        "title": "Build tools",
    9675                        "status": "validating",
    9676                        "validation_status": "pending",
    9677                        "validation_evidence": "Need to verify cmake and compiler paths before building.",
    9678                    }],
    9679                },
    9680            },
    9681        )
    9682
    9683        result = run_one_step(
    9684            job_id,
    9685            config=config,
    9686            db=db,
    9687            llm=ScriptedLLM([
    9688                LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={
    9689                    "command": "printf 'cmake compiler ok\\n'",
    9690                    "timeout_seconds": 5,
    9691                })])
    9692            ]),
    9693        )
    9694
    9695        assert result.status == "completed"
    9696        assert result.tool_name == "shell_exec"
    9697    finally:
    9698        db.close()
    9699
    9700
    9701def test_run_one_step_blocks_non_matching_pending_milestone_action(tmp_path):
    9702    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9703    db = AgentDB(tmp_path / "state.db")
    9704    try:
    9705        job_id = db.create_job(
    9706            "Validate a pending milestone",
    9707            title="roadmap-pending-unrelated",
    9708            metadata={
    9709                "roadmap": {
    9710                    "title": "Generic Roadmap",
    9711                    "status": "validating",
    9712                    "milestones": [{
    9713                        "title": "Environment baseline",
    9714                        "status": "validating",
    9715                        "validation_status": "pending",
    9716                        "next_action": "Validate candidate files with a shell probe.",
    9717                        "evidence_needed": "Shell output showing candidate file status.",
    9718                    }],
    9719                },
    9720            },
    9721        )
    9722
    9723        result = run_one_step(
    9724            job_id,
    9725            config=config,
    9726            db=db,
    9727            llm=ScriptedLLM([
    9728                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={
    9729                    "query": "unrelated topic",
    9730                    "limit": 5,
    9731                })])
    9732            ]),
    9733        )
    9734
    9735        assert result.status == "blocked"
    9736        assert result.result["error"] == "milestone validation required"
    9737    finally:
    9738        db.close()
    9739
    9740
    9741def test_run_one_step_blocks_wrong_milestone_validation_when_gate_is_active(tmp_path):
    9742    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9743    db = AgentDB(tmp_path / "state.db")
    9744    try:
    9745        job_id = db.create_job(
    9746            "Validate the active milestone only",
    9747            title="roadmap-wrong-milestone",
    9748            metadata={
    9749                "roadmap": {
    9750                    "title": "Generic Roadmap",
    9751                    "status": "validating",
    9752                    "milestones": [{
    9753                        "title": "Current milestone",
    9754                        "status": "validating",
    9755                        "validation_status": "pending",
    9756                    }],
    9757                },
    9758            },
    9759        )
    9760
    9761        result = run_one_step(
    9762            job_id,
    9763            config=config,
    9764            db=db,
    9765            llm=ScriptedLLM([
    9766                LLMResponse(tool_calls=[ToolCall(name="record_milestone_validation", arguments={
    9767                    "milestone": "Different milestone",
    9768                    "validation_status": "passed",
    9769                })])
    9770            ]),
    9771        )
    9772
    9773        assert result.status == "blocked"
    9774        assert result.result["error"] == "current milestone validation required"
    9775        roadmap = db.get_job(job_id)["metadata"]["roadmap"]
    9776        assert [milestone["title"] for milestone in roadmap["milestones"]] == ["Current milestone"]
    9777    finally:
    9778        db.close()
    9779
    9780
    9781def test_run_one_step_normalizes_matching_validation_to_active_milestone(tmp_path):
    9782    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9783    db = AgentDB(tmp_path / "state.db")
    9784    try:
    9785        job_id = db.create_job(
    9786            "Validate the active milestone from matching evidence",
    9787            title="roadmap-normalize-milestone-validation",
    9788            metadata={
    9789                "roadmap": {
    9790                    "title": "Generic Roadmap",
    9791                    "status": "validating",
    9792                    "milestones": [{
    9793                        "title": "Environment baseline evidence: check build tools",
    9794                        "status": "validating",
    9795                        "validation_status": "pending",
    9796                        "validation_evidence": "Need to verify cmake, compiler, and candidate files before building.",
    9797                    }],
    9798                },
    9799            },
    9800        )
    9801
    9802        result = run_one_step(
    9803            job_id,
    9804            config=config,
    9805            db=db,
    9806            llm=ScriptedLLM([
    9807                LLMResponse(tool_calls=[ToolCall(name="record_milestone_validation", arguments={
    9808                    "milestone": "Validate candidate files and build environment",
    9809                    "validation_status": "blocked",
    9810                    "result": "cmake path failed, compiler still needs verification, and candidate file status is unclear.",
    9811                    "evidence": "shell output showed missing cmake path and file checks are still needed.",
    9812                    "issues": ["cmake path missing", "candidate file status unresolved"],
    9813                })])
    9814            ]),
    9815        )
    9816
    9817        assert result.status == "completed"
    9818        assert result.tool_name == "record_milestone_validation"
    9819        roadmap = db.get_job(job_id)["metadata"]["roadmap"]
    9820        assert [milestone["title"] for milestone in roadmap["milestones"]] == [
    9821            "Environment baseline evidence: check build tools"
    9822        ]
    9823        milestone = roadmap["milestones"][0]
    9824        assert milestone["validation_status"] == "blocked"
    9825        assert milestone["metadata"]["normalized_from_milestone"] == "Validate candidate files and build environment"
    9826        assert milestone["metadata"]["normalized_to_active_gate"] is True
    9827    finally:
    9828        db.close()
    9829
    9830
    9831def test_run_one_step_blocks_task_churn_when_roadmap_stalls(tmp_path):
    9832    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9833    db = AgentDB(tmp_path / "state.db")
    9834    try:
    9835        job_id = db.create_job(
    9836            "Keep roadmap aligned with broad work",
    9837            title="roadmap-stale",
    9838            metadata={
    9839                "roadmap": {
    9840                    "title": "Generic Roadmap",
    9841                    "status": "planned",
    9842                    "milestones": [{
    9843                        "title": "Foundation",
    9844                        "status": "planned",
    9845                        "validation_status": "not_started",
    9846                    }],
    9847                },
    9848                "task_queue": [{"title": f"Task {index}", "status": "done"} for index in range(8)],
    9849            },
    9850        )
    9851        run_id = db.start_run(job_id, model="fake")
    9852        for index in range(2):
    9853            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
    9854            db.finish_step(step_id, status="completed", summary=f"artifact {index}", output_data={"success": True})
    9855
    9856        result = run_one_step(
    9857            job_id,
    9858            config=config,
    9859            db=db,
    9860            llm=ScriptedLLM([
    9861                LLMResponse(tool_calls=[ToolCall(name="record_tasks", arguments={
    9862                    "tasks": [{"title": "More task churn", "status": "open"}]
    9863                })])
    9864            ]),
    9865        )
    9866
    9867        assert result.status == "blocked"
    9868        assert result.result["error"] == "roadmap update required"
    9869        assert result.result["blocked_tool"] == "record_tasks"
    9870    finally:
    9871        db.close()
    9872
    9873
    9874def test_run_one_step_allows_roadmap_update_when_roadmap_stalls(tmp_path):
    9875    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9876    db = AgentDB(tmp_path / "state.db")
    9877    try:
    9878        job_id = db.create_job(
    9879            "Update stale roadmap",
    9880            title="roadmap-update",
    9881            metadata={
    9882                "roadmap": {
    9883                    "title": "Generic Roadmap",
    9884                    "status": "planned",
    9885                    "milestones": [{
    9886                        "title": "Foundation",
    9887                        "status": "planned",
    9888                        "validation_status": "not_started",
    9889                    }],
    9890                },
    9891                "task_queue": [{"title": f"Task {index}", "status": "done"} for index in range(8)],
    9892            },
    9893        )
    9894        run_id = db.start_run(job_id, model="fake")
    9895        for index in range(2):
    9896            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="write_artifact")
    9897            db.finish_step(step_id, status="completed", summary=f"artifact {index}", output_data={"success": True})
    9898
    9899        result = run_one_step(
    9900            job_id,
    9901            config=config,
    9902            db=db,
    9903            llm=ScriptedLLM([
    9904                LLMResponse(tool_calls=[ToolCall(name="record_roadmap", arguments={
    9905                    "title": "Generic Roadmap",
    9906                    "status": "active",
    9907                    "current_milestone": "Foundation",
    9908                    "milestones": [{
    9909                        "title": "Foundation",
    9910                        "status": "active",
    9911                        "validation_status": "pending",
    9912                        "acceptance_criteria": "evidence reviewed",
    9913                    }],
    9914                })])
    9915            ]),
    9916        )
    9917
    9918        assert result.status == "completed"
    9919        assert result.tool_name == "record_roadmap"
    9920        roadmap = db.get_job(job_id)["metadata"]["roadmap"]
    9921        assert roadmap["status"] == "active"
    9922        assert roadmap["milestones"][0]["validation_status"] == "pending"
    9923    finally:
    9924        db.close()
    9925
    9926
    9927def test_run_one_step_blocks_branch_work_when_tasks_are_exhausted(tmp_path):
    9928    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9929    db = AgentDB(tmp_path / "state.db")
    9930    try:
    9931        job_id = db.create_job(
    9932            "Keep improving without looping",
    9933            title="exhausted",
    9934            metadata={"task_queue": [{"title": "First branch", "status": "done", "priority": 5}]},
    9935        )
    9936
    9937        result = run_one_step(
    9938            job_id,
    9939            config=config,
    9940            db=db,
    9941            llm=ScriptedLLM([
    9942                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "same broad topic", "limit": 5})])
    9943            ]),
    9944        )
    9945
    9946        assert result.status == "blocked"
    9947        assert result.result["error"] == "task branch required before more work"
    9948        assert result.result["blocked_tool"] == "web_search"
    9949        assert result.result["recoverable"] is True
    9950    finally:
    9951        db.close()
    9952
    9953
    9954def test_run_one_step_allows_record_tasks_when_tasks_are_exhausted(tmp_path):
    9955    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9956    db = AgentDB(tmp_path / "state.db")
    9957    try:
    9958        job_id = db.create_job(
    9959            "Keep improving by opening branches",
    9960            title="branch",
    9961            metadata={"task_queue": [{"title": "First branch", "status": "done", "priority": 5}]},
    9962        )
    9963
    9964        result = run_one_step(
    9965            job_id,
    9966            config=config,
    9967            db=db,
    9968            llm=ScriptedLLM([
    9969                LLMResponse(tool_calls=[ToolCall(name="record_tasks", arguments={
    9970                    "tasks": [{"title": "Next branch", "status": "open", "priority": 6}]
    9971                })])
    9972            ]),
    9973        )
    9974
    9975        assert result.status == "completed"
    9976        assert result.tool_name == "record_tasks"
    9977        job = db.get_job(job_id)
    9978        assert any(task["title"] == "Next branch" and task["status"] == "open" for task in job["metadata"]["task_queue"])
    9979    finally:
    9980        db.close()
    9981
    9982
    9983def test_run_one_step_blocks_new_tasks_when_queue_is_saturated(tmp_path):
    9984    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    9985    db = AgentDB(tmp_path / "state.db")
    9986    try:
    9987        job_id = db.create_job(
    9988            "Finish existing work",
    9989            title="saturated",
    9990            kind="generic",
    9991            metadata={
    9992                "task_queue": [
    9993                    {"title": f"Open branch {index}", "status": "open", "priority": index}
    9994                    for index in range(40)
    9995                ]
    9996            },
    9997        )
    9998
    9999        result = run_one_step(
    10000            job_id,
    10001            config=config,
    10002            db=db,
    10003            llm=ScriptedLLM([
    10004                LLMResponse(tool_calls=[
    10005                    ToolCall(name="record_tasks", arguments={"tasks": [{"title": "Yet another branch", "status": "open"}]})
    10006                ])
    10007            ]),
    10008        )
    10009
    10010        assert result.status == "blocked"
    10011        assert result.result["error"] == "task queue saturated"
    10012        assert result.result["task_queue"]["open_count"] == 40
    10013        job = db.get_job(job_id)
    10014        pressure = job["metadata"]["task_backlog_pressure"]
    10015        assert pressure["source"] == "blocked_record_tasks"
    10016        assert pressure["open_count"] == 40
    10017        assert pressure["reason"] == "too many open tasks"
    10018    finally:
    10019        db.close()
    10020
    10021
    10022def test_run_one_step_blocks_batch_that_would_saturate_task_queue(tmp_path):
    10023    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10024    db = AgentDB(tmp_path / "state.db")
    10025    try:
    10026        job_id = db.create_job(
    10027            "Keep long-running work focused",
    10028            title="projected-sprawl",
    10029            kind="generic",
    10030            metadata={
    10031                "task_queue": [
    10032                    {"title": f"Existing branch {index}", "status": "done", "priority": index}
    10033                    for index in range(74)
    10034                ]
    10035            },
    10036        )
    10037
    10038        result = run_one_step(
    10039            job_id,
    10040            config=config,
    10041            db=db,
    10042            llm=ScriptedLLM([
    10043                LLMResponse(tool_calls=[
    10044                    ToolCall(
    10045                        name="record_tasks",
    10046                        arguments={
    10047                            "tasks": [
    10048                                {"title": f"New branch {index}", "status": "open"}
    10049                                for index in range(10)
    10050                            ]
    10051                        },
    10052                    )
    10053                ])
    10054            ]),
    10055        )
    10056
    10057        assert result.status == "blocked"
    10058        assert result.result["error"] == "task queue saturated"
    10059        assert result.result["task_queue"]["reason"] == "total task queue is too large"
    10060        assert result.result["task_queue"]["projected_total_count"] == 84
    10061        job = db.get_job(job_id)
    10062        assert len(job["metadata"]["task_queue"]) == 74
    10063    finally:
    10064        db.close()
    10065
    10066
    10067def test_run_one_step_executes_accounting_before_saturated_record_tasks(tmp_path):
    10068    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10069    db = AgentDB(tmp_path / "state.db")
    10070    try:
    10071        job_id = db.create_job(
    10072            "Keep useful recovery state",
    10073            title="saturated-batch-order",
    10074            kind="generic",
    10075            metadata={
    10076                "task_queue": [
    10077                    {"title": f"Existing branch {index}", "status": "done", "priority": index}
    10078                    for index in range(84)
    10079                ]
    10080            },
    10081        )
    10082
    10083        result = run_one_step(
    10084            job_id,
    10085            config=config,
    10086            db=db,
    10087            llm=ScriptedLLM([
    10088                LLMResponse(tool_calls=[
    10089                    ToolCall(
    10090                        name="record_tasks",
    10091                        arguments={"tasks": [{"title": "New blocked branch", "status": "open"}]},
    10092                    ),
    10093                    ToolCall(
    10094                        name="record_lesson",
    10095                        arguments={"lesson": "Use the existing branch before adding more tasks.", "category": "strategy"},
    10096                    ),
    10097                ])
    10098            ]),
    10099        )
    10100
    10101        tool_steps = [step for step in db.list_steps(job_id=job_id) if step.get("kind") == "tool"]
    10102        assert [step["tool_name"] for step in tool_steps[-2:]] == ["record_lesson", "record_tasks"]
    10103        assert result.status == "blocked"
    10104        assert result.result["error"] == "task queue saturated"
    10105        lessons = db.get_job(job_id)["metadata"].get("lessons") or []
    10106        assert any("existing branch" in str(lesson.get("lesson") or "") for lesson in lessons)
    10107    finally:
    10108        db.close()
    10109
    10110
    10111def test_run_one_step_blocks_batch_that_would_saturate_open_tasks(tmp_path):
    10112    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10113    db = AgentDB(tmp_path / "state.db")
    10114    try:
    10115        job_id = db.create_job(
    10116            "Execute current branches before planning more",
    10117            title="projected-open-sprawl",
    10118            kind="generic",
    10119            metadata={
    10120                "task_queue": [
    10121                    {"title": f"Open branch {index}", "status": "open", "priority": index}
    10122                    for index in range(35)
    10123                ]
    10124            },
    10125        )
    10126
    10127        result = run_one_step(
    10128            job_id,
    10129            config=config,
    10130            db=db,
    10131            llm=ScriptedLLM([
    10132                LLMResponse(tool_calls=[
    10133                    ToolCall(
    10134                        name="record_tasks",
    10135                        arguments={
    10136                            "tasks": [
    10137                                {"title": f"New open branch {index}", "status": "open"}
    10138                                for index in range(5)
    10139                            ]
    10140                        },
    10141                    )
    10142                ])
    10143            ]),
    10144        )
    10145
    10146        assert result.status == "blocked"
    10147        assert result.result["error"] == "task queue saturated"
    10148        assert result.result["task_queue"]["reason"] == "too many open tasks"
    10149        assert result.result["task_queue"]["projected_open_count"] == 40
    10150        job = db.get_job(job_id)
    10151        assert len(job["metadata"]["task_queue"]) == 35
    10152    finally:
    10153        db.close()
    10154
    10155
    10156def test_run_one_step_ignores_guard_recovery_tasks_for_queue_saturation(tmp_path):
    10157    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10158    db = AgentDB(tmp_path / "state.db")
    10159    try:
    10160        job_id = db.create_job(
    10161            "Continue objective work after guard recovery",
    10162            title="guard-task-sprawl",
    10163            kind="generic",
    10164            metadata={
    10165                "task_queue": [
    10166                    {
    10167                        "title": f"Resolve guard: recoverable blocker {index}",
    10168                        "status": "open",
    10169                        "priority": 9,
    10170                        "metadata": {"guard_recovery": {"error": f"recoverable blocker {index}"}},
    10171                    }
    10172                    for index in range(45)
    10173                ]
    10174            },
    10175        )
    10176
    10177        result = run_one_step(
    10178            job_id,
    10179            config=config,
    10180            db=db,
    10181            llm=ScriptedLLM([
    10182                LLMResponse(tool_calls=[
    10183                    ToolCall(name="record_tasks", arguments={"tasks": [{"title": "Run next objective branch", "status": "open"}]})
    10184                ])
    10185            ]),
    10186        )
    10187
    10188        assert result.status == "completed"
    10189        assert result.tool_name == "record_tasks"
    10190        job = db.get_job(job_id)
    10191        assert any(task["title"] == "Run next objective branch" for task in job["metadata"]["task_queue"])
    10192    finally:
    10193        db.close()
    10194
    10195
    10196def test_run_one_step_ignores_guard_recovery_tasks_for_total_sprawl(tmp_path):
    10197    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10198    db = AgentDB(tmp_path / "state.db")
    10199    try:
    10200        job_id = db.create_job(
    10201            "Continue objective work after many recovered guards",
    10202            title="guard-total-sprawl",
    10203            kind="generic",
    10204            metadata={
    10205                "task_queue": [
    10206                    {
    10207                        "title": f"Resolve guard: recovered blocker {index}",
    10208                        "status": "done",
    10209                        "priority": 9,
    10210                        "metadata": {"guard_recovery": {"error": f"recovered blocker {index}"}},
    10211                    }
    10212                    for index in range(85)
    10213                ]
    10214            },
    10215        )
    10216
    10217        result = run_one_step(
    10218            job_id,
    10219            config=config,
    10220            db=db,
    10221            llm=ScriptedLLM([
    10222                LLMResponse(tool_calls=[
    10223                    ToolCall(name="record_tasks", arguments={"tasks": [{"title": "Fresh objective branch", "status": "open"}]})
    10224                ])
    10225            ]),
    10226        )
    10227
    10228        assert result.status == "completed"
    10229        assert result.tool_name == "record_tasks"
    10230        job = db.get_job(job_id)
    10231        assert any(task["title"] == "Fresh objective branch" for task in job["metadata"]["task_queue"])
    10232    finally:
    10233        db.close()
    10234
    10235
    10236def test_run_one_step_blocks_read_only_shell_churn(tmp_path):
    10237    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10238    db = AgentDB(tmp_path / "state.db")
    10239    try:
    10240        job_id = db.create_job("Choose from discovered candidates", title="read-only-churn", kind="generic")
    10241        for command in [
    10242            "find /tmp/work -type f | head",
    10243            "ls -lah /tmp/work",
    10244            "curl -s https://example.test/api/list | head -100",
    10245        ]:
    10246            run_id = db.start_run(job_id, model="test")
    10247            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec", input_data={"arguments": {"command": command}})
    10248            db.finish_step(step_id, status="completed", output_data={"success": True, "returncode": 0, "stdout": "candidate-a\ncandidate-b"})
    10249            db.finish_run(run_id, "completed")
    10250
    10251        result = run_one_step(
    10252            job_id,
    10253            config=config,
    10254            db=db,
    10255            llm=ScriptedLLM([
    10256                LLMResponse(tool_calls=[
    10257                    ToolCall(name="shell_exec", arguments={"command": "curl -s https://example.test/api/list?page=2"})
    10258                ])
    10259            ]),
    10260        )
    10261
    10262        assert result.status == "blocked"
    10263        assert result.result["error"] == "action decision required"
    10264        assert result.result["read_only_shell_churn"]["read_only_shell_count"] == 3
    10265    finally:
    10266        db.close()
    10267
    10268
    10269def test_run_one_step_allows_action_after_read_only_shell_churn(tmp_path):
    10270    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10271    db = AgentDB(tmp_path / "state.db")
    10272    try:
    10273        job_id = db.create_job("Act after discovered candidates", title="read-only-to-action", kind="generic")
    10274        for command in [
    10275            "find /tmp/work -type f | head",
    10276            "ls -lah /tmp/work",
    10277            "curl -s https://example.test/api/list | head -100",
    10278        ]:
    10279            run_id = db.start_run(job_id, model="test")
    10280            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec", input_data={"arguments": {"command": command}})
    10281            db.finish_step(step_id, status="completed", output_data={"success": True, "returncode": 0, "stdout": "candidate-a\ncandidate-b"})
    10282            db.finish_run(run_id, "completed")
    10283
    10284        result = run_one_step(
    10285            job_id,
    10286            config=config,
    10287            db=db,
    10288            llm=ScriptedLLM([
    10289                LLMResponse(tool_calls=[
    10290                    ToolCall(name="shell_exec", arguments={"command": "python run_candidate.py --input candidate-a"})
    10291                ])
    10292            ]),
    10293            registry=SuccessRegistry(),
    10294        )
    10295
    10296        assert result.status == "completed"
    10297        assert result.tool_name == "shell_exec"
    10298    finally:
    10299        db.close()
    10300
    10301
    10302def test_run_one_step_allows_read_only_shell_after_durable_decision(tmp_path):
    10303    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10304    db = AgentDB(tmp_path / "state.db")
    10305    try:
    10306        job_id = db.create_job("Recover from inspection churn", title="read-only-decision", kind="generic")
    10307        for command in [
    10308            "find /tmp/work -type f | head",
    10309            "ls -lah /tmp/work",
    10310            "curl -s https://example.test/api/list | head -100",
    10311        ]:
    10312            run_id = db.start_run(job_id, model="test")
    10313            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec", input_data={"arguments": {"command": command}})
    10314            db.finish_step(step_id, status="completed", output_data={"success": True, "returncode": 0, "stdout": "candidate-a\ncandidate-b"})
    10315            db.finish_run(run_id, "completed")
    10316
    10317        run_id = db.start_run(job_id, model="test")
    10318        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="record_lesson")
    10319        db.finish_step(
    10320            step_id,
    10321            status="completed",
    10322            output_data={"success": True, "lesson": {"category": "decision", "lesson": "Use candidate-a and inspect its exact metadata next."}},
    10323        )
    10324        db.finish_run(run_id, "completed")
    10325
    10326        result = run_one_step(
    10327            job_id,
    10328            config=config,
    10329            db=db,
    10330            llm=ScriptedLLM([
    10331                LLMResponse(tool_calls=[
    10332                    ToolCall(name="shell_exec", arguments={"command": "ls -lah /tmp/work/candidate-a"})
    10333                ])
    10334            ]),
    10335            registry=SuccessRegistry(),
    10336        )
    10337
    10338        assert result.status == "completed"
    10339        assert result.tool_name == "shell_exec"
    10340    finally:
    10341        db.close()
    10342
    10343
    10344def test_run_one_step_allows_explicit_download_after_read_only_shell_churn(tmp_path):
    10345    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10346    db = AgentDB(tmp_path / "state.db")
    10347    try:
    10348        job_id = db.create_job("Download selected candidate", title="read-only-to-download", kind="generic")
    10349        for command in [
    10350            "find /tmp/work -type f | head",
    10351            "ls -lah /tmp/work",
    10352            "curl -s https://example.test/api/list | head -100",
    10353        ]:
    10354            run_id = db.start_run(job_id, model="test")
    10355            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec", input_data={"arguments": {"command": command}})
    10356            db.finish_step(step_id, status="completed", output_data={"success": True, "returncode": 0, "stdout": "candidate-a\ncandidate-b"})
    10357            db.finish_run(run_id, "completed")
    10358
    10359        result = run_one_step(
    10360            job_id,
    10361            config=config,
    10362            db=db,
    10363            llm=ScriptedLLM([
    10364                LLMResponse(tool_calls=[
    10365                    ToolCall(name="shell_exec", arguments={"command": "curl -L -o /tmp/candidate.bin https://example.test/candidate.bin"})
    10366                ])
    10367            ]),
    10368            registry=SuccessRegistry(),
    10369        )
    10370
    10371        assert result.status == "completed"
    10372        assert result.tool_name == "shell_exec"
    10373    finally:
    10374        db.close()
    10375
    10376
    10377def test_run_one_step_blocks_new_tasks_when_queue_sprawls(tmp_path):
    10378    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10379    db = AgentDB(tmp_path / "state.db")
    10380    try:
    10381        job_id = db.create_job(
    10382            "Consolidate long-running work",
    10383            title="task-sprawl",
    10384            kind="generic",
    10385            metadata={
    10386                "task_queue": [
    10387                    {"title": f"Completed branch {index}", "status": "done", "priority": index}
    10388                    for index in range(80)
    10389                ]
    10390            },
    10391        )
    10392
    10393        result = run_one_step(
    10394            job_id,
    10395            config=config,
    10396            db=db,
    10397            llm=ScriptedLLM([
    10398                LLMResponse(tool_calls=[
    10399                    ToolCall(name="record_tasks", arguments={"tasks": [{"title": "New branch", "status": "open"}]})
    10400                ])
    10401            ]),
    10402        )
    10403
    10404        assert result.status == "blocked"
    10405        assert result.result["error"] == "task queue saturated"
    10406        assert result.result["task_queue"]["reason"] == "total task queue is too large"
    10407        assert result.result["task_queue"]["total_count"] == 80
    10408    finally:
    10409        db.close()
    10410
    10411
    10412def test_recent_task_saturation_keeps_record_tasks_for_existing_updates(tmp_path):
    10413    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10414    db = AgentDB(tmp_path / "state.db")
    10415    try:
    10416        job_id = db.create_job(
    10417            "Execute existing work",
    10418            title="saturated-tools",
    10419            kind="generic",
    10420            metadata={
    10421                "task_queue": [
    10422                    {"title": f"Open branch {index}", "status": "open", "priority": index}
    10423                    for index in range(40)
    10424                ]
    10425            },
    10426        )
    10427        first = run_one_step(
    10428            job_id,
    10429            config=config,
    10430            db=db,
    10431            llm=ScriptedLLM([
    10432                LLMResponse(tool_calls=[
    10433                    ToolCall(name="record_tasks", arguments={"tasks": [{"title": "New branch", "status": "open"}]})
    10434                ])
    10435            ]),
    10436        )
    10437        assert first.status == "blocked"
    10438        llm = CapturingLLM(LLMResponse(tool_calls=[ToolCall(name="record_lesson", arguments={"lesson": "execute existing work"})]))
    10439
    10440        run_one_step(job_id, config=config, db=db, llm=llm)
    10441
    10442        tool_names = {tool["function"]["name"] for tool in llm.tools}
    10443        prompt = llm.messages[-1]["content"]
    10444        assert "Task queue saturation" in prompt
    10445        assert "Do not create new task branches" in prompt
    10446        assert "Existing runnable task titles" in prompt
    10447        assert "Open branch 0" in prompt
    10448        assert "record_tasks only to update existing task titles" in prompt
    10449        assert "record_tasks" in tool_names
    10450        assert "record_lesson" in tool_names
    10451        assert "shell_exec" in tool_names
    10452    finally:
    10453        db.close()
    10454
    10455
    10456def test_repeated_task_saturation_temporarily_suppresses_record_tasks(tmp_path):
    10457    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10458    db = AgentDB(tmp_path / "state.db")
    10459    try:
    10460        job_id = db.create_job(
    10461            "Execute existing work",
    10462            title="repeated-saturation",
    10463            kind="generic",
    10464            metadata={
    10465                "task_queue": [
    10466                    {"title": f"Open branch {index}", "status": "open", "priority": index}
    10467                    for index in range(40)
    10468                ]
    10469            },
    10470        )
    10471        for title in ("New branch one", "New branch two"):
    10472            blocked = run_one_step(
    10473                job_id,
    10474                config=config,
    10475                db=db,
    10476                llm=ScriptedLLM([
    10477                    LLMResponse(tool_calls=[ToolCall(name="record_tasks", arguments={"tasks": [{"title": title, "status": "open"}]})])
    10478                ]),
    10479            )
    10480            assert blocked.status == "blocked"
    10481            assert blocked.result["error"] == "task queue saturated"
    10482
    10483        llm = CapturingLLM(LLMResponse(tool_calls=[ToolCall(name="record_lesson", arguments={"lesson": "execute existing branch"})]))
    10484        run_one_step(job_id, config=config, db=db, llm=llm)
    10485
    10486        tool_names = {tool["function"]["name"] for tool in llm.tools}
    10487        assert "record_tasks" not in tool_names
    10488        assert "record_lesson" in tool_names
    10489        assert "shell_exec" in tool_names
    10490    finally:
    10491        db.close()
    10492
    10493
    10494def test_chronic_backlog_suppresses_new_task_planning_tool(tmp_path):
    10495    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10496    db = AgentDB(tmp_path / "state.db")
    10497    try:
    10498        job_id = db.create_job(
    10499            "Execute existing work",
    10500            title="chronic-backlog-tools",
    10501            kind="generic",
    10502            metadata={
    10503                "task_queue": [
    10504                    {"title": f"Open branch {index}", "status": "open", "priority": index}
    10505                    for index in range(82)
    10506                ]
    10507            },
    10508        )
    10509        llm = CapturingLLM(LLMResponse(tool_calls=[ToolCall(name="record_lesson", arguments={"lesson": "execute existing work"})]))
    10510
    10511        run_one_step(job_id, config=config, db=db, llm=llm)
    10512
    10513        tool_names = {tool["function"]["name"] for tool in llm.tools}
    10514        prompt = llm.messages[-1]["content"]
    10515        assert "Current execution focus" in prompt
    10516        assert "backlog=82 tasks" in prompt
    10517        assert "record_tasks" not in tool_names
    10518        assert "record_lesson" in tool_names
    10519        assert "shell_exec" in tool_names
    10520    finally:
    10521        db.close()
    10522
    10523
    10524def test_run_one_step_allows_existing_task_update_when_queue_is_saturated(tmp_path):
    10525    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10526    db = AgentDB(tmp_path / "state.db")
    10527    try:
    10528        job_id = db.create_job(
    10529            "Finish existing work",
    10530            title="saturated",
    10531            kind="generic",
    10532            metadata={
    10533                "task_queue": [
    10534                    {"title": f"Open branch {index}", "status": "open", "priority": index}
    10535                    for index in range(40)
    10536                ]
    10537            },
    10538        )
    10539
    10540        result = run_one_step(
    10541            job_id,
    10542            config=config,
    10543            db=db,
    10544            llm=ScriptedLLM([
    10545                LLMResponse(tool_calls=[
    10546                    ToolCall(name="record_tasks", arguments={"tasks": [{"title": "Open branch 0", "status": "active"}]})
    10547                ])
    10548            ]),
    10549        )
    10550
    10551        assert result.status == "completed"
    10552        assert result.tool_name == "record_tasks"
    10553        job = db.get_job(job_id)
    10554        assert job["metadata"]["task_queue"][0]["status"] == "active"
    10555    finally:
    10556        db.close()
    10557
    10558
    10559def test_run_one_step_allows_semantic_task_update_when_queue_is_saturated(tmp_path):
    10560    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10561    db = AgentDB(tmp_path / "state.db")
    10562    try:
    10563        job_id = db.create_job(
    10564            "Finish existing work",
    10565            title="semantic-saturated",
    10566            kind="generic",
    10567            metadata={
    10568                "task_queue": [
    10569                    {
    10570                        "title": "Validate model files and run baseline benchmark",
    10571                        "status": "open",
    10572                        "priority": 5,
    10573                    },
    10574                    *[
    10575                        {"title": f"Completed branch {index}", "status": "done", "priority": index}
    10576                        for index in range(81)
    10577                    ],
    10578                ]
    10579            },
    10580        )
    10581
    10582        result = run_one_step(
    10583            job_id,
    10584            config=config,
    10585            db=db,
    10586            llm=ScriptedLLM([
    10587                LLMResponse(tool_calls=[
    10588                    ToolCall(
    10589                        name="record_tasks",
    10590                        arguments={
    10591                            "tasks": [{
    10592                                "title": "Validate candidate model files and run baseline benchmark",
    10593                                "status": "active",
    10594                                "priority": 10,
    10595                            }]
    10596                        },
    10597                    )
    10598                ])
    10599            ]),
    10600        )
    10601
    10602        assert result.status == "completed"
    10603        assert result.tool_name == "record_tasks"
    10604        job = db.get_job(job_id)
    10605        task = job["metadata"]["task_queue"][0]
    10606        assert task["title"] == "Validate model files and run baseline benchmark"
    10607        assert task["status"] == "active"
    10608        assert task["metadata"]["original_title"] == "Validate candidate model files and run baseline benchmark"
    10609    finally:
    10610        db.close()
    10611
    10612
    10613def test_run_one_step_auto_records_anti_bot_browser_source(tmp_path):
    10614    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10615    db = AgentDB(tmp_path / "state.db")
    10616    try:
    10617        job_id = db.create_job("Avoid blocked browser pages", title="guard")
    10618
    10619        result = run_one_step(
    10620            job_id,
    10621            config=config,
    10622            db=db,
    10623            llm=ScriptedLLM([
    10624                LLMResponse(tool_calls=[ToolCall(name="browser_snapshot", arguments={"full": True})])
    10625            ]),
    10626            registry=AntiBotBrowserRegistry(),
    10627        )
    10628        job = db.get_job(job_id)
    10629        source = job["metadata"]["source_ledger"][0]
    10630
    10631        assert result.status == "completed"
    10632        assert result.result["source_warning"] == "captcha/anti-bot block"
    10633        assert source["source"] == "https://source.example/search"
    10634        assert source["fail_count"] == 1
    10635        assert source["usefulness_score"] == 0.02
    10636        assert job["metadata"]["last_lesson"]["category"] == "source_quality"
    10637    finally:
    10638        db.close()
    10639
    10640
    10641def test_run_one_step_blocks_misleading_artifact_after_anti_bot_snapshot(tmp_path):
    10642    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10643    db = AgentDB(tmp_path / "state.db")
    10644    try:
    10645        job_id = db.create_job("Do not invent findings from blocked pages", title="guard")
    10646        run_id = db.start_run(job_id)
    10647        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="browser_snapshot")
    10648        db.finish_step(
    10649            step_id,
    10650            status="completed",
    10651            output_data={
    10652                "success": True,
    10653                "data": {
    10654                    "origin": "https://source.example/search",
    10655                    "snapshot": 'Iframe "Security CAPTCHA" You have been blocked.',
    10656                },
    10657            },
    10658            summary="browser_snapshot returned 1250 chars",
    10659        )
    10660        db.finish_run(run_id, "completed")
    10661
    10662        result = run_one_step(
    10663            job_id,
    10664            config=config,
    10665            db=db,
    10666            llm=ScriptedLLM([
    10667                LLMResponse(tool_calls=[ToolCall(
    10668                    name="write_artifact",
    10669                    arguments={
    10670                        "title": "Directory finding source",
    10671                        "summary": "Contains result listings for finding extraction",
    10672                        "content": "This source contains reusable findings.",
    10673                    },
    10674                )])
    10675            ]),
    10676        )
    10677        job = db.get_job(job_id)
    10678
    10679        assert result.status == "blocked"
    10680        assert result.result["error"] == "misleading blocked-source artifact blocked"
    10681        assert result.result["auto_source_record"]["source"]["source"] == "https://source.example/search"
    10682        assert db.list_artifacts(job_id) == []
    10683        assert job["metadata"]["source_ledger"][0]["warnings"] == ["captcha/anti-bot block"]
    10684    finally:
    10685        db.close()
    10686
    10687
    10688def test_run_one_step_allows_blocked_source_artifact_when_acknowledged(tmp_path):
    10689    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10690    db = AgentDB(tmp_path / "state.db")
    10691    try:
    10692        job_id = db.create_job("Save blocked source notes", title="guard")
    10693        run_id = db.start_run(job_id)
    10694        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="browser_snapshot")
    10695        db.finish_step(
    10696            step_id,
    10697            status="completed",
    10698            output_data={
    10699                "success": True,
    10700                "data": {
    10701                    "origin": "https://source.example/search",
    10702                    "snapshot": 'Iframe "Security CAPTCHA" You have been blocked.',
    10703                },
    10704            },
    10705        )
    10706        db.finish_run(run_id, "completed")
    10707
    10708        result = run_one_step(
    10709            job_id,
    10710            config=config,
    10711            db=db,
    10712            llm=ScriptedLLM([
    10713                LLMResponse(tool_calls=[ToolCall(
    10714                    name="write_artifact",
    10715                    arguments={
    10716                        "title": "Blocked source note",
    10717                        "summary": "Blocked by CAPTCHA; not usable as finding evidence",
    10718                        "content": "The page showed a CAPTCHA and no usable evidence was visible.",
    10719                    },
    10720                )])
    10721            ]),
    10722        )
    10723
    10724        assert result.status == "completed"
    10725        assert db.list_artifacts(job_id)[0]["title"] == "Blocked source note"
    10726    finally:
    10727        db.close()
    10728
    10729
    10730def test_run_one_step_blocks_browser_loop_after_anti_bot_snapshot(tmp_path):
    10731    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10732    db = AgentDB(tmp_path / "state.db")
    10733    try:
    10734        job_id = db.create_job("Pivot after blocked browser pages", title="guard")
    10735        run_id = db.start_run(job_id)
    10736        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="browser_snapshot")
    10737        db.finish_step(
    10738            step_id,
    10739            status="completed",
    10740            output_data={
    10741                "success": True,
    10742                "data": {
    10743                    "origin": "https://source.example/search",
    10744                    "snapshot": 'Iframe "Security CAPTCHA" You have been blocked.',
    10745                },
    10746            },
    10747        )
    10748        db.finish_run(run_id, "completed")
    10749
    10750        result = run_one_step(
    10751            job_id,
    10752            config=config,
    10753            db=db,
    10754            llm=ScriptedLLM([
    10755                LLMResponse(tool_calls=[ToolCall(name="browser_scroll", arguments={"direction": "down"})])
    10756            ]),
    10757        )
    10758
    10759        assert result.status == "blocked"
    10760        assert result.result["error"] == "anti-bot source loop blocked"
    10761        assert result.result["auto_source_record"]["source"]["fail_count"] == 1
    10762    finally:
    10763        db.close()
    10764
    10765
    10766def test_run_one_step_blocks_known_bad_browser_source_from_ledger(tmp_path):
    10767    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10768    db = AgentDB(tmp_path / "state.db")
    10769    try:
    10770        job_id = db.create_job("Avoid sources already scored as bad", title="guard")
    10771        db.append_source_record(
    10772            job_id,
    10773            "https://blocked.example/search",
    10774            source_type="blocked_browser_source",
    10775            usefulness_score=0.02,
    10776            fail_count_delta=1,
    10777            warnings=["captcha/anti-bot block"],
    10778            outcome="blocked; pivot",
    10779        )
    10780
    10781        result = run_one_step(
    10782            job_id,
    10783            config=config,
    10784            db=db,
    10785            llm=ScriptedLLM([
    10786                LLMResponse(tool_calls=[ToolCall(name="browser_navigate", arguments={"url": "https://www.blocked.example/search?page=2"})])
    10787            ]),
    10788        )
    10789        job = db.get_job(job_id)
    10790
    10791        assert result.status == "blocked"
    10792        assert result.result["error"] == "known bad source blocked"
    10793        assert result.result["known_bad_source"]["source"] == "https://blocked.example/search"
    10794        assert job["metadata"]["last_agent_update"]["category"] == "blocked"
    10795    finally:
    10796        db.close()
    10797
    10798
    10799def test_run_one_step_blocks_known_bad_extract_source_from_ledger(tmp_path):
    10800    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10801    db = AgentDB(tmp_path / "state.db")
    10802    try:
    10803        job_id = db.create_job("Avoid extracting bad sources", title="guard")
    10804        db.append_source_record(
    10805            job_id,
    10806            "https://lowyield.example/source",
    10807            source_type="web_source",
    10808            usefulness_score=0.05,
    10809            fail_count_delta=2,
    10810            outcome="no useful candidates",
    10811        )
    10812
    10813        result = run_one_step(
    10814            job_id,
    10815            config=config,
    10816            db=db,
    10817            llm=ScriptedLLM([
    10818                LLMResponse(tool_calls=[ToolCall(
    10819                    name="web_extract",
    10820                    arguments={"urls": ["https://lowyield.example/source?retry=1"]},
    10821                )])
    10822            ]),
    10823        )
    10824
    10825        assert result.status == "blocked"
    10826        assert result.result["error"] == "known bad source blocked"
    10827        assert result.result["known_bad_source"]["fail_count"] == 2
    10828    finally:
    10829        db.close()
    10830
    10831
    10832def test_run_one_step_allows_child_url_when_bad_web_source_is_domain_root(tmp_path):
    10833    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10834    db = AgentDB(tmp_path / "state.db")
    10835    try:
    10836        job_id = db.create_job("Avoid over-broad domain source blocks", title="guard")
    10837        db.append_source_record(
    10838            job_id,
    10839            "https://source.example",
    10840            source_type="web_source",
    10841            usefulness_score=0.05,
    10842            fail_count_delta=1,
    10843            outcome="root health check failed",
    10844        )
    10845
    10846        result = run_one_step(
    10847            job_id,
    10848            config=config,
    10849            db=db,
    10850            llm=ScriptedLLM([
    10851                LLMResponse(tool_calls=[ToolCall(
    10852                    name="web_extract",
    10853                    arguments={"urls": ["https://source.example/api/public/models"]},
    10854                )])
    10855            ]),
    10856            registry=SuccessRegistry(),
    10857        )
    10858
    10859        assert result.status == "completed"
    10860        assert result.tool_name == "web_extract"
    10861    finally:
    10862        db.close()
    10863
    10864
    10865def test_run_one_step_records_failed_shell_url_source(tmp_path):
    10866    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10867    db = AgentDB(tmp_path / "state.db")
    10868    try:
    10869        job_id = db.create_job("Avoid broken shell URL sources", title="guard")
    10870        url = "https://source.example/api/private/tree/main"
    10871
    10872        result = run_one_step(
    10873            job_id,
    10874            config=config,
    10875            db=db,
    10876            llm=ScriptedLLM([
    10877                LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": f"curl -s {url}"})])
    10878            ]),
    10879            registry=FailedUrlShellRegistry(),
    10880        )
    10881        sources = db.get_job(job_id)["metadata"]["source_ledger"]
    10882        source = sources[0]
    10883
    10884        assert result.status == "failed"
    10885        assert source["source"] == url
    10886        assert source["source_type"] == "shell_exec"
    10887        assert source["fail_count"] == 1
    10888        assert source["usefulness_score"] == 0.01
    10889        assert source["metadata"]["failure_kind"] == "auth_or_http"
    10890    finally:
    10891        db.close()
    10892
    10893
    10894def test_run_one_step_records_pathful_failed_shell_urls_not_root_health_checks(tmp_path):
    10895    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10896    db = AgentDB(tmp_path / "state.db")
    10897    try:
    10898        job_id = db.create_job("Avoid poisoning whole hosts from mixed probes", title="guard")
    10899        bad_url = "https://source.example/api/private/tree/main"
    10900
    10901        result = run_one_step(
    10902            job_id,
    10903            config=config,
    10904            db=db,
    10905            llm=ScriptedLLM([
    10906                LLMResponse(tool_calls=[ToolCall(
    10907                    name="shell_exec",
    10908                    arguments={"command": f"curl -sI https://source.example && curl -s {bad_url}"},
    10909                )])
    10910            ]),
    10911            registry=FailedUrlShellRegistry(),
    10912        )
    10913        sources = db.get_job(job_id)["metadata"]["source_ledger"]
    10914
    10915        assert result.status == "failed"
    10916        assert [source["source"] for source in sources] == [bad_url]
    10917    finally:
    10918        db.close()
    10919
    10920
    10921def test_run_one_step_blocks_known_bad_shell_source_family(tmp_path):
    10922    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10923    db = AgentDB(tmp_path / "state.db")
    10924    try:
    10925        job_id = db.create_job("Pivot from failed source family", title="guard")
    10926
    10927        first = run_one_step(
    10928            job_id,
    10929            config=config,
    10930            db=db,
    10931            llm=ScriptedLLM([
    10932                LLMResponse(tool_calls=[ToolCall(
    10933                    name="shell_exec",
    10934                    arguments={"command": "curl -L https://source.example/downloads/private/model-a.bin"},
    10935                )])
    10936            ]),
    10937            registry=FailedUrlShellRegistry(),
    10938        )
    10939        blocked = run_one_step(
    10940            job_id,
    10941            config=config,
    10942            db=db,
    10943            llm=ScriptedLLM([
    10944                LLMResponse(tool_calls=[ToolCall(
    10945                    name="shell_exec",
    10946                    arguments={"command": "curl -L https://source.example/downloads/private/model-b.bin"},
    10947                )])
    10948            ]),
    10949        )
    10950        allowed = run_one_step(
    10951            job_id,
    10952            config=config,
    10953            db=db,
    10954            llm=ScriptedLLM([
    10955                LLMResponse(tool_calls=[ToolCall(
    10956                    name="shell_exec",
    10957                    arguments={"command": "curl -L https://source.example/downloads/public/model-b.bin"},
    10958                )])
    10959            ]),
    10960            registry=SuccessRegistry(),
    10961        )
    10962
    10963        assert first.status == "failed"
    10964        assert blocked.status == "blocked"
    10965        assert blocked.result["error"] == "known bad source blocked"
    10966        assert blocked.result["known_bad_source"]["source"] == "https://source.example/downloads/private"
    10967        assert allowed.status == "completed"
    10968    finally:
    10969        db.close()
    10970
    10971
    10972def test_run_one_step_derives_bad_shell_source_family_from_exact_failure(tmp_path):
    10973    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    10974    db = AgentDB(tmp_path / "state.db")
    10975    try:
    10976        job_id = db.create_job("Pivot from exact failed file source", title="guard")
    10977        db.append_source_record(
    10978            job_id,
    10979            "https://source.example/downloads/private/model-a.bin",
    10980            source_type="shell_exec",
    10981            usefulness_score=0.01,
    10982            fail_count_delta=1,
    10983            warnings=["auth failure"],
    10984            outcome="401 Unauthorized",
    10985        )
    10986
    10987        blocked = run_one_step(
    10988            job_id,
    10989            config=config,
    10990            db=db,
    10991            llm=ScriptedLLM([
    10992                LLMResponse(tool_calls=[ToolCall(
    10993                    name="shell_exec",
    10994                    arguments={"command": "curl -L https://source.example/downloads/private/model-b.bin"},
    10995                )])
    10996            ]),
    10997        )
    10998
    10999        assert blocked.status == "blocked"
    11000        assert blocked.result["error"] == "known bad source blocked"
    11001        assert blocked.result["known_bad_source"]["source"] == "https://source.example/downloads/private"
    11002        assert blocked.result["known_bad_source"]["metadata"]["source_family_from"].endswith("/model-a.bin")
    11003    finally:
    11004        db.close()
    11005
    11006
    11007def test_run_one_step_does_not_block_entire_host_after_auth_source_families(tmp_path):
    11008    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    11009    db = AgentDB(tmp_path / "state.db")
    11010    try:
    11011        job_id = db.create_job("Pivot from repeated host auth failures", title="guard")
    11012        for source in (
    11013            "https://source.example/private/a/model.bin",
    11014            "https://source.example/private/b/model.bin",
    11015            "https://source.example/private/c/model.bin",
    11016        ):
    11017            db.append_source_record(
    11018                job_id,
    11019                source,
    11020                source_type="shell_exec",
    11021                usefulness_score=0.01,
    11022                fail_count_delta=1,
    11023                warnings=["401 unauthorized"],
    11024                outcome="HTTP 401 Unauthorized",
    11025                metadata={"failure_kind": "auth_or_http"},
    11026            )
    11027
    11028        allowed_same_host = run_one_step(
    11029            job_id,
    11030            config=config,
    11031            db=db,
    11032            llm=ScriptedLLM([
    11033                LLMResponse(tool_calls=[ToolCall(
    11034                    name="shell_exec",
    11035                    arguments={"command": "curl -L https://source.example/private/d/model.bin"},
    11036                )])
    11037            ]),
    11038            registry=SuccessRegistry(),
    11039        )
    11040        allowed = run_one_step(
    11041            job_id,
    11042            config=config,
    11043            db=db,
    11044            llm=ScriptedLLM([
    11045                LLMResponse(tool_calls=[ToolCall(
    11046                    name="shell_exec",
    11047                    arguments={"command": "curl -L https://other.example/private/d/model.bin"},
    11048                )])
    11049            ]),
    11050            registry=SuccessRegistry(),
    11051        )
    11052
    11053        assert allowed_same_host.status == "completed"
    11054        assert allowed.status == "completed"
    11055    finally:
    11056        db.close()
    11057
    11058
    11059def test_run_one_step_blocks_known_bad_shell_source_path(tmp_path):
    11060    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    11061    db = AgentDB(tmp_path / "state.db")
    11062    try:
    11063        job_id = db.create_job("Pivot from failed shell URL source", title="guard")
    11064        db.append_source_record(
    11065            job_id,
    11066            "https://source.example/api/private/tree/main",
    11067            source_type="shell_exec",
    11068            usefulness_score=0.01,
    11069            fail_count_delta=1,
    11070            warnings=["auth failure"],
    11071            outcome="401 Unauthorized",
    11072        )
    11073
    11074        blocked = run_one_step(
    11075            job_id,
    11076            config=config,
    11077            db=db,
    11078            llm=ScriptedLLM([
    11079                LLMResponse(tool_calls=[ToolCall(
    11080                    name="shell_exec",
    11081                    arguments={"command": "curl -s 'https://source.example/api/private/tree/main?recursive=true'"},
    11082                )])
    11083            ]),
    11084        )
    11085        allowed = run_one_step(
    11086            job_id,
    11087            config=config,
    11088            db=db,
    11089            llm=ScriptedLLM([
    11090                LLMResponse(tool_calls=[ToolCall(
    11091                    name="shell_exec",
    11092                    arguments={"command": "curl -s 'https://source.example/api/public/models'"},
    11093                )])
    11094            ]),
    11095            registry=SuccessRegistry(),
    11096        )
    11097
    11098        assert blocked.status == "blocked"
    11099        assert blocked.result["error"] == "known bad source blocked"
    11100        assert blocked.result["known_bad_source"]["source"] == "https://source.example/api/private/tree/main"
    11101        assert allowed.status == "completed"
    11102    finally:
    11103        db.close()
    11104
    11105
    11106def test_run_one_step_allows_mixed_shell_command_with_bad_root_health_check(tmp_path):
    11107    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    11108    db = AgentDB(tmp_path / "state.db")
    11109    try:
    11110        job_id = db.create_job("Avoid over-broad shell root source blocks", title="guard")
    11111        db.append_source_record(
    11112            job_id,
    11113            "https://source.example",
    11114            source_type="shell_exec",
    11115            usefulness_score=0.01,
    11116            fail_count_delta=1,
    11117            warnings=["root health check failed earlier"],
    11118            outcome="HTTP failure",
    11119        )
    11120
    11121        result = run_one_step(
    11122            job_id,
    11123            config=config,
    11124            db=db,
    11125            llm=ScriptedLLM([
    11126                LLMResponse(tool_calls=[ToolCall(
    11127                    name="shell_exec",
    11128                    arguments={"command": "curl -sI https://source.example && curl -s https://source.example/api/public/models"},
    11129                )])
    11130            ]),
    11131            registry=SuccessRegistry(),
    11132        )
    11133
    11134        assert result.status == "completed"
    11135        assert result.tool_name == "shell_exec"
    11136    finally:
    11137        db.close()
    11138
    11139
    11140def test_run_one_step_saves_unpersisted_evidence_before_known_bad_source_block(tmp_path):
    11141    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    11142    db = AgentDB(tmp_path / "state.db")
    11143    try:
    11144        job_id = db.create_job("Evidence checkpoint still wins", title="guard")
    11145        db.append_source_record(
    11146            job_id,
    11147            "https://blocked.example/search",
    11148            source_type="blocked_browser_source",
    11149            usefulness_score=0.02,
    11150            fail_count_delta=1,
    11151            warnings=["captcha/anti-bot block"],
    11152            outcome="blocked; pivot",
    11153        )
    11154        run_id = db.start_run(job_id)
    11155        step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="browser_snapshot")
    11156        db.finish_step(
    11157            step_id,
    11158            status="completed",
    11159            output_data={
    11160                "success": True,
    11161                "data": {"origin": "https://useful.example"},
    11162                "snapshot": "Useful source evidence. " * 80,
    11163            },
    11164        )
    11165        db.finish_run(run_id, "completed")
    11166
    11167        result = run_one_step(
    11168            job_id,
    11169            config=config,
    11170            db=db,
    11171            llm=ScriptedLLM([
    11172                LLMResponse(tool_calls=[ToolCall(name="browser_navigate", arguments={"url": "https://blocked.example/search"})])
    11173            ]),
    11174        )
    11175
    11176        assert result.status == "blocked"
    11177        assert result.result["error"] == "artifact required before more research"
    11178        assert "auto_checkpoint" in result.result
    11179        assert result.result["auto_checkpoint"]["artifact_id"]
    11180    finally:
    11181        db.close()
    11182
    11183
    11184def test_run_one_step_blocks_search_streak(tmp_path):
    11185    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    11186    db = AgentDB(tmp_path / "state.db")
    11187    try:
    11188        job_id = db.create_job("Do not search forever", title="guard")
    11189        for query in ("alpha findings", "beta findings", "gamma findings"):
    11190            run_id = db.start_run(job_id)
    11191            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="web_search", input_data={"arguments": {"query": query}})
    11192            db.finish_step(step_id, status="completed", output_data={"success": True, "query": query, "results": []})
    11193            db.finish_run(run_id, "completed")
    11194
    11195        result = run_one_step(
    11196            job_id,
    11197            config=config,
    11198            db=db,
    11199            llm=ScriptedLLM([
    11200                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "delta findings", "limit": 5})])
    11201            ]),
    11202        )
    11203
    11204        assert result.status == "blocked"
    11205        assert result.result["error"] == "search loop blocked"
    11206        assert result.result["recent_search_streak"] == 3
    11207    finally:
    11208        db.close()
    11209
    11210
    11211def test_run_one_step_blocks_similar_search_query(tmp_path):
    11212    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    11213    db = AgentDB(tmp_path / "state.db")
    11214    try:
    11215        job_id = db.create_job("Avoid query rewrites", title="guard")
    11216        run_id = db.start_run(job_id)
    11217        step_id = db.add_step(
    11218            job_id=job_id,
    11219            run_id=run_id,
    11220            kind="tool",
    11221            tool_name="web_search",
    11222            input_data={"arguments": {"query": "target digital marketing research"}},
    11223        )
    11224        db.finish_step(step_id, status="completed", output_data={"success": True, "query": "target digital marketing research", "results": []})
    11225        db.finish_run(run_id, "completed")
    11226
    11227        result = run_one_step(
    11228            job_id,
    11229            config=config,
    11230            db=db,
    11231            llm=ScriptedLLM([
    11232                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "target marketing digital research", "limit": 5})])
    11233            ]),
    11234        )
    11235
    11236        assert result.status == "blocked"
    11237        assert result.result["error"] == "similar search query blocked"
    11238    finally:
    11239        db.close()
    11240
    11241
    11242def test_run_one_step_reflects_every_fixed_interval(tmp_path):
    11243    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    11244    db = AgentDB(tmp_path / "state.db")
    11245    try:
    11246        job_id = db.create_job("Reflect over work", title="reflect")
    11247        for index in range(12):
    11248            run_id = db.start_run(job_id)
    11249            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="web_search")
    11250            db.finish_step(step_id, status="completed", summary=f"step {index}", output_data={"success": True})
    11251            db.finish_run(run_id, "completed")
    11252
    11253        result = run_one_step(
    11254            job_id,
    11255            config=config,
    11256            db=db,
    11257            llm=ScriptedLLM([
    11258                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "should not be used"})])
    11259            ]),
    11260        )
    11261        job = db.get_job(job_id)
    11262
    11263        assert result.tool_name == "reflect"
    11264        assert result.status == "completed"
    11265        assert job["metadata"]["reflections"]
    11266        assert job["metadata"]["last_agent_update"]["category"] == "plan"
    11267        assert "Lessons learned:" in build_messages(job, db.list_steps(job_id=job_id))[-1]["content"]
    11268    finally:
    11269        db.close()
    11270
    11271
    11272def test_reflection_does_not_repeat_existing_strategy_lesson(tmp_path):
    11273    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    11274    db = AgentDB(tmp_path / "state.db")
    11275    strategy = "Choose the next branch from durable evidence, then record the result as findings, tasks, experiments, sources, or memory."
    11276    try:
    11277        job_id = db.create_job("Reflect over repeated work", title="reflect")
    11278        db.append_lesson(job_id, strategy, category="strategy")
    11279        for index in range(12):
    11280            run_id = db.start_run(job_id)
    11281            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="web_search")
    11282            db.finish_step(step_id, status="completed", summary=f"step {index}", output_data={"success": True})
    11283            db.finish_run(run_id, "completed")
    11284
    11285        result = run_one_step(
    11286            job_id,
    11287            config=config,
    11288            db=db,
    11289            llm=ScriptedLLM([
    11290                LLMResponse(tool_calls=[ToolCall(name="web_search", arguments={"query": "should not be used"})])
    11291            ]),
    11292        )
    11293        job = db.get_job(job_id)
    11294
    11295        assert result.tool_name == "reflect"
    11296        assert result.result["lesson_recorded"] is False
    11297        assert len(job["metadata"]["lessons"]) == 1
    11298        assert job["metadata"]["lessons"][0].get("seen_count") is None
    11299    finally:
    11300        db.close()
    11301
    11302
    11303def test_reflection_strategy_uses_current_operator_state(tmp_path):
    11304    config = AppConfig(runtime=RuntimeConfig(home=tmp_path))
    11305    db = AgentDB(tmp_path / "state.db")
    11306    try:
    11307        job_id = db.create_job(
    11308            "Reflect over operator context",
    11309            title="reflect",
    11310            metadata={
    11311                "operator_messages": [
    11312                    {
    11313                        "id": "op_1",
    11314                        "mode": "steer",
    11315                        "message": "Use the corrected target before continuing.",
    11316                        "created_at": "2026-01-01T00:00:00+00:00",
    11317                    }
    11318                ]
    11319            },
    11320        )
    11321        for index in range(12):
    11322            run_id = db.start_run(job_id)
    11323            step_id = db.add_step(job_id=job_id, run_id=run_id, kind="tool", tool_name="shell_exec")
    11324            db.finish_step(step_id, status="completed", summary=f"step {index}", output_data={"success": True})
    11325            db.finish_run(run_id, "completed")
    11326
    11327        result = run_one_step(
    11328            job_id,
    11329            config=config,
    11330            db=db,
    11331            llm=ScriptedLLM([
    11332                LLMResponse(tool_calls=[ToolCall(name="shell_exec", arguments={"command": "should not run"})])
    11333            ]),
    11334        )
    11335
    11336        assert result.tool_name == "reflect"
    11337        assert "Incorporate or supersede active operator context" in result.result["reflection"]["strategy"]
    11338    finally:
    11339        db.close()

Verification

## Test Coverage Map

Test files included in the source index.

### tests/nipux_cli/test_artifacts.py

2 tests · 34 lines

test_artifact_store_writes_reads_and_searches, test_artifact_store_rejects_paths_outside_home

### tests/nipux_cli/test_browser_web.py

8 tests · 118 lines

test_session_name_is_stable_and_safe, test_long_session_name_is_short_and_hashed, test_strip_html_removes_scripts_and_keeps_text, test_browser_marks_anti_bot_interstitial_as_warning, test_browser_marks_captcha_block_as_warning, test_web_extract_marks_anti_bot_pages_as_warning, test_browser_tool_uses_native_wrapper, ...

### tests/nipux_cli/test_cli.py

184 tests · 4970 lines

test_cli_has_operator_commands, test_cli_version_flag, test_main_catches_keyboard_interrupt_without_traceback, test_python_module_entrypoint_uses_cli_main, test_init_openrouter_writes_secret_free_config_and_env_template, test_init_defaults_to_local_endpoint, test_init_openrouter_defaults_to_generic_route, test_shell...

### tests/nipux_cli/test_cli_model_preflight.py

5 tests · 86 lines

test_remote_model_preflight_blocks_rejected_auth, test_remote_model_preflight_allows_recovery_monitor_for_quota, test_remote_model_preflight_skips_fake_runs, test_model_preflight_checks_local_endpoints, test_start_does_not_spawn_daemon_when_model_preflight_fails

### tests/nipux_cli/test_compression.py

1 tests · 101 lines

test_refresh_memory_index_includes_durable_progress_ledgers

### tests/nipux_cli/test_config.py

6 tests · 143 lines

test_load_config_defaults_to_local_endpoint, test_load_config_from_yaml, test_load_config_reads_local_env_file, test_load_config_tightens_local_env_permissions, test_default_config_yaml_allows_provider_template_without_secret, test_config_example_matches_default_local_endpoint

### tests/nipux_cli/test_daemon.py

30 tests · 560 lines

test_single_instance_lock_rejects_second_holder, test_daemon_lock_status_reports_free_lock, test_lock_metadata_can_be_updated_while_held, test_lock_metadata_update_restores_missing_process_fields, test_daemon_lock_heartbeat_updates_while_worker_turn_runs, test_stop_daemon_recovers_pidless_lock_from_process_list, tes...

### tests/nipux_cli/test_dashboard.py

3 tests · 86 lines

test_dashboard_collects_jobs_steps_and_artifacts, test_overview_marks_idle_daemon_as_ready_for_work, test_overview_marks_old_heartbeat_as_busy_for_running_step

### tests/nipux_cli/test_db.py

19 tests · 609 lines

test_db_job_run_step_and_artifact_roundtrip, test_create_job_uses_unique_readable_slug_ids, test_step_numbers_increment_across_runs_for_a_job, test_job_token_usage_aggregates_message_usage, test_append_operator_message_roundtrip, test_claim_operator_messages_marks_one_message_at_a_time, test_acknowledge_operator_mes...

### tests/nipux_cli/test_digest.py

1 tests · 43 lines

test_daily_digest_includes_ledgers_lessons_sources_and_strategy

### tests/nipux_cli/test_doctor.py

5 tests · 157 lines

test_doctor_checks_local_runtime_without_model_call, test_doctor_warns_when_remote_model_key_is_missing, test_doctor_reports_openrouter_auth_failure, test_doctor_reports_generation_limit_after_model_listing, test_doctor_reports_nested_provider_generation_error

### tests/nipux_cli/test_generic_runtime_audit.py

1 tests · 35 lines

test_runtime_code_has_no_task_specific_literals

### tests/nipux_cli/test_live_memory_graph_smoke.py

2 tests · 37 lines

test_live_memory_graph_smoke_fails_cleanly_without_key, test_live_memory_graph_smoke_seed_pushes_generic_consolidation

### tests/nipux_cli/test_llm.py

5 tests · 151 lines

test_chat_llm_requires_tool_choice_for_worker_actions, test_chat_llm_retries_without_tool_choice_when_provider_rejects_it, test_chat_llm_complete_response_returns_usage, test_chat_llm_disables_provider_sdk_retries, test_openrouter_generation_usage_enriches_cost_and_tokens

### tests/nipux_cli/test_measurement.py

2 tests · 32 lines

test_measurement_candidates_extract_markdown_table_unit_columns, test_measurement_candidates_extract_generic_table_metrics

### tests/nipux_cli/test_metric_format.py

2 tests · 11 lines

test_format_metric_value_spaces_named_units, test_format_metric_value_keeps_attached_symbol_units

### tests/nipux_cli/test_operator_context.py

3 tests · 30 lines

test_conversation_only_operator_messages_do_not_enter_worker_prompt, test_actionable_operator_messages_remain_worker_constraints, test_inactive_prompt_operator_ids_returns_only_conversation_active_messages

### tests/nipux_cli/test_planning.py

8 tests · 90 lines

test_initial_task_contracts_are_generic_and_complete, test_initial_roadmap_uses_valid_generic_contracts, test_initial_plan_adapts_to_measurable_objectives, test_initial_plan_adapts_to_deliverable_objectives, test_initial_plan_treats_generated_files_as_deliverables, test_initial_plan_adapts_to_monitoring_objectives, ...

### tests/nipux_cli/test_progress.py

7 tests · 214 lines

test_progress_checkpoint_reports_deltas_and_recent_durable_work, test_progress_checkpoint_for_saved_output_is_concise, test_progress_checkpoint_without_delta_is_activity_not_progress, test_progress_checkpoint_counts_existing_record_updates_as_progress, test_progress_checkpoint_ignores_non_substantive_record_touches,...

### tests/nipux_cli/test_project_atlas.py

2 tests · 46 lines

test_project_atlas_generator_maps_prompts_tools_and_source_without_self_embedding, test_project_atlas_redacts_secret_assignments_from_rendered_source

### tests/nipux_cli/test_provider_errors.py

2 tests · 21 lines

test_provider_action_required_detects_payload_and_status_text, test_provider_rate_limited_detects_transient_rate_text

### tests/nipux_cli/test_templates.py

1 tests · 15 lines

test_generic_template_pushes_artifacts_and_updates

### tests/nipux_cli/test_tools.py

69 tests · 2166 lines

test_static_tool_surface_is_focused, test_tool_registry_validates_required_arguments, test_tool_registry_blocks_truncated_reference_arguments, test_tool_access_config_filters_worker_schema_and_blocks_calls, test_artifact_tools_roundtrip, test_read_artifact_missing_ref_returns_valid_recent_refs, test_defer_job_record...

### tests/nipux_cli/test_uninstall.py

7 tests · 136 lines

test_uninstall_plan_includes_runtime_and_legacy_state, test_uninstall_plan_includes_configured_runtime_home, test_uninstall_runtime_removes_state_and_service_files, test_uninstall_runtime_dry_run_keeps_files, test_uninstall_installed_tool_uses_uv_when_available, test_uninstall_installed_tool_falls_back_to_safe_uv_pa...

### tests/nipux_cli/test_worker.py

296 tests · 11339 lines

test_system_prompt_is_contract_first_not_research_first, test_run_one_step_executes_scripted_tool_call, test_run_one_step_records_estimated_usage_for_scripted_model, test_run_one_step_blocks_content_only_worker_turn, test_run_one_step_repairs_content_only_worker_turn_with_tool_retry, test_run_one_step_recovers_repea...

Audit cues

## Review Points

Generated signals for where to inspect next.

### Large modules

tests/nipux_cli/test_worker.py (11339 lines), nipux_cli/worker.py (7538 lines), tests/nipux_cli/test_cli.py (4970 lines), nipux_cli/cli.py (3188 lines), nipux_cli/db.py (2752 lines)

### Prompt surfaces

25 prompt/instruction-like strings were extracted. Inspect this section after any agent-behavior change.

### Tool surface

29 tools are exposed to the worker. Review descriptions whenever generic behavior changes.

### Symbol map

2158 symbols were parsed. Large modules are candidates for refactoring once behavior stabilizes.