# Demos

The `demos/` folder contains self-contained project scenarios used to exercise and validate the full `AITeamFlow` pipeline. Each demo provides a natural-language project spec as input; the system is expected to autonomously produce working, tested, deployable code as output.

---

## Demo catalog

| # | Directory | Team profile | Scope | Business use case |
| - | --------- | ------------ | ----- | ----------------- |
| 0 | `00_smoke_test` | `prototype` | **Setup validator** — single Python function + one test; minimum token spend | [BUSINESS_USE_CASE.md](../demos/00_smoke_test/BUSINESS_USE_CASE.md) |
| 1 | `01_hello_world` | `full` (default) | Minimal Flask REST API with health, items CRUD, pytest, Dockerfile | [BUSINESS_USE_CASE.md](../demos/01_hello_world/BUSINESS_USE_CASE.md) |
| 2 | `02_todo_app` | `full` | Full-stack TODO app — Flask + SQLite backend, HTML/JS frontend | [BUSINESS_USE_CASE.md](../demos/02_todo_app/BUSINESS_USE_CASE.md) |
| 3 | `03_data_pipeline` | `full` | ETL pipeline — CSV ingest, validate/transform, SQLite load, CLI report | [BUSINESS_USE_CASE.md](../demos/03_data_pipeline/BUSINESS_USE_CASE.md) |
| 4 | `04_ml_api` | `full` | FastAPI ML inference service — scikit-learn model, predict/health/metrics endpoints | [BUSINESS_USE_CASE.md](../demos/04_ml_api/BUSINESS_USE_CASE.md) |
| 5 | `05_microservices` | `full` | Three-service system — API Gateway, User Service, Notification Service + docker-compose | [BUSINESS_USE_CASE.md](../demos/05_microservices/BUSINESS_USE_CASE.md) |
| 6 | `06_karpathy_optimization` | `research-optimizer` | AutoOptimizer Loop — iterative metric-driven optimization with keep/revert and RAG lesson injection | [BUSINESS_USE_CASE.md](../demos/06_karpathy_optimization/BUSINESS_USE_CASE.md) |

Team profiles are documented in [TEAM_PROFILES.md](TEAM_PROFILES.md). Set `team_profile` in `input.json` or pass `--team` to `run_demo.py` / `compare_backends.py`.

---

## Per-demo file layout

```text
demos/NN_name/
  BUSINESS_USE_CASE.md   # Business context: problem, users, objectives, success criteria.
  input.json             # Required. Project spec fed to the flow.
  expected_output.json   # Recommended. Acceptance contract (status, artifacts, summary).
  run_report.json        # Generated by the e2e test / capture_demo.py on a successful run.
  failure_report.json    # Generated by the e2e test when the run fails.
  RESULTS.md             # Generated by capture_demo.py — human-readable pass/fail summary.
  output/                # Generated source files produced by the flow (git-ignored).
  failure/               # Generated failure artifacts snapshot on a bad capture run (git-ignored).
```

### Committed fixtures (checked in)

| File | Purpose | Who creates it |
| ---- | ------- | -------------- |
| `BUSINESS_USE_CASE.md` | Stakeholder-facing problem statement, personas, objectives, risks | Hand-authored |
| `input.json` | Project description, name, stack passed to `run_ai_team()` | Hand-authored |
| `expected_output.json` | High-level acceptance contract; documents intended artifacts and status | Hand-authored |

### Generated on run (not committed, git-ignored via `demos/.gitignore`)

| File / Dir | Purpose | Who creates it |
| ---------- | ------- | -------------- |
| `output/` | Generated source files (app.py, tests, Dockerfile, etc.) | `tests/e2e/test_e2e_hello_world.py` on success |
| `run_report.json` | Machine-readable run summary: duration, retries, phase, files generated | `tests/e2e/test_e2e_hello_world.py` and `scripts/capture_demo.py` |
| `failure_report.json` | Structured failure snapshot: exception type, message, phase, last agent output | `tests/e2e/test_e2e_hello_world.py` on failure |
| `RESULTS.md` | Human-readable report: generated files, test results, lint, Docker, smoke test | `scripts/capture_demo.py` |
| `failure/` | Timestamped copy of output + `capture_result.json` + `issue_template.md` on capture failure | `scripts/capture_demo.py` |

> **Showcase note:** generated run artifacts are intentionally ignored by default. If a
> demo result is meant to support a public post or release, publish a curated summary
> such as `docs/SHOWCASE_RESULTS.md` instead of committing raw generated workspaces.

---

## `input.json` schema

```json
{
  "project_name": "snake_case_name",
  "description": "Full natural-language spec passed to the flow as the project description.",
  "team_profile": "prototype",
  "requirements": ["Flask", "pytest"],
  "stack": ["Flask", "SQLite", "HTML/JS"]
}
```

| Field | Required | Purpose |
| ----- | -------- | ------- |
| `project_name` | Recommended | Short id for logs and reports |
| `description` | Required* | Natural-language spec for the flow |
| `team_profile` | Optional | Profile key from `team_profiles.yaml` (default `full`; CLI `--team` overrides) |
| `requirements` | Optional | Python packages hint |
| `stack` | Optional | Technology labels |

\*Or `project_name` + `stack` if `description` is omitted.

`demo_input.py` reads `input.json` for `description` and `team_profile`. If `project_description.txt` exists it takes precedence for the description only — team profile still comes from `input.json` when present.

## `expected_output.json` schema

```json
{
  "status": "success",
  "artifacts": ["app.py", "tests/", "requirements.txt", "README.md", "Dockerfile"],
  "summary": "One-sentence description of what was produced."
}
```

No production code currently reads `expected_output.json` — it is documentation-level contract for human reviewers and future automated validators.

---

## Running a demo

```bash
# Validate setup first — cheapest profile-aware run (prototype crew via LangGraph)
poetry run python scripts/run_demo.py demos/00_smoke_test --skip-estimate --backend langgraph

# Run a demo end-to-end (requires OPENROUTER_API_KEY in .env)
poetry run python scripts/run_demo.py demos/01_hello_world

# Skip the cost-estimate prompt (useful in CI or scripted runs)
poetry run python scripts/run_demo.py demos/02_todo_app --skip-estimate

# Use Rich TUI for live agent progress
poetry run python scripts/run_demo.py demos/01_hello_world --monitor

# Run with log monitoring (stops on first runtime error)
./scripts/monitor_demo.sh demos/01_hello_world
```

### Demo 06 — AutoOptimizer Loop

Demo 06 uses the `ai-team optimize` subcommand rather than `scripts/run_demo.py`:

```bash
# Build demo 02 first, or point the optimizer at another existing workspace.
# Demo 06's input references the todo API produced by demo 02.
poetry run python scripts/run_demo.py demos/02_todo_app --skip-estimate

ai-team optimize ./demos/02_todo_app/output \
  --metric demos/06_karpathy_optimization/metric.yaml \
  --strategy demos/06_karpathy_optimization/strategy.md \
  --backend claude-agent-sdk \
  --budget 2.00 \
  --max-experiments 10

# Results are written to the target workspace under logs/experiments.jsonl
```

The demo ships with:

| File | Purpose |
| ---- | ------- |
| `demos/06_karpathy_optimization/input.json` | Workspace description (for reference) |
| `demos/06_karpathy_optimization/metric.yaml` | `test_pass_rate` metric via pytest |
| `demos/06_karpathy_optimization/strategy.md` | Prioritised optimization hints for the agent |
| `demos/06_karpathy_optimization/expected_output.json` | Acceptance contract (min 1 experiment, no regression) |

## Capturing and verifying demo output

After the flow completes and files are written to `output/`, run the capture script to verify, lint, build, and smoke-test:

```bash
# Verify output, run pytest + ruff + Docker build + smoke test, write RESULTS.md
poetry run python scripts/capture_demo.py \
  --output-dir demos/01_hello_world/output \
  --run-report demos/01_hello_world/run_report.json \
  --demo-id 01_hello_world

# Skip Docker build and smoke test (faster local check)
poetry run python scripts/capture_demo.py \
  --output-dir demos/01_hello_world/output \
  --skip-docker
```

`capture_demo.py` runs these verification stages in order and stops at the first failure:

1. Required file presence check
2. `pytest` with coverage (minimum 80%)
3. `ruff` lint (fails if >50 violations)
4. Docker image build
5. Container smoke test (health, items CRUD endpoints)

## Cross-backend comparison

Run the same demo spec through multiple backends and compare output quality, cost, latency, and token usage:

```bash
poetry run python scripts/compare_backends.py demos/01_hello_world --env dev
poetry run python scripts/compare_backends.py demos/01_hello_world --team backend-api --markdown out.md
```

---

## Adding a new demo

1. Create `demos/NN_name/` (use the next sequential number).
2. Write `BUSINESS_USE_CASE.md` (problem, users, business objectives, success criteria).
3. Write `input.json` with `project_name`, `description`, and optionally `requirements` / `stack`.
4. Write `expected_output.json` documenting the intended artifacts and a one-line summary.
5. Run `scripts/run_demo.py demos/NN_name` to exercise the flow.
6. Run `scripts/capture_demo.py --output-dir demos/NN_name/output --demo-id NN_name` to verify and generate `RESULTS.md`.
7. Commit `BUSINESS_USE_CASE.md`, `input.json`, `expected_output.json`, and (optionally) the generated `RESULTS.md` as a reference baseline.
