# v2 Phase C Implementation Plan

> **Status (2026-05-10):** Tasks 1–13 shipped as **v2.0.0a3** (the bundle infrastructure + four-tool collapse + cross-tool sharing assertion + housekeeping). Tasks 14–29 — `get_section` (#7), the seven-stage `synthesize` pipeline (#10), Phase C goldens, cross-cutting test updates, and release work for those — are **deferred to a4**. The Phase C TypedDicts for `get_section` and `synthesize` shipped in a3 as forward-declared contract surface so a4 can land the implementations without re-litigating wire shapes. To resume: pick up at Task 14 in a fresh session.

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
>
> **Read first:** [`docs/superpowers/specs/2026-05-09-v2-phase-c-retrieval-primitives-design.md`](../specs/2026-05-09-v2-phase-c-retrieval-primitives-design.md). The spec is the source of truth for the bundle shape, the four-tool collapse rules, the `get_section`/`synthesize` wire formats, and the housekeeping items folded in. Each task below points at the relevant section.

**Goal:** Land Phase C (#11 EntryBundle, #7 `get_section`, #10 `synthesize`) as one bundled PR off `v2-phase-c`. The four content-shape tools collapse to bundle slicers; new tools `get_section` and `synthesize` (`zim_query.synthesize=True`) ship; tag the squash-merge as `v2.0.0a3`.

**Architecture:** New module `openzim_mcp/bundle.py` provides a pure-function `extract_entry_bundle()` (one HTML parse → `EntryBundle` value) plus a cache-aware `get_or_build_bundle()` keyed at `bundle:v2c:{validated_path}:{entry_path}`. New module `openzim_mcp/synthesize.py` provides the `synthesize_query()` pipeline (per-archive search → RRF fusion for multi-archive → libzim snippet extraction → bundle-based section attribution → citation rendering → budget enforcement). `tool_schemas.py` gains `EntryBundle`, `SectionMeta`, `InfoboxField`, `InfoboxData`, `LinkBuckets`, `TocHeading`, `GetSectionResponse`, `SynthesizeResponse`, `Citation`, `SynthesizePassage`. `TableOfContentsResponse.toc` tightens from `list[dict[str, Any]]` to `list[TocHeading]` (the heading field `id` is renamed to `section_id` — wire-format break in v2 alpha). The four legacy per-tool cache prefixes (`summary_data:`, `toc_data:`, `structure_data:`, `links_full:v2b:`) disappear; one bundle key replaces all four.

**Tech Stack:** Python 3.12+, FastMCP (via `mcp.server.fastmcp`), libzim, BeautifulSoup, html2text, `tiktoken` (already in deps from Phase A), `pytest` + `pytest-asyncio`. No new dependencies.

---

## Plan structure

The work decomposes into eight phases, executed in order:

1. **Branch & housekeeping** (Tasks 1–3) — verify branch, mypy override cleanup, GitHub labels.
2. **Foundation** (Tasks 4–5) — Phase C TypedDicts, SynthesizeConfig.
3. **Bundle infrastructure** (Tasks 6–8) — `bundle.py`, cache-aware accessor, eviction tests.
4. **Four-tool collapse** (Tasks 9–13) — one task per tool plus a cross-tool shared-bundle test.
5. **`get_section`** (Tasks 14–15) — data layer, tool registration, miss-path test.
6. **`synthesize`** (Tasks 16–22) — pipeline stages built in order, then wired to `zim_query`.
7. **Goldens + cross-cutting test updates** (Tasks 23–26) — Phase C goldens, `test_structured_tool_output` / `test_response_contract` / `test_tool_schemas` extensions.
8. **Docs + release** (Tasks 27–29) — README, CHANGELOG, verify, open PR.

Each task is self-contained: a fresh subagent should be able to execute it by reading only this plan + the spec. When in doubt, refer to the spec for shape details.

> **Wire shape note (cross-cutting, carried from Phase B):** FastMCP wraps `Union[<SuccessResponse>, ToolErrorPayload]` returns in a uniform `{"result": ...}` envelope on the wire. Tests should assert against `structured.get("result", structured)` (the wrapper-tolerant unwrap) — **NEVER** `assert "result" not in structured`. Apply this pattern to every test in Tasks 14–22 and Task 23.

> **Lint discipline (cross-cutting):** CI lint = `make lint` (flake8 + isort + black). NOT ruff. After any `.py` edit, run `uv run black openzim_mcp tests && uv run isort openzim_mcp tests` before committing. The pre-commit hook enforces this.

---

## Task 1: Verify `v2-phase-c` branch state

**Files:** none (git operations only)

**Why:** The spec commit (b6ecbb6) and the self-review-checklist removal (f7d30aa) already created the `v2-phase-c` branch off `main`. This task verifies state before any code changes land.

- [ ] **Step 1: Verify on `v2-phase-c` with the two expected commits**

```bash
git status --porcelain
git branch --show-current
git log --oneline main..v2-phase-c
```

Expected: empty `git status --porcelain`; `v2-phase-c` from `git branch --show-current`; two commits ahead of main (`f7d30aa docs(v2): drop spec self-review checklist post-approval` and `b6ecbb6 docs(v2): Phase C retrieval-primitives design spec`). If branch is missing or dirty, stop and ask the user.

- [ ] **Step 2: Confirm spec exists at expected path**

```bash
ls -la docs/superpowers/specs/2026-05-09-v2-phase-c-retrieval-primitives-design.md
```

Expected: file exists, ~10-12 KB. If missing, stop.

---

## Task 2: Remove stale `[[tool.mypy.overrides]] module = ['libzim']` from `pyproject.toml`

**Files:**
- Modify: `pyproject.toml` (mypy.overrides section)

**Why:** mypy now flags this block as "unused section" — the `libzim` typeshed annotations are sufficient. Folded into Phase C per the spec's housekeeping section.

- [ ] **Step 1: Inspect current overrides block**

```bash
grep -nA 4 'tool.mypy.overrides' pyproject.toml
```

Expected: identifies a block like:

```toml
[[tool.mypy.overrides]]
module = ["libzim"]
ignore_missing_imports = true
```

- [ ] **Step 2: Remove the libzim override block**

Open `pyproject.toml` and delete the `[[tool.mypy.overrides]]` block whose `module = ["libzim"]`. Leave any other override blocks (e.g., for tests) untouched.

- [ ] **Step 3: Verify mypy still passes**

```bash
uv run mypy openzim_mcp
```

Expected: PASS (no regressions; the previously-flagged "unused section" warning is gone). If mypy now flags missing imports for `libzim`, restore the block and stop — the user needs to investigate.

- [ ] **Step 4: Commit**

```bash
git add pyproject.toml
git commit -m "chore(pyproject): remove stale libzim mypy override

mypy now resolves libzim types without the override; the section was
flagged as 'unused' and contributes nothing. Folded into Phase C
housekeeping per the design spec."
```

---

## Task 3: Add `v2` / `v2-phase-c` GitHub labels (and apply `v2-phase-b` retroactively to PR #111)

**Files:** none (gh CLI only)

**Why:** Folded into Phase C per the spec's housekeeping section. Labels were missing for v2 work; PR #111 was opened before any v2-* labels existed.

- [ ] **Step 1: Check existing labels**

```bash
gh label list --search v2
```

Expected: zero or partial output. The labels `v2`, `v2-phase-a`, `v2-phase-b`, `v2-phase-c` may or may not exist.

- [ ] **Step 2: Create missing labels (idempotent — `--force` is OK)**

```bash
gh label create v2          --description "v2 effort" --color C5DEF5 --force
gh label create v2-phase-a  --description "v2 Phase A — quality wins" --color BFD4F2 --force
gh label create v2-phase-b  --description "v2 Phase B — response contract" --color BFD4F2 --force
gh label create v2-phase-c  --description "v2 Phase C — retrieval primitives" --color BFD4F2 --force
```

Expected: each command prints either "Created label" or "Updated label". Continue on either.

- [ ] **Step 3: Apply `v2` and `v2-phase-b` to the merged PR #111**

```bash
gh pr edit 111 --add-label v2 --add-label v2-phase-b
```

Expected: prints the PR URL on success. Verify via:

```bash
gh pr view 111 --json labels
```

Expected: JSON with both `v2` and `v2-phase-b` labels listed.

- [ ] **Step 4: No commit — labels are GitHub-side only**

Skip. There is no source change to commit for this task.

---

## Task 4: Add Phase C TypedDicts to `openzim_mcp/tool_schemas.py`

**Files:**
- Modify: `openzim_mcp/tool_schemas.py` (append + tighten `TableOfContentsResponse`)
- Modify: `tests/test_tool_schemas.py` (add Phase C structural tests)

**Why:** Centralizing Phase C's TypedDicts up-front lets every later task `from openzim_mcp.tool_schemas import EntryBundle` etc. without forward-reference juggling. The `TableOfContentsResponse.toc` tightening (the `id` → `section_id` rename + `list[TocHeading]`) is a wire-format break — it lands here so that the four-tool collapse tasks can build on the new shape.

The new TypedDicts (per spec § "Data shapes"):

| Name | Purpose |
|---|---|
| `SectionMeta` | One section's metadata in a bundle. |
| `InfoboxField` | One row of an infobox key/value table. |
| `InfoboxData` | Whole infobox: title + list of fields. |
| `LinkBuckets` | Internal/external/media link buckets in a bundle (uses existing `LinkItem`). |
| `EntryBundle` | The bundle itself. |
| `TocHeading` | One TOC heading: `section_id`, `text`, `level`, `id_source`, `children`. |
| `GetSectionResponse` | `get_section` wire response. |
| `Citation` | One synthesize citation (cite_id, archive, entry_path, …). |
| `SynthesizePassage` | One passage from synthesize (cite_id, text_markdown, rank, score). |
| `SynthesizeResponse` | `synthesize` mode wire response. |

- [ ] **Step 1: Write failing structural tests (extending `tests/test_tool_schemas.py`)**

Append to `tests/test_tool_schemas.py`:

```python
# ---------------------------------------------------------------------------
# Phase C TypedDicts — structural assertions
# ---------------------------------------------------------------------------

def test_section_meta_has_required_fields() -> None:
    hints = get_type_hints(ts.SectionMeta)
    for key in ("id", "title", "level", "char_start", "char_end", "parent_id"):
        assert key in hints, f"SectionMeta missing {key}"


def test_entry_bundle_has_required_fields() -> None:
    hints = get_type_hints(ts.EntryBundle)
    for key in (
        "entry_path", "title", "content_type", "word_count", "char_count",
        "rendered_markdown", "sections", "links", "infobox",
    ):
        assert key in hints, f"EntryBundle missing {key}"


def test_link_buckets_has_three_categories() -> None:
    hints = get_type_hints(ts.LinkBuckets)
    for key in ("internal", "external", "media"):
        assert key in hints, f"LinkBuckets missing {key}"


def test_infobox_data_shape() -> None:
    field_hints = get_type_hints(ts.InfoboxField)
    assert {"label", "value"} <= set(field_hints.keys())
    data_hints = get_type_hints(ts.InfoboxData)
    assert "fields" in data_hints
    assert "title" in data_hints  # NotRequired but get_type_hints surfaces it


def test_toc_heading_uses_section_id_not_id() -> None:
    hints = get_type_hints(ts.TocHeading)
    assert "section_id" in hints, "TocHeading must use section_id (Phase C rename)"
    assert "id" not in hints, "TocHeading must not retain the old `id` field"
    for key in ("text", "level", "id_source", "children"):
        assert key in hints, f"TocHeading missing {key}"


def test_table_of_contents_response_uses_typed_toc() -> None:
    """Phase C tightens toc from list[dict[str, Any]] to list[TocHeading]."""
    hints = get_type_hints(ts.TableOfContentsResponse)
    # The annotation should be list[TocHeading] (or List[TocHeading]).
    toc_annotation = hints["toc"]
    # repr captures both new-style and List[]-style.
    assert "TocHeading" in repr(toc_annotation), (
        f"TableOfContentsResponse.toc must be list[TocHeading], got {toc_annotation!r}"
    )


def test_get_section_response_has_required_fields() -> None:
    hints = get_type_hints(ts.GetSectionResponse)
    for key in (
        "entry_path", "title", "section_id", "section_title", "level",
        "parent_id", "content_markdown", "char_count", "word_count",
        "truncated", "_meta",
    ):
        assert key in hints, f"GetSectionResponse missing {key}"


def test_citation_has_required_fields() -> None:
    hints = get_type_hints(ts.Citation)
    for key in (
        "cite_id", "archive", "entry_path", "title",
        "section_id", "section_title",
    ):
        assert key in hints, f"Citation missing {key}"


def test_synthesize_passage_has_required_fields() -> None:
    hints = get_type_hints(ts.SynthesizePassage)
    for key in ("cite_id", "text_markdown", "rank", "score"):
        assert key in hints, f"SynthesizePassage missing {key}"


def test_synthesize_response_has_required_fields() -> None:
    hints = get_type_hints(ts.SynthesizeResponse)
    for key in (
        "query", "answer_markdown", "passages", "citations",
        "archives_searched", "fallback_used",
        "total_chars", "total_words", "_meta",
    ):
        assert key in hints, f"SynthesizeResponse missing {key}"
```

- [ ] **Step 2: Run the new tests and confirm they fail**

```bash
uv run pytest tests/test_tool_schemas.py -k 'section_meta or entry_bundle or link_buckets or infobox_data or toc_heading or table_of_contents_response_uses_typed or get_section_response or citation_has or synthesize_passage or synthesize_response' -v
```

Expected: every test FAILs with `AttributeError: module 'openzim_mcp.tool_schemas' has no attribute '<TypeName>'` (or `KeyError: 'section_id'` for the TocHeading test).

- [ ] **Step 3: Add the TypedDicts to `openzim_mcp/tool_schemas.py`**

Append (just before `class ToolErrorPayload(TypedDict):` — keep error-envelope last per Phase B convention; if structure differs, append before the trailing `__all__` if any):

```python
# ---------------------------------------------------------------------------
# Phase C — bundle internals
# ---------------------------------------------------------------------------

class SectionMeta(TypedDict):
    """One section's metadata in an EntryBundle.

    Section IDs come from openzim_mcp.content_processor.resolve_heading_id.
    char_start / char_end are offsets into EntryBundle.rendered_markdown.
    """
    id: str
    title: str
    level: int
    char_start: int
    char_end: int
    parent_id: NotRequired[Optional[str]]


class InfoboxField(TypedDict):
    label: str
    value: str


class InfoboxData(TypedDict):
    title: NotRequired[str]
    fields: list[InfoboxField]


class LinkBuckets(TypedDict):
    internal: list[LinkItem]
    external: list[LinkItem]
    media: list[LinkItem]


class EntryBundle(TypedDict):
    """Single-parse intermediate for content-shape tools.

    First touch of an entry produces this bundle; subsequent calls to
    get_entry_summary, get_table_of_contents, get_article_structure,
    extract_article_links, and get_section all slice into it without
    re-parsing the HTML. Stored under cache key
    'bundle:v2c:{validated_path}:{entry_path}'.
    """
    entry_path: str
    title: str
    content_type: str
    word_count: int
    char_count: int
    rendered_markdown: str
    sections: list[SectionMeta]
    links: LinkBuckets
    infobox: Optional[InfoboxData]


# ---------------------------------------------------------------------------
# Phase C — TOC heading (replaces list[dict[str, Any]] in TableOfContentsResponse)
# ---------------------------------------------------------------------------

class TocHeading(TypedDict):
    """One heading in get_table_of_contents output.

    `section_id` is the value to pass to get_section(section_id=...).
    Renamed from the old `id` field for clarity.
    """
    section_id: str
    text: str
    level: int
    id_source: str   # "id" | "descendant_anchor" | "preceding_anchor" | "slug"
    children: list["TocHeading"]


# ---------------------------------------------------------------------------
# Phase C — get_section response
# ---------------------------------------------------------------------------

class GetSectionResponse(TypedDict):
    entry_path: str
    title: str
    section_id: str
    section_title: str
    level: int
    parent_id: Optional[str]
    content_markdown: str
    char_count: int
    word_count: int
    truncated: bool
    _meta: MetaEnvelope


# ---------------------------------------------------------------------------
# Phase C — synthesize response
# ---------------------------------------------------------------------------

class Citation(TypedDict):
    cite_id: str
    archive: str
    entry_path: str
    title: str
    section_id: Optional[str]
    section_title: Optional[str]


class SynthesizePassage(TypedDict):
    cite_id: str
    text_markdown: str
    rank: int
    score: float


class SynthesizeResponse(TypedDict):
    query: str
    answer_markdown: str
    passages: list[SynthesizePassage]
    citations: list[Citation]
    archives_searched: list[str]
    fallback_used: Literal["xapian_score", "rrf_fusion", "reranker"]
    total_chars: int
    total_words: int
    _meta: MetaEnvelope
```

(If `Literal` is not yet imported, add `from typing import Literal` at the file top alongside the other typing imports.)

- [ ] **Step 4: Tighten `TableOfContentsResponse.toc`**

Find the existing `TableOfContentsResponse` (around `tool_schemas.py:351`). Change:

```python
toc: list[dict[str, Any]]
```

to:

```python
toc: list[TocHeading]
```

`TocHeading` must be defined above this class — either move the new `TocHeading` definition above `TableOfContentsResponse` in the file, or use the string-form forward reference `toc: list["TocHeading"]`. The former is cleaner.

- [ ] **Step 5: Re-run the structural tests**

```bash
uv run pytest tests/test_tool_schemas.py -v
```

Expected: every test PASSes (the original Phase B tests + all 10 new Phase C tests).

- [ ] **Step 6: Run mypy**

```bash
uv run mypy openzim_mcp
```

Expected: PASS. (The `TableOfContentsResponse.toc` tightening may surface latent type issues in code that builds TOC dicts — those are addressed in Task 10. For now, mypy should be clean because nothing yet writes a `TocHeading`-incompatible dict to that field.)

- [ ] **Step 7: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/tool_schemas.py tests/test_tool_schemas.py
git commit -m "feat(v2): add Phase C TypedDicts and tighten TocHeading

Adds EntryBundle, SectionMeta, InfoboxField, InfoboxData, LinkBuckets,
TocHeading, GetSectionResponse, Citation, SynthesizePassage,
SynthesizeResponse. Tightens TableOfContentsResponse.toc from
list[dict[str, Any]] to list[TocHeading].

Wire-format break: TOC heading field 'id' is renamed to 'section_id'
(the value get_section(section_id=...) consumes). v2 alpha — clean
break per v2 policy.

Spec: docs/superpowers/specs/2026-05-09-v2-phase-c-retrieval-primitives-design.md"
```

---

## Task 5: Add `SynthesizeConfig` block to `openzim_mcp/config.py`

**Files:**
- Modify: `openzim_mcp/config.py` (new `SynthesizeConfig` model + add to root config)
- Modify: `tests/test_config.py` (new test asserting defaults)

**Why:** Phase C's synthesize tunables (`top_n`, `per_archive_k`, `output_char_budget`) need a typed config home so users can override via the existing config-file mechanism. Defaults per spec § "Defaults": `top_n=5`, `per_archive_k=10`, `output_char_budget=4800`.

- [ ] **Step 1: Inspect existing config structure**

```bash
grep -nE 'class .*Config\(.*\):|content:.*Config' openzim_mcp/config.py | head -10
```

Expected: identifies the root config model (likely `OpenZimMcpConfig`) and sub-models (e.g., `ContentConfig`). Note the field-style (Pydantic model with `Field(default=...)`).

- [ ] **Step 2: Write failing test**

Add to `tests/test_config.py`:

```python
def test_synthesize_config_defaults() -> None:
    """Phase C — synthesize tunables exist with documented defaults."""
    from openzim_mcp.config import OpenZimMcpConfig
    cfg = OpenZimMcpConfig()
    assert cfg.synthesize.top_n == 5
    assert cfg.synthesize.per_archive_k == 10
    assert cfg.synthesize.output_char_budget == 4800
```

- [ ] **Step 3: Run the test, confirm it fails**

```bash
uv run pytest tests/test_config.py::test_synthesize_config_defaults -v
```

Expected: FAIL with `AttributeError: 'OpenZimMcpConfig' object has no attribute 'synthesize'`.

- [ ] **Step 4: Add `SynthesizeConfig` to `openzim_mcp/config.py`**

Add (near the other sub-config classes — e.g., next to `ContentConfig`):

```python
class SynthesizeConfig(BaseModel):
    """Phase C: tunables for `zim_query(synthesize=True)`.

    All knobs are advisory — the synthesize pipeline obeys these as
    soft budgets (e.g., output_char_budget truncates the *last* passage
    rather than refusing to include it).
    """

    top_n: int = Field(default=5, ge=1, le=50, description="Final passages returned.")
    per_archive_k: int = Field(default=10, ge=1, le=100, description="Top-K from each archive before fusion.")
    output_char_budget: int = Field(default=4800, ge=500, le=20000, description="Soft cap on answer_markdown chars (~1200 tokens).")
```

Then add `synthesize: SynthesizeConfig = Field(default_factory=SynthesizeConfig)` as a field on the root `OpenZimMcpConfig` class (next to the existing `content: ContentConfig = ...` field).

- [ ] **Step 5: Run the test, confirm it passes**

```bash
uv run pytest tests/test_config.py::test_synthesize_config_defaults -v
```

Expected: PASS.

- [ ] **Step 6: Run mypy and full config tests**

```bash
uv run mypy openzim_mcp
uv run pytest tests/test_config.py -v
```

Expected: both PASS.

- [ ] **Step 7: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/config.py tests/test_config.py
git commit -m "feat(v2): add SynthesizeConfig block (top_n, per_archive_k, budget)

Defaults per Phase C spec: top_n=5, per_archive_k=10,
output_char_budget=4800. Pydantic-validated with sane upper bounds.
Wired onto OpenZimMcpConfig.synthesize.

Spec § Defaults."
```

---

## Task 6: Create `openzim_mcp/bundle.py` — pure-function `extract_entry_bundle()`

**Files:**
- Create: `openzim_mcp/bundle.py`
- Create: `tests/test_bundle.py`

**Why:** The bundle extractor is the heart of #11. It runs ONE HTML parse and produces an `EntryBundle` with `rendered_markdown`, `sections` (with `char_start`/`char_end` into `rendered_markdown`), `links` (internal/external/media buckets), and `infobox`. This task lands the pure function (no caching, no ContentService dependency); Task 7 adds `get_or_build_bundle()` (the cache-aware accessor).

The post-render text-matching algorithm for section char_ranges (per spec § "New modules" → step 6):

1. Run the existing renderer to produce `rendered_markdown`.
2. Get the heading list from `_build_headings(soup)` (already defined in `content_processor.py` — produces `[{level, text, anchor, id_source, position}]` in document order).
3. For each heading in document order, search `rendered_markdown` starting from `last_match_end` for the pattern `^#{level} re.escape(text_normalized)$` (multiline). `text_normalized` is the heading text after html2text-style whitespace collapse.
4. Record `char_start` = match.start(); `char_end` = next match's `char_start` (or `len(rendered_markdown)` for the last heading).
5. If a heading can't be located (extreme edge case), skip it and log a warning.

- [ ] **Step 1: Write failing tests for `extract_entry_bundle`**

Create `tests/test_bundle.py`:

```python
"""Tests for openzim_mcp.bundle.

Bundle determinism, structural invariants, and offset-correctness for
the post-render text-matching algorithm. The cache-aware accessor
(get_or_build_bundle) has its own tests in this file.
"""

from __future__ import annotations

from pathlib import Path
from typing import Any
from unittest.mock import MagicMock

import pytest

from openzim_mcp.bundle import extract_entry_bundle, get_or_build_bundle
from openzim_mcp.cache import OpenZimMcpCache
from openzim_mcp.content_processor import ContentProcessor


SAMPLE_HTML = """\
<html><body>
<h1>Berlin</h1>
<p>Berlin is the capital and largest city of Germany.</p>
<h2>Geography</h2>
<p>Berlin's terrain is generally flat. The Spree River runs through the city.</p>
<h3>Climate</h3>
<p>Humid continental, transitioning to oceanic in milder areas.</p>
<h2>History</h2>
<p>Founded in the 13th century, Berlin has a long and complex history.</p>
<a href="A/Spree_River">Spree</a>
<a href="https://en.wikipedia.org/wiki/Berlin">External link</a>
<img src="berlin.jpg" alt="Berlin">
</body></html>
"""


def _make_archive_with_entry(html: str, *, title: str = "Berlin",
                              entry_path: str = "A/Berlin",
                              mime: str = "text/html") -> MagicMock:
    """Build a minimal libzim Archive mock returning the given HTML."""
    item = MagicMock()
    item.content = html.encode("utf-8")
    item.mimetype = mime
    entry = MagicMock()
    entry.title = title
    entry.path = entry_path
    entry.get_item.return_value = item
    archive = MagicMock()
    archive.get_entry_by_path.return_value = entry
    return archive


@pytest.fixture
def cp() -> ContentProcessor:
    return ContentProcessor()


def test_bundle_has_rendered_markdown(cp: ContentProcessor) -> None:
    archive = _make_archive_with_entry(SAMPLE_HTML)
    bundle = extract_entry_bundle(archive, "A/Berlin", content_processor=cp)
    assert bundle["entry_path"] == "A/Berlin"
    assert bundle["title"] == "Berlin"
    assert "Berlin is the capital" in bundle["rendered_markdown"]


def test_bundle_sections_have_offsets(cp: ContentProcessor) -> None:
    archive = _make_archive_with_entry(SAMPLE_HTML)
    bundle = extract_entry_bundle(archive, "A/Berlin", content_processor=cp)
    sections = bundle["sections"]
    titles = [s["title"] for s in sections]
    # The four headings in SAMPLE_HTML in document order.
    assert titles == ["Berlin", "Geography", "Climate", "History"]
    levels = [s["level"] for s in sections]
    assert levels == [1, 2, 3, 2]


def test_bundle_section_offsets_are_sorted_and_disjoint(cp: ContentProcessor) -> None:
    archive = _make_archive_with_entry(SAMPLE_HTML)
    bundle = extract_entry_bundle(archive, "A/Berlin", content_processor=cp)
    sections = bundle["sections"]
    # char_start ascending
    starts = [s["char_start"] for s in sections]
    assert starts == sorted(starts)
    # 0 <= char_start < char_end <= len(rendered_markdown)
    md_len = len(bundle["rendered_markdown"])
    for s in sections:
        assert 0 <= s["char_start"] < s["char_end"] <= md_len


def test_bundle_slice_returns_section_content(cp: ContentProcessor) -> None:
    archive = _make_archive_with_entry(SAMPLE_HTML)
    bundle = extract_entry_bundle(archive, "A/Berlin", content_processor=cp)
    md = bundle["rendered_markdown"]
    geography = next(s for s in bundle["sections"] if s["title"] == "Geography")
    slice_text = md[geography["char_start"]:geography["char_end"]]
    assert "Geography" in slice_text
    assert "Spree" in slice_text
    # Must not contain the next heading's title (History)
    assert "History" not in slice_text


def test_bundle_section_ids_unique(cp: ContentProcessor) -> None:
    archive = _make_archive_with_entry(SAMPLE_HTML)
    bundle = extract_entry_bundle(archive, "A/Berlin", content_processor=cp)
    ids = [s["id"] for s in bundle["sections"]]
    assert len(ids) == len(set(ids)), f"Duplicate section IDs: {ids}"


def test_bundle_links_categorized(cp: ContentProcessor) -> None:
    archive = _make_archive_with_entry(SAMPLE_HTML)
    bundle = extract_entry_bundle(archive, "A/Berlin", content_processor=cp)
    links = bundle["links"]
    # Spree is internal (relative href)
    assert any(li["href"] == "A/Spree_River" for li in links["internal"])
    # External link
    assert any("wikipedia.org" in li["href"] for li in links["external"])
    # Media link
    assert any(li["href"] == "berlin.jpg" for li in links["media"])


def test_bundle_is_deterministic(cp: ContentProcessor) -> None:
    """Same input → identical bundle. Required for cache-eviction safety."""
    archive1 = _make_archive_with_entry(SAMPLE_HTML)
    archive2 = _make_archive_with_entry(SAMPLE_HTML)
    b1 = extract_entry_bundle(archive1, "A/Berlin", content_processor=cp)
    b2 = extract_entry_bundle(archive2, "A/Berlin", content_processor=cp)
    assert b1 == b2


def test_bundle_handles_repeated_heading_text(cp: ContentProcessor) -> None:
    """Two headings with identical text are disambiguated by document order."""
    html = """\
<html><body>
<h2>References</h2><p>First refs section</p>
<h2>External Links</h2><p>Some links</p>
<h2>References</h2><p>Second refs section (duplicate title)</p>
</body></html>
"""
    archive = _make_archive_with_entry(html)
    bundle = extract_entry_bundle(archive, "A/Test", content_processor=cp)
    titles = [s["title"] for s in bundle["sections"]]
    assert titles == ["References", "External Links", "References"]
    starts = [s["char_start"] for s in bundle["sections"]]
    assert starts[0] < starts[1] < starts[2]


def test_bundle_handles_markdown_significant_chars_in_heading(cp: ContentProcessor) -> None:
    """Headings with *, _, [, ] etc. survive html2text + regex matching."""
    html = """\
<html><body>
<h2>C++ programming</h2><p>About C++</p>
<h2>Python (programming language)</h2><p>About Python</p>
</body></html>
"""
    archive = _make_archive_with_entry(html)
    bundle = extract_entry_bundle(archive, "A/Test", content_processor=cp)
    titles = [s["title"] for s in bundle["sections"]]
    assert "C++ programming" in titles
    assert "Python (programming language)" in titles


def test_bundle_word_and_char_counts(cp: ContentProcessor) -> None:
    archive = _make_archive_with_entry(SAMPLE_HTML)
    bundle = extract_entry_bundle(archive, "A/Berlin", content_processor=cp)
    md = bundle["rendered_markdown"]
    assert bundle["char_count"] == len(md)
    assert bundle["word_count"] == len(md.split())
```

- [ ] **Step 2: Run tests, confirm they fail with import error**

```bash
uv run pytest tests/test_bundle.py -v
```

Expected: every test FAILs with `ModuleNotFoundError: No module named 'openzim_mcp.bundle'`.

- [ ] **Step 3: Create `openzim_mcp/bundle.py`**

```python
"""Per-entry bundle extraction.

Phase C #11: First touch of an entry runs ONE HTML parse → produces a
single EntryBundle value cached under 'bundle:v2c:{validated_path}:{entry_path}'.
The four content-shape tools (get_entry_summary, get_table_of_contents,
get_article_structure, extract_article_links) and get_section all slice
into the bundle without re-parsing.

This module is intentionally pure: extract_entry_bundle takes an open
archive and returns the bundle. The cache-aware accessor
get_or_build_bundle handles cache lookups and is the entry point used
by the data-layer methods.
"""

from __future__ import annotations

import logging
import re
from pathlib import Path
from typing import TYPE_CHECKING, Optional, cast

from openzim_mcp.tool_schemas import (
    EntryBundle,
    InfoboxData,
    InfoboxField,
    LinkBuckets,
    LinkItem,
    SectionMeta,
)

if TYPE_CHECKING:
    from libzim.reader import Archive

    from openzim_mcp.cache import OpenZimMcpCache
    from openzim_mcp.content_processor import ContentProcessor

logger = logging.getLogger(__name__)


_BUNDLE_KEY_PREFIX = "bundle:v2c"


def _bundle_cache_key(validated_path: "Path", entry_path: str) -> str:
    return f"{_BUNDLE_KEY_PREFIX}:{validated_path}:{entry_path}"


def _normalize_heading_text(text: str) -> str:
    """Match html2text's whitespace handling: collapse runs of whitespace."""
    return " ".join(text.split())


def _resolve_entry_html(archive: "Archive", entry_path: str) -> tuple[str, str, str]:
    """Fetch the entry's HTML, returning (title, mimetype, html).

    Raises whatever the libzim layer raises; callers wrap.
    """
    entry = archive.get_entry_by_path(entry_path)
    item = entry.get_item()
    title = entry.title or "Untitled"
    mime = item.mimetype or ""
    html = bytes(item.content).decode("utf-8", errors="replace")
    return title, mime, html


def _extract_infobox(soup: "Any", content_processor: "ContentProcessor") -> Optional[InfoboxData]:
    """Extract the first infobox as InfoboxData, or None if absent.

    Uses ContentProcessor.extract_infobox (which already handles the
    selectors and KV-row extraction). Returns None if no infobox is found.
    Mutates `soup` to remove the extracted infobox.
    """
    rows = content_processor.extract_infobox(soup)
    if not rows:
        return None
    fields: list[InfoboxField] = [
        {"label": r["label"], "value": r["value"]} for r in rows
    ]
    return cast("InfoboxData", {"fields": fields})


def _categorize_links(links: list[LinkItem]) -> LinkBuckets:
    """Split a flat link list into internal/external/media buckets.

    LinkItem already has the typing info we need; we route by a simple
    heuristic that matches what the existing extract_html_links does.
    """
    internal: list[LinkItem] = []
    external: list[LinkItem] = []
    media: list[LinkItem] = []
    for li in links:
        href = li.get("href", "")
        link_type = li.get("type", "")
        if link_type == "media" or any(
            href.lower().endswith(ext)
            for ext in (".jpg", ".jpeg", ".png", ".gif", ".svg", ".webp",
                        ".mp4", ".webm", ".ogg", ".mp3", ".wav", ".pdf")
        ):
            media.append(li)
        elif href.startswith(("http://", "https://", "//")):
            external.append(li)
        else:
            internal.append(li)
    return cast("LinkBuckets", {"internal": internal, "external": external, "media": media})


def _compute_section_offsets(
    rendered_markdown: str,
    headings: list[dict],
) -> list[SectionMeta]:
    """Locate each heading in rendered_markdown and emit SectionMeta.

    Iterates headings in document order, searching for `^#{level} <text>$`
    starting from the last match end. On match-failure, the heading is
    skipped (logged at warning level).
    """
    sections: list[SectionMeta] = []
    cursor = 0
    parent_stack: list[tuple[int, str]] = []  # (level, section_id)
    matches: list[tuple[int, str, int, str]] = []  # (level, text, char_start, section_id)

    for h in headings:
        level = int(h["level"])
        text = _normalize_heading_text(h.get("text", ""))
        section_id = h.get("anchor") or h.get("id") or ""
        if not text or not section_id:
            continue
        # Build the regex: ^#{level} <escaped text>$ (multiline).
        pattern = re.compile(
            rf"^{'#' * level} {re.escape(text)}\s*$",
            re.MULTILINE,
        )
        match = pattern.search(rendered_markdown, cursor)
        if match is None:
            logger.warning(
                "Bundle: could not locate heading %r (level %d) in rendered markdown",
                text, level,
            )
            continue
        matches.append((level, text, match.start(), section_id))
        cursor = match.end()

    # Second pass: compute char_end (next match start, or len(markdown))
    md_len = len(rendered_markdown)
    for i, (level, text, char_start, section_id) in enumerate(matches):
        char_end = matches[i + 1][2] if i + 1 < len(matches) else md_len
        # Resolve parent_id: last heading on the stack with strictly lower level.
        while parent_stack and parent_stack[-1][0] >= level:
            parent_stack.pop()
        parent_id = parent_stack[-1][1] if parent_stack else None
        sections.append(cast("SectionMeta", {
            "id": section_id,
            "title": text,
            "level": level,
            "char_start": char_start,
            "char_end": char_end,
            "parent_id": parent_id,
        }))
        parent_stack.append((level, section_id))

    return sections


def extract_entry_bundle(
    archive: "Archive",
    entry_path: str,
    *,
    content_processor: "ContentProcessor",
) -> EntryBundle:
    """Run the single HTML parse and produce the bundle.

    Pure: no caching, no I/O beyond the archive read. Callers that want
    caching should use get_or_build_bundle.
    """
    from bs4 import BeautifulSoup  # local import to keep top-level fast

    from openzim_mcp.content_processor import _build_headings

    title, mime, html = _resolve_entry_html(archive, entry_path)

    # Bundle currently only meaningful for HTML entries; non-HTML returns
    # an empty-shape bundle so consumers can detect the case via empty
    # sections list.
    if not mime.startswith("text/html"):
        empty: EntryBundle = cast("EntryBundle", {
            "entry_path": entry_path,
            "title": title,
            "content_type": mime,
            "word_count": 0,
            "char_count": 0,
            "rendered_markdown": "",
            "sections": [],
            "links": cast("LinkBuckets", {"internal": [], "external": [], "media": []}),
            "infobox": None,
        })
        return empty

    soup = BeautifulSoup(html, "html.parser")
    # Headings extracted BEFORE soup is mutated (extract_infobox / render).
    headings = _build_headings(soup)
    # Links extracted BEFORE infobox extraction strips parts of the soup.
    raw_links = content_processor.extract_html_links(html)
    link_buckets = _categorize_links(raw_links)
    # Infobox: also mutates soup to remove the infobox so html2text doesn't
    # pipe-soup it. Returns None if no infobox.
    infobox = _extract_infobox(soup, content_processor)
    # Render to markdown (mutates soup further).
    rendered = content_processor._render_soup_to_text(soup, compact=False)
    sections = _compute_section_offsets(rendered, headings)

    bundle: EntryBundle = cast("EntryBundle", {
        "entry_path": entry_path,
        "title": title,
        "content_type": mime,
        "word_count": len(rendered.split()),
        "char_count": len(rendered),
        "rendered_markdown": rendered,
        "sections": sections,
        "links": link_buckets,
        "infobox": infobox,
    })
    return bundle


def get_or_build_bundle(
    archive: "Archive",
    entry_path: str,
    *,
    cache: "OpenZimMcpCache",
    validated_path: "Path",
    content_processor: "ContentProcessor",
) -> EntryBundle:
    """Cache-aware bundle accessor. Builds on miss; returns cached on hit."""
    key = _bundle_cache_key(validated_path, entry_path)
    cached = cache.get(key)
    if cached is not None:
        logger.debug("Bundle cache hit: %s", entry_path)
        return cast("EntryBundle", cached)
    logger.debug("Bundle cache miss: %s — building", entry_path)
    bundle = extract_entry_bundle(archive, entry_path, content_processor=content_processor)
    cache.set(key, bundle)
    return bundle
```

Note: `_build_headings` is a module-private function in `content_processor.py` (per Explore agent at line 154). If it's not exported, add it to a new `__all__` or just import via `from openzim_mcp.content_processor import _build_headings` (Python permits this; the leading underscore is convention, not enforcement). The implementer should verify the import works in Step 4.

- [ ] **Step 4: Run the tests**

```bash
uv run pytest tests/test_bundle.py -v
```

Expected: every test PASSes. If `_build_headings` import fails, either (a) export it explicitly via `__all__` in `content_processor.py`, or (b) keep the leading-underscore import (it works regardless of `__all__`).

If a test fails, the most likely cause is the regex's whitespace handling in `_compute_section_offsets`. Inspect a failing case interactively:

```bash
uv run python -c "from tests.test_bundle import SAMPLE_HTML, _make_archive_with_entry; from openzim_mcp.bundle import extract_entry_bundle; from openzim_mcp.content_processor import ContentProcessor; b = extract_entry_bundle(_make_archive_with_entry(SAMPLE_HTML), 'A/Berlin', content_processor=ContentProcessor()); print(b['rendered_markdown'][:400])"
```

…then adjust the regex or normalization to match what html2text emitted.

- [ ] **Step 5: Run mypy**

```bash
uv run mypy openzim_mcp
```

Expected: PASS.

- [ ] **Step 6: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/bundle.py tests/test_bundle.py
git commit -m "feat(v2): add EntryBundle extractor (#11 foundation)

Pure-function extract_entry_bundle runs ONE HTML parse and produces a
SectionMeta list (with char_start/char_end into rendered_markdown via
post-render text matching), categorized link buckets, and an optional
infobox. Cache-aware get_or_build_bundle uses the existing OpenZimMcpCache
under key 'bundle:v2c:{validated_path}:{entry_path}'.

Section offsets are computed by scanning rendered_markdown for
^#{level} <heading text>$ in document order; repeated headings are
disambiguated by document-order matching from the last cursor position.

Spec § Architecture > New modules > openzim_mcp/bundle.py."
```

---

## Task 7: Add cache-hit / cache-miss tests for `get_or_build_bundle`

**Files:**
- Modify: `tests/test_bundle.py` (append cache-aware accessor tests)

**Why:** Task 6 added `get_or_build_bundle` and a smoke import via the structural test, but the cache-hit-vs-miss behavior wasn't asserted. This task adds those assertions, plus an eviction-round-trip test that proves bundles are deterministic across rebuild.

- [ ] **Step 1: Append failing tests to `tests/test_bundle.py`**

```python
# ---------------------------------------------------------------------------
# get_or_build_bundle: cache-hit / cache-miss / eviction-round-trip
# ---------------------------------------------------------------------------

def test_get_or_build_bundle_cache_miss_then_hit(cp: ContentProcessor, tmp_path: Path) -> None:
    """First call builds; second call returns cached without re-parsing."""
    cache = OpenZimMcpCache(max_size=128, ttl_seconds=300)
    archive = _make_archive_with_entry(SAMPLE_HTML)
    validated_path = tmp_path / "test.zim"
    validated_path.touch()

    # First call: cache miss → build
    initial_misses = cache.stats()["misses"]
    bundle1 = get_or_build_bundle(
        archive, "A/Berlin",
        cache=cache, validated_path=validated_path, content_processor=cp,
    )
    after_first = cache.stats()
    assert after_first["misses"] == initial_misses + 1
    assert archive.get_entry_by_path.call_count == 1

    # Second call: cache hit → no archive access
    bundle2 = get_or_build_bundle(
        archive, "A/Berlin",
        cache=cache, validated_path=validated_path, content_processor=cp,
    )
    after_second = cache.stats()
    assert after_second["misses"] == after_first["misses"]   # no new miss
    assert after_second["hits"] >= after_first["hits"] + 1
    assert archive.get_entry_by_path.call_count == 1          # still one archive read
    assert bundle1 == bundle2


def test_get_or_build_bundle_eviction_rebuild_identical(cp: ContentProcessor, tmp_path: Path) -> None:
    """After eviction, rebuild produces a bundle == the original."""
    cache = OpenZimMcpCache(max_size=128, ttl_seconds=300)
    archive = _make_archive_with_entry(SAMPLE_HTML)
    validated_path = tmp_path / "test.zim"
    validated_path.touch()

    bundle1 = get_or_build_bundle(
        archive, "A/Berlin",
        cache=cache, validated_path=validated_path, content_processor=cp,
    )

    # Force eviction by clearing the cache entirely
    cache.clear()

    bundle2 = get_or_build_bundle(
        archive, "A/Berlin",
        cache=cache, validated_path=validated_path, content_processor=cp,
    )

    assert bundle1 == bundle2  # determinism — required for cursor-survival


def test_get_or_build_bundle_different_paths_different_keys(cp: ContentProcessor, tmp_path: Path) -> None:
    """Two different validated_paths produce two cache entries."""
    cache = OpenZimMcpCache(max_size=128, ttl_seconds=300)
    archive_a = _make_archive_with_entry(SAMPLE_HTML)
    archive_b = _make_archive_with_entry(SAMPLE_HTML)
    path_a = tmp_path / "a.zim"
    path_b = tmp_path / "b.zim"
    path_a.touch()
    path_b.touch()

    get_or_build_bundle(archive_a, "A/Berlin",
                        cache=cache, validated_path=path_a, content_processor=cp)
    get_or_build_bundle(archive_b, "A/Berlin",
                        cache=cache, validated_path=path_b, content_processor=cp)

    # Both archives were touched (different cache keys)
    assert archive_a.get_entry_by_path.call_count == 1
    assert archive_b.get_entry_by_path.call_count == 1
    assert cache.stats()["misses"] >= 2
```

- [ ] **Step 2: Run tests, confirm they pass**

```bash
uv run pytest tests/test_bundle.py -v
```

Expected: all (Task 6 + Task 7) tests PASS. If `OpenZimMcpCache.stats()` returns differently-named keys, adjust the assertions to match the actual stats keys (the public API was confirmed by the Explore agent: `size, max_size, ttl, hit_rate, hits, misses`).

- [ ] **Step 3: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add tests/test_bundle.py
git commit -m "test(v2): assert bundle cache hit/miss + eviction-rebuild determinism

get_or_build_bundle: first call → miss + build; second call → hit (no
archive access); after cache.clear(), rebuild produces an identical
bundle (required for cursor-survival in extract_article_links pagination)."
```

---

## Task 8: Add bundle-invariant tests (sorted, disjoint, parent-child range nesting)

**Files:**
- Modify: `tests/test_bundle.py` (append invariant tests)

**Why:** The spec calls out invariants that the implementation must uphold. Some are checked by Task 6's tests; this task adds the parent/child range-nesting invariant explicitly and a more aggressive disjoint-at-same-level check.

- [ ] **Step 1: Append tests**

```python
# ---------------------------------------------------------------------------
# Bundle invariants from spec § "Invariants"
# ---------------------------------------------------------------------------

DEEP_NESTED_HTML = """\
<html><body>
<h1>Top</h1><p>Top intro.</p>
<h2>Section A</h2><p>A intro.</p>
<h3>A.1</h3><p>A.1 body.</p>
<h3>A.2</h3><p>A.2 body.</p>
<h2>Section B</h2><p>B intro.</p>
<h3>B.1</h3><p>B.1 body.</p>
<h4>B.1.a</h4><p>B.1.a body.</p>
</body></html>
"""


def test_bundle_parent_child_range_nesting(cp: ContentProcessor) -> None:
    archive = _make_archive_with_entry(DEEP_NESTED_HTML)
    bundle = extract_entry_bundle(archive, "A/Test", content_processor=cp)
    sections = bundle["sections"]
    by_id = {s["id"]: s for s in sections}

    for s in sections:
        if s.get("parent_id"):
            parent = by_id[s["parent_id"]]
            assert parent["char_start"] <= s["char_start"]
            assert s["char_end"] <= parent["char_end"]
            assert parent["level"] < s["level"]


def test_bundle_same_level_disjoint(cp: ContentProcessor) -> None:
    """Two h2s under the same h1 must not overlap each other."""
    archive = _make_archive_with_entry(DEEP_NESTED_HTML)
    bundle = extract_entry_bundle(archive, "A/Test", content_processor=cp)
    sections = bundle["sections"]
    h2s = [s for s in sections if s["level"] == 2]
    assert len(h2s) == 2
    a, b = h2s
    assert a["char_end"] <= b["char_start"]
```

- [ ] **Step 2: Run tests**

```bash
uv run pytest tests/test_bundle.py -v
```

Expected: all PASS. If the parent_id resolution in `_compute_section_offsets` is wrong (e.g., level-1 wrapping behavior), the parent-child test will fail and the implementer should adjust the parent_stack logic in `bundle.py`.

- [ ] **Step 3: Commit**

```bash
git add tests/test_bundle.py
git commit -m "test(v2): assert bundle invariants — parent/child nesting + same-level disjoint

DEEP_NESTED_HTML covers a 4-level hierarchy. Asserts parent.char_start
<= child.char_start < child.char_end <= parent.char_end, and that
sibling sections at the same level have non-overlapping char ranges.
Spec § Invariants."
```

---

## Task 9: Collapse `get_entry_summary` to a bundle slicer

**Files:**
- Modify: `openzim_mcp/zim/content.py:1213` (`_extract_entry_summary_data`)
- Modify: `openzim_mcp/zim/content.py:1111` (`get_entry_summary_data` — remove the legacy `summary_data:` cache prefix lookup since the bundle becomes the cache)
- Modify: `tests/test_get_entry_summary.py` (assertions unchanged — wire format stable)

**Why:** First of the four tools to collapse. After this task, `get_entry_summary` calls `get_or_build_bundle()` and slices `rendered_markdown[:max_chars]` (or first-N-words equivalent), populating the existing `EntrySummaryResponse` shape. Wire format unchanged.

- [ ] **Step 1: Inspect current `_extract_entry_summary_data`**

```bash
sed -n '1213,1280p' openzim_mcp/zim/content.py
```

Expected: a method that calls `_resolve_entry_with_fallback`, `process_mime_content`, and constructs an `EntrySummaryResponse`-compatible dict. Note the `max_words` parameter and the existing `summary_data:` cache key in `get_entry_summary_data` (around line 1111).

- [ ] **Step 2: Replace the body of `_extract_entry_summary_data` with a bundle-based slice**

Modify `_extract_entry_summary_data` (the data-layer method that previously parsed HTML) to call `get_or_build_bundle` and slice. The new body:

```python
def _extract_entry_summary_data(
    self,
    archive: "Archive",
    validated_path: Path,
    entry_path: str,
    max_words: int,
    *,
    compact: bool = False,
) -> EntrySummaryResponse:
    from openzim_mcp.bundle import get_or_build_bundle

    bundle = get_or_build_bundle(
        archive, entry_path,
        cache=self.cache,
        validated_path=validated_path,
        content_processor=self.content_processor,
    )

    md = bundle["rendered_markdown"]
    # Lead summary = first section's content (sections[0] = the entry's title h1
    # or first heading). If there are no sections, fall back to a leading slice.
    if bundle["sections"]:
        first = bundle["sections"][0]
        summary_md = md[first["char_start"]:first["char_end"]]
    else:
        summary_md = md

    # Truncate to max_words. Word boundary, not char.
    words = summary_md.split()
    is_truncated = len(words) > max_words
    if is_truncated:
        summary_md = " ".join(words[:max_words])

    payload = cast("EntrySummaryResponse", {
        "path": bundle["entry_path"],
        "title": bundle["title"],
        "summary": summary_md,
        "word_count": min(len(words), max_words),
        "content_type": bundle["content_type"],
        "is_truncated": is_truncated,
    })
    return cast("EntrySummaryResponse", attach_meta(payload))
```

- [ ] **Step 3: Remove the legacy `summary_data:` cache key from `get_entry_summary_data`**

The bundle is now the cache. Delete any code in `get_entry_summary_data` (around line 1111) that prefixes a `summary_data:` cache key, calls `cache.get()` with that prefix, or `cache.set()`s with that prefix. The bundle's own cache (under `bundle:v2c:`) handles the caching transparently inside `get_or_build_bundle`.

- [ ] **Step 4: Run existing entry-summary tests**

```bash
uv run pytest tests/test_get_entry_summary.py -v
```

Expected: all PASS. Wire-format-unchanged is the goal — if any assertion changed values, investigate and fix the slicer (likely the lead-section boundary heuristic). If the existing tests assumed a specific lead snippet shape that the bundle-based slice doesn't reproduce, update the slicer (not the tests) to match.

- [ ] **Step 5: Verify the bundle is being used (not the legacy parse path)**

```bash
uv run pytest tests/test_get_entry_summary.py -v --tb=short
uv run pytest tests/test_bundle.py -v
```

Both should PASS; mypy clean:

```bash
uv run mypy openzim_mcp
```

- [ ] **Step 6: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/zim/content.py
git commit -m "refactor(v2): get_entry_summary slices the EntryBundle (#11)

_extract_entry_summary_data now calls get_or_build_bundle and slices
rendered_markdown[sections[0].char_start:char_end] instead of running
its own process_mime_content. Removes the legacy 'summary_data:' cache
prefix; the bundle ('bundle:v2c:') is now the cache.

Wire format unchanged."
```

---

## Task 10: Collapse `get_table_of_contents` to a bundle slicer + apply TOC `id`→`section_id` rename

**Files:**
- Modify: `openzim_mcp/zim/structure.py:478` (`_extract_table_of_contents_data`)
- Modify: `openzim_mcp/zim/structure.py:517` (`_build_hierarchical_toc` — DELETE; bundle replaces it)
- Modify: `openzim_mcp/zim/structure.py:579` (`_headings_to_tree` — KEEP, but feed it bundle SectionMeta entries instead of fresh-parsed dicts; restructure as `_sections_to_toc_tree`)
- Modify: `openzim_mcp/zim/structure.py:410` (`get_table_of_contents_data` — remove `toc_data:` cache prefix)
- Modify: `tests/test_get_table_of_contents.py` (update assertions for `id`→`section_id` rename + flatter shape)

**Why:** Second tool. This is the wire-format-break task: `id` → `section_id`, and `toc: list[dict[str, Any]]` → `list[TocHeading]`. The existing `_build_hierarchical_toc` is replaced by a bundle-based path. The hierarchical assembly (`_headings_to_tree`) is reused but operates on bundle SectionMeta.

- [ ] **Step 1: Replace `_extract_table_of_contents_data`**

```python
def _extract_table_of_contents_data(
    self, archive: "Archive", entry_path: str
) -> TableOfContentsResponse:
    """Build TOC by slicing the EntryBundle's sections list."""
    from openzim_mcp.bundle import get_or_build_bundle

    # validated_path resolution stays as in the existing helper; reuse it:
    validated_path = self.path_validator.validate_path(self._zim_file_path)
    validated_path = self.path_validator.validate_zim_file(validated_path)

    bundle = get_or_build_bundle(
        archive, entry_path,
        cache=self.cache,
        validated_path=validated_path,
        content_processor=self.content_processor,
    )

    payload: TableOfContentsResponse = cast("TableOfContentsResponse", {
        "title": bundle["title"],
        "path": bundle["entry_path"],
        "content_type": bundle["content_type"],
        "toc": _sections_to_toc_tree(bundle["sections"]),
        "heading_count": len(bundle["sections"]),
        "max_depth": max((s["level"] for s in bundle["sections"]), default=0),
    })
    if not bundle["content_type"].startswith("text/html"):
        payload["message"] = (
            f"TOC extraction requires HTML content, got: {bundle['content_type']}"
        )
    elif not bundle["sections"]:
        payload["message"] = "No headings found in article"
    return cast("TableOfContentsResponse", attach_meta(payload))
```

(Note: `self._zim_file_path` may need to be threaded through differently. Check the existing call-site in `get_table_of_contents_data` — it usually has `validated_path` already in scope. If so, pass it as a parameter to `_extract_table_of_contents_data` instead of re-validating. The existing pattern in other `_extract_*_data` methods is the source of truth.)

- [ ] **Step 2: Add `_sections_to_toc_tree` (replaces `_build_hierarchical_toc` + `_headings_to_tree`)**

```python
def _sections_to_toc_tree(sections: list[SectionMeta]) -> list[TocHeading]:
    """Build a hierarchical TOC tree from a flat SectionMeta list.

    Uses a stack to nest headings by level. Each TocHeading has the
    Phase C field name `section_id` (renamed from the old `id`).
    """
    root: list[TocHeading] = []
    # Stack entries: (level, children_list_to_append_to)
    stack: list[tuple[int, list[TocHeading]]] = [(0, root)]

    for s in sections:
        node: TocHeading = cast("TocHeading", {
            "section_id": s["id"],
            "text": s["title"],
            "level": s["level"],
            "id_source": "",  # filled below if upstream tracked it
            "children": [],
        })
        while stack and stack[-1][0] >= s["level"]:
            stack.pop()
        if stack:
            stack[-1][1].append(node)
        else:
            root.append(node)
        stack.append((s["level"], node["children"]))

    return root
```

`id_source` provenance: the bundle's SectionMeta doesn't currently carry `id_source` (it was dropped when we wrote `bundle.py` Step 3 to keep the bundle minimal). Two options:

1. **Drop `id_source` from `TocHeading` entirely** — it's a debugging aid, not a contract surface. (Simpler. Recommended.)
2. **Add `id_source` to `SectionMeta`** so the bundle preserves it. Round-trip from `_build_headings`.

Pick (1): remove `id_source` from `TocHeading` (Task 4) and from this builder, OR keep it in `TocHeading` as `NotRequired[str]` and emit `""` from this builder. The cleanest end-state is to remove it entirely from `TocHeading`. Apply that edit to `tool_schemas.py` here as well: drop the `id_source: str` line from `TocHeading`. Update `tests/test_tool_schemas.py::test_toc_heading_uses_section_id_not_id` to drop `id_source` from the required-keys assertion.

- [ ] **Step 3: Delete `_build_hierarchical_toc` and `_headings_to_tree` from `structure.py`**

These are now dead code. Remove both functions. If any other call-site references them, update it to call the bundle-based path.

- [ ] **Step 4: Remove the `toc_data:` cache prefix from `get_table_of_contents_data`**

In `get_table_of_contents_data` (around line 410), delete code that prefixes a `toc_data:` cache key, calls `cache.get()` with it, or `cache.set()`s with it. The bundle handles caching.

- [ ] **Step 5: Update `tests/test_get_table_of_contents.py`**

Find existing assertions on `heading["id"]` and rename to `heading["section_id"]`. Find any test that called `heading["id_source"]` and remove it (the field is gone). If a test snapshotted the entire TOC dict, regenerate the snapshot with the new shape.

```bash
grep -nE 'heading\["id"\]|heading\["id_source"\]|"id":|"id_source":' tests/test_get_table_of_contents.py
```

For each match, update to `section_id` (or remove the `id_source` reference). If the test asserts the dict shape exactly, update the expected dict.

- [ ] **Step 6: Run TOC + bundle + tool-schemas tests**

```bash
uv run pytest tests/test_get_table_of_contents.py tests/test_bundle.py tests/test_tool_schemas.py -v
```

Expected: all PASS. If TOC shape mismatches surface, the cause is usually the `_sections_to_toc_tree` builder — verify against `DEEP_NESTED_HTML` fixture in `test_bundle.py`.

- [ ] **Step 7: Run mypy + lint + commit**

```bash
uv run mypy openzim_mcp
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/zim/structure.py openzim_mcp/tool_schemas.py tests/test_get_table_of_contents.py tests/test_tool_schemas.py
git commit -m "feat(v2)!: collapse get_table_of_contents to bundle slicer; rename id→section_id

_extract_table_of_contents_data now slices EntryBundle.sections via
_sections_to_toc_tree (replaces the deleted _build_hierarchical_toc +
_headings_to_tree). Wire format break: TOC heading field 'id' is renamed
to 'section_id' and toc tightens from list[dict[str, Any]] to
list[TocHeading]. Drops the id_source field (debugging-only, not a
contract surface).

Removes the legacy 'toc_data:' cache prefix; bundle handles caching.

Spec § Behavior > #11 + Wire-format break."
```

---

## Task 11: Collapse `get_article_structure` to a bundle slicer

**Files:**
- Modify: `openzim_mcp/zim/structure.py:122` (`_extract_article_structure_data`)
- Modify: `openzim_mcp/zim/structure.py:57` (`get_article_structure_data` — remove `structure_data:` cache prefix)
- Modify: `tests/test_get_article_structure.py` (assertions unchanged — wire format stable)

**Why:** Third tool. The current implementation runs its own HTML parse and section-preview extraction. After collapse, it derives `headings` and `sections` (with previews) from the bundle's flat `sections` list, slicing `rendered_markdown[s.char_start : s.char_start+300]` for each preview.

- [ ] **Step 1: Inspect current method**

```bash
sed -n '122,180p' openzim_mcp/zim/structure.py
```

Expected: a method that decodes raw HTML and calls `content_processor.extract_html_structure()`. Note the response shape (`ArticleStructureResponse` per `tool_schemas.py:363`).

- [ ] **Step 2: Replace the body of `_extract_article_structure_data`**

```python
def _extract_article_structure_data(
    self,
    archive: "Archive",
    validated_path: Path,
    entry_path: str,
) -> ArticleStructureResponse:
    """Article structure derived from the EntryBundle."""
    from openzim_mcp.bundle import get_or_build_bundle

    bundle = get_or_build_bundle(
        archive, entry_path,
        cache=self.cache,
        validated_path=validated_path,
        content_processor=self.content_processor,
    )

    md = bundle["rendered_markdown"]
    PREVIEW_CHARS = 300

    # Flat headings list (matches existing wire shape).
    headings = [
        {
            "id": s["id"],
            "text": s["title"],
            "level": s["level"],
            "position": i,
        }
        for i, s in enumerate(bundle["sections"])
    ]

    # Sections list with content preview from the bundle's char ranges.
    sections = [
        {
            "title": s["title"],
            "level": s["level"],
            "content_preview": md[s["char_start"]:s["char_start"] + PREVIEW_CHARS],
        }
        for s in bundle["sections"]
    ]

    payload: ArticleStructureResponse = cast("ArticleStructureResponse", {
        "title": bundle["title"],
        "path": bundle["entry_path"],
        "content_type": bundle["content_type"],
        "headings": headings,
        "sections": sections,
        "metadata": {},  # preserve existing key; populate as the legacy code did
        "word_count": bundle["word_count"],
        "character_count": bundle["char_count"],
    })
    return cast("ArticleStructureResponse", attach_meta(payload))
```

(If the legacy `metadata` field in `ArticleStructureResponse` was populated with anything more substantive than `{}`, replicate it from `bundle` fields. Inspect what the existing code emits to confirm.)

- [ ] **Step 3: Remove the `structure_data:` cache prefix from `get_article_structure_data`**

Delete cache.get/set calls that prefix `structure_data:` in the data-layer method around line 57.

- [ ] **Step 4: Run tests**

```bash
uv run pytest tests/test_get_article_structure.py -v
```

Expected: all PASS. If the `metadata` key shape is asserted by tests and now diverges, replicate the legacy metadata population from the bundle.

- [ ] **Step 5: Run mypy + lint + commit**

```bash
uv run mypy openzim_mcp
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/zim/structure.py
git commit -m "refactor(v2): get_article_structure slices the EntryBundle (#11)

_extract_article_structure_data derives headings and section previews
from bundle.sections (with previews via rendered_markdown[char_start :
char_start+300]). Removes the legacy 'structure_data:' cache prefix.

Wire format unchanged."
```

---

## Task 12: Collapse `extract_article_links` to a bundle slicer

**Files:**
- Modify: `openzim_mcp/zim/structure.py:178` (`extract_article_links_data`)
- Modify: `openzim_mcp/zim/structure.py:348` (`_get_or_load_link_extraction` — DELETE)
- Modify: `openzim_mcp/zim/structure.py:371` (`_load_link_extraction` — DELETE)
- Modify: `tests/test_extract_article_links.py` (assertions unchanged — wire format stable)

**Why:** Fourth and last collapse. The current implementation has its own load-once helper (`_load_link_extraction`) and a separate cache prefix (`links_full:v2b:`). After collapse, both are deleted; `extract_article_links_data` reads from `bundle["links"][kind]` and paginates in memory using the existing `Cursor` API.

- [ ] **Step 1: Inspect current `extract_article_links_data`**

```bash
sed -n '178,300p' openzim_mcp/zim/structure.py
```

Expected: method that builds the cursor `{o, l, ep, k}` and slices the bundle's link list. After Task 12, the only change is the source of `links` — the bundle, not `_get_or_load_link_extraction`.

- [ ] **Step 2: Replace the body of `extract_article_links_data`**

The function signature stays. Replace the link-loading section with bundle access:

```python
def extract_article_links_data(
    self,
    zim_file_path: str,
    entry_path: str,
    *,
    kind: str = "internal",
    limit: int = 50,
    offset: int = 0,
    cursor: Optional[str] = None,
) -> Union[LinksResponse, ToolErrorPayload]:
    """Extract one category of links from an entry, paginated.

    Phase C: links come from the EntryBundle. Cursor (Phase B) is
    unchanged: {o, l, ep, k}.
    """
    from openzim_mcp.bundle import get_or_build_bundle
    # ... existing kind-validation, cursor-decode logic stays ...

    validated_path = self.path_validator.validate_path(zim_file_path)
    validated_path = self.path_validator.validate_zim_file(validated_path)

    with _zim_ops_mod.zim_archive(validated_path) as archive:
        bundle = get_or_build_bundle(
            archive, entry_path,
            cache=self.cache,
            validated_path=validated_path,
            content_processor=self.content_processor,
        )

    all_links_for_kind = bundle["links"][kind]
    total = len(all_links_for_kind)
    page = all_links_for_kind[offset : offset + limit]
    done = (offset + len(page)) >= total
    next_cursor = None
    if not done:
        next_cursor = Cursor.encode(
            tool="extract_article_links",
            state={"o": offset + limit, "l": limit, "ep": entry_path, "k": kind},
        )
    category_totals = {
        "internal": len(bundle["links"]["internal"]),
        "external": len(bundle["links"]["external"]),
        "media": len(bundle["links"]["media"]),
    }

    payload: LinksResponse = cast("LinksResponse", {
        "title": bundle["title"],
        "path": bundle["entry_path"],
        "content_type": bundle["content_type"],
        "kind": kind,
        "results": page,
        "next_cursor": next_cursor,
        "total": total,
        "done": done,
        "page_info": {
            "offset": offset,
            "limit": limit,
            "returned_count": len(page),
        },
        "category_totals": category_totals,
    })
    return cast("LinksResponse", attach_meta(payload))
```

The exact existing structure (kind validation, cursor-decode handling, error envelope) should be preserved verbatim — the ONLY change is the source of links data. Do not rewrite the cursor logic — port it from the current implementation as-is.

- [ ] **Step 3: Delete `_get_or_load_link_extraction` and `_load_link_extraction`**

Both are dead code after Task 12. Remove them (lines 348-409 approximately).

- [ ] **Step 4: Run tests**

```bash
uv run pytest tests/test_extract_article_links.py -v
```

Expected: all PASS. If pagination assertions fail, the most likely cause is a mismatch between the bundle's link order and the legacy load-and-cache order. Inspect both, align the order in `_categorize_links` (in `bundle.py`) if necessary.

- [ ] **Step 5: Run mypy + lint + commit**

```bash
uv run mypy openzim_mcp
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/zim/structure.py
git commit -m "refactor(v2): extract_article_links slices the EntryBundle (#11)

extract_article_links_data reads bundle.links[kind] and paginates
in-memory via the Phase B Cursor (state unchanged: {o, l, ep, k}).
Removes the legacy 'links_full:v2b:' cache prefix and the dead
_get_or_load_link_extraction / _load_link_extraction helpers.

Wire format unchanged."
```

---

## Task 13: Cross-tool shared-bundle test (1 miss → N hits)

**Files:**
- Modify: `tests/test_structured_tool_output.py` (add cross-tool test)

**Why:** Proves the four collapsed tools share a single bundle build per entry. Without this assertion, a future regression that re-introduces per-tool parsing would silently degrade performance.

- [ ] **Step 1: Append failing test**

Find the existing `tests/test_structured_tool_output.py`. Append:

```python
def test_phase_c_bundle_shared_across_four_tools(monkeypatch, tmp_path) -> None:
    """All four collapsed tools share one EntryBundle per entry.

    First call to any tool builds the bundle; the other three calls hit
    the bundle cache. This is the load-bearing performance assertion for
    Phase C #11 — if it ever fails, someone re-introduced per-tool parsing.
    """
    from openzim_mcp.bundle import _bundle_cache_key
    # Use the existing test-fixture-creating helper from this file.
    # (See sibling tests for the canonical setup; reuse rather than rewrite.)
    handler = _make_test_handler(tmp_path)  # existing helper in this file
    cache = handler.cache
    initial_bundle_misses = sum(
        1 for _ in cache._cache.keys()
        if str(_).startswith("bundle:v2c:")
    )

    # Call all four tools on the same entry.
    handler.get_entry_summary("test.zim", "A/Berlin", max_words=200)
    handler.get_table_of_contents("test.zim", "A/Berlin")
    handler.get_article_structure("test.zim", "A/Berlin")
    handler.extract_article_links("test.zim", "A/Berlin", kind="internal", limit=50)

    # Exactly one bundle key exists in cache for this (path, entry).
    bundle_keys = [
        k for k in cache._cache.keys()
        if str(k).startswith("bundle:v2c:") and "A/Berlin" in str(k)
    ]
    assert len(bundle_keys) == 1, (
        f"Expected 1 shared bundle key, found {len(bundle_keys)}: {bundle_keys}"
    )

    # And the legacy per-tool prefixes are GONE.
    legacy_prefixes = ("summary_data:", "toc_data:", "structure_data:", "links_full:")
    for k in cache._cache.keys():
        for prefix in legacy_prefixes:
            assert not str(k).startswith(prefix), (
                f"Legacy cache prefix {prefix!r} still in use; expected only bundle:v2c:"
            )
```

(`_make_test_handler` is illustrative — use whatever fixture pattern the file already uses to construct a handler with cache + a test ZIM. The point of the test is the cache-key assertion at the end.)

- [ ] **Step 2: Run the test, confirm it passes**

```bash
uv run pytest tests/test_structured_tool_output.py::test_phase_c_bundle_shared_across_four_tools -v
```

Expected: PASS. If the assertion fires (`Expected 1 shared bundle key, found 0`), the bundle isn't being built — verify Tasks 9-12 actually swapped to `get_or_build_bundle`. If the legacy-prefix assertion fires, find the offending tool and drop its cache code (Task 9-12 step "Remove the X cache prefix").

- [ ] **Step 3: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add tests/test_structured_tool_output.py
git commit -m "test(v2): assert all four collapsed tools share one bundle per entry

Calls get_entry_summary, get_table_of_contents, get_article_structure,
extract_article_links on the same entry; asserts (a) exactly one
bundle:v2c: cache key exists for that (path, entry), and (b) none of
the legacy per-tool prefixes (summary_data:, toc_data:, structure_data:,
links_full:) appear. Load-bearing perf assertion for Phase C #11."
```

---

## Task 14: Add `_get_section_data` data-layer method

**Files:**
- Modify: `openzim_mcp/zim/structure.py` (add `get_section_data` + `_get_section_data` methods next to TOC code)
- Create: `tests/test_get_section.py`

**Why:** First half of #7. The data-layer method builds the bundle, finds the section, slices `rendered_markdown[char_start:char_end]`, applies `max_chars`, returns `GetSectionResponse`. The MCP tool registration comes in Task 15.

- [ ] **Step 1: Write failing tests**

Create `tests/test_get_section.py`:

```python
"""Tests for openzim_mcp.zim.structure.get_section_data."""

from __future__ import annotations

from pathlib import Path
from unittest.mock import MagicMock

import pytest

from tests.test_bundle import SAMPLE_HTML, _make_archive_with_entry  # reuse fixtures


@pytest.fixture
def handler(tmp_path: Path):
    """Construct a StructureToolsHandler around a single in-memory archive."""
    # Use the canonical handler-construction helper used by the rest of the
    # test suite (sibling files in tests/). Adjust import to match.
    from tests.helpers import make_structure_handler  # if exists; else inline
    return make_structure_handler(tmp_path, html=SAMPLE_HTML, entry_path="A/Berlin")


def test_get_section_returns_geography(handler) -> None:
    response = handler.get_section_data(
        "test.zim", "A/Berlin", section_id="geography"
    )
    # GetSectionResponse shape
    assert response["entry_path"] == "A/Berlin"
    assert response["title"] == "Berlin"
    assert response["section_id"] == "geography"
    assert response["section_title"] == "Geography"
    assert response["level"] == 2
    assert "Spree" in response["content_markdown"]
    assert response["truncated"] is False
    assert response["char_count"] == len(response["content_markdown"])
    assert response["word_count"] == len(response["content_markdown"].split())


def test_get_section_unknown_id_returns_tool_error(handler) -> None:
    response = handler.get_section_data(
        "test.zim", "A/Berlin", section_id="this-does-not-exist"
    )
    assert response.get("error") is True, f"Expected ToolErrorPayload, got: {response}"
    # available_section_ids must be present so the model can self-correct
    assert "available_section_ids" in response
    assert "geography" in response["available_section_ids"]


def test_get_section_max_chars_truncates(handler) -> None:
    response = handler.get_section_data(
        "test.zim", "A/Berlin", section_id="geography", max_chars=20
    )
    assert response["truncated"] is True
    assert len(response["content_markdown"]) <= 20


def test_get_section_meta_envelope_present(handler) -> None:
    response = handler.get_section_data(
        "test.zim", "A/Berlin", section_id="geography"
    )
    assert "_meta" in response
    assert "tokens_est" in response["_meta"]
```

(If the helpers don't exist, build the handler inline in a fixture using the same pattern as existing test files. The structure handler is constructed with a `cache`, `path_validator`, `content_processor`. See `tests/test_get_table_of_contents.py` for the pattern in this repo.)

- [ ] **Step 2: Run tests, confirm they fail**

```bash
uv run pytest tests/test_get_section.py -v
```

Expected: FAIL with `AttributeError: 'StructureToolsHandler' object has no attribute 'get_section_data'`.

- [ ] **Step 3: Add `get_section_data` and `_get_section_data` to `openzim_mcp/zim/structure.py`**

Place these methods next to `get_table_of_contents_data`:

```python
def get_section_data(
    self,
    zim_file_path: str,
    entry_path: str,
    section_id: str,
    *,
    max_chars: Optional[int] = None,
) -> Union[GetSectionResponse, ToolErrorPayload]:
    """Public entry point for the get_section tool. Returns the typed
    response or a ToolErrorPayload on entry-not-found / section-not-found."""
    try:
        validated_path = self.path_validator.validate_path(zim_file_path)
        validated_path = self.path_validator.validate_zim_file(validated_path)
        with _zim_ops_mod.zim_archive(validated_path) as archive:
            return self._get_section_data(
                archive, validated_path, entry_path, section_id, max_chars
            )
    except OpenZimMcpFileNotFoundError as e:
        return tool_error("file_not_found", message=str(e))
    except OpenZimMcpArchiveError as e:
        return tool_error("entry_not_found", message=str(e))


def _get_section_data(
    self,
    archive: "Archive",
    validated_path: Path,
    entry_path: str,
    section_id: str,
    max_chars: Optional[int],
) -> Union[GetSectionResponse, ToolErrorPayload]:
    from openzim_mcp.bundle import get_or_build_bundle

    bundle = get_or_build_bundle(
        archive, entry_path,
        cache=self.cache,
        validated_path=validated_path,
        content_processor=self.content_processor,
    )

    section = next(
        (s for s in bundle["sections"] if s["id"] == section_id),
        None,
    )
    if section is None:
        return tool_error(
            "section_not_found",
            message=(
                f"No section with id={section_id!r} in entry {entry_path!r}. "
                f"Available section IDs: see available_section_ids."
            ),
            available_section_ids=[s["id"] for s in bundle["sections"]],
        )

    body = bundle["rendered_markdown"][section["char_start"]:section["char_end"]]
    cap = max_chars if max_chars is not None else self.config.content.max_content_length
    truncated = len(body) > cap
    if truncated:
        body = body[:cap]

    payload: GetSectionResponse = cast("GetSectionResponse", {
        "entry_path": bundle["entry_path"],
        "title": bundle["title"],
        "section_id": section["id"],
        "section_title": section["title"],
        "level": section["level"],
        "parent_id": section.get("parent_id"),
        "content_markdown": body,
        "char_count": len(body),
        "word_count": len(body.split()),
        "truncated": truncated,
    })
    return cast("GetSectionResponse", attach_meta(payload))
```

(The `tool_error()` helper accepts arbitrary kwargs that flow into the payload — Phase B established this. If `available_section_ids` is rejected, drop into the message instead. Verify the `tool_error` signature in `openzim_mcp/errors.py` or `tool_schemas.py`.)

- [ ] **Step 4: Run tests, confirm they pass**

```bash
uv run pytest tests/test_get_section.py -v
```

Expected: all PASS. If `available_section_ids` doesn't survive in the ToolErrorPayload, adjust `tool_error()` to accept extra fields or attach the list to a different field.

- [ ] **Step 5: Run mypy + lint + commit**

```bash
uv run mypy openzim_mcp
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/zim/structure.py tests/test_get_section.py
git commit -m "feat(v2): add get_section data layer (#7)

get_section_data is the public entry point; _get_section_data slices
EntryBundle.sections by section_id and returns GetSectionResponse, or
tool_error('section_not_found', available_section_ids=[...]) on miss.

Tool registration comes in Task 15."
```

---

## Task 15: Register the `get_section` MCP tool

**Files:**
- Modify: `openzim_mcp/tools/structure_tools.py` (add `@server.tool()` registration)
- Modify: `tests/test_structured_tool_output.py` (add integration assertions for `get_section`)

**Why:** Wires the data-layer method (Task 14) into FastMCP via the established Phase B pattern: thin wrapper → `_data` method → `Union[GetSectionResponse, ToolErrorPayload]` return annotation. After this task, `get_section` appears in tool listings.

- [ ] **Step 1: Inspect existing tool registrations in `openzim_mcp/tools/structure_tools.py`**

```bash
grep -nE '@server\.|async def' openzim_mcp/tools/structure_tools.py | head -20
```

Expected: pattern of `@server.mcp.tool()` decorators wrapping async functions that delegate to `handler.<method>_data(...)`. Match this pattern for `get_section`.

- [ ] **Step 2: Add the registration**

In `openzim_mcp/tools/structure_tools.py`, near `get_table_of_contents`:

```python
@server.mcp.tool()
async def get_section(
    zim_file_path: str,
    entry_path: str,
    section_id: str,
    max_chars: Optional[int] = None,
) -> Union[GetSectionResponse, ToolErrorPayload]:
    """Fetch a single section from an entry by its section_id.

    section_id values come from get_table_of_contents (TocHeading.section_id)
    or get_article_structure (heading[].id). Returns the section body as
    markdown plus metadata. Sections are sized for direct consumption
    (≈500-1500 tokens); a max_chars cap truncates the body and sets
    truncated=True.

    On a miss (no section with that id), returns a ToolErrorPayload with
    error="section_not_found" and available_section_ids in the payload so
    the model can self-correct.

    Args:
        zim_file_path: Path to the ZIM file.
        entry_path: Entry path (e.g., "A/Berlin").
        section_id: Section identifier from TOC.
        max_chars: Optional cap on section body chars (default uses
                   config.content.max_content_length).
    """
    return await asyncio.to_thread(
        handler.get_section_data,
        zim_file_path,
        entry_path,
        section_id,
        max_chars=max_chars,
    )
```

(Check the actual registration pattern — `handler` may be `structure_handler`, the `@server.mcp.tool()` may be `@server.tool()`. Match what TOC / article_structure use in this file.)

- [ ] **Step 3: Add integration test in `tests/test_structured_tool_output.py`**

```python
def test_get_section_returns_structured_content(structured_client) -> None:
    """End-to-end: call get_section via the MCP client surface, verify shape."""
    response = structured_client.call_tool(
        "get_section",
        {"zim_file_path": "test.zim", "entry_path": "A/Berlin",
         "section_id": "geography"},
    )
    structured = response.structuredContent
    payload = structured.get("result", structured)  # wrapper-tolerant unwrap
    assert payload["section_id"] == "geography"
    assert payload["section_title"] == "Geography"
    assert "_meta" in payload
```

(Use whatever `structured_client` / call-tool fixture this file already provides. The wrapper-tolerant unwrap is mandatory per the Phase B note at the top of the plan.)

- [ ] **Step 4: Run tests**

```bash
uv run pytest tests/test_structured_tool_output.py::test_get_section_returns_structured_content tests/test_get_section.py -v
```

Expected: all PASS. If the registration's import or signature is wrong, the test will fail at the call_tool step — adjust the registration to match the file's existing patterns.

- [ ] **Step 5: Run mypy + full structure tests + lint + commit**

```bash
uv run mypy openzim_mcp
uv run pytest tests/test_get_section.py tests/test_structured_tool_output.py -v
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/tools/structure_tools.py tests/test_structured_tool_output.py
git commit -m "feat(v2): register get_section MCP tool (#7)

@server.tool() get_section(zim_file_path, entry_path, section_id, max_chars=None)
returns Union[GetSectionResponse, ToolErrorPayload]. Wraps the data
layer added in Task 14. Adds end-to-end test via the structured client
with the wrapper-tolerant unwrap pattern."
```

---

## Task 16: Create `openzim_mcp/synthesize.py` with RRF helper + skeleton

**Files:**
- Create: `openzim_mcp/synthesize.py`
- Create: `tests/test_synthesize.py` (RRF helper tests)

**Why:** First synthesize task. Lands the module skeleton, the `synthesize_query` signature stub, and the pure RRF (Reciprocal Rank Fusion) helper with deterministic tests. Subsequent tasks fill in the pipeline stages.

- [ ] **Step 1: Write failing tests for the RRF helper**

Create `tests/test_synthesize.py`:

```python
"""Tests for openzim_mcp.synthesize."""

from __future__ import annotations

import pytest

from openzim_mcp.synthesize import _rrf_fuse


def test_rrf_fuse_single_ranking_preserves_order() -> None:
    """One ranking → output is the same order, with k-decayed scores."""
    rankings = [
        [("A/Doc1", 0.9), ("A/Doc2", 0.7), ("A/Doc3", 0.5)],
    ]
    fused = _rrf_fuse(rankings, k=60)
    paths = [p for p, _ in fused]
    assert paths == ["A/Doc1", "A/Doc2", "A/Doc3"]


def test_rrf_fuse_two_rankings_unifies() -> None:
    """Doc that appears high in both rankings beats one that appears in just one."""
    rankings = [
        [("A/Doc1", 0.9), ("A/Doc2", 0.7), ("A/Doc3", 0.5)],
        [("A/Doc1", 0.8), ("A/Doc4", 0.6), ("A/Doc2", 0.4)],
    ]
    fused = _rrf_fuse(rankings, k=60)
    paths = [p for p, _ in fused]
    # Doc1 appears at rank 1 in both → highest fused score.
    assert paths[0] == "A/Doc1"
    # Doc2 appears at ranks 2 + 3 → next.
    assert paths[1] == "A/Doc2"


def test_rrf_fuse_empty_rankings_returns_empty() -> None:
    assert _rrf_fuse([], k=60) == []


def test_rrf_fuse_score_formula() -> None:
    """Score(d) = sum over rankings of 1/(k + rank(d))."""
    rankings = [
        [("A/Doc1", 0.9)],   # rank 1 in this ranking
        [("A/Doc1", 0.5)],   # rank 1 in this ranking too
    ]
    fused = _rrf_fuse(rankings, k=60)
    expected = 1.0 / (60 + 1) + 1.0 / (60 + 1)
    assert fused[0][1] == pytest.approx(expected)
```

- [ ] **Step 2: Run tests, confirm they fail**

```bash
uv run pytest tests/test_synthesize.py -v
```

Expected: FAIL with `ModuleNotFoundError: No module named 'openzim_mcp.synthesize'`.

- [ ] **Step 3: Create `openzim_mcp/synthesize.py`**

```python
"""Server-side synthesize mode for zim_query.

Phase C #10: when zim_query is called with synthesize=True, this module
runs the pipeline:

    per-archive search → fuse (RRF for multi-archive) → rerank (identity
    in Phase C; cross-encoder hook in Phase D) → passage extraction via
    libzim.SearchIterator.getSnippet → section attribution via the
    EntryBundle → citation rendering → budget enforcement

Returns SynthesizeResponse (defined in tool_schemas.py).
"""

from __future__ import annotations

import logging
from collections import defaultdict
from pathlib import Path
from typing import TYPE_CHECKING, Optional, cast

from openzim_mcp.tool_schemas import (
    Citation,
    SynthesizePassage,
    SynthesizeResponse,
)

if TYPE_CHECKING:
    from libzim.reader import Archive

    from openzim_mcp.cache import OpenZimMcpCache
    from openzim_mcp.config import SynthesizeConfig
    from openzim_mcp.content_processor import ContentProcessor

logger = logging.getLogger(__name__)


# ---------------------------------------------------------------------------
# RRF helper — Reciprocal Rank Fusion
# ---------------------------------------------------------------------------

def _rrf_fuse(
    rankings: list[list[tuple[str, float]]],
    *,
    k: int = 60,
) -> list[tuple[str, float]]:
    """Combine multiple per-archive rankings into a single fused ranking.

    For each document, score = sum over rankings of 1 / (k + rank). k=60
    is the standard from the Cormack et al. reference. Documents missing
    from a ranking contribute 0 from that ranking.

    Args:
        rankings: list of per-source rankings; each ranking is a list of
                  (entry_path, source_score) in rank order.
        k: smoothing constant.

    Returns:
        list of (entry_path, fused_score) sorted by fused_score descending.
    """
    if not rankings:
        return []
    scores: dict[str, float] = defaultdict(float)
    for ranking in rankings:
        for rank, (path, _src_score) in enumerate(ranking, start=1):
            scores[path] += 1.0 / (k + rank)
    fused = sorted(scores.items(), key=lambda x: x[1], reverse=True)
    return fused
```

- [ ] **Step 4: Run tests**

```bash
uv run pytest tests/test_synthesize.py -v
```

Expected: all PASS.

- [ ] **Step 5: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/synthesize.py tests/test_synthesize.py
git commit -m "feat(v2): add openzim_mcp/synthesize.py skeleton + RRF helper

_rrf_fuse implements the standard k=60 Reciprocal Rank Fusion over
per-source rankings. Pure, deterministic, no archive deps. The pipeline
stages (per-archive search, passage extraction, attribution, citation
rendering, budget enforcement) land in Tasks 17-21."
```

---

## Task 17: Add per-archive search stage to `synthesize_query`

**Files:**
- Modify: `openzim_mcp/synthesize.py` (add `_per_archive_search` and `synthesize_query` signature)
- Modify: `tests/test_synthesize.py` (add tests for single-archive + multi-archive search)

**Why:** The first stage of the synthesize pipeline. Single-archive uses `search_zim_file` (top-K by Xapian); multi-archive uses `search_all` (per-archive top-K). This task lands the search wrapper and a deterministic test using mock archives.

- [ ] **Step 1: Write failing tests**

Append to `tests/test_synthesize.py`:

```python
from unittest.mock import MagicMock

from openzim_mcp.config import SynthesizeConfig


def test_per_archive_search_single_archive() -> None:
    """Single archive → list[(entry_path, snippet, score)] from Xapian."""
    from openzim_mcp.synthesize import _per_archive_search

    archive = MagicMock()
    # Mock the search-iterator return shape used by openzim_mcp.zim.search
    archive.basename = "wikipedia_en_simple"

    # _per_archive_search calls into the existing search infrastructure;
    # it should be testable via a search-handler mock.
    search_handler = MagicMock()
    search_handler.search_top_k.return_value = [
        {"path": "A/Berlin", "snippet": "...", "score": 0.9},
        {"path": "A/Munich", "snippet": "...", "score": 0.7},
    ]

    results = _per_archive_search(
        archive, search_handler=search_handler, query="german cities", k=5,
    )
    assert len(results) == 2
    assert results[0]["path"] == "A/Berlin"


def test_synthesize_query_signature_exists() -> None:
    """synthesize_query is callable; signature stable from this task on."""
    from openzim_mcp.synthesize import synthesize_query
    import inspect
    sig = inspect.signature(synthesize_query)
    assert {"query", "archives", "cache", "content_processor", "config"} <= set(sig.parameters)
```

- [ ] **Step 2: Run tests, confirm they fail**

```bash
uv run pytest tests/test_synthesize.py::test_per_archive_search_single_archive tests/test_synthesize.py::test_synthesize_query_signature_exists -v
```

Expected: FAIL with `ImportError: cannot import name '_per_archive_search'` and `synthesize_query`.

- [ ] **Step 3: Add `_per_archive_search` and `synthesize_query` skeleton to `openzim_mcp/synthesize.py`**

Append to `openzim_mcp/synthesize.py`:

```python
# ---------------------------------------------------------------------------
# Pipeline stage 1: per-archive search
# ---------------------------------------------------------------------------

def _per_archive_search(
    archive: "Archive",
    *,
    search_handler,  # SearchToolsHandler — typed loosely to avoid circular import
    query: str,
    k: int,
) -> list[dict]:
    """Run Xapian search on one archive, return top-K hits as dicts.

    Hit shape: {"path": str, "snippet": str, "score": float}.
    """
    return search_handler.search_top_k(archive, query, k=k)


# ---------------------------------------------------------------------------
# synthesize_query — main entry point (skeleton; stages added in Tasks 18-21)
# ---------------------------------------------------------------------------

def synthesize_query(
    query: str,
    *,
    archives: list[tuple["Archive", "Path"]],   # (archive, validated_path) pairs
    search_handler,
    cache: "OpenZimMcpCache",
    content_processor: "ContentProcessor",
    config: "SynthesizeConfig",
) -> SynthesizeResponse:
    """Run the synthesize pipeline. Stages added incrementally in Tasks 18-21."""
    # Stage 1: per-archive search
    per_archive_results: list[list[dict]] = []
    archives_searched: list[str] = []
    for archive, validated_path in archives:
        archive_name = validated_path.stem
        archives_searched.append(archive_name)
        per_archive_results.append(
            _per_archive_search(
                archive, search_handler=search_handler,
                query=query, k=config.per_archive_k,
            )
        )

    # Subsequent stages added in Tasks 18-21 below this line.
    # For now, return a minimal SynthesizeResponse so the module imports cleanly.
    return cast("SynthesizeResponse", {
        "query": query,
        "answer_markdown": "",
        "passages": [],
        "citations": [],
        "archives_searched": archives_searched,
        "fallback_used": "xapian_score" if len(archives) == 1 else "rrf_fusion",
        "total_chars": 0,
        "total_words": 0,
        "_meta": {},
    })
```

(`SearchToolsHandler.search_top_k` is the helper this task assumes exists. It does not — Task 17b is to add it. To keep tasks atomic, do step 4 below.)

- [ ] **Step 4: Add `search_top_k` helper to `openzim_mcp/zim/search.py`**

```bash
grep -nE 'class SearchToolsHandler|def search_zim_file|def search_all' openzim_mcp/zim/search.py | head
```

Pick a stable insertion point near `search_zim_file`. Add a small wrapper that returns the canonical `[{path, snippet, score}]` shape used by synthesize:

```python
def search_top_k(self, archive: "Archive", query: str, *, k: int) -> list[dict]:
    """Top-K Xapian results from an open archive as a flat dict list.

    Used by synthesize.py as the primary stage of the pipeline. Snippet
    is the libzim getSnippet() output (Phase A #1).
    """
    from libzim.search import Query, Searcher
    searcher = Searcher(archive)
    search = searcher.search(Query().set_query(query))
    iterator = iter(search.getResults(0, k))
    hits: list[dict] = []
    while True:
        try:
            entry_id = next(iterator)
        except StopIteration:
            break
        # libzim's SearchIterator exposes getSnippet/getScore on the iterator;
        # the API may differ slightly. Match the existing search_zim_file
        # implementation in this file for snippet/score extraction.
        snippet = iterator.getSnippet() if hasattr(iterator, "getSnippet") else ""
        score = float(iterator.getScore()) if hasattr(iterator, "getScore") else 0.0
        path = entry_id  # entry_id IS the path in libzim 9+
        hits.append({"path": path, "snippet": snippet, "score": score})
    return hits
```

(The exact libzim API for snippet/score may differ — check what `search_zim_file_data` already uses in this file and replicate.)

- [ ] **Step 5: Run tests**

```bash
uv run pytest tests/test_synthesize.py -v
```

Expected: PASS for the two new tests in this task. The mock-based test exercises the wrapper without touching libzim.

- [ ] **Step 6: Run mypy + lint + commit**

```bash
uv run mypy openzim_mcp
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/synthesize.py openzim_mcp/zim/search.py tests/test_synthesize.py
git commit -m "feat(v2): synthesize stage 1 — per-archive Xapian search

Adds SearchToolsHandler.search_top_k wrapper returning {path, snippet,
score}; _per_archive_search calls it from openzim_mcp.synthesize. The
synthesize_query skeleton drives stage 1 against (archive,
validated_path) pairs and emits a minimal SynthesizeResponse —
subsequent stages land in Tasks 18-21."
```

---

## Task 18: Add passage extraction stage (libzim snippet → SynthesizePassage)

**Files:**
- Modify: `openzim_mcp/synthesize.py` (add `_extract_passages` stage; integrate into `synthesize_query`)
- Modify: `tests/test_synthesize.py` (add passage-extraction tests)

**Why:** Stage 4 of the pipeline (per spec § "Pipeline"). Takes the post-fusion hit list and converts each hit into a `SynthesizePassage` with `text_markdown` (the libzim snippet, possibly markdown-rendered if it comes back as HTML), `rank`, and `score`. Citation attribution (Stage 5) comes in Task 19.

- [ ] **Step 1: Write failing tests**

```python
def test_extract_passages_renders_markdown_from_snippets() -> None:
    """libzim snippets may come back as HTML; passages contain markdown."""
    from openzim_mcp.synthesize import _extract_passages
    hits = [
        {"path": "A/Berlin", "snippet": "<p>Berlin is the capital.</p>", "score": 0.9},
        {"path": "A/Munich", "snippet": "<p>Munich is in Bavaria.</p>", "score": 0.7},
    ]
    cp = MagicMock()
    cp.html_to_plain_text.side_effect = lambda html: html.replace("<p>", "").replace("</p>", "")

    passages = _extract_passages(hits, archive_name="wikipedia_en_simple",
                                  content_processor=cp)
    assert len(passages) == 2
    assert passages[0]["rank"] == 1
    assert passages[0]["text_markdown"] == "Berlin is the capital."
    assert passages[0]["cite_id"] == "wikipedia_en_simple/A/Berlin"  # no #section yet
    assert passages[0]["score"] == 0.9


def test_extract_passages_preserves_rank_order() -> None:
    from openzim_mcp.synthesize import _extract_passages
    hits = [{"path": f"A/Doc{i}", "snippet": f"<p>doc {i}</p>", "score": 1.0 - 0.1*i}
            for i in range(5)]
    cp = MagicMock()
    cp.html_to_plain_text.side_effect = lambda html: html
    passages = _extract_passages(hits, archive_name="test", content_processor=cp)
    assert [p["rank"] for p in passages] == [1, 2, 3, 4, 5]
```

- [ ] **Step 2: Run tests, confirm they fail**

```bash
uv run pytest tests/test_synthesize.py::test_extract_passages_renders_markdown_from_snippets tests/test_synthesize.py::test_extract_passages_preserves_rank_order -v
```

Expected: FAIL with `ImportError: cannot import name '_extract_passages'`.

- [ ] **Step 3: Add `_extract_passages` to `openzim_mcp/synthesize.py`**

```python
# ---------------------------------------------------------------------------
# Pipeline stage 4: passage extraction
# ---------------------------------------------------------------------------

def _extract_passages(
    hits: list[dict],
    *,
    archive_name: str,
    content_processor: "ContentProcessor",
) -> list[SynthesizePassage]:
    """Convert hit dicts into SynthesizePassage entries.

    Each hit's snippet (potentially HTML from libzim.getSnippet) is
    rendered to markdown. cite_id is the entry-level form
    f"{archive_name}/{path}" — the "#section_id" suffix is added by
    Stage 5 (_attribute_sections) once the bundle is consulted.
    """
    passages: list[SynthesizePassage] = []
    for rank, hit in enumerate(hits, start=1):
        snippet = hit.get("snippet") or ""
        # Render HTML snippet to markdown text (use the same html_to_plain_text
        # path as content_processor uses elsewhere; the snippet is small so the
        # render is cheap).
        text = content_processor.html_to_plain_text(snippet) if snippet else ""
        text = text.strip()
        cite_id = f"{archive_name}/{hit['path']}"
        passages.append(cast("SynthesizePassage", {
            "cite_id": cite_id,
            "text_markdown": text,
            "rank": rank,
            "score": float(hit.get("score", 0.0)),
        }))
    return passages
```

(`ContentProcessor.html_to_plain_text` is the standard rendering helper the rest of the codebase uses. If the actual method name is different, match it — `grep -n 'def html_to' openzim_mcp/content_processor.py`.)

- [ ] **Step 4: Run tests**

```bash
uv run pytest tests/test_synthesize.py -v
```

Expected: all PASS.

- [ ] **Step 5: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/synthesize.py tests/test_synthesize.py
git commit -m "feat(v2): synthesize stage 4 — passage extraction

_extract_passages converts hit dicts into SynthesizePassage with
markdown-rendered snippets, sequential rank, and entry-level cite_id
(archive/entry_path; the #section suffix is added by Stage 5 in Task 19)."
```

---

## Task 19: Add section attribution stage (`_attribute_sections`)

**Files:**
- Modify: `openzim_mcp/synthesize.py` (add `_attribute_sections`)
- Modify: `tests/test_synthesize.py` (add tests)

**Why:** Stage 5 of the pipeline. For each passage, look up its entry's `EntryBundle` and find the section whose char_range contains the snippet text. Update `cite_id` to `archive/entry_path#section_id`. On bundle-build failure for a hit, drop the section attribution silently — keep the passage, leave its cite_id at the entry level.

- [ ] **Step 1: Write failing tests**

```python
def test_attribute_sections_adds_section_id_when_match_in_first_section() -> None:
    """Passage text found in section 0 → cite_id gets #<section_id>."""
    from openzim_mcp.synthesize import _attribute_sections

    bundle = {
        "rendered_markdown": "# Berlin\n\nIntro paragraph.\n\n## Geography\n\nGeography text here.\n",
        "sections": [
            {"id": "berlin", "title": "Berlin", "level": 1,
             "char_start": 0, "char_end": 38, "parent_id": None},
            {"id": "geography", "title": "Geography", "level": 2,
             "char_start": 38, "char_end": 75, "parent_id": "berlin"},
        ],
    }
    passages = [
        cast("SynthesizePassage", {
            "cite_id": "wiki/A/Berlin",
            "text_markdown": "Geography text here.",
            "rank": 1, "score": 0.9,
        }),
    ]
    bundle_lookup = lambda path: bundle if path == "A/Berlin" else None

    attributed = _attribute_sections(passages, bundle_lookup=bundle_lookup,
                                      hit_paths=["A/Berlin"])
    assert attributed[0]["cite_id"] == "wiki/A/Berlin#geography"


def test_attribute_sections_drops_section_on_bundle_failure() -> None:
    """Bundle build fails → cite_id stays at entry level; passage not dropped."""
    from openzim_mcp.synthesize import _attribute_sections

    def bundle_lookup(path):
        raise RuntimeError("simulated archive read failure")

    passages = [
        cast("SynthesizePassage", {
            "cite_id": "wiki/A/Berlin",
            "text_markdown": "Anything",
            "rank": 1, "score": 0.5,
        }),
    ]
    attributed = _attribute_sections(passages, bundle_lookup=bundle_lookup,
                                      hit_paths=["A/Berlin"])
    assert len(attributed) == 1
    assert attributed[0]["cite_id"] == "wiki/A/Berlin"   # unchanged


def test_attribute_sections_no_match_keeps_entry_level() -> None:
    """Passage text not located in any section → cite_id stays at entry level."""
    from openzim_mcp.synthesize import _attribute_sections

    bundle = {
        "rendered_markdown": "Whole article body here.",
        "sections": [
            {"id": "intro", "title": "Intro", "level": 1,
             "char_start": 0, "char_end": 24, "parent_id": None},
        ],
    }
    bundle_lookup = lambda path: bundle
    passages = [
        cast("SynthesizePassage", {
            "cite_id": "wiki/A/Berlin",
            "text_markdown": "totally unrelated string",
            "rank": 1, "score": 0.1,
        }),
    ]
    attributed = _attribute_sections(passages, bundle_lookup=bundle_lookup,
                                      hit_paths=["A/Berlin"])
    assert attributed[0]["cite_id"] == "wiki/A/Berlin"
```

- [ ] **Step 2: Run tests, confirm they fail**

```bash
uv run pytest tests/test_synthesize.py -k 'attribute' -v
```

Expected: FAIL with `ImportError: cannot import name '_attribute_sections'`.

- [ ] **Step 3: Add `_attribute_sections` to `openzim_mcp/synthesize.py`**

```python
# ---------------------------------------------------------------------------
# Pipeline stage 5: section attribution
# ---------------------------------------------------------------------------

def _attribute_sections(
    passages: list[SynthesizePassage],
    *,
    bundle_lookup,   # callable: entry_path -> EntryBundle (raises on failure)
    hit_paths: list[str],
) -> list[SynthesizePassage]:
    """For each passage, find its containing section in the bundle and append #section_id.

    On bundle-build failure for an entry, leave the cite_id at entry level
    (no #section suffix). The passage is never dropped — section attribution
    is best-effort.
    """
    attributed: list[SynthesizePassage] = []
    for passage, hit_path in zip(passages, hit_paths):
        new_cite_id = passage["cite_id"]
        try:
            bundle = bundle_lookup(hit_path)
        except Exception as e:
            logger.info(
                "Bundle build failed for %s during synthesize attribution: %s",
                hit_path, e,
            )
            attributed.append(passage)
            continue

        if bundle is None:
            attributed.append(passage)
            continue

        # Locate the passage text in rendered_markdown; find the section
        # whose char_range contains the match start.
        md = bundle.get("rendered_markdown", "")
        passage_text = passage["text_markdown"]
        if not passage_text or not md:
            attributed.append(passage)
            continue

        # Substring-find the first occurrence (passage texts are typically
        # 100-300 chars and the entry is ~30-80 KB; .find() is fast enough).
        pos = md.find(passage_text)
        if pos < 0:
            attributed.append(passage)
            continue

        for section in bundle.get("sections", []):
            if section["char_start"] <= pos < section["char_end"]:
                new_cite_id = f"{passage['cite_id']}#{section['id']}"
                break

        new_passage = dict(passage)
        new_passage["cite_id"] = new_cite_id
        attributed.append(cast("SynthesizePassage", new_passage))
    return attributed
```

- [ ] **Step 4: Run tests**

```bash
uv run pytest tests/test_synthesize.py -v
```

Expected: all PASS.

- [ ] **Step 5: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/synthesize.py tests/test_synthesize.py
git commit -m "feat(v2): synthesize stage 5 — section attribution via bundle

_attribute_sections looks up each passage's bundle, finds the section
whose char_range contains the passage text, and appends #section_id to
cite_id. Bundle build failures are caught and logged at info level;
the passage is kept but stays at entry-level citation."
```

---

## Task 20: Add citation rendering + budget enforcement stages

**Files:**
- Modify: `openzim_mcp/synthesize.py` (add `_render_answer`, `_enforce_budget`, `_build_citations`)
- Modify: `tests/test_synthesize.py` (add tests)

**Why:** Stages 6 and 7. `_render_answer` concatenates passages into `answer_markdown` with inline `[cite: ...]` markers. `_enforce_budget` truncates the last passage if `output_char_budget` is exceeded. `_build_citations` expands cite_ids into the structured `Citation` list.

- [ ] **Step 1: Write failing tests**

```python
def test_render_answer_joins_passages_with_citations() -> None:
    from openzim_mcp.synthesize import _render_answer
    passages = [
        cast("SynthesizePassage", {
            "cite_id": "wiki/A/Berlin#geography",
            "text_markdown": "Berlin is mostly flat.",
            "rank": 1, "score": 0.9,
        }),
        cast("SynthesizePassage", {
            "cite_id": "wiki/A/Berlin#climate",
            "text_markdown": "Climate is humid continental.",
            "rank": 2, "score": 0.7,
        }),
    ]
    md = _render_answer(passages)
    assert "Berlin is mostly flat." in md
    assert "[cite: wiki/A/Berlin#geography]" in md
    assert "Climate is humid continental." in md
    assert "[cite: wiki/A/Berlin#climate]" in md


def test_enforce_budget_truncates_last_passage() -> None:
    from openzim_mcp.synthesize import _enforce_budget
    passages = [
        cast("SynthesizePassage", {"cite_id": "x/y", "text_markdown": "A" * 50,
                                    "rank": 1, "score": 1.0}),
        cast("SynthesizePassage", {"cite_id": "x/y", "text_markdown": "B" * 50,
                                    "rank": 2, "score": 0.9}),
    ]
    capped = _enforce_budget(passages, char_budget=70)
    # First passage is 50 chars; budget allows 70. Second passage truncates to 20.
    assert len(capped) == 2
    assert len(capped[0]["text_markdown"]) == 50
    assert len(capped[1]["text_markdown"]) == 20


def test_build_citations_dedupes_by_entry() -> None:
    from openzim_mcp.synthesize import _build_citations
    passages = [
        cast("SynthesizePassage", {"cite_id": "wiki/A/Berlin#geography", "text_markdown": "",
                                    "rank": 1, "score": 0.9}),
        cast("SynthesizePassage", {"cite_id": "wiki/A/Berlin#climate", "text_markdown": "",
                                    "rank": 2, "score": 0.7}),
        cast("SynthesizePassage", {"cite_id": "wiki/A/Munich#geography", "text_markdown": "",
                                    "rank": 3, "score": 0.5}),
    ]
    bundle_titles = {"A/Berlin": "Berlin", "A/Munich": "Munich"}
    bundle_section_titles = {
        ("A/Berlin", "geography"): "Geography",
        ("A/Berlin", "climate"): "Climate",
        ("A/Munich", "geography"): "Geography",
    }
    citations = _build_citations(
        passages, archive_titles=bundle_titles,
        section_titles=bundle_section_titles,
    )
    # 3 unique cite_ids → 3 citations
    assert len(citations) == 3
    cite_ids = [c["cite_id"] for c in citations]
    assert "wiki/A/Berlin#geography" in cite_ids
    berlin_geo = next(c for c in citations if c["cite_id"] == "wiki/A/Berlin#geography")
    assert berlin_geo["title"] == "Berlin"
    assert berlin_geo["section_title"] == "Geography"
    assert berlin_geo["section_id"] == "geography"
    assert berlin_geo["entry_path"] == "A/Berlin"
    assert berlin_geo["archive"] == "wiki"
```

- [ ] **Step 2: Run, confirm fails**

```bash
uv run pytest tests/test_synthesize.py -k 'render_answer or enforce_budget or build_citations' -v
```

Expected: ImportError for the three new helpers.

- [ ] **Step 3: Add the three helpers to `openzim_mcp/synthesize.py`**

```python
# ---------------------------------------------------------------------------
# Pipeline stages 6-7: render + enforce budget; helper to build Citations
# ---------------------------------------------------------------------------

def _render_answer(passages: list[SynthesizePassage]) -> str:
    """Concatenate passages with inline [cite: ...] markers."""
    parts: list[str] = []
    for p in passages:
        parts.append(f"{p['text_markdown']}\n[cite: {p['cite_id']}]")
    return "\n\n".join(parts)


def _enforce_budget(
    passages: list[SynthesizePassage],
    *,
    char_budget: int,
) -> list[SynthesizePassage]:
    """Truncate passages so total text_markdown chars <= char_budget.

    Iterates in rank order; the last passage that doesn't fit is truncated
    to fit the remaining budget. Subsequent passages are dropped if budget
    is exhausted. Citation markers are NOT counted against the budget
    (they're small and rendering happens after this stage).
    """
    budget = char_budget
    capped: list[SynthesizePassage] = []
    for p in passages:
        body = p["text_markdown"]
        if len(body) <= budget:
            capped.append(p)
            budget -= len(body)
            continue
        # Truncate this passage to fit, then stop.
        if budget > 0:
            new_p = dict(p)
            new_p["text_markdown"] = body[:budget]
            capped.append(cast("SynthesizePassage", new_p))
        break
    return capped


def _parse_cite_id(cite_id: str) -> tuple[str, str, Optional[str]]:
    """Decompose 'archive/entry_path#section_id' into its parts.

    archive/entry_path is everything before '#'; section_id is the suffix
    after '#' (or None if absent). The archive identifier is the FIRST
    path segment (basename of the .zim, minus extension).
    """
    base, _, section_id = cite_id.partition("#")
    archive, _, entry_path = base.partition("/")
    return archive, entry_path, (section_id or None)


def _build_citations(
    passages: list[SynthesizePassage],
    *,
    archive_titles: dict[str, str],          # entry_path -> entry title
    section_titles: dict[tuple[str, str], str],  # (entry_path, section_id) -> section title
) -> list[Citation]:
    """Expand passages' cite_ids into Citation entries; dedupe by cite_id."""
    seen: dict[str, Citation] = {}
    for p in passages:
        cite_id = p["cite_id"]
        if cite_id in seen:
            continue
        archive, entry_path, section_id = _parse_cite_id(cite_id)
        title = archive_titles.get(entry_path, entry_path)
        section_title = (
            section_titles.get((entry_path, section_id))
            if section_id else None
        )
        seen[cite_id] = cast("Citation", {
            "cite_id": cite_id,
            "archive": archive,
            "entry_path": entry_path,
            "title": title,
            "section_id": section_id,
            "section_title": section_title,
        })
    return list(seen.values())
```

- [ ] **Step 4: Run tests**

```bash
uv run pytest tests/test_synthesize.py -v
```

Expected: all PASS.

- [ ] **Step 5: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/synthesize.py tests/test_synthesize.py
git commit -m "feat(v2): synthesize stages 6-7 — render answer + enforce budget + build citations

_render_answer joins passages with [cite: ...] markers. _enforce_budget
truncates passages so total text_markdown chars stay under
output_char_budget; truncates the last fitting passage and drops the rest.
_build_citations dedupes by cite_id and expands to the wire-format
Citation TypedDict (archive, entry_path, title, section_id, section_title)."
```

---

## Task 21: Wire the full `synthesize_query` pipeline + zero-hits / fallback paths

**Files:**
- Modify: `openzim_mcp/synthesize.py` (replace skeleton `synthesize_query` with the full pipeline)
- Modify: `tests/test_synthesize.py` (add end-to-end tests for fallback_used, zero-hits, multi-archive)

**Why:** Glue task. Replaces the skeleton from Task 17 with the real pipeline using the helpers from Tasks 16-20. Covers: `fallback_used` set correctly per single/multi-archive; zero-hits returns empty response with `_meta.reason="0_hits"`; multi-archive uses RRF.

- [ ] **Step 1: Write failing end-to-end tests**

```python
def test_synthesize_query_zero_hits_returns_empty_with_reason(
    cp, monkeypatch
) -> None:
    from openzim_mcp.synthesize import synthesize_query

    archive = MagicMock()
    archive.basename = "test"
    cache = MagicMock()
    config = SynthesizeConfig()

    search_handler = MagicMock()
    search_handler.search_top_k.return_value = []  # zero hits

    response = synthesize_query(
        "no results query",
        archives=[(archive, Path("test.zim"))],
        search_handler=search_handler,
        cache=cache, content_processor=cp, config=config,
    )
    assert response["passages"] == []
    assert response["citations"] == []
    assert response["answer_markdown"] == ""
    assert response["_meta"].get("reason") == "0_hits"


def test_synthesize_query_single_archive_uses_xapian_score_fallback(cp) -> None:
    from openzim_mcp.synthesize import synthesize_query

    archive = MagicMock()
    cache = MagicMock()
    cache.get.return_value = None  # force bundle build
    config = SynthesizeConfig()

    search_handler = MagicMock()
    search_handler.search_top_k.return_value = [
        {"path": "A/Berlin", "snippet": "<p>Berlin info</p>", "score": 0.9},
    ]

    # Bundle build path returns a minimal bundle so attribution finds the section.
    monkey_bundle = {
        "entry_path": "A/Berlin", "title": "Berlin", "content_type": "text/html",
        "word_count": 1, "char_count": 100,
        "rendered_markdown": "Berlin info\n",
        "sections": [{"id": "berlin", "title": "Berlin", "level": 1,
                      "char_start": 0, "char_end": 12, "parent_id": None}],
        "links": {"internal": [], "external": [], "media": []},
        "infobox": None,
    }
    # synthesize_query's bundle access goes through get_or_build_bundle —
    # mock it via the cache returning the bundle on second access.
    cache.get.side_effect = lambda key: monkey_bundle if "A/Berlin" in key else None

    response = synthesize_query(
        "berlin",
        archives=[(archive, Path("test.zim"))],
        search_handler=search_handler,
        cache=cache, content_processor=cp, config=config,
    )
    assert response["fallback_used"] == "xapian_score"
    assert response["archives_searched"] == ["test"]


def test_synthesize_query_multi_archive_uses_rrf_fusion(cp) -> None:
    from openzim_mcp.synthesize import synthesize_query

    a1 = MagicMock(); a2 = MagicMock()
    search_handler = MagicMock()
    search_handler.search_top_k.side_effect = [
        [{"path": "A/Berlin", "snippet": "", "score": 0.9}],
        [{"path": "A/Berlin", "snippet": "", "score": 0.5}],  # both archives have it
    ]
    cache = MagicMock()
    cache.get.return_value = None

    response = synthesize_query(
        "berlin",
        archives=[(a1, Path("wiki1.zim")), (a2, Path("wiki2.zim"))],
        search_handler=search_handler,
        cache=cache, content_processor=cp, config=SynthesizeConfig(),
    )
    assert response["fallback_used"] == "rrf_fusion"
    assert sorted(response["archives_searched"]) == ["wiki1", "wiki2"]
```

- [ ] **Step 2: Replace `synthesize_query` body in `openzim_mcp/synthesize.py`**

Replace the Task 17 skeleton with the full pipeline:

```python
def synthesize_query(
    query: str,
    *,
    archives: list[tuple["Archive", "Path"]],
    search_handler,
    cache: "OpenZimMcpCache",
    content_processor: "ContentProcessor",
    config: "SynthesizeConfig",
) -> SynthesizeResponse:
    """Run the synthesize pipeline end-to-end."""
    from openzim_mcp.bundle import get_or_build_bundle

    # Stage 1: per-archive search
    per_archive_hits: list[list[dict]] = []
    archives_searched: list[str] = []
    archive_by_name: dict[str, tuple["Archive", "Path"]] = {}
    for archive, validated_path in archives:
        archive_name = validated_path.stem
        archives_searched.append(archive_name)
        archive_by_name[archive_name] = (archive, validated_path)
        per_archive_hits.append(
            _per_archive_search(
                archive, search_handler=search_handler,
                query=query, k=config.per_archive_k,
            )
        )

    # Flatten + maintain hit→archive mapping for citations
    hit_to_archive: dict[tuple[str, str], dict] = {}
    for archive_name, hits in zip(archives_searched, per_archive_hits):
        for hit in hits:
            hit_to_archive[(archive_name, hit["path"])] = hit

    # Stage 2-3: fuse + (identity) rerank
    is_multi = len(archives) > 1
    if is_multi:
        # Build per-archive rankings: list[list[(path, score)]]
        per_archive_rankings = [
            [(hit["path"], hit["score"]) for hit in hits]
            for hits in per_archive_hits
        ]
        fused = _rrf_fuse(per_archive_rankings, k=60)[: config.top_n]
        fallback_used = "rrf_fusion"
        # For each fused path, pick the highest-ranked source ranking.
        # (Could pick more cleverly but the cite_id only needs ONE archive.)
        top_hits: list[tuple[str, dict]] = []   # (archive_name, hit dict)
        for path, fused_score in fused:
            for archive_name in archives_searched:
                if (archive_name, path) in hit_to_archive:
                    h = dict(hit_to_archive[(archive_name, path)])
                    h["score"] = fused_score   # override with fused score
                    top_hits.append((archive_name, h))
                    break
    else:
        archive_name = archives_searched[0] if archives_searched else "unknown"
        top_hits = [(archive_name, hit) for hit in per_archive_hits[0][: config.top_n]] if per_archive_hits else []
        fallback_used = "xapian_score"

    # Zero hits short-circuit
    if not top_hits:
        return cast("SynthesizeResponse", {
            "query": query,
            "answer_markdown": "",
            "passages": [],
            "citations": [],
            "archives_searched": archives_searched,
            "fallback_used": fallback_used,
            "total_chars": 0,
            "total_words": 0,
            "_meta": {"reason": "0_hits"},
        })

    # Stage 4: passage extraction (per-archive grouped, then concatenated)
    all_passages: list[SynthesizePassage] = []
    all_paths: list[str] = []
    for archive_name, hit in top_hits:
        passages = _extract_passages([hit], archive_name=archive_name,
                                       content_processor=content_processor)
        all_passages.extend(passages)
        all_paths.append(hit["path"])
    # Re-rank passages 1..N
    for i, p in enumerate(all_passages, start=1):
        p["rank"] = i

    # Stage 5: section attribution
    archive_for_path: dict[str, tuple["Archive", "Path"]] = {}
    for archive_name, hit in top_hits:
        archive_for_path[hit["path"]] = archive_by_name[archive_name]

    def bundle_lookup(entry_path: str):
        archive, validated_path = archive_for_path.get(entry_path, (None, None))
        if archive is None:
            return None
        return get_or_build_bundle(
            archive, entry_path,
            cache=cache, validated_path=validated_path,
            content_processor=content_processor,
        )

    attributed = _attribute_sections(
        all_passages, bundle_lookup=bundle_lookup, hit_paths=all_paths,
    )

    # Stage 7: budget enforcement
    capped = _enforce_budget(attributed, char_budget=config.output_char_budget)

    # Stage 6: render + build citations
    answer_md = _render_answer(capped)

    # Build per-entry title / per-(entry, section) title lookups for citations
    archive_titles: dict[str, str] = {}
    section_titles: dict[tuple[str, str], str] = {}
    for archive_name, hit in top_hits:
        try:
            b = bundle_lookup(hit["path"])
        except Exception:
            continue
        if b is None:
            continue
        archive_titles[hit["path"]] = b["title"]
        for s in b["sections"]:
            section_titles[(hit["path"], s["id"])] = s["title"]

    citations = _build_citations(
        capped, archive_titles=archive_titles, section_titles=section_titles,
    )

    return cast("SynthesizeResponse", {
        "query": query,
        "answer_markdown": answer_md,
        "passages": capped,
        "citations": citations,
        "archives_searched": archives_searched,
        "fallback_used": fallback_used,
        "total_chars": len(answer_md),
        "total_words": len(answer_md.split()),
        "_meta": {},
    })
```

- [ ] **Step 3: Run tests**

```bash
uv run pytest tests/test_synthesize.py -v
```

Expected: all PASS.

- [ ] **Step 4: Run mypy + lint + commit**

```bash
uv run mypy openzim_mcp
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/synthesize.py tests/test_synthesize.py
git commit -m "feat(v2): wire full synthesize pipeline + zero-hits + fallback paths

synthesize_query now runs all 7 stages: per-archive search, RRF fusion
(multi-archive only; single-archive skips), passage extraction,
section attribution, budget enforcement, answer rendering, citation
building. fallback_used is 'xapian_score' for single-archive,
'rrf_fusion' for multi-archive (Phase D will start emitting 'reranker').
Zero hits returns empty response with _meta.reason='0_hits'."
```

---

## Task 22: Wire `synthesize` into `zim_query` (server.py + simple_tools.py) + integration test

**Files:**
- Modify: `openzim_mcp/server.py:204` (add `synthesize: bool = False` param to `zim_query`; expand return-type annotation)
- Modify: `openzim_mcp/simple_tools.py:127` (`handle_zim_query` — branch on synthesize=True)
- Modify: `tests/test_structured_tool_output.py` (add end-to-end synthesize integration test)

**Why:** Final synthesize task. Adds the public surface. `zim_query`'s return type expands to `Union[str, SynthesizeResponse, ToolErrorPayload]`. Non-synthesize calls keep returning the markdown string (Phase F's concern to unify). Synthesize=True dispatches directly to `_handle_synthesize_query` in `simple_tools.py`, bypassing intent classification.

- [ ] **Step 1: Modify `openzim_mcp/server.py:204` zim_query**

Update the signature and return type:

```python
@self.mcp.tool()
async def zim_query(
    query: str,
    zim_file_path: Optional[str] = None,
    limit: Optional[int] = None,
    offset: int = 0,
    max_content_length: Optional[int] = None,
    compact: bool = True,
    compact_budget: Optional[Any] = None,
    synthesize: bool = False,
) -> Union[str, SynthesizeResponse, ToolErrorPayload]:
    """... (existing docstring) ...

    NEW (Phase C):
        synthesize: When True, runs the synthesize pipeline — top-N
            libzim snippets concatenated with [cite:
            archive/entry_path#section_id] markers + structured citation
            list. Returns SynthesizeResponse instead of markdown. No LLM
            generation; pure retrieval + concatenation. Falls back to
            Xapian score (single-archive) or RRF fusion (multi-archive)
            until Phase D's cross-encoder reranker ships.
    """
    try:
        options: Dict[str, Any] = {
            "limit": limit if limit is not None else 3,
            "max_content_length": (
                max_content_length if max_content_length is not None else 4000
            ),
            "compact": compact,
            "synthesize": synthesize,
        }
        if offset != 0:
            options["offset"] = offset
        if compact_budget is not None:
            options["compact_budget"] = compact_budget

        if self.simple_tools_handler:
            handler = self.simple_tools_handler
            return await asyncio.to_thread(
                handler.handle_zim_query, query, zim_file_path, options
            )
        else:
            return "Error: Simple tools handler not initialized"
    except Exception as e:
        logger.error(f"Error in zim_query: {e}")
        return self._create_enhanced_error_message(
            operation="zim_query",
            error=e,
            context=f"Query: {query}, File: {zim_file_path}",
        )
```

(Add `from openzim_mcp.tool_schemas import SynthesizeResponse, ToolErrorPayload` at the top of `server.py` if not already imported.)

- [ ] **Step 2: Modify `openzim_mcp/simple_tools.py:127` handle_zim_query**

Add a branch at the top of `handle_zim_query` that checks for `options["synthesize"]` and dispatches to a new `_handle_synthesize_query`:

```python
def handle_zim_query(
    self,
    query: str,
    zim_file_path: Optional[str] = None,
    options: Optional[Dict[str, Any]] = None,
) -> Union[str, SynthesizeResponse, ToolErrorPayload]:
    options = options or {}
    if options.get("synthesize"):
        return self._handle_synthesize_query(query, zim_file_path, options)
    # existing intent-based dispatch follows
    ...
```

Then add `_handle_synthesize_query` to the same class:

```python
def _handle_synthesize_query(
    self,
    query: str,
    zim_file_path: Optional[str],
    options: Dict[str, Any],
) -> Union[SynthesizeResponse, ToolErrorPayload]:
    """Phase C: dispatch to synthesize pipeline."""
    from openzim_mcp.synthesize import synthesize_query

    # Resolve archive list: single-archive when zim_file_path is set,
    # multi-archive otherwise (zim_query semantics).
    archives_to_open: list[Path] = []
    if zim_file_path:
        validated = self.path_validator.validate_path(zim_file_path)
        validated = self.path_validator.validate_zim_file(validated)
        archives_to_open = [validated]
    else:
        # use existing multi-archive resolution; whatever search_all uses
        archives_to_open = self.config.list_zim_files()  # adapt to actual API

    if not archives_to_open:
        return tool_error("no_archives_available",
                          message="No ZIM archives configured.")

    archives: list[tuple["Archive", Path]] = []
    open_handles = []  # for cleanup
    for vp in archives_to_open:
        try:
            archive = _zim_ops_mod.zim_archive(vp).__enter__()
            open_handles.append((archive, vp))
            archives.append((archive, vp))
        except Exception as e:
            logger.warning("Could not open archive %s for synthesize: %s", vp, e)
            continue

    if not archives:
        return tool_error("no_archives_available",
                          message="All configured archives failed to open.")

    try:
        response = synthesize_query(
            query,
            archives=archives,
            search_handler=self.search_handler,
            cache=self.cache,
            content_processor=self.content_processor,
            config=self.config.synthesize,
        )
        return response
    finally:
        # Cleanly close all opened archives
        for archive, _vp in open_handles:
            try:
                _zim_ops_mod.zim_archive(_vp).__exit__(None, None, None)
            except Exception:
                pass
```

(`self.search_handler` is a placeholder — wire it from whatever the SimpleToolsHandler holds today. If it doesn't have a search-tools handler, instantiate one inline or pass it via constructor.)

- [ ] **Step 3: Write integration test**

Append to `tests/test_structured_tool_output.py`:

```python
def test_zim_query_synthesize_returns_structured_response(structured_client) -> None:
    response = structured_client.call_tool(
        "zim_query",
        {"query": "berlin geography",
         "zim_file_path": "test.zim",
         "synthesize": True},
    )
    structured = response.structuredContent
    payload = structured.get("result", structured)
    assert "answer_markdown" in payload
    assert "passages" in payload
    assert "citations" in payload
    assert "archives_searched" in payload
    assert payload["fallback_used"] in ("xapian_score", "rrf_fusion")


def test_zim_query_no_synthesize_returns_string(structured_client) -> None:
    """Without synthesize=True, zim_query returns markdown string (unchanged)."""
    response = structured_client.call_tool(
        "zim_query",
        {"query": "list available ZIM files"},
    )
    # No structuredContent — text payload
    assert response.content[0].text  # markdown string
```

- [ ] **Step 4: Run tests**

```bash
uv run pytest tests/test_structured_tool_output.py tests/test_synthesize.py -v
```

Expected: all PASS.

- [ ] **Step 5: Run mypy + lint + commit**

```bash
uv run mypy openzim_mcp
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add openzim_mcp/server.py openzim_mcp/simple_tools.py tests/test_structured_tool_output.py
git commit -m "feat(v2): wire synthesize=True into zim_query (#10)

zim_query gains a synthesize: bool = False param; return type expands
to Union[str, SynthesizeResponse, ToolErrorPayload]. Non-synthesize
calls keep returning markdown string (Phase F unification). Synthesize
calls bypass intent classification and dispatch directly to
_handle_synthesize_query, which assembles the archive list (single or
multi per zim_query semantics), opens them, and runs synthesize_query."
```

---

## Task 23: Phase C golden snapshot tests

**Files:**
- Create: `tests/test_golden_v2_phase_c.py`

**Why:** Locks in the wire format for `get_section`, `synthesize`, and the four collapsed tools' post-bundle output. Phase B established the golden-snapshot pattern; Phase C adds 5-7 new snapshots.

- [ ] **Step 1: Inspect Phase B's golden test format**

```bash
sed -n '1,80p' tests/test_golden_v2_phase_b.py
```

Expected: tests use a `_canonicalize_payload` helper that strips volatile fields (`_meta.tokens_est` etc.) and compares to a JSON snapshot stored on disk under `tests/goldens/v2/phase_b/`.

- [ ] **Step 2: Create `tests/test_golden_v2_phase_c.py`**

```python
"""Phase C golden snapshots — wire format pinning for #11/#7/#10."""

from __future__ import annotations

import json
from pathlib import Path
from typing import Any

import pytest

from tests.test_golden_v2_phase_b import _canonicalize_payload  # reuse


GOLDENS_DIR = Path(__file__).parent / "goldens" / "v2" / "phase_c"


def _load_golden(name: str) -> Any:
    return json.loads((GOLDENS_DIR / f"{name}.json").read_text())


def _save_golden(name: str, payload: Any) -> None:
    GOLDENS_DIR.mkdir(parents=True, exist_ok=True)
    (GOLDENS_DIR / f"{name}.json").write_text(
        json.dumps(payload, indent=2, sort_keys=True)
    )


@pytest.fixture
def golden_handler(tmp_path):
    """Construct a handler over a known test ZIM (use the existing fixture)."""
    from tests.helpers import make_full_handler  # or the equivalent
    return make_full_handler(tmp_path)


@pytest.mark.parametrize("entry_path,section_id", [
    ("A/Berlin", "geography"),
    ("A/Berlin", "climate"),
    ("A/Munich", "history"),
])
def test_golden_get_section(golden_handler, entry_path, section_id) -> None:
    response = golden_handler.get_section_data(
        "test.zim", entry_path, section_id=section_id,
    )
    name = f"get_section_{entry_path.replace('/', '_')}_{section_id}"
    canon = _canonicalize_payload(response)
    if not (GOLDENS_DIR / f"{name}.json").exists():
        _save_golden(name, canon)
        pytest.fail(f"Created golden {name}.json — re-run to verify.")
    assert canon == _load_golden(name)


@pytest.mark.parametrize("query", [
    "berlin geography",
    "german cities",
    "european capitals",
])
def test_golden_synthesize(golden_handler, query) -> None:
    response = golden_handler.handle_zim_query(
        query, "test.zim", {"synthesize": True, "compact": False},
    )
    name = f"synthesize_{query.replace(' ', '_')}"
    canon = _canonicalize_payload(response)
    if not (GOLDENS_DIR / f"{name}.json").exists():
        _save_golden(name, canon)
        pytest.fail(f"Created golden {name}.json — re-run to verify.")
    assert canon == _load_golden(name)
```

- [ ] **Step 3: First run creates the goldens (intentionally fails)**

```bash
uv run pytest tests/test_golden_v2_phase_c.py -v
```

Expected: FAILs with `Created golden <name>.json — re-run to verify.` for each parameter case.

- [ ] **Step 4: Inspect the created goldens, verify they look reasonable**

```bash
ls tests/goldens/v2/phase_c/
cat tests/goldens/v2/phase_c/get_section_A_Berlin_geography.json | head -30
```

Sanity-check the JSON: `section_id` matches the input, `_meta` is present, the body is non-trivial markdown. If any look wrong, debug the corresponding handler before re-running.

- [ ] **Step 5: Re-run, expect PASS**

```bash
uv run pytest tests/test_golden_v2_phase_c.py -v
```

Expected: all PASS now that goldens exist on disk.

- [ ] **Step 6: Commit**

```bash
git add tests/test_golden_v2_phase_c.py tests/goldens/v2/phase_c/
git commit -m "test(v2): Phase C golden snapshots (get_section + synthesize)

Three get_section snapshots across two entries; three synthesize
snapshots across deterministic queries. Reuses Phase B's
_canonicalize_payload to strip volatile fields. Spec § Testing."
```

---

## Task 24: Update `tests/test_response_contract.py` to exempt `get_section` and `synthesize`

**Files:**
- Modify: `tests/test_response_contract.py`

**Why:** Phase B's contract regression test asserts that every list-returning tool emits `{results, next_cursor, total, done, page_info}`. `get_section` returns a single section (not a list) and `synthesize` returns a fixed top-N (not paginated). Both must be exempted, but both must still carry `_meta`.

- [ ] **Step 1: Inspect current contract test**

```bash
grep -nE 'def test_|EXEMPT|_meta' tests/test_response_contract.py | head -20
```

Expected: a parameterized test or a list of (tool_name, expected_contract) tuples; an existing exemption pattern.

- [ ] **Step 2: Add `get_section` and `synthesize` to the exemption list AND assert `_meta`**

Find the existing exemption set and add `"get_section"`. For `synthesize`, the response shape is structured but doesn't follow the pagination contract — exempt it from the contract check while asserting `_meta` independently.

```python
# In whatever data structure lists tool→contract assertions:
NON_PAGINATED_TOOLS = {
    # existing entries unchanged
    "get_section",       # Phase C: single-section, not list-paginated
    "synthesize",        # Phase C: fixed top-N, not list-paginated
}

def test_get_section_response_carries_meta_envelope(handler) -> None:
    response = handler.get_section_data(
        "test.zim", "A/Berlin", section_id="geography",
    )
    assert "_meta" in response
    assert isinstance(response["_meta"], dict)


def test_synthesize_response_carries_meta_envelope(handler) -> None:
    response = handler.handle_zim_query(
        "berlin", "test.zim", {"synthesize": True},
    )
    assert "_meta" in response
```

- [ ] **Step 3: Run tests**

```bash
uv run pytest tests/test_response_contract.py -v
```

Expected: all PASS.

- [ ] **Step 4: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add tests/test_response_contract.py
git commit -m "test(v2): exempt get_section + synthesize from list-pagination contract

Phase C adds two tools that don't fit the list-paginated contract:
get_section (single section) and synthesize (fixed top-N). Both still
carry _meta — explicit assertions added."
```

---

## Task 25: Update `tests/test_structured_tool_output.py` for TOC rename

**Files:**
- Modify: `tests/test_structured_tool_output.py`

**Why:** Task 10 renamed TOC heading `id` → `section_id`. The structured-tool-output test for `get_table_of_contents` may still assert on the old `id` field. Update.

- [ ] **Step 1: Find every assertion against TOC heading shape**

```bash
grep -nE 'heading\["id"\]|"id":|toc.*\["id"\]' tests/test_structured_tool_output.py
```

For each match, decide: is this asserting on the TOC heading id (must rename to `section_id`) or on a different `id` field (entry id, etc.)?

- [ ] **Step 2: Rename each TOC heading id reference**

Replace each `heading["id"]` (or analog) with `heading["section_id"]`. If the test asserts the TOC dict shape exactly, update the expected shape to use `section_id`.

- [ ] **Step 3: Run tests**

```bash
uv run pytest tests/test_structured_tool_output.py -v
```

Expected: all PASS.

- [ ] **Step 4: Lint + commit**

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
git add tests/test_structured_tool_output.py
git commit -m "test(v2): update TOC structured-output assertions for id→section_id rename"
```

---

## Task 26: Final mypy + full pytest sweep

**Files:** none (verification only)

**Why:** Catch any cross-cutting type or test breakage from the cumulative Phase C changes before docs/release.

- [ ] **Step 1: Run mypy on the full source tree**

```bash
uv run mypy openzim_mcp
```

Expected: PASS. If any errors surface, fix them in the offending file. Common issues from Phase C:
- `cast("X", ...)` missing on dict literals returning TypedDicts
- `_meta` envelope shape mismatches
- TOC tightening (Task 4) surfacing unsuited callsites that build the old loose dict — convert to `TocHeading`-shaped dicts.

- [ ] **Step 2: Run full pytest**

```bash
uv run pytest -x -v
```

Expected: every test PASSES. `-x` stops at the first failure; investigate, fix, re-run.

- [ ] **Step 3: Run lint sweep**

```bash
make lint
```

Expected: PASS (flake8 + isort + black). If any failures, run:

```bash
uv run black openzim_mcp tests
uv run isort openzim_mcp tests
make lint
```

- [ ] **Step 4: No commit (verification task only)**

If fixes were needed, commit them under appropriate `fix(v2): ...` messages and re-verify.

---

## Task 27: README + CHANGELOG entries for v2.0.0a3

**Files:**
- Modify: `README.md` (Phase C section; mark Phase C shipped)
- Modify: `CHANGELOG.md` (add `## [2.0.0a3]` block)

**Why:** Document the v2.0.0a3 release. Mirrors what Phase A and Phase B did.

- [ ] **Step 1: Add CHANGELOG `## [2.0.0a3]` block**

Open `CHANGELOG.md`. Insert a new section above `## [2.0.0a2]`:

```markdown
## [2.0.0a3] — 2026-05-09 (alpha pre-release)

v2 Phase C: new retrieval primitives. Adds the EntryBundle, the
`get_section` tool, and the synthesize mode on `zim_query`. **One
wire-format break** (TOC heading field rename).

### New tool — `get_section`

```
get_section(zim_file_path, entry_path, section_id, *, max_chars=None)
  → Union[GetSectionResponse, ToolErrorPayload]
```

Returns a single section's body (~500-1500 tokens — small-model sweet
spot per parent-document-retrieval research) plus full metadata.
section_id values come from `get_table_of_contents` (TocHeading.section_id).
On miss, returns `tool_error("section_not_found", available_section_ids=[...])`
so the model can self-correct.

### `zim_query` gains `synthesize: bool = False` parameter

When True, runs the synthesize pipeline: per-archive Xapian search →
RRF fusion (multi-archive) → libzim snippet extraction → bundle-based
section attribution → citation rendering. Returns a structured
`SynthesizeResponse`:

```python
{
    "query": str,
    "answer_markdown": str,        # passages + inline [cite: ...] markers
    "passages": list[SynthesizePassage],
    "citations": list[Citation],
    "archives_searched": list[str],
    "fallback_used": Literal["xapian_score", "rrf_fusion", "reranker"],
    "total_chars": int,
    "total_words": int,
    "_meta": MetaEnvelope,
}
```

Pure retrieval + concatenation. No LLM generation. Phase D's cross-encoder
reranker will inject before passage extraction; Phase C ships with the
identity rerank stage and `fallback_used` records the actual fusion mode.

### Breaking — `get_table_of_contents`

| Field | Before | After |
|---|---|---|
| TOC heading identifier | `heading["id"]` | `heading["section_id"]` |
| TOC list element type | `dict[str, Any]` | `TocHeading` TypedDict |

The value is unchanged (still `resolve_heading_id()`'s output with slug
fallback). The new name is what `get_section(section_id=...)` consumes.
The old `id_source` field is dropped (debugging-only, not a contract surface).

### #11 EntryBundle (internal — no wire impact)

Four content-shape tools (`get_entry_summary`, `get_table_of_contents`,
`get_article_structure`, `extract_article_links`) collapse to thin
slicers over a single shared `EntryBundle` value cached at
`bundle:v2c:{validated_path}:{entry_path}`. First touch parses HTML
once; subsequent calls (across all four tools) hit the bundle cache.

Removed legacy per-tool cache prefixes: `summary_data:`, `toc_data:`,
`structure_data:`, `links_full:v2b:`. Wire formats unchanged for the
three non-TOC tools.

### Other

- New module: `openzim_mcp/bundle.py` — `extract_entry_bundle`, `get_or_build_bundle`.
- New module: `openzim_mcp/synthesize.py` — `synthesize_query` plus pipeline-stage helpers.
- New TypedDicts in `openzim_mcp/tool_schemas.py`: `EntryBundle`, `SectionMeta`, `InfoboxField`, `InfoboxData`, `LinkBuckets`, `TocHeading`, `GetSectionResponse`, `SynthesizeResponse`, `Citation`, `SynthesizePassage`.
- New config block: `OpenZimMcpConfig.synthesize` with `top_n`, `per_archive_k`, `output_char_budget`.
- New tests: `tests/test_bundle.py`, `tests/test_get_section.py`, `tests/test_synthesize.py`, `tests/test_golden_v2_phase_c.py`.
- The Phase A `_meta` envelope is unchanged; new tools attach it on every response.
- Housekeeping: removed stale `[[tool.mypy.overrides]] module = ['libzim']` block from `pyproject.toml`. Added GitHub labels `v2`, `v2-phase-c`; applied `v2-phase-b` retroactively to PR #111.
```

- [ ] **Step 2: Update README's v2 section**

Find the README section that summarizes v2 progress (likely near the top or in a "What's new" block). Add a brief v2.0.0a3 entry pointing at Phase C's new tools and the TOC rename.

- [ ] **Step 3: Update `docs/v2/README.md` Phase C status row** (already updated to "In Design" in the spec commit; bump to "Shipped (v2.0.0a3)")

```bash
grep -n 'Phase C' docs/v2/README.md
```

Find the `| **C** | New retrieval primitives | #7, #10, #11 | In Design |` row. Update to:

```markdown
| **C** | New retrieval primitives | #7, #10, #11 | **Shipped (v2.0.0a3)** | [2026-05-09-v2-phase-c-retrieval-primitives-design.md](../superpowers/specs/2026-05-09-v2-phase-c-retrieval-primitives-design.md) |
```

- [ ] **Step 4: Commit**

```bash
git add README.md CHANGELOG.md docs/v2/README.md
git commit -m "docs(v2): CHANGELOG + README entries for v2.0.0a3 (Phase C)"
```

---

## Task 28: Final verification before opening PR

**Files:** none (verification only)

**Why:** Last gate before pushing to origin. Ensures everything passes from a clean state.

- [ ] **Step 1: Verify clean working tree**

```bash
git status --porcelain
```

Expected: empty.

- [ ] **Step 2: Verify all checks pass from a fresh state**

```bash
make lint
uv run mypy openzim_mcp
uv run pytest -x -v
```

Expected: all three PASS. If anything fails, fix and amend the latest commit only if appropriate (per CLAUDE.md: never amend without explicit ask — the implementer should add a fix commit instead).

- [ ] **Step 3: Inspect commit log**

```bash
git log --oneline main..v2-phase-c
```

Expected: a sequence of focused commits — spec, housekeeping, foundations, per-tool tasks, synthesize stages, golden tests, docs. Roughly 25-30 commits.

---

## Task 29: Push branch and open PR

**Files:** none (git/gh operations)

**Why:** Publish Phase C for review.

- [ ] **Step 1: Push the branch to origin**

```bash
git push -u origin v2-phase-c
```

Expected: success message with new branch tracking origin/v2-phase-c.

- [ ] **Step 2: Open the PR**

```bash
gh pr create \
  --base main \
  --head v2-phase-c \
  --title "feat(v2)!: Phase C — retrieval primitives (#7, #10, #11)" \
  --label v2 \
  --label v2-phase-c \
  --body-file - <<'EOF'
## Phase C of the v2 effort — new retrieval primitives

**Tag target:** v2.0.0a3 (alpha pre-release)
**Spec:** docs/superpowers/specs/2026-05-09-v2-phase-c-retrieval-primitives-design.md

### What's in scope (#11 + #7 + #10)

- **#11 EntryBundle.** First touch of an entry runs ONE HTML parse → produces a `EntryBundle` value cached in the existing LRU. The four content-shape tools (`get_entry_summary`, `get_table_of_contents`, `get_article_structure`, `extract_article_links`) collapse from independent HTML re-parsers to thin slicers over the bundle. Legacy per-tool cache prefixes removed.
- **#7 `get_section`.** New tool that fetches a single section by `section_id` (≈500-1500 tokens). Section IDs come from TOC. On miss, returns `tool_error("section_not_found", available_section_ids=[...])` so the model can self-correct.
- **#10 `synthesize` mode.** `zim_query` gains `synthesize: bool = False`. When True, returns a structured `SynthesizeResponse` with top-N libzim snippets concatenated with `[cite: archive/entry_path#section_id]` markers + structured citation list. Pure retrieval + concatenation. RRF fusion for multi-archive. Phase D's cross-encoder reranker will plug in between extraction and assembly.

### Wire-format breaks

- **`get_table_of_contents`.** TOC heading field renamed `id` → `section_id`; `toc` tightened from `list[dict[str, Any]]` to `list[TocHeading]`. v2 alpha — clean break per v2 policy.

### Test surface

- New: `tests/test_bundle.py`, `tests/test_get_section.py`, `tests/test_synthesize.py`, `tests/test_golden_v2_phase_c.py`.
- Extended: `tests/test_structured_tool_output.py`, `tests/test_response_contract.py`, `tests/test_tool_schemas.py`.
- mypy clean, lint clean, all goldens stable.

### Release plan

- Merge into `main`.
- Tag `v2.0.0a3` after merge (matches Phase A/B sequence).
- v2 final tag waits for Phases D, E, F.
EOF
```

Expected: prints the new PR URL on success.

- [ ] **Step 3: Verify PR was created with the right labels**

```bash
gh pr view --json number,labels,state
```

Expected: state is OPEN, labels include `v2` and `v2-phase-c`.

- [ ] **Step 4: Wait for CI and address feedback**

Once the PR is open, CI runs (analyze, perf, security, sonar, bandit, test matrix). Address any failures with new commits on `v2-phase-c`. After approval and squash-merge: tag the squash-merge commit as `v2.0.0a3` and `gh release create v2.0.0a3 --prerelease`.
