# Changelog

All notable changes to OpenContracts will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Changed

- **Corpus CAML article (README) reading layer — typography, contrast, and spacing (2026-06).**
  The corpus home README.caml view felt crowded and low-contrast on mobile
  because its prose blocks render through the shared chat `MarkdownMessageRenderer`,
  whose 8px paragraph spacing and missing heading rules are tuned for chat
  bubbles, not long-form articles. Added a long-form reading layer scoped to
  `article > section` in `frontend/src/components/corpuses/caml/CamlArticleFrame.tsx`:
  body line-height 1.72 at slate[800] for contrast, generous heading rhythm
  (`h2` top margin 2.25em with a teal hairline rule, `h3` 1.75em), a 720px
  measure cap, and a mobile gutter bump from 1rem to 1.25rem. The library-owned
  article header (serif title, eyebrow/dek, hero) is intentionally left
  untouched so its elegant muted-lead styling survives.

### Fixed

- **Ugly white line in PDF annotation highlights when the bounding box is hidden (2026-06).**
  `SelectionInfo` (the annotation label container) in
  `frontend/src/components/annotator/display/components/Containers.tsx`
  is absolutely positioned just above the highlight's top edge and spans the
  full annotation width. When the bounding box was not displayed
  (`showBoundingBox === false`) its background fell back to an *opaque* white
  (`rgba(255, 255, 255, 0.9)`), rendering a full-width white bar across the top
  of every highlight — even when labels were turned off entirely. The
  equivalent search-result highlight (`SearchResult.tsx`) already used a
  transparent fallback (`rgba(255, 255, 255, 0.0)`) and had no such artifact.
  Fix: when the bounding box is hidden, render the `SelectionInfo` background as
  `transparent` instead of opaque white, matching the search-result pattern. The
  colored label "tab" is preserved unchanged when the bounding box *is*
  displayed, and the label pill itself (`LabelTagContainer`) keeps its own
  colored background, so labels remain readable.

- **Corpus Chat — "Invalid Date" on every server-loaded message.** `CorpusChat`
  rendered `new Date(msg.createdAt).toLocaleString()`
  (`frontend/src/components/corpuses/CorpusChat.tsx:348`) but `GET_CHAT_MESSAGES`
  (`frontend/src/graphql/queries.ts`) never selected `createdAt`, so every
  message loaded from the server showed "Invalid Date" (the conversation-list
  timestamps were unaffected). Added the `createdAt` field to the query; the
  `ChatMessageType` TS type already declared it, so no type changes were needed.
- **Corpus Chat — stacked double header on desktop/tablet.** Opening a
  conversation from the "Conversation History" (VIEW) flow rendered both the
  outer "Conversation History" header and `CorpusChat`'s inner "Conversation"
  header (two stacked rows, two redundant Home buttons), shrinking the message
  scroll area to ~38% of viewport height on a 1366×680 laptop (~1 message
  visible). `CorpusQueryView` now mirrors CorpusChat's list/conversation mode in
  the VIEW flow via `onViewModeChange` and suppresses the outer header whenever a
  single conversation is open — recovering ~71px and leaving one header (the
  ASK/search-expanded flow already did this). Regression test added in
  `frontend/src/views/__tests__/CorpusQueryView.handlers.test.tsx`.
- **Discover landing search box stacked its icon on a separate line from the input on mobile (2026-06).**
  The `@os-legal/ui` `SearchBox` ships a `@media (max-width: 480px)` rule that
  sets `.oc-search-box__input { flex: 1 1 100% }`, forcing the input onto its
  own full-width row and stranding the magnifying-glass icon alone on the line
  above the placeholder. Added a higher-specificity override in the
  `SearchContainer` styled-component
  (`frontend/src/components/landing/NewHeroSection.tsx`) that restores
  `flex: 1 1 auto` on the input below `SMALL_MOBILE_BREAKPOINT` (480px), so the
  icon and input share one row again while the submit button still wraps to its
  own full-width row. Note: the upstream `SearchBox` mobile rule is unchanged
  through the latest `@os-legal/ui` 0.1.19, so this consumer-side override is
  version-independent and is retained after the bump noted under Changed.

- **Alarming "Reconnecting…" / network-error toast pile-up on mobile screen-unlock (2026-06).**
  On mobile, locking the screen suspends the page and drops in-flight queries
  and WebSockets; on unlock a burst of failures fired before connectivity
  re-established, stacking a column of red toasts ("Network error.",
  repeated "ERROR Unable to fetch corpuses.", "Failed to refresh data. Please
  reload the page."). Two root causes: (1) the per-query error toasts in the
  corpus views (`frontend/src/views/Corpuses.tsx`,
  `frontend/src/components/{documents,extracts,annotations,analyses,research}/*Cards.tsx`)
  were fired from the component render body **without a `toastId`**, so each
  re-render while `error` stayed truthy stacked a brand-new toast; (2)
  `frontend/src/graphql/errorLink.ts` and
  `frontend/src/components/network/NetworkStatusHandler.tsx` showed network/refetch
  error toasts unconditionally, even during the brief, expected reconnect window.
  Fix: a shared "reconnecting" grace window (`isReconnectingVar` in
  `frontend/src/graphql/cache.ts`, helpers in
  `frontend/src/utils/networkNotifications.ts`). `NetworkStatusHandler` arms the
  window around its reconnect refetch (and a 10s safety auto-disarm); while armed
  — or while `navigator.onLine` is false — `notifyTransientNetworkError` suppresses
  the alarming toasts in favour of the single calm "Reconnecting…" indicator. All
  transient network toasts now carry a stable `toastId` so they de-duplicate
  instead of stacking; genuine persistent failures still surface once (after the
  window closes). The red "Failed to refresh data. Please reload the page." toast
  on a reconnect refetch failure was removed. Tests:
  `frontend/src/utils/__tests__/networkNotifications.test.ts` and updated
  `frontend/src/components/network/__tests__/NetworkStatusHandler.test.tsx`.

- **Deep-research and conversation-memory Celery tasks were never registered on the worker (2026-06).**
  `run_deep_research` (`opencontractserver/tasks/research_tasks.py`) and the
  memory tasks `check_conversations_for_curation` / `curate_corpus_memory`
  (`opencontractserver/tasks/memory_tasks.py`) were absent from
  `opencontractserver/tasks/__init__.py`. The Celery worker registers central
  tasks only as a side effect of that package's `__init__` running at boot
  (the `@shared_task` decorator registers a task when its module is imported);
  these three modules were reachable only through lazy, function-local
  producer-side imports (`research/services/research_reports.py`) or by
  name via `CELERY_BEAT_SCHEDULE`. A real worker would therefore reject the
  queued message with _"Received unregistered task"_, leaving deep-research
  reports stuck and the conversation-curation beat job non-functional. The
  defect was masked in the unit suite because `config/settings/test.py` runs
  Celery in `CELERY_TASK_ALWAYS_EAGER` mode, where `.delay()` executes inline
  in the process that just lazily imported the module. Fix: import the three
  tasks in `opencontractserver/tasks/__init__.py` (and add them to `__all__`),
  matching the registration mechanism every other shipped task relies on.
  Added a subprocess-based regression test
  (`opencontractserver/tests/architecture/test_celery_task_registration.py`)
  that boots a worker in a clean interpreter and asserts these task names are
  present in `app.tasks`.

### Changed

- **Mobile DocumentKnowledgeBase layout cleanup (2026-06).** Removed the
  full-width control band that sat between the header and the page on mobile —
  it ate vertical space and read as wasteful. The Sections / Find / fit-width
  controls now float as a single compact frosted pill over the viewer's
  top-right corner (`frontend/src/components/knowledge_base/document/layouts/mobile/MobileDocToolbar.tsx`,
  rendered as an overlay inside `ViewerArea` in
  `frontend/.../layouts/MobileDocumentLayout.tsx`), so the document fills the
  whole surface. Also deepened the document backdrop on mobile from the
  near-white `#f7f9f9` to a new `VIEWER_CANVAS` token (`#e4e9f0`,
  `frontend/src/assets/configurations/designTokens.ts`), applied in the
  `@media (max-width: 768px)` branch of `PDFContainer`
  (`frontend/src/components/annotator/display/viewer/DocumentViewer.tsx`), so
  the white page sheet reads as a floating sheet with real contrast instead of
  washed-out near-white-on-near-white.

- **Corpus/document chat — message readability.** Capped the message column at
  60rem and centered it (`frontend/src/components/widgets/chat/ChatMessage.styles.ts`,
  `MessageContainer`) so messages no longer span the full width of a wide
  corpus-chat viewport (which produced very long, hard-to-scan line lengths), and
  gave the user ("You") bubble a solid, clearly-visible neutral background +
  border in place of the near-invisible `rgba(247, 248, 249, 0.5)`. Both changes
  apply to the corpus chat and the document knowledge-base chat (shared
  component); blue remains reserved for the assistant's identity.
- **Corpus Chat — scroll-to-bottom behavior.** Opening or switching conversations
  now jumps instantly to the latest message instead of playing a long smooth
  scroll animation across the whole history; smooth scrolling is reserved for
  single incremental appends / streamed tokens
  (`frontend/src/components/corpuses/CorpusChat.tsx`).
- **Bumped `@os-legal/ui` 0.1.16 → 0.1.19** (`frontend/package.json`,
  `frontend/yarn.lock`). The upstream `SearchBox` mobile layout is unchanged in
  0.1.19, so the consumer-side icon/input override above is still required.
- **Scoped admin (superuser) data access — admins are no longer omniscient over user data (2026-06).**
  Previously a `is_superuser` account received a blanket bypass throughout the
  permission layer: it could READ every row of every data model and pass every
  `user_can` check regardless of ownership, sharing, `is_public`, or corpus
  membership. An admin's access to **user data** (corpuses, documents,
  annotations, relationships, notes, conversations, analyses, extracts,
  datacells, folders, feedback, user profiles, badge awards, agents, research
  reports, ingestion sources, …) is now computed **identically to a normal
  user**. The `if user.is_superuser: return all()/True` short-circuits were
  removed from every visibility/authorization path:
  - **Managers / querysets** (`opencontractserver/shared/Managers.py`,
    `shared/QuerySets.py`): `BaseVisibilityManager`, `PermissionQuerySet`,
    `DocumentQuerySet`, `AnnotationQuerySet`, `NoteQuerySet`,
    `UserFeedbackQuerySet`, `PermissionedTreeQuerySet`, `RelationshipManager`
    `visible_to_user`; `UserProfileManager.visible_to_user`
    (`users/models.py`). Superusers now also receive the permission prefetch +
    `distinct()` previously skipped for them.
  - **Single-object checks**: `_default_user_can`
    (`utils/permissioning.py`), `AnnotationManager`/`NoteManager`/
    `RelationshipManager.user_can`. `get_users_permissions_for_obj` no longer
    grants superusers the full permission set; `resolve_my_permissions`
    (`config/graphql/.../mixins.py`) no longer injects a synthetic `superuser`
    permission.
  - **Service layer**: `AnnotationService` (`_compute_effective_permissions`,
    privacy filtering, `get_corpus_annotations`), `RelationshipService`,
    `Conversation`/`ChatMessage` visibility (`conversations/models.py`),
    `ConversationService`, `UserService.get_visible_users`, `MetadataService`,
    `ExtractService`, `AnalysisService`, `DocumentRelationship` service,
    `AgentConfiguration`/`AgentActionResult`/`ResearchReport` managers,
    `BadgeService.get_visible_user_badges`.
  - **GraphQL resolvers** (`config/graphql/`): extract document count/list,
    `ResearchReport` `myPermissions`, `@mention` corpus/document autocomplete
    (`search_queries.py`), `Document.can_retry` (`document_types.py`), label
    deletion (`label_mutations.py`).
  - **Badge awarding** (`config/graphql/badge_mutations.py`): now authorizes the
    awarder first (corpus CRUD for corpus badges, superuser for global badges)
    and resolves the recipient with a direct, unfiltered lookup, so it no longer
    depends on the removed `UserProfileManager` superuser bypass while preserving
    the IDOR contract.
  - **Retained for superusers** (deliberate): (1) the **structural-write
    break-glass** — superusers may still write structural annotations/
    relationships (`AnnotationManager`/`RelationshipManager.user_can`); (2)
    **moderation** of conversations/threads (`Conversation`/`Corpus.can_moderate`
    - moderation-action resolvers); (3) **admin-only configuration/restriction
      gates** in mutations/services (`PipelineSettings`, `Badge`/`CorpusCategory`
      management, "create global agents", "make analyses public", worker-upload
      provisioning). The break-glass for inspecting/repairing arbitrary user data
      is the **Django admin site** (`is_staff`), unaffected by these changes.
      Permanent document deletion now requires normal DELETE permission (no
      superuser override). Follow-up: an explicit, audited support/impersonation
      mechanism may be added later.
  - **Tests**: the authorization-invariant suite and ~30 permission/optimizer/
    service test modules were updated so a no-grant superuser is asserted to be
    a "stranger" for data (with positive grant-path cases and the retained
    structural-write break-glass). Docs: `docs/permissioning/consolidated_permissioning_guide.md`.

### Fixed

- **Deep research reports rendered fabricated `example.com` hyperlinks.** The
  deep-research agent is given only corpus-internal retrieval tools (no web
  access — see `opencontractserver/research/constants.py`
  `DEEP_RESEARCH_READ_ONLY_TOOLS`), yet it was asked to write a polished
  markdown report. With no real URLs to hand, the model fabricated markdown
  hyperlinks — overwhelmingly the canonical `https://example.com` placeholder —
  and nothing stripped them, so the frontend `SafeMarkdown` renderer
  (`frontend/src/components/knowledge_base/markdown/SafeMarkdown.tsx`, which
  allows `https:`) turned them into live, misleading anchors. The agent's only
  sanctioned attribution channel is the `<cite ids="…">` tag, which the system
  renders to footnotes. Fixed in two layers: (1) `build_deep_research_system_prompt`
  (`opencontractserver/research/constants.py`) now explicitly forbids hyperlinks
  / URLs and names `example.com` as a placeholder to avoid; (2) a new
  `_strip_fabricated_links` helper in
  `opencontractserver/research/services/research_reports.py` downgrades any
  externally-resolvable markdown link (`scheme://`, `//host`, `mailto:`/`tel:`,
  or a bare domain) to its plain label inside `ResearchReportService.finalize`,
  applied to both the executive summary and the rendered body so no fabricated
  link survives into stored content (covers the normal and salvage paths).
  In-app relative links (`/d/…`), fragment anchors (`#…`), `<cite>` tags, and
  `[^n]` footnote markers are left untouched. Tests in
  `opencontractserver/tests/research/test_research_report_service.py`.
- **MCP `search_corpus` returned duplicate hits and dead-end passages; bad
  tool arguments leaked a raw `TypeError`.** Surfaced while evaluating the
  public MCP server (`cite.opensource.legal/mcp`). Three fixes:
  - **Duplicate hits.** The annotation→embedding join can return one row per
    stored vector, so a single annotation surfaced multiple times with an
    identical score (observed: the same passage filling 3 of 4 result slots).
    `search_corpus` (`opencontractserver/mcp/tools.py`) now over-fetches
    candidates (`MCP_SEARCH_CANDIDATE_MULTIPLIER`/`_MAX` in
    `opencontractserver/constants/mcp.py`) and de-duplicates the merged feed
    (`_dedupe_search_hits`) after sorting by score — keeping the highest-scoring
    instance — before capping at `limit`.
  - **Dead-end passages.** Passage search hits omitted any identifier, so a hit
    could not be bridged back to the `annotation://` resource or
    `list_annotations`. `format_search_passage`
    (`opencontractserver/mcp/formatters.py`) now includes `annotation_id`
    (matching the `annotation_id` key used by relationship nodes); it is also
    the stable key the dedup logic uses.
  - **Raw `TypeError` on bad arguments.** An unknown tool argument (e.g. `page`
    on `get_document_text`) raised a bare `TypeError` deep in `sync_to_async`,
    escaping the structured-error branch as an opaque transport error that
    leaked the Python function signature. A new `_safe_tool_arguments`
    (`opencontractserver/mcp/server.py`), used by both the global and scoped
    dispatchers, drops the server-injected `user` and rejects unknown arguments
    as a `ValidationError` — surfaced as a structured `{"error": ...}` payload
    that names the offending argument and lists the valid ones. The scoped tool
    wrapper now carries `functools.wraps` so validation sees the wrapped tool's
    real signature rather than the wrapper's `(**kwargs)`.
  - Corpus-level retrieval sparseness on machine-ingested imports (the SpaceX
    S-1) is tracked separately in #1883 (V2 import skips structural ingestion +
    PAWLs re-encoding) and is not addressed here.
- **Deep-research code-review follow-ups (#1864).** Addressed the actionable
  items from the PR #1836 review:

  - `frontend/src/graphql/mutations.ts` — `CancelResearchReportOutput.obj.status`
    was typed as the bare `string`; narrowed to `JobStatus | string` to match
    `ResearchReportType.status` so callers comparing against `JobStatus` values
    get type checking on this payload.
  - `frontend/tests/ResearchReportDetailTestWrapper.tsx` — the single detail
    mock now carries `maxUsageCount: Number.POSITIVE_INFINITY` (mirroring
    `CorpusResearchReportCardsTestWrapper`). The detail view uses
    `notifyOnNetworkStatusChange` and can refetch/poll, so a fixed-bucket mock
    risked a "No more mocked responses" error if a future test exercised a
    non-terminal state.
  - `opencontractserver/research/models.py` — documented that
    `ResearchReport.duration_seconds` is always a computed `@property` (never a
    stored field), since the status-tool duration tests set `started_at` /
    `completed_at` directly and depend on that invariant. Added a guard test
    (`test_research_report_model.py::test_duration_seconds_computed_from_timestamps`).
  - Reviewer items that needed no change, after verification: async catch blocks
    in `ResearchReportDetail.tsx` / `StartResearchModal.tsx` already log via
    `console.error`; the `slug` column is already indexed (`unique=True` +
    `db_index=True`); and the snake_case reactive-var locals in
    `CorpusResearchReportCards.tsx` (`opened_corpus`, `research_search_term`,
    `auth_token`) intentionally match the established codebase convention used by
    the sibling `CorpusExtractCards`/`CorpusAnalysesCards` and avoid shadowing the
    imported camelCase reactive vars — renaming them would have been a regression.

- **WebSocket 1011 reconnect churn on a 5-second beat: channels-redis 4.3.0 vs.
  redis-py 8.0 default `socket_timeout=5` (#1886).** Every consumer's idle
  channel-layer receive loop crashed ~every 5s, so Daphne closed the socket with
  code **1011** and the browser reconnected, went idle, and tripped again —
  continuous churn on `/ws/notification-updates/`, `/ws/agent-chat/`, and every
  other consumer. Root cause: `redis` (redis-py) is unpinned in
  `requirements/base.txt` and floated to **8.0.0**, which changed the default
  `socket_timeout` from `None` to **5s**. channels-redis' receive loop issues a
  5s _server-side_ blocking pop (`bzpopmin`/`brpop`, `brpop_timeout=5`); with a
  5s _client_ read timeout the client's deadline fires before the server's nil
  reply returns, so redis-py raises `redis.exceptions.TimeoutError` out of the
  consumer (`retry_on_timeout=False`, and channels-redis doesn't catch it). The
  client loses the race on every idle cycle, which is why the failures were
  idle-only and landed on a clean 5s beat (a local send→receive round-trip
  returns before 5s, so it never exercised the path). **Fix:** the
  `CHANNEL_LAYERS` host is now a dict with an explicit `socket_timeout: None` in
  both `config/settings/base.py` (`{"host": …, "port": …, "socket_timeout":
None}`) and `config/settings/test_integration.py` (`{"address": REDIS_URL,
"socket_timeout": None}`), restoring the historical no-client-read-deadline
  behavior these long-lived idle WebSockets require. redis-py is **not** pinned
  back below 8.0. Regression test:
  `opencontractserver/tests/test_redis_integration.py::TestChannelsRedisLayer::test_idle_receive_survives_blocking_pop_window`
  delivers a message only after the first 5s blocking-pop cycle, so a regressed
  config (client read timeout ≤ `brpop_timeout`) raises `redis.TimeoutError` and
  fails the test.

- **`AnnotationFilterMode` export filtering silently ignored the requested mode
  (#1868).** `AnnotationFilterMode` (`opencontractserver/types/enums.py`) was a
  plain `Enum`, not a `str`-mixin enum like every sibling in the file, so a
  member never compared equal to its string value. `build_label_lookups`
  (`opencontractserver/utils/etl.py`) compounded this by mixing comparison
  styles: bare string literals at the mode-selection branches
  (`== "ANALYSES_ONLY"`, `== "CORPUS_LABELSET_PLUS_ANALYSES"`) but enum-member
  membership at the corpus-labelset augmentation branch
  (`in (AnnotationFilterMode.CORPUS_LABELSET_ONLY, …)`). As a result **no single
  argument type was handled correctly**:

  - A **string** argument (what the GraphQL export mutation and Celery tasks
    deliver — `config/graphql/document_mutations.py:644,672`) satisfied the
    selection branches but silently **skipped the corpus-labelset augmentation**,
    so labels defined in a corpus's label set but not referenced by any
    annotation were dropped from the export's label lookups, breaking
    roundtrip-safety (fork ≡ export+import).
  - An **enum-member** argument failed the string selection branches and fell
    through to `CORPUS_LABELSET_ONLY` for `ANALYSES_ONLY` /
    `CORPUS_LABELSET_PLUS_ANALYSES`.

  Separately, `build_document_export` compared only against enum members and
  hit its `else: raise ValueError` for a **string** argument; that exception is
  caught by the function's own `except`, returning `("", "", None, {}, {})`, so
  the **V2 export path** (`package_corpus_export_v2` →`build_corpus_v2_zip`,
  which forwards the GraphQL string unchanged) silently **dropped every
  document** from the archive — for all modes, including the default.

  Fix: `AnnotationFilterMode` is now `class AnnotationFilterMode(str, enum.Enum)`,
  and `build_label_lookups`'s string-literal comparisons are normalized to enum
  members so both ETL functions compare consistently. Members and their string
  values are now interchangeable, so both the in-process (enum) and
  GraphQL/Celery (string) call paths hit the same branches. Both ETL function
  signatures now accept `AnnotationFilterMode | str` to match runtime reality,
  retiring the `# type: ignore[arg-type]` / `cast(AnnotationFilterMode, …)`
  workarounds that prior callers had used. Regression coverage added in
  `opencontractserver/tests/test_corpus_export_with_analysis_filters.py`
  (`BuildLabelLookupsStringEnumEquivalenceTestCase`) and
  `opencontractserver/tests/test_types.py`
  (`TestAnnotationFilterMode.test_is_string_enum`). Observable effect: the
  string-driven export paths now include the corpus's full label set in their
  label lookups (matching the enum-driven fork path) instead of only the
  referenced labels. `test_filter_modes_change_annotation_count`'s
  `build_label_lookups` assertions were updated to verify the per-mode label
  _membership_ (which the fix corrects) rather than brittle raw counts; the
  `build_document_export` counts are unchanged. Both ETL functions also now
  normalize their `annotation_filter_mode` argument to an
  `AnnotationFilterMode` member at the boundary, so an invalid mode string
  raises a clear `ValueError` instead of silently falling through to
  `CORPUS_LABELSET_ONLY`; covered by
  `BuildLabelLookupsStringEnumEquivalenceTestCase.test_invalid_mode_raises_valueerror`.

- **Location Tagger: non-string geocoding hints crashed the tool (#1871,
  follow-up to #1822).** `add_annotations_from_exact_strings`
  (`opencontractserver/llms/tools/core_tools/annotations.py`) forwarded the
  LLM-supplied `hints` mapping straight to the geocoder. The values come from
  raw model output, so a model emitting `{"country": 123}` (a non-string)
  reached `resolve_place` → `_normalise`, where `str.strip().lower()` raised
  `AttributeError` deep in the geocoding service instead of failing gracefully
  at the tool boundary. Hint keys/values are now coerced to `str` and `None`
  values dropped (so `{"country": null}` reads as "no hint", not the literal
  `"None"`), making the declared `dict[str, str]` contract true at runtime.
  Part of the #1822 code-review follow-up, which also: moved a per-annotation
  assertion inside its loop in `test_location_tagger_agent.py` (the post-loop
  check only validated the last, order-non-deterministic row); decoupled
  `test_city_without_hints_*` from the geocoder's population tie-break (already
  pinned by `test_geocoding_service.ResolvePlaceCityTests`); narrowed
  `LABEL_TEXT_TO_GEOCODE_LABEL_TYPE` to a `Literal` value type; and documented
  the no-superuser migration-skip recovery path in
  `docs/agents/location_tagger.md`.
- **Corpus chat duplicate-response loop on reconnect**
  (`frontend/src/components/corpuses/CorpusChat.tsx`). Submitting a query from
  the corpus home search bar navigated into the chat view and auto-sent the
  `initialQuery` once the WebSocket opened. The auto-send effect had no
  "already sent" guard, so every `wsReady` false→true transition — i.e. every
  WebSocket reconnect (the brief "reconnecting" flash) — re-injected the user's
  message and triggered another assistant response, repeating indefinitely. The
  initial query is now tracked by a ref (`sentInitialQueryRef`) and sent exactly
  once; the ref is only stamped after a successful `wsSend`, so a socket that
  drops before the first send still delivers the query once the connection is
  stable. Regression test added in `frontend/tests/CorpusChat.ct.tsx`
  ("initialQuery is NOT re-sent when the WebSocket reconnects").

### Added

- **Admin management of registered LLMs in System Settings.** Superusers can
  now view, enable/disable, and pick a system-wide default for the registered
  LLM providers from the admin **System Settings → Pipeline Configuration**
  page. LLM providers were already discovered by the pipeline registry and
  exposed over GraphQL (`pipelineComponents.llmProviders`), but the frontend
  never surfaced them.
  - **Frontend** — the Component Library now lists LLM providers (Anthropic,
    OpenAI, Google, Ollama) as a fourth stage with suggested-model chips, an
    "API key" indicator, and the same enable/disable toggle used by
    parsers/embedders/thumbnailers
    (`frontend/src/components/admin/system_settings/ComponentLibrary.tsx`,
    `config.ts`, `types.ts`). A new **Default LLM** row + picker modal lets
    admins choose the install-wide default model spec
    (`frontend/src/components/admin/SystemSettings.tsx`,
    `system_settings/FiletypeDefaults.tsx`). `GET_PIPELINE_COMPONENTS` now
    fetches `llmProviders`; settings queries/mutations carry `defaultLlm`
    (`system_settings/graphql.ts`). Fixed a latent bug where toggling any
    component while in the "all enabled" state would have silently dropped
    every LLM provider from the rebuilt enabled list — LLM providers are now
    included in that computation.
  - **Backend** — new `PipelineSettings.default_llm` field + migration
    (`opencontractserver/documents/models.py`,
    `documents/migrations/0040_add_default_llm_to_pipeline_settings.py`),
    surfaced and mutable via GraphQL (`config/graphql/pipeline_types.py`,
    `pipeline_queries.py`, `pipeline_settings_mutations.py`; validated with
    `validate_model_spec` and stored in canonical `provider:model` form).
    `resolve_model_spec` gained a `settings_default` tier so the configured
    default is consulted _before_ Django's `DEFAULT_LLM` / `OPENAI_MODEL`
    (`opencontractserver/llms/llm_registry.py`); it is threaded in by the agent
    factory (async, via `database_sync_to_async`) and `Corpus.save()` through a
    new `get_default_llm_spec()` helper
    (`opencontractserver/pipeline/utils.py`,
    `llms/agents/agent_factory.py`, `corpuses/models.py`). The resolver itself
    stays ORM-free and async-safe. Precedence is unchanged otherwise: per-call →
    per-agent → per-corpus → this default → Django settings.
  - **Fixed** — `PipelineSettings.get_instance()` and
    `ResetPipelineSettingsMutation` could assign `None` to the NOT NULL
    `default_llm` column when Django's `DEFAULT_LLM` setting was explicitly set
    to `None` (e.g. in tests exercising the legacy fallback), raising an
    `IntegrityError`; both now coerce the setting to `""`
    (`opencontractserver/documents/models.py` line ~1371,
    `config/graphql/pipeline_settings_mutations.py`). The reset mutation now
    also resets/returns `default_reranker` (previously omitted from the reset
    path and its response). `normalise_model_spec` was moved inside the
    `validate_model_spec` try/except in `UpdatePipelineSettingsMutation` so an
    unexpected raise returns a clean `ok=False` response rather than a 500.
    Added stable `data-testid`s (`edit-default-llm`, `edit-default-embedder`,
    `library-filter-*`) to replace positional/text selectors in the component
    tests, and dropped the untyped `settingsKey` carried into
    `LIBRARY_STAGE_CONFIG` by the previous `STAGE_CONFIG` spread.
- **Corpus Home "Map" view — per-corpus geographic annotations (#1821).** A
  fifth Corpus Home view mode (`landing` / `details` / `discussions` /
  `article` / **`map`**) that plots a corpus's geographic annotations, reusing
  the #1820 `AnnotationMap` and the #1819 `geographicAnnotationsForCorpus`
  query.

  - **`CorpusMapView`** (`frontend/src/components/corpuses/CorpusMapView.tsx`) —
    wraps `AnnotationMap` with the corpus-scoped pins query, a back-to-overview
    header badged with the place count, an empty state pointing users at the
    Location Tagger agent, and pin-panel document links that resolve through the
    permission-filtered `document(id:)` redirect query. Performance: corpus pin
    sets are bounded/aggregated, so it fetches the whole corpus **once**
    (`bbox: null`, all label types) with no per-pan refetch, selecting only the
    lightweight pin fields.
  - **Map toggle + count badge** (`CorpusHome/CorpusMapToggle.tsx`, rendered in
    `CorpusLandingView`'s top bar) — "Map · N places", muted when the corpus has
    no geo annotations but still clickable so users reach the empty-state
    guidance. Its count query reuses `corpusGeoInitialVariables`, so it shares an
    Apollo cache entry with `CorpusMapView` — the badge warms the map and opening
    it costs no extra round-trip.
  - **Routing** — `view=map` and a `pin=<place>` deep-link param are parsed by
    `CentralRouteManager` into the `corpusDetailView` / new `corpusMapPin`
    reactive vars (no direct URL touches in components). `?view=map&pin=Paris`
    opens the map zoomed to Paris with its side panel open; selecting a pin
    reflects the place into the URL (`updateCorpusMapPinParam`, replace-mode) so
    the map is shareable and survives refresh.
  - **`AnnotationMap` additive props** (`fitToPins`, `focusPinName`) — opt-in
    imperative framing via an internal `MapController` (auto-fit to the coarsest
    band on first load; fly-to + select for the deep-link place). Both default
    off, so Discover and existing callers are unaffected. New `bandZoomRange` /
    `coarsestBand` helpers keep the framed zoom inside the band that has pins.
    The auto-fit now passes `MAP_FIT_PADDING_PX` to `getBoundsZoom` so framed
    pins are not flush against the viewport edges (the constant was previously
    imported but never applied), and a deep-link focus consumes the one-shot
    auto-fit (`didFitRef`) so clearing `?pin=` cannot retroactively re-fit an
    already-framed map.
  - **Tests** — `frontend/tests/CorpusMapView.ct.tsx` (pins render + corpus-doc
    panel, empty state, error placeholder, `?pin=Paris` deep link),
    `frontend/tests/CorpusMapToggle.ct.tsx` (with-places / no-places /
    loading badge states), and `zoomBands.test.ts` coverage for the new
    helpers. Screenshots:
    `corpus--map-view--with-pins`, `corpus--map-view--empty`,
    `corpus--map-view--load-error`, `corpus--map-toggle--with-places`,
    `corpus--map-toggle--no-places`.

- **Reusable `AnnotationMap` (Leaflet) component + Discover "Map" tab (#1820).**
  A caller-agnostic React map that visualises geographic document annotations,
  plus its first integration on the Discover page. Builds on the #1819 backend
  (`globalGeographicAnnotations` / `geographicAnnotationsForCorpus`).
  - **Reusable component** (`frontend/src/components/maps/AnnotationMap.tsx`) —
    Leaflet `MapContainer` + OpenStreetMap `TileLayer` (no API key, required
    attribution) with `leaflet.markercluster` clustering. It takes `pins` and
    fires `onBoundsChange(bbox, zoom)` / `onPinClick(pin)`; it holds no
    Discover- or corpus-specific logic so #1821 (Corpus Home) can reuse it.
    Clicking a pin opens a side panel with the place name, document count, and
    sample-document links. The component is route-agnostic: it surfaces the
    click via an `onSelectDocument(documentId)` callback, and the Discover
    wiring resolves the document's Relay global id to its canonical `/d/...`
    URL (via the existing id-redirect query) before navigating. The client
    shows only the pins matching the current zoom band — country/state/city
    (`frontend/src/components/maps/zoomBands.ts`) — since the server returns all
    label types. Accessible: region role + aria-label, keyboard-focusable
    markers with descriptive labels, and `prefers-reduced-motion` disables
    zoom/pan and marker animations. Fixes the Leaflet default-marker-icon
    bundler problem by importing the icon assets and setting `L.Icon.Default`.
  - **Discover "Map" tab** (`frontend/src/views/DiscoverSearchResults.tsx`,
    `frontend/src/components/maps/DiscoverMapPanel.tsx`) — a new tab that feeds
    `AnnotationMap` from `globalGeographicAnnotations`
    (`frontend/src/graphql/queries/geographicAnnotations.ts`, selecting only the
    six pin fields). Bbox/zoom-driven refetches are debounced 300 ms and only
    fire on pan/zoom; the map viewport is persisted to the URL (`?lat&lng&z`)
    via `useSearchParams` for deep-linking and refresh persistence.
  - **Constants** (`frontend/src/assets/configurations/constants.ts`) — tile
    URL/attribution, default center/zoom, min/max zoom, cluster radius, map
    height, zoom-band thresholds, debounce ms, and the lowercase
    `country`/`state`/`city` label-type literals the GraphQL API expects.
  - **Dependencies** (`frontend/package.json`) — `leaflet`, `react-leaflet`
    (v4 for React 18), `leaflet.markercluster`, `@types/leaflet`,
    `@types/leaflet.markercluster`.
  - **Tests** (`frontend/tests/AnnotationMap.ct.tsx`,
    `frontend/tests/DiscoverMapPanel.ct.tsx`) — Playwright CT coverage that pins
    render at the right coordinates, a pin click opens the panel / invokes
    `onPinClick` / fires the document-open callback / closes the panel, the
    zoom→label-type band selection picks the right pins, and the Discover panel
    renders pins from the mocked query.
  - **Review follow-ups (#1820)**: added `keyArgs` for
    `globalGeographicAnnotations` / `geographicAnnotationsForCorpus` in
    `cache.ts` so panning isolates cache entries by `bbox`/`zoom`/`labelTypes`;
    exported `GetGeographicAnnotationsForCorpus{Input,Output}` for the #1821
    consumer; introduced the `GeoLabelType` union and typed
    `GeographicAnnotationPin.labelType` with it; moved `pluralizeDocuments` into
    `utils/formatters.ts` and the map URL-param keys into `constants.ts`;
    swapped the deprecated marker `keypress` listener for `keydown`; froze the
    initial map view with `useState` instead of `useRef().current`; and wrapped
    the map's document-resolve navigation in a `try/catch`. Added a
    `docScreenshot` for the pin side-panel state.
  - **Second review follow-ups (#1820)**: typed `GeographicAnnotationsInput.labelTypes`
    as `GeoLabelType[]` (was `string[]`) so invalid label-type strings can't reach
    the server; extracted the antimeridian-aware viewport-centre math into a pure,
    exported `bboxCenter` helper (`zoomBands.ts`) and reused it in
    `DiscoverMapPanel`; exported `readMapViewFromParams` for direct testing; added
    unit tests covering `labelTypeForZoom` boundary bands, `bboxCenter`
    antimeridian crossings, and `readMapViewFromParams` URL-restoration edge cases;
    and corrected a misleading CLAUDE.md-pitfall-#8 comment in
    `DiscoverMapPanelTestWrapper`.
- **Location Tagger default agent (#1822)** — a new global, public default agent
  that auto-creates geocoded `OC_COUNTRY` / `OC_STATE` / `OC_CITY` annotations so
  documents populate the Discover (#1820) and Corpus Home (#1821) maps at scale.
  Built on the existing corpus-actions framework (configure with an `ADD_DOCUMENT`
  trigger to auto-tag uploads, or `MANUAL_BATCH` to backfill a corpus).
  - **Default agent registration**
    (`opencontractserver/agents/migrations/0015_create_location_tagger_agent.py`,
    `DEFAULT_LOCATION_TAGGER_INSTRUCTIONS` in `config/settings/base.py`): a
    globe-badged `Location Tagger` agent wired to the
    `add_annotations_from_exact_strings` tool. Idempotent and slug-safe (mirrors
    `0002`/`0004`).
  - **Geocoding integrated into `add_annotations_from_exact_strings`**
    (`opencontractserver/llms/tools/core_tools/annotations.py`): each item gained
    an optional `hints` field (`{"country": "US", "state": "TX"}`). When
    `label_text` is one of the OC\_\* geographic labels the span is resolved via the
    offline geocoder (#1819) and `{canonical_name, lat, lng, admin_codes,
geocoded}` is written to `Annotation.data`. Non-geographic labels are
    untouched — fully backward-compatible.
  - **Shared geocoding-data builder** (`build_geocoded_annotation_data` +
    `LABEL_TEXT_TO_GEOCODE_LABEL_TYPE` in
    `opencontractserver/annotations/services/geographic_service.py`): single source
    of truth for the `Annotation.data` payload, now reused by both the GraphQL
    geographic mutations (`config/graphql/annotation_mutations.py`) and the agent
    tool so map aggregation sees one identical shape (DRY — removes the duplicated
    inline dict from the mutation).
  - **Docs & tests**: `docs/agents/location_tagger.md` (configuration + worked
    example + limitations), `docs/test_scripts/location_tagger_end_to_end.md`
    (manual upload → corpus action → map pins), and
    `opencontractserver/tests/test_location_tagger_agent.py` (agent registration,
    geocoding + hint disambiguation, ungeocodable-miss sentinel, non-geographic
    backward-compatibility).
  - **Review follow-ups (#1822)**: tightened the `parsed_items` hint tuple type
    to `dict[str, str] | None`; the `0015` migration now logs a warning when it
    falls back to the concise built-in prompt (settings prompt absent) so the
    degraded state is visible; added tests pinning the migration→settings
    `system_instructions` wiring and proving geocoding is resolved once per item
    and reused for every occurrence (not re-resolved per match).
  - **Blank-span guard (#1822 review)**: `add_annotations_from_exact_strings`
    now skips items whose `exact_string` is blank/whitespace-only. Besides
    wasting a geocoder call, `doc_text.find("")` returns the search start so the
    occurrence loop never advanced — a blank span would have spun forever.
    Mirrors the blank-`raw_text` guard in `_create_geographic_annotation`.
- **MCP knowledge-tool UX — make a corpus usable as a low-friction tool**
  (`opencontractserver/mcp/`, issues #1858–#1862). A hands-on evaluation of the
  live public MCP interface found search returned nothing, full text crashed on
  cloud storage (fixed separately in the `read_field_file_text` work), and
  annotations were not content-searchable. This initiative addresses the
  remaining gaps:

  - **`search_corpus` → unified passage + block feed** (#1858, #1862): now
    searches **annotation** embeddings (passages) instead of document-level
    vectors, and merges in **`OC_SUBTREE_GROUP` relationship "blocks"**
    (ancestor + full descendant subtree) from `CoreRelationshipVectorStore`.
    Each result is tagged `type: "passage" | "block"`. New `granularity`
    (`passage`/`block`/`both`) and `structural` (tri-state) params. Fixes the
    prior bug where an empty vector result returned immediately and the text
    fallback was dead code — the fallback now runs whenever the vector path is
    empty/absent/errors (`opencontractserver/mcp/tools.py`).
  - **`list_annotations` content search** (#1859): new `text_contains` and
    `structural` filters, results ordered by `(page, id)`, and a leaner payload
    (`format_annotation` drops `color`/`created`; keeps the `structural` flag).
  - **`list_relationships` tool** (#1862): new tool returning labeled
    `source → target` edges via `RelationshipService`, filterable by
    `structural`/`label_text`, document- or corpus-scoped. Adds
    `RelationshipService.get_corpus_relationships`
    (`opencontractserver/annotations/services/relationship_service.py`).
  - **`get_document_text` bounded slicing** (#1860): new `char_offset`/`max_chars`
    params with `next_offset`/`truncated`/`total_chars` in the response, so long
    documents no longer blow the context window in one call. **Behavior change:**
    a call with no `max_chars` now returns at most the first
    `MCP_DOCUMENT_TEXT_DEFAULT_CHARS` (50 000) characters instead of the full
    extracted text; the response carries `truncated`/`next_offset` so callers can
    page through the remainder. MCP clients that relied on a single call returning
    the whole document must now follow `next_offset` (or pass an explicit
    `max_chars`, hard-capped at `MCP_DOCUMENT_TEXT_MAX_CHARS` = 200 000).
  - **Polish** (#1861): `get_corpus_info` now surfaces only labels actually used
    on the corpus's annotations (not the full seeded label set); not-found
    errors (`Document`/`Corpus.DoesNotExist`) are humanized into actionable
    messages that name the remediation tool instead of leaking raw Django
    exception strings; tool descriptions sharpened. New size constants in
    `opencontractserver/constants/mcp.py`.
  - Tests: `opencontractserver/mcp/tests/test_mcp.py` (passage/block feed,
    granularity, structural filter, annotation content search, relationship
    enumeration, in-use labels, error formatting). Design + plan:
    `docs/development/2026-05-31-mcp-knowledge-tool-ux-design.md`.

- **MCP interactive sign-in for Claude web/desktop and ChatGPT.** Added an
  authenticated MCP entrypoint at `/mcp/me/` (`opencontractserver/mcp/server.py`)
  that returns `401 + WWW-Authenticate` to unauthenticated callers, triggering
  the OAuth 2.1 + PKCE flow in interactive clients; once signed in it serves the
  user's private + public resources. The public `/mcp/` endpoint is unchanged
  (anonymous = public only) and a valid bearer token is still honored on either.
  Both share the global stateless MCP server (auth is per-request via the
  `_mcp_user` ContextVar).
- **RFC 9728 path-based protected-resource metadata** at
  `/.well-known/oauth-protected-resource/mcp/me` (alongside the existing root
  document), plus a `cite-authenticated` server advertised in
  `/.well-known/mcp.json` when `USE_AUTH0=True`
  (`opencontractserver/discovery/views.py`, `opencontractserver/discovery/urls.py`).
- **CORS for the MCP endpoints.** `/mcp*` bypasses Django middleware, so CORS is
  now enforced inside the MCP ASGI app: `OPTIONS` preflight, an allow-list via
  the new `MCP_CORS_ALLOWED_ORIGINS` setting (defaults to Claude, ChatGPT, and
  the MCP Inspector; merges in `CORS_ALLOWED_ORIGINS`), and exposing
  `WWW-Authenticate` / `Mcp-Session-Id` (`config/settings/base.py`). The CORS
  `send`-wrapping runs before both the rate-limit check and JWT validation, so
  401 (missing **and** expired-token) and 429 responses all carry
  `Access-Control-Allow-Origin` — pinned by
  `test_invalid_token_401_carries_cors_origin_for_allowlisted_origin` and
  `test_rate_limited_429_carries_cors_origin_for_allowlisted_origin`
  (`opencontractserver/mcp/tests/test_mcp.py`). `Access-Control-Allow-Credentials`
  is intentionally omitted (MCP is Bearer-token, not cookie, auth);
  `_OAUTH_PROTECTED_RESOURCES` is a `frozenset` to make its membership-test
  intent explicit.
  - **Review follow-ups (#1842)**: the CORS `send`-wrapper now folds
    `Vary: Origin` into a pre-existing `Vary` header (e.g. `Accept-Encoding`)
    instead of dropping it on a membership check, so caches/CDNs still vary on
    `Origin` (regression test
    `test_cors_vary_folds_into_existing_vary_header`). The MCP Inspector
    loopback origins (`http://localhost:6274`, `http://127.0.0.1:6274`) are now
    only in the `MCP_CORS_ALLOWED_ORIGINS` default under `DEBUG`, so they never
    ship in a production default; operators can still set them explicitly. Added
    an inline note on why the `send`-wrap must precede the rate-limit/JWT
    branches.
- **Chunked (resumable) uploads for large files** — work around the 100 MB
  per-request body ceiling that upstream proxies (Cloudflare) impose on the
  document-import REST endpoints. The client slices a file into sub-100 MB
  parts, uploads each independently, and the server reassembles them before
  handing the whole file to the **existing** import services (so there is one
  import code path per kind, chunked or not).
  - **Backend models** (`opencontractserver/document_imports/models.py`):
    `ChunkedUploadSession` + `ChunkedUploadPart`. Parts persist through Django
    storage (not local disk) so any web/worker process can reassemble a
    session. Initial migration `0001_initial`.
  - **REST endpoints** (`opencontractserver/document_imports/views.py`,
    `urls.py`): `POST /api/imports/chunked/start/`,
    `PUT|POST /api/imports/chunked/<id>/parts/<index>/`,
    `POST /api/imports/chunked/<id>/complete/`, and
    `GET /api/imports/chunked/<id>/` (progress, for resuming). `complete`
    returns the **same** response shape as the matching single-request
    endpoint. Part PUTs use a looser throttle scope (`document_import_chunks`,
    5000/hour) than the whole-file `document_imports` scope (120/hour).
  - **Service layer** (`opencontractserver/document_imports/services.py`):
    `start_chunked_upload` / `store_chunk` / `complete_chunked_upload` plus a
    streaming reassembler (peak memory bounded to one 8 MB block regardless of
    file size) and `purge_stale_chunked_uploads`. All four import kinds are
    supported (single document + the three zip flows). Per-user session
    isolation (IDOR-safe 404s), arithmetic + integrity validation, and
    fast-fail permission gating at `start`.
  - **Periodic cleanup** (`opencontractserver/document_imports/tasks.py`,
    `config/settings/base.py` `CELERY_BEAT_SCHEDULE`): hourly
    `purge_stale_chunked_uploads` GCs abandoned sessions and their stored parts
    after `CHUNKED_UPLOAD_STALE_HOURS` (default 24h).
  - **New settings** (`config/settings/base.py`):
    `CHUNKED_UPLOAD_PART_MAX_BYTES` (default 90 MB, must stay below the proxy
    cap), `CHUNKED_UPLOAD_MAX_PARTS`, `CHUNKED_UPLOAD_STALE_HOURS`, and the
    `document_import_chunks` throttle rate.
  - **Frontend** (`frontend/src/utils/importHttp.ts`): the four public import
    helpers transparently route files above `UPLOAD.CHUNK_THRESHOLD_BYTES`
    (50 MB) through the chunked protocol — call sites and response handling are
    unchanged. The artificial 100 MB dropzone cap was raised
    (`constants.ts`: `MAX_FILE_SIZE_BYTES` → 2 GB for single documents;
    bulk-zip dropzone now uses the 500 MB `MAX_IMPORT_ZIP_BYTES` cap via new
    `FileDropZone` `maxSizeBytes`/`maxSizeDisplay` props).
  - **Resilient frontend transport** (`frontend/src/utils/importHttp.ts`,
    `constants.ts`): parts now upload with bounded concurrency
    (`UPLOAD.CHUNK_CONCURRENCY`, default 4) instead of strictly sequentially,
    each part retries with exponential backoff on transient/5xx/network
    failures (`CHUNK_MAX_ATTEMPTS`, `CHUNK_RETRY_BASE_DELAY_MS`) while 4xx
    client errors fail fast, and an optional `onProgress(fraction)` callback
    on every public import helper drives a progress bar for large uploads.
  - **GC race hardening** (`opencontractserver/document_imports/services.py`):
    `purge_stale_chunked_uploads` no longer purges `ASSEMBLING` sessions inside
    the normal stale window (which could delete parts out from under a live
    `complete` reassembly); they are reclaimed only after a longer grace window
    (`CHUNKED_UPLOAD_ASSEMBLING_GRACE_HOURS`, default 6h) so a crashed
    mid-assembly worker is still cleaned up. The Celery task now also accepts
    `completed_retention_days` so an operator can override retention at enqueue
    time. The single-document reassembly's whole-file-in-RAM tradeoff has since
    been removed — it now streams (see the issue #1843 entry under **Fixed**).
  - **Tests**: `opencontractserver/tests/test_document_imports_chunked.py`
    (round-trip byte-exact reassembly, validation, IDOR isolation, integrity
    checks, zip kinds, stale-session GC, ASSEMBLING grace window) and the
    `importHttp chunked transport` suite in
    `frontend/src/utils/__tests__/importHttp.test.ts` (concurrency cap,
    per-part retry, 4xx fail-fast, progress reporting).

### Fixed

- **MCP `search_corpus` dropped the document pointer on structural passage
  hits.** Structural annotations carry `document_id=NULL` and reach their
  document only through `structural_set` (mirrored by the document's
  `structural_annotation_set`), but `format_search_passage`
  (`opencontractserver/mcp/formatters.py`) read only `annotation.document` and
  therefore returned `document_slug=null` / `document_title=""` for every
  structural hit — handing an AI agent a search result it could not navigate
  back to a document (observed live on the DGCL corpus at
  `cite.opensource.legal/mcp`). `search_corpus`
  (`opencontractserver/mcp/tools.py`) now builds a corpus-scoped
  `structural_set_id -> (slug, title)` lookup for structural passages and
  threads it into the formatter, so structural hits resolve their document the
  same way block hits already do. The lookup is corpus-scoped (not a lazy
  per-annotation query) because a structural set can be shared across
  documents/corpuses, so the slug must be chosen within the caller's corpus.
  Regression guard:
  `MCPSearchCorpusStructuralSlugTest` in `opencontractserver/mcp/tests/test_mcp.py`.

- **Canonical-CAML corpus home regressions: missing tab toggle on mobile +
  dropped corpus cover image** (`frontend/src/components/corpuses/CorpusHome.tsx`,
  `frontend/src/components/corpuses/CorpusHome/CorpusArticleView.tsx`). After the
  canonical-CAML refactor made a `Readme.CAML` article the corpus home page,
  `CorpusArticleView` replaced `CorpusLandingView` and lost two affordances the
  landing view provided automatically:
  - The Explore/Manage **mode toggle** (the control that reveals the corpus
    sidebar tabs) was rendered only as a bottom `position: fixed`
    `FloatingModeToggle` inside the article's `overflowY: auto` scroll
    container. Fixed-position-in-scroll-container is unreliable on mobile
    (notably iOS), so the toggle disappeared there. It now renders inline in the
    article's **sticky toolbar** (`data-testid="…-mode-toggle"`), which stays
    visible on every viewport. The redundant floating toggle was removed from
    the article path (the landing path keeps its floating toggle), and the now
    dead `$stackAboveChat` prop was dropped.
  - The **corpus cover image** (`corpus.icon`) only appeared if the CAML body
    explicitly referenced `corpus://icon` / `corpus://current`. Backfilled
    articles never do, so corpuses silently lost their cover image on the home
    page. `CorpusArticleView` now auto-renders the corpus icon above the article
    body when the CAML does not already reference it — matching the auto-hero
    behavior of `CorpusLandingView`.
- **Canonical-CAML backfill migration `0054` crashed on cloud storage
  (`AttributeError: 'bytes' object has no attribute 'encode'`).** Production
  `manage.py migrate` aborted at
  `corpuses/migrations/0054_canonical_caml_backfill.py:184`
  (`ContentFile(body.encode("utf-8"))`). Root cause: S3Boto3Storage /
  GoogleCloudStorage (django-storages #382) silently return `bytes` from a
  text-mode (`"r"`) `FieldFile.read()` **without raising**, so the readers'
  `except`-guarded binary fallbacks never fired and `bytes` flowed into the
  `str`-only `body.encode(...)`. Invisible locally because `FileSystemStorage`
  honors text mode and returns `str`.

  - **Migration readers normalise to `str`** (`_read_md_description`,
    `_read_caml_doc_body`) via a new module-local `_coerce_to_text` helper,
    inlined to keep the historical migration self-contained. The migration is
    atomic (rolled back cleanly on the crash) and idempotent, so a re-run after
    this fix completes the backfill.
  - **`description_cache.read_caml_body` had the identical latent bug** and now
    delegates to the canonical `opencontractserver.utils.files.read_field_file_text`
    (errors=`ignore`), removing the duplicated hand-rolled fallback. This also
    fixes the live cache-refresh signal handler and the GraphQL
    `descriptionRevisions` facade, which read CAML bodies on cloud deployments.
  - Regression tests in
    `opencontractserver/tests/test_corpus_canonical_caml_migration.py` exercise
    both the bytes and str paths for the migration readers and `read_caml_body`.

- **Research report `finalize()` now writes content and provenance atomically**
  (`opencontractserver/research/services/research_reports.py`,
  `ResearchReportService.finalize`). The terminal content/status `save()` and the
  `source_annotations` / `source_documents` M2M `set()` calls were previously
  issued as separate, unguarded queries: a worker crash or Celery soft-time-limit
  between them could leave a `COMPLETED` report whose footnote citations had no
  backing provenance rows (content with empty provenance). Both writes are now
  wrapped in a single `transaction.atomic()` block so they commit or roll back
  together; the citation rendering above stays outside the block as pure
  computation.
- **Research report state no longer bleeds across route transitions**
  (`frontend/src/routing/CentralRouteManager.tsx`). The `openedResearchReport`
  reactive var was cleared in the browse/extract/thread/labelset/user branches but
  not when navigating from `/research/:slug` straight to a corpus or document route
  (which changes neither `corpusId` nor `documentId`), so stale report state could
  persist. Phase 1 now clears it up front whenever the resolved route type is not
  `research` — a single source of truth that also covers browser back/forward.
  Regression-guarded by a new test in
  `frontend/src/routing/__tests__/centralRouteDiscipline.test.ts`.
- **In-run history compaction follow-up (issue #1824).** Code-review fixes for
  the pydantic-ai `shrink_old_artifacts_processor` shipped in #1817.
  - **Structured tool returns now shrink to valid JSON**
    (`opencontractserver/llms/history_processors.py`). `ToolReturnPart.content`
    can be a dict/list; the shrink path stringified it with `str()` (Python
    repr — single-quoted keys, `True`/`None`), which the model can misparse. A
    new `_stringify_tool_content` helper serialises non-strings with
    `json.dumps(content, default=str)` (falling back to `str()` only if JSON
    serialisation fails) and is shared by both the token estimator and the
    shrink, so the pre-shrink estimate matches the post-shrink payload.
  - **Strengthened `test_thinking_only_modelresponse_is_not_emptied`**
    (`opencontractserver/tests/test_history_processors.py`). The old fixture had
    no shrinkable content in the older prefix, so the test passed via the no-op
    early-return rather than the empty-parts guard it names. It now places a
    large `ToolReturnPart` alongside the thinking-only response so the shrink
    actually runs, and asserts (via telemetry) that the guard — not an early
    return — is why the `ThinkingPart` survives. Added
    `test_non_string_tool_return_serialized_as_json` to pin the JSON fix.
- **MCP `WWW-Authenticate` base-URL hardening** (`opencontractserver/mcp/server.py`)
  — the 401 challenge now prefers the trusted `MCP_PUBLIC_BASE_URL` over the
  request `Host` header (MCP bypasses `ALLOWED_HOSTS`), falling back to the
  previous sanitized-Host derivation. A configured value that survives
  quote/CR/LF stripping but is still not a valid `scheme://host` (a typo such
  as a trailing `;junk`, or a header-injection attempt) is now rejected via a
  `re.fullmatch` guard and the challenge degrades to a realm-only `Bearer`
  value rather than emitting a mangled URL or trusting the request `Host`.
  Refreshed `docs/mcp/README.md`, which previously stated "no authentication."
- **Chunked single-document import no longer buffers the whole file in RAM at
  `complete` (issue #1843).** Reassembling a chunked single-document upload
  previously did `file_bytes = tmp.read()` and passed the whole assembled file
  to `import_document_for_user` → `Corpus.import_content` →
  `documents.versioning.import_document`, which only accepted `content: bytes`.
  Because `complete` runs synchronously in the web process and
  `MAX_DOCUMENT_IMPORT_SIZE_BYTES` defaults to 5 GB (frontend accepts 2 GB), a
  single `complete` call could spike a web worker by 2–5 GB and starve other
  workers. A streaming file-like is now threaded all the way through:
  - **`documents/versioning.py`**: `import_document` accepts `content=None`
    plus a type-routed `content_file` (and still honours explicit
    `pdf_file` / `txt_file`); the content hash is computed by streaming the
    file via the new `compute_sha256_for_file` helper instead of from a `bytes`
    blob. Existing `content=bytes` callers are unchanged.
  - **`corpuses/models.py`**: `Corpus.import_content` accepts a `content_file`
    file object as a streaming alternative to `content` bytes (exactly one is
    required) and forwards it to `import_document`.
  - **`document_imports/services.py`**: `import_document_for_user` accepts a
    `file_obj` (sniffing only an 8 KB header for MIME detection rather than
    reading the whole file), and `complete_chunked_upload`'s DOCUMENT branch
    now passes `File(tmp)` instead of `tmp.read()`. Peak memory for a chunked
    single-document import is now O(block) — matching the three ZIP kinds —
    instead of ~the whole file size.
  - **Review hardening**: `import_document` now raises `ValueError` when a
    caller passes both `content_file` and the matching explicit
    `pdf_file`/`txt_file` (previously `content_file` was silently dropped),
    mirroring `import_content`'s mutual-exclusion; the MIME-sniff narrowing in
    `import_document_for_user` no longer uses an `assert` (stripped under
    `python -O`) and the streamed-hash call site documents its intentional
    double read. New test:
    `StreamingImportTestCase::test_import_document_rejects_content_file_and_explicit_file`.
  - **Tests**: streaming-hash and file-routing unit tests in
    `test_document_versioning.py` (`StreamingImportTestCase`) and chunked
    HTTP tests in `test_document_imports_chunked.py` (streamed-not-buffered
    contract, streamed-hash correctness, plain-text round trip).
- **Flaky test isolation** (`opencontractserver/tests/research/test_research_report_model.py`, issue #1845): `test_visible_to_user_superuser_sees_all` asserted a global `visible_to_user(admin).count() == 2`. Because the superuser branch returns `.all()`, a sibling `TransactionTestCase` that commits `ResearchReport` rows broke the count under a sequential whole-`research/`-directory run (CI stayed green only because xdist isolates files to separate worker DBs). Scoped the assertion to the test's own rows. Test-only; no production impact.
- **Corpus document versioning & path/folder audit — correctness and performance fixes.** A sweep of the dual-tree versioning (`DocumentPath`) and `CorpusFolder` path system surfaced several correctness gaps and N+1s. Regression coverage: `opencontractserver/tests/test_versioning_paths_audit.py`.
  - **`Corpus.add_document` no longer silently supersedes a colliding document** (`opencontractserver/corpuses/models.py`). When the auto-/caller-supplied path (e.g. `/documents/<title>`) collided with an existing active document, the occupant was marked `is_current=False` and the new doc became a "version" of an unrelated content tree — the first document silently vanished from the corpus. `add_document` now disambiguates the path (`/documents/Report` → `/documents/Report_1`) via `CorpusPathService._disambiguate_path`, always creating an independent root path (`parent=None`, `version_number=1`). `add_document` is not a versioning entry point; `import_content` remains the path-versioning surface.
  - **Folder rename / move now reconciles `DocumentPath.path` strings** (`opencontractserver/corpuses/services/folders.py`, `services/paths.py`). `update_folder` (rename) and `move_folder` changed `CorpusFolder.get_path()` but left the folder-derived `path` strings of contained documents (and descendant-folder documents) stale — drifting from the location that document _moves_ derive. New `CorpusPathService.reconcile_paths_after_folder_change` rewrites every folder-derived active path under the affected subtree to the new prefix as immutable MOVED history nodes (batch deactivate + `bulk_create` + signal dispatch), inside the same transaction as the folder mutation. Non-folder-derived paths (e.g. an upload's `/documents/<title>`) are intentionally left untouched.
  - **Single source of truth for lifecycle-action inference** (`opencontractserver/documents/models.py` `DocumentPath.infer_action`). Replaces three divergent copies (`versioning.get_path_history`, `DocumentType.resolve_path_history`, `DocumentPathType.resolve_action`) that disagreed on folder-only MOVED detection and RESTORED handling. `get_path_history` keeps its legacy label vocabulary (`IMPORTED`→`CREATED`, plus the `UNKNOWN` no-op refinement) for backward compatibility while delegating the logic.
  - **Folder name collisions return a graceful error instead of HTTP 500** (`services/folders.py`). `create_folder` (concurrent-create TOCTOU) and `move_folder` (colliding destination sibling) now catch the `unique_folder_name_per_parent` `IntegrityError` and return `"A folder named '…' already exists in this location"`; `move_folder` also prechecks the destination.
  - **Restore disambiguates when the original path was reused while trashed** (`services/lifecycle.py`, `documents/versioning.py`). Restoring a soft-deleted document whose path had since been taken by a new upload raised an uncaught `unique_active_path_per_corpus` `IntegrityError`. Both restore paths now disambiguate the restored path (`/report.pdf` → `/report_1.pdf`) so both documents survive.
  - **`version_number` / `last_modified` resolvers share one cached query** (`config/graphql/document_types.py`). Both read the same current `DocumentPath`; a request-scoped cache collapses the per-document 2N to a single shared lookup. `has_version_history` now uses `parent_id` (zero extra queries instead of fetching the parent row). `resolve_path_history` resolves predecessors from an in-memory index (removes the per-node `.parent` N+1).
  - **`resolve_version_history` is scoped to `visible_to_user`** (`config/graphql/document_types.py`) — it no longer leaks version metadata (creator, hash, size) for documents hidden from the caller, matching `resolve_corpus_versions`.
  - **`get_document_folder` tolerates more than one active path** (`services/folder_documents.py`) — switched from `.get()` (could raise `MultipleObjectsReturned`, HTTP 500) to a deterministic `.order_by("-created").first()`.
  - **`get_filesystem_at_time` is scoped to the target corpus** (`documents/versioning.py`) — the outer query was an unscoped full-table scan relying solely on a correlated subquery; it now filters `corpus=corpus`.
  - **CI green-up follow-up** (`opencontractserver/tests/test_caml_review_tools.py`): the new leading-`/` precondition on `CorpusPathService.disambiguate_path` exposed a test-only misuse — the `_create_caml_doc` fixture built the corpus's `Readme.CAML` article through `corpus.add_document(path="Readme.CAML")`. `add_document` is the disambiguating _new-independent-document_ API and now (correctly) rejects the reserved, slashless CAML sentinel path. The fixture now mirrors the production canonical write path (`documents.versioning.import_document` with `path=CAML_ARTICLE_TITLE`, version-up semantics, no disambiguation), matching `backfill_caml_doc_for_corpus` and the CAML edit tool. Production was never affected — it never routes the CAML path through `add_document`/`disambiguate_path`.
- **Corpus category management — PR #1837 follow-up (issue #1847).** Code-review fixes for the runtime-configurable corpus-category admin feature.
  - **`CorpusCategoryService.update_category` no longer issues a no-op write** (`opencontractserver/corpuses/services/corpus_category_service.py`). An update with all-`None` kwargs previously still ran a `save()` that only bumped `modified`; it now short-circuits to a successful no-op. Regression: `test_update_with_no_fields_is_noop` (asserted via `assertNumQueries(0)`).
  - **Category `description` now has a 2,000-char soft cap** — new `MAX_CATEGORY_DESCRIPTION_LENGTH`, mirrored between `opencontractserver/constants/corpus_categories.py` and `frontend/src/assets/configurations/constants.ts`. Validated in `CorpusCategoryService._validate_fields` (consistent with the name/icon checks, since the model field is an unbounded `TextField`) and enforced as a `maxLength` on the description textarea. Regression: `test_create_rejects_overly_long_description`, `test_update_rejects_overly_long_description`.
  - **Delete now shows in-flight feedback** (`frontend/src/components/widgets/modals/ConfirmModal.tsx`, `frontend/src/components/admin/corpus_categories/CorpusCategoryManagement.tsx`). `ConfirmModal` gained an additive, backward-compatible `confirmLoading` prop that keeps the dialog open with a spinner and disabled buttons until the async action settles (the previous design auto-closed on confirm, so a spinner on the delete button could never have been seen); the delete mutation's loading state now drives it.
  - **Test / nit cleanups**: `test_create_uses_default_icon_and_color` asserts against the shared default constants instead of hardcoded `"folder"` / `"#3B82F6"` literals; the `corpus_mutations` import in `config/graphql/mutations.py` regained its section comment (the category and folder imports kept theirs).

### Security

- **SmartLabelListMutation IDOR fix** (`config/graphql/smart_label_mutations.py`)
  — the mutation fetched a corpus by raw PK (`Corpus.objects.get(pk=...)`) and
  returned its labelset's labels with **no corpus-READ gate**, so any
  authenticated user could enumerate a private corpus's label taxonomy. It now
  routes the fetch through `BaseService.get_or_none(Corpus, pk, user,
PermissionTypes.READ, request=info.context)` and returns the IDOR-safe
  "Corpus not found" response for both missing and unreadable corpuses. This
  bug pre-dated the Phase 6 service-layer refactor (it was not a regression).
  Regression test added: `test_smart_label_list_denies_unreadable_corpus`.

### Fixed

- **MCP document full-text retrieval crashed on cloud storage**
  (`opencontractserver/mcp/tools.py::get_document_text`,
  `opencontractserver/mcp/resources.py::get_document_resource`) — both read
  `document.txt_extract_file` via `.open("r").read()` and placed the raw result
  into the dict that the MCP dispatcher serializes with `json.dumps(...)`
  (`opencontractserver/mcp/server.py:447` and `:1066`). On AWS/GCP deployments
  django-storages backends (`S3Boto3Storage`, `GoogleCloudStorage`) return
  `bytes` from text-mode reads (django-storages #382), so `json.dumps` raised
  `TypeError: Object of type bytes is not JSON serializable` and downstream MCP
  clients could not retrieve document full text. LOCAL `FileSystemStorage`
  returns `str`, which is why dev and the test suite never reproduced it. Both
  call sites now decode through the new `read_field_file_text()` helper
  (`opencontractserver/utils/files.py`), which centralizes the
  `isinstance(..., bytes)` decode workaround previously duplicated in
  `tasks/embeddings_task.py` and `pipeline/parsers/oc_markdown_parser.py`. The
  same routing also fixes a latent instance of this bug in the agent memory
  reader's happy path
  (`opencontractserver/agents/memory.py::read_memory_content`, which decoded
  bytes only in its exception fallback, not its primary `.open("r").read()`
  branch). Regression test:
  `test_get_document_text_handles_bytes_from_cloud_storage`
  (`opencontractserver/mcp/tests/test_mcp.py`); helper unit tests:
  `opencontractserver/tests/test_files_utils.py`.
- **Routed the remaining seven production `txt_extract_file.open("r")` call
  sites through `read_field_file_text()`** so they no longer break on cloud
  storage the same way: `pipeline/base/thumbnailer.py`, `utils/export_v2.py`,
  `utils/extraction_grounding.py`, and the four LLM core tools
  (`llms/tools/core_tools/{pii,document_indexing,search,annotations}.py`). The
  LLM tools are the highest-impact because they feed `doc_text` directly into
  agent context, where a raw `bytes` value would crash the downstream call or
  produce garbage. The two MCP read sites and the four LLM core tools now pass
  `errors="replace"` so a few undecodable bytes substitute `U+FFFD` instead of
  raising `UnicodeDecodeError`: at the MCP sites it would otherwise be caught
  by the surrounding `except` and silently return an empty document; in the
  agent tools it keeps the tool fault-tolerant (per the CLAUDE.md agent
  tool-fault-tolerance rule) and stays internally consistent because match /
  annotation positions are computed against the same decoded string. The
  pipeline/export sites (`thumbnailer`, `export_v2`, `extraction_grounding`)
  retain strict decoding to fail fast on genuinely corrupt files. Regression
  test for the resource path:
  `test_get_document_resource_handles_bytes_from_cloud_storage`
  (`opencontractserver/mcp/tests/test_mcp.py`). `read_field_file_text()`'s
  parameter is typed as a structural `Protocol` rather than the concrete
  `FieldFile`, so duck-typed test doubles type-check without `# type: ignore`.

### Changed

- **DRY: single definition of the compaction kick-in point** (issue #1824)
  (`opencontractserver/llms/context_guardrails.py`). New
  `context_window_and_threshold(model_name, threshold_ratio)` replaces the
  `int(get_context_window_for_model(...) * ratio)` triplication across
  `should_compact`, `compact_message_history`, and the in-run
  `shrink_old_artifacts_processor`, so turn-level and in-run compaction derive
  the threshold identically from one place. Documented (finding #3) that
  `IN_RUN_TOOL_RETURN_TARGET_CHARS` bounds the preserved prefix, not the final
  string — the appended trim notice adds a small fixed overhead — and added a
  comment confirming the `_on_in_run_shrink` closure's captured message IDs are
  stable for the turn (finding #5).
- **`BaseService.filter_visible_qs` now fails closed** (`opencontractserver/shared/services/base.py`)
  — an input lacking a `visible_to_user` method previously passed through
  **unfiltered** (fail-open, a latent row-leak); it now raises `TypeError`.
  Real callers always pass a permissioned QuerySet/manager, so this only
  surfaces genuine wiring bugs. Test updated:
  `test_filter_visible_qs_rejects_exotic_input`.
- **Architecture audit scans `config/graphql/` recursively**
  (`opencontractserver/shared/architecture_audit.py`) — `iter_graphql_modules`
  switched from `glob("*.py")` to `rglob("*.py")` so resolver logic in
  subpackages (e.g. `permissioning/permission_annotator/`) is covered by the
  `opencontracts.E001` check and the pytest invariant. (Subpackages were clean;
  this closes the blind spot.)
- **`config/graphql/document_types.py::resolve_can_restore`** migrated off a raw
  `Corpus.objects.get(...)` fetch-then-check onto `BaseService.get_or_none`
  (behaviour-preserving; removes a service-layer bypass).

### Documentation

- Clarified across `CLAUDE.md` (rule 7), `docs/architecture/query_permission_patterns.md`,
  and `docs/development/architecture_invariants.md` that the `opencontracts.E001`
  enforcement scope is **`config/graphql/` only** — MCP tools, LLM tools, REST
  views, and user-context Celery tasks remain _policy_, not mechanically
  scanned, and still contain correct-but-inline Tier-0 calls.
- Documented that `request=` is a no-op on the queryset helpers
  (`filter_visible` / `filter_visible_qs`); the Tier-2 permission cache only
  applies to the single-object helpers.

### Added

- **Deep-research frontend + chat status tool (completes the backend-only v1 from #1814).** The deep-research agent shipped backend-only: the completion chat message linked to `/research/{slug}` but no route resolved it, the `RESEARCH_REPORT_*` notifications were never surfaced on the frontend, and there was no UI to read a finished report. This adds the full frontend surface plus a backend status tool so a user can trigger (conversationally), monitor, and read reports.
  - **Backend — slug resolver + chat status tool.** `config/graphql/research_queries.py` adds `researchReportBySlug(slug)` (creator-only via `BaseService.filter_visible`; IDOR-safe null for non-owners/unknown slugs) so the frontend can resolve the `/research/{slug}` route the completion chat message links to. `opencontractserver/llms/tools/research_tools.py` adds `check_deep_research_status` (read-only chat tool) backed by `ResearchReportService.list_recent_for_corpus`, registered on the corpus chat agent alongside `start_deep_research` in `opencontractserver/llms/agents/pydantic_ai_agents.py`; both are filtered out of the read-only research agent via `restrict_tool_names`, so a follow-up like "is my research done?" returns status/progress + the report link instead of a dead end.
  - **Frontend GraphQL + cache** (`frontend/src/graphql/{queries,mutations,cache}.ts`, `frontend/src/types/graphql-api.ts`): `GET_RESEARCH_REPORT`, `RESOLVE_RESEARCH_REPORT_BY_SLUG`, `GET_RESEARCH_REPORTS` (relay), `START_RESEARCH_REPORT`, `CANCEL_RESEARCH_REPORT`; `JobStatus` / `ResearchReportType` / `ResearchReportListItem` types; the `openedResearchReport` reactive var; and `researchReports: relayStylePagination(["corpusId", "status"])` + `researchReportBySlug` cache policies (keyArgs use the GraphQL field-argument names per CLAUDE.md #15).
  - **Routing** (`frontend/src/utils/navigationUtils.ts`, `frontend/src/routing/CentralRouteManager.tsx`, `frontend/src/App.tsx`, `frontend/src/components/routes/ResearchReportRoute.tsx`): a standalone `/research/:slug` route resolved by CentralRouteManager Phase 1 (slug-based, with a Phase-3 canonical-redirect guard like threads/labelsets), honoring the existing backend link. `openedResearchReport` is added to the routing write-discipline allowlist (`centralRouteDiscipline.test.ts`).
  - **Views** (`frontend/src/views/ResearchReportDetail.tsx`, `frontend/src/views/ResearchTabContent.tsx`, `frontend/src/components/research/{CorpusResearchReportCards,ResearchReportListCard}.tsx`): a report detail view (markdown body with footnote citations via `SafeMarkdown`; Report / Citations / Sources / Run-details tabs; live polling + a Cancel control while running; citation rows deep-link to the cited annotation) and a corpus **"Research" tab** (`frontend/src/views/Corpuses.tsx`) listing reports. A secondary `StartResearchModal` provides an explicit kickoff; the primary trigger remains the corpus chat agent.
  - **Notifications** (`frontend/src/hooks/useNotificationWebSocket.ts`, `frontend/src/hooks/useJobNotifications.tsx`, `frontend/src/components/notifications/JobNotificationToast.tsx`, `frontend/src/utils/jobNotificationCacheUpdates.ts`, `frontend/src/hooks/useResearchCompletionNotification.ts`): `RESEARCH_REPORT_*` added to the notification type union + job-toast system (the toast is clickable and deep-links to `/research/:slug`), terminal Apollo-cache updates, and a completion hook the detail view uses to refetch + stop polling on the terminal flip.
  - **Tests**: backend slug-resolver + status-tool/service tests (`opencontractserver/tests/research/test_research_report_queries.py`, `test_research_status_tool.py`); frontend unit tests for route parsing + research utils (`frontend/src/utils/__tests__/`) and a Playwright component test for the detail view (`frontend/tests/ResearchReportDetail.ct.tsx`).
  - **Review hardening**: removed the dead `RESEARCH_REPORT_PROGRESS` member from the notification type union (no v1 emitter/handler — add it when wired); `getResearchStatus` now dev-warns on an unrecognized `JobStatus` instead of silently rendering "Queued"; the chat status tool's duration string now mirrors the frontend `formatResearchDuration` (sub-minute reads "42s", not "0m 42s") with a regression test; `StartResearchModal`'s optional title input is capped at `MAX_RESEARCH_TITLE_CHARS` (255, mirroring the model column); citation rows key off the unique footnote instead of the array index; and an anonymous-user rejection test pins the `@login_required` gate on `researchReportBySlug`.
- **Runtime-configurable corpus categories (tags) — superuser CRUD via GraphQL + in-app admin panel.** Corpus categories (the "Case Law", "Contracts", "Legislation", "Knowledge" tags shown on the Discover page and in corpus settings) were already stored in the `CorpusCategory` model but could only be created/edited by editing the `0035_seed_default_categories` migration or using Django admin. Superusers can now manage the full set at runtime from within the app.

  - **Superuser-only GraphQL mutations** (`config/graphql/corpus_category_mutations.py`) — `createCorpusCategory`, `updateCorpusCategory`, `deleteCorpusCategory`. Each is gated on `info.context.user.is_superuser` (returns `ok=false`/message for non-superusers, mirroring the pipeline-settings mutations), validates name (non-empty, unique, ≤255 chars), icon length, and hex color format, and is rate-limited at `WRITE_LIGHT`. Deleting a category cleans up the `Corpus.categories` M2M links automatically but leaves the corpuses themselves intact. Registered in `config/graphql/mutations.py`. The existing `corpusCategories` query already exposes the read side. Updated the `CorpusCategoryType` docstring (`config/graphql/corpus_types.py`) which previously claimed categories were "managed via Django Admin only".
  - **Service layer** (`opencontractserver/corpuses/services/corpus_category_service.py`) — the mutations are thin GraphQL wrappers (superuser gate + Relay global-id parsing only); validation, unique-name enforcement, and all `CorpusCategory.objects.*` access live in the new `CorpusCategoryService` (re-exported from `opencontractserver/corpuses/services/__init__.py`), per CLAUDE.md rule 7. Shared bounds/defaults (`MAX_CATEGORY_NAME_LENGTH`, `MAX_CATEGORY_ICON_LENGTH`, `DEFAULT_CATEGORY_ICON`, `DEFAULT_CATEGORY_COLOR`) moved to `opencontractserver/constants/corpus_categories.py`; the frontend defaults moved to `frontend/src/assets/configurations/constants.ts` (`DEFAULT_CATEGORY_ICON` / `DEFAULT_CATEGORY_COLOR`) so the two sides no longer drift.
  - **In-app admin UI** (`frontend/src/components/admin/corpus_categories/`) — new `CorpusCategoryManagement` page (route `/admin/corpus-categories`, registered in `App.tsx`) listing categories in a table with create/edit/delete via a modal form (name, description, icon, color picker, sort order) and a delete confirmation. Superuser-gated via the `backendUserObj` reactive var. A "Corpus Categories" card was added to the `GlobalSettingsPanel` admin landing grid. GraphQL operations + types live in `frontend/src/components/admin/corpus_categories/graphql.ts`.
  - **Tests** (`opencontractserver/tests/test_corpus_category.py::TestCorpusCategoryManagementMutations`) — create/update/delete happy paths, default icon/color on create, superuser gating for all three mutations, duplicate-name rejection (create + update), blank-name and invalid-color rejection, not-found handling, and confirmation that deleting a category removes only the M2M link, not the corpus.

- **OC_COUNTRY / OC_STATE / OC_CITY annotation conventions + offline geocoding service (foundation — issue #1819).** Lays the platform groundwork for the map UI tracked in follow-up issues #1820 (reusable `<AnnotationMap />` + Discover integration) and #1821 (Corpus Home map view). This PR ships the backend foundation only — the React component and Corpus Home view are intentionally scoped to the follow-ups.

  - **Constants** (`opencontractserver/constants/annotations.py`, `frontend/src/assets/configurations/constants.ts`) — three new label conventions (`OC_COUNTRY`, `OC_STATE`, `OC_CITY`) with a coherent blue→teal colour ramp (country deepest at `#0E3A5F`, city lightest at `#3E92CC`) so a mixed-type map cluster reads at a glance. Backend constants carry the colour / icon / description; the frontend mirrors the label names + colour family per the existing "Keep in sync" convention used for `OC_URL`.
  - **Annotation `data` JSONField + migration `annotations/0075_annotation_data.py`** — new structured-metadata sidecar distinct from `Annotation.json` (which stores positional bounds / span offsets). Nullable so existing rows need no backfill; first consumer is the geocoding pipeline which stamps `{canonical_name, lat, lng, admin_codes, geocoded}` here. The column is generic enough for future label conventions (planned: extraction provenance, citation anchoring) to reuse instead of each minting another sparse typed field.
  - **Offline geocoding utility** (`opencontractserver/utils/geocoding/`) — pure-Python, network-free place resolution against a bundled reference dataset. `data/countries.json` (~190 rows, ISO 3166-1 complete, alpha-2 + alpha-3 + common aliases + centroids), `data/us_states.json` (50 + DC + 5 territories, USPS codes + AP-style abbreviations), `data/cities.json` (~150 curated GeoNames `cities1000` rows including the disambiguation cases needed by the resolver — multiple "Paris" rows, multiple "Springfield" rows, multiple "London" rows). The CC BY 4.0 GeoNames attribution lives at `docs/credits/geonames.md` along with the regeneration recipe for when broader coverage is required. **Resolver contract** (`service.py::resolve_place`): exact (case-insensitive) → alias (including ISO codes / USPS abbreviations / dotted forms like "U.S.") → fuzzy (`difflib.SequenceMatcher.ratio()` ≥ `0.85`, tie-break by population). `country_hint` / `state_hint` narrow the candidate pool BEFORE matching so a hinted ambiguous span (e.g. `"Paris"` + `state_hint="TX"`) prefers the right row even when multiple rows are exact name matches. Returns frozen `ResolvedPlace(canonical_name, label_type, lat, lng, admin_codes)` on hit, `None` on miss. Indexes are built lazily under a module-level lock on first call — JSON load runs at most once per process. The issue's suggested `rapidfuzz` is intentionally NOT added as a dependency (the stdlib `difflib` clears the bar for the simple typo class we care about; transpositions stay below threshold by design to keep false-positive rate manageable).
  - **`GeographicAnnotationService`** (`opencontractserver/annotations/services/geographic_service.py`, re-exported from `annotations/services/__init__.py`) — the canonical entry point for map aggregation per CLAUDE.md rule 7 (never compose `visible_to_user` / `user_can` inline in resolvers). Two visibility modes: `aggregate_for_corpus(user, corpus, *, bbox, label_types, request)` routes through `CorpusDocumentService.get_corpus_documents_visible_to_user` so a private document inside a public/shared corpus does NOT contribute pins to a viewer who lacks document-level READ (the documented MIN-permission semantic, #1682); `aggregate_global(user, *, bbox, label_types, request)` uses `Annotation.objects.visible_to_user(user)` for the Discover surface. Both return `list[GeographicPin]` (a frozen dataclass — `canonical_name, label_type, lat, lng, document_count, sample_document_ids`). Pins are grouped by `(label_type, canonical_name, lat, lng)` and ship a bounded `sample_document_ids` preview (cap = 5) rather than the full annotation row set. Rows with `data['geocoded'] is False` are excluded so the ungeocodable-text path (resolver miss) survives as a user annotation but doesn't pollute the map.
  - **GraphQL mutations** (`config/graphql/annotation_mutations.py`, registered in `config/graphql/mutations.py`) — `addCountryAnnotation`, `addStateAnnotation`, `addCityAnnotation`. Each mirrors `AddUrlAnnotation`'s contract (validate parents via `_resolve_annotation_parents` → ensure the OC\_\* label on the corpus → write the annotation with `structural=True`) and additionally calls `resolve_place` and stamps the result into `Annotation.data`. The shared body `_create_geographic_annotation` centralises the flow so the three wrappers stay one-liner-thin. `country_hint` / `state_hint` arguments on `addStateAnnotation` and `addCityAnnotation` mirror the resolver's disambiguation surface. **Resolver-miss contract** (per the issue spec): the annotation is created regardless of resolver outcome — the user's labelling work survives — but `data['geocoded']` is False and the mutation's `geocoded` field returns False so the client can surface the warning. Labels created by these mutations are marked `read_only=True` so the platform's structural-write-deny rule applies (only superusers can edit later).
  - **GraphQL queries** (`config/graphql/annotation_queries.py::AnnotationQueryMixin`) — `geographicAnnotationsForCorpus(corpusId, bbox?, zoom?, labelTypes?)` and `globalGeographicAnnotations(bbox?, zoom?, labelTypes?)`. Both return `[GeographicAnnotationPinType]` — a new GraphQL type (`canonicalName, labelType, lat, lng, documentCount, sampleDocumentIds`) that mirrors the service dataclass one-to-one. `BBoxInputType` (south/west/north/east floats) is shared by both queries; the resolver wraps it into the service's `BBox` dataclass which handles antimeridian-crossing boxes (`west > east` → two longitude ranges unioned). `sample_document_ids` are encoded as Relay global IDs (`to_global_id("DocumentType", pk)`) so the frontend can deep-link straight to the document viewer. `zoom` is accepted for forward compatibility with the cluster-threshold logic in #1820 but is not consumed server-side today — the resolver returns all label types and the client picks the right one at the current cluster threshold. Both queries route through `BaseService.get_or_none` / the service layer so the same IDOR-safe "empty list = doesn't exist OR no permission" response surfaces uniformly.
  - **Tests**:
    - `opencontractserver/tests/test_geocoding_service.py` — exhaustive coverage of `resolve_place` per label type: exact / alias (alpha-2 / alpha-3 / dotted / USPS) / fuzzy / no-match / case-insensitive / empty input / disambiguation via hints (Paris/FR vs Paris/TX, Springfield IL/MO, London/CA vs London/GB) / graceful fallback when an unrecognised hint would otherwise zero the candidate pool.
    - `opencontractserver/tests/test_geographic_annotation_mutations.py` — happy path / outsider-permission rejection / ungeocodable-text creates-annotation-with-`geocoded:False` / label `read_only` flag / label idempotency on repeat / state alias / city disambiguation via hints, all three Add\*Annotation mutations.
    - `opencontractserver/tests/test_geographic_annotation_service.py` — fixture with public + private document in a shared corpus pins the MIN(document, corpus) semantic: viewer with corpus READ + no document-level READ on the private doc sees the France pin with `documentCount=1`, NOT 2. Also covers bbox filtering (Europe-only box excludes Tokyo), antimeridian-crossing bbox (`_bbox_contains` handles the wrap-around), label-type filtering, ungeocodable-row exclusion from aggregation, sample-ID cap, unknown-label-type rejection, and the IDOR-safe empty-list response for a user with no corpus permission.
    - `docs/test_scripts/geographic_annotations_smoke.md` — manual smoke test exercising the full mutation → service → aggregation loop against a live shell, used as the integration gate before the foundation is consumed by the map UI.

- **Runtime LLM configuration — per-corpus default model + per-agent override + pipeline-component-style provider registry.** Previously, the only knob for LLM selection was a per-call `agents.for_document(model="…")` string that bypassed the codebase entirely (no validation, no persistence, no per-corpus default); the global model came from `settings.OPENAI_MODEL` and could only be changed by redeploying. Three changes together let users pick a model per corpus, override it per agent, and discover what's available without redeploying.
  - **`BaseLLMProvider` pipeline component + four shipped providers** (`opencontractserver/pipeline/base/llm_provider.py`, `opencontractserver/pipeline/llm_providers/{openai,anthropic,google,ollama}_provider.py`). New abstract pipeline component class mirroring `BaseEmbedder`'s shape: each subclass declares a `provider_key` (pydantic-ai's prefix — `"openai"`, `"anthropic"`, `"google-gla"`, `"ollama"`), a `supported_models` tuple of suggested bare model names exposed to the UI, and a `requires_api_key: bool`. Concrete providers carry no behaviour — pydantic-ai owns client instantiation; these classes exist so the rest of the system has a discovery surface to enumerate and validate against.
  - **`PipelineComponentRegistry` extended with the new component type** (`opencontractserver/pipeline/registry.py`). `ComponentType.LLM_PROVIDER` added to the enum, `_llm_providers` / `_llm_providers_by_key` storage and a discovery block in `_discover_all_components()`, plus `get_all_llm_providers_cached()` / `get_llm_provider_by_key_cached()` module-level helpers and three new metadata fields (`provider_key`, `supported_models`, `requires_api_key`) on `PipelineComponentDefinition` (set only for LLM providers; absent on other component types). The registry log line and `get_all_components_cached()` dict both pick up the new bucket.
  - **`opencontractserver/llms/llm_registry.py` — model-spec parsing, validation, and the priority-chain resolver.** `parse_model_spec("anthropic:claude-opus-4-6") → ("anthropic", "claude-opus-4-6")`; bare strings (no colon) default to the `openai` provider so legacy `OPENAI_MODEL="gpt-4o"` values keep working. `normalise_model_spec` returns the canonical `"{provider}:{model}"` form. `validate_model_spec` raises `LLMProviderNotRegistered` for unknown providers (model names themselves are not strictly enforced — the `supported_models` tuple is a UI suggestion, not a runtime gate). `resolve_model_spec(explicit=..., agent_preferred=..., corpus_preferred=...)` walks the documented priority chain: per-call → per-agent → per-corpus → `settings.DEFAULT_LLM` → legacy `settings.OPENAI_MODEL` → hard fallback `"gpt-4o"`.
  - **`Corpus.preferred_llm` + `Corpus.created_with_llm` audit field** (`opencontractserver/corpuses/models.py`, migration `corpuses/0052_corpus_preferred_llm.py`). New `CharField(128)` for the corpus's preferred pydantic-ai model spec and an editable=False audit-trail column stamped at creation. `Corpus.save()` validates and normalises `preferred_llm` on every write via the new registry (`ValidationError` on malformed spec / unknown provider) and freezes `created_with_llm` to the resolved default at first save. The audit column is backfilled for existing rows in the migration's `RunPython` step. Unlike `preferred_embedder`, `preferred_llm` is freely mutable — swapping LLMs doesn't invalidate stored data.
  - **`AgentConfiguration.preferred_llm`** (`opencontractserver/agents/models.py`, migration `agents/0014_agentconfiguration_preferred_llm.py`). New nullable `CharField(128)` so a single agent (e.g. the doc-summarizer @-mention) can override the corpus default with its own model. Same `save()`-time validation as `Corpus.preferred_llm`. Null means "fall back to corpus default" — preserves pre-feature behaviour for every existing agent row.
  - **Agent factory threads the resolver through** (`opencontractserver/llms/agents/agent_factory.py`). `UnifiedAgentFactory.create_document_agent` and `create_corpus_agent` now resolve the corpus object BEFORE building `AgentConfig`, then call `resolve_model_spec(explicit=model, agent_preferred=agent_preferred_llm, corpus_preferred=corpus.preferred_llm)` and pass the result as `model_name=` to `get_default_config()`. `agents.for_*` exposes a dedicated `agent_preferred_llm=` kwarg so the per-agent slot is exercised by production code (not just tests): `agent_tasks.generate_agent_response` (the @-mention task) and `delegation_tools.build_delegation_tool` (the sub-agent @-mention) forward `agent_preferred_llm=agent_config.preferred_llm`, which means a per-call `model=` still wins over the agent's persisted preference.
  - **`get_context_window_for_model` handles provider-prefixed specs** (`opencontractserver/llms/context_guardrails.py`). Strips the pydantic-ai provider prefix before the lookup table check so `"anthropic:claude-opus-4-6"` resolves to the same 200K window as the bare `"claude-opus-4-6"`. Without this fix, the resolver's canonical-form normalisation (always prefixed) would have collapsed every model to `DEFAULT_CONTEXT_WINDOW` and broken compaction sizing for Claude/Gemini.
  - **GraphQL surface** (`config/graphql/{pipeline_types,pipeline_queries,corpus_mutations,agent_mutations,agent_types,serializers}.py`).
    - `PipelineComponentType` gains `providerKey` / `supportedModels` / `requiresApiKey` (set only for LLM providers); `PipelineComponentsType` gains an `llmProviders` field that surfaces the registered providers via the existing `pipelineComponents` query. The query resolver pulls `llm_providers` directly off the registry (bypasses the non-superuser enabled-components filter — LLM providers are metadata, not credentials).
    - `CreateCorpusMutation` and `UpdateCorpusMutation` accept `preferred_llm: String`; `CorpusSerializer` exposes `preferred_llm` (writable) and `created_with_llm` (read-only).
    - `CreateAgentConfigurationMutation` accepts `preferred_llm`; `UpdateAgentConfigurationMutation` accepts `preferred_llm` (set/replace) plus a separate `clear_preferred_llm: Boolean` (explicit reset back to the corpus default — avoids empty-string ambiguity). Both flow through `AgentConfigurationService.create_agent` / `update_agent`, which convert `ValidationError` raised by `AgentConfiguration.save()` into `ServiceResult.failure` with a clean message. `AgentConfigurationType` adds `preferred_llm` to its exposed fields.
  - **Tests** (`opencontractserver/tests/test_llm_runtime_config.py`). Covers spec parsing edge cases, normalisation, validator (known/unknown providers, malformed strings), the four-rung resolver priority chain, registry discovery of the four shipped providers, `Corpus.preferred_llm` save-path validation + `created_with_llm` audit-stamp + audit-field immutability, `AgentConfiguration.preferred_llm` save-path validation, the prefix-stripping fix in `get_context_window_for_model`, and an end-to-end scenario asserting the resolver winning order with real `Corpus` rows.

### Changed

- **Corpus description storage consolidated to a single canonical source — the `Readme.CAML` Document body.** The corpus description was previously stored in four places that drifted independently (the `Corpus.md_description` `FileField`, the `Corpus.description` plain-text projection, the `Corpus.description_preview` short excerpt added by PR #1805, and the `Readme.CAML` `Document` rendered by `CorpusArticleView`). After this refactor:
  - **One canonical source:** the corpus's `Readme.CAML` Document body. All reads and writes converge on it.
  - **Auto-maintained read-only caches:** `Corpus.description`, `Corpus.description_preview`, and a new `Corpus.readme_caml_document` FK are refreshed by a `Document.post_save` / `DocumentPath.post_save` signal handler (`opencontractserver/corpuses/signals.py`). Direct writes to the cache columns are silently overwritten on the next CAML save — codified by `test_direct_cache_write_is_overwritten_on_next_caml_save`.
  - **GraphQL surface unchanged:** `description`, `descriptionPreview`, `mdDescription` keep their observable contracts (only the underlying storage shifts; `mdDescription` URL now points at the CAML doc's `txt_extract_file` instead of the removed `md_description` FileField). New additive `readmeCamlDocument` field exposes the canonical Document directly for clients that want revision history / permissions / version-tree access.
  - **One write mechanism:** both the editor (`CorpusService.update_description`) and the agent's targeted-replacement tool (`aapply_caml_article_edit`) now route through `opencontractserver.documents.versioning.import_document`, which creates a new Document sharing the existing `version_tree_id` and atomically flips `DocumentPath.is_current` flags. Frontend deep-links resolve to the head of the version tree via `corpus.readmeCamlDocument` (or DocumentPath join), no longer to a specific Document pk.
  - **Revision history** uses the existing `Document.version_tree_id` siblings — `CorpusDescriptionRevisionType` becomes a `graphene.ObjectType` facade exposing the historical `id / version / author / snapshot / created` shape so the frontend revision-history UI renders unchanged. The `diff` field is dropped (clients that need diffs compute via `difflib` from successive snapshots).
- **Export schema bumped to V3** (`opencontractserver/tasks/export_tasks_v2.py`, `opencontractserver/utils/export_v2.py`, `opencontractserver/types/dicts.py`, `opencontractserver/utils/validate_export.py`). V3 drops `md_description` and `md_description_revisions` from the top-level payload — the Readme.CAML Document rides in `annotated_docs` like any other Document, so revisions appear as version-tree siblings without extra plumbing. V2 imports remain supported via a back-compat shim (`import_md_description_revisions`) that synthesizes a Readme.CAML Document from `md_description` and replays revisions as version-tree siblings (preserving historical `created` timestamps), plus runs `oc-import://` link rewriting on the synthesized body. Validator gates required fields by version: V2 still requires both legacy fields, V3 forbids them.

### Removed

- `Corpus.md_description` `FileField`, `CorpusDescriptionRevision` model + table, and the `Corpus.update_description` / `_read_md_description_content` / `_summarize_for_preview` / `_markdown_to_plain_text` model helpers (replaced by the canonical-CAML write path through `import_document` and by `opencontractserver/corpuses/services/description_cache.py`). The `description_preview` branch of `Corpus.save()` is also removed — the Document signal handler owns cache refresh now.
- `LandingMarkdownSection` styled component in `frontend/src/components/corpuses/CorpusHome/CorpusLandingView.tsx` (PR #1805 workaround). Rich content renders via the existing `CorpusArticleView` short-circuit; the landing hero still shows the short `descriptionPreview` subtitle.
- `package_md_description_revisions` helper in `opencontractserver/utils/export_v2.py` and the `DescriptionRevisionExport` TypedDict in `opencontractserver/types/dicts.py`.

### Technical Details

- **Migrations:** `0053_add_readme_caml_fk.py` (schema-only — adds the cache FK), `0054_canonical_caml_backfill.py` (data migration — for every Corpus with non-empty `md_description`, creates a Readme.CAML Document + DocumentPath, replays each `CorpusDescriptionRevision` row as a version-tree sibling, populates the cache columns; idempotent via lookup-before-create), `0055_drop_legacy_description_storage.py` (column + table drop; forward-only). Large deployments should schedule `0054` during a maintenance window.
- **Spec:** `docs/superpowers/specs/2026-05-27-canonical-caml-description-refactor-design.md` covers §4.1 source of truth, §4.4 signal flow, §4.5 GraphQL surface, §4.6 frontend impact, §4.7 agent tools, §4.8 V2 → V3 export, §4.9 data migration.
- **Replaces PR #1805** — the `description_preview` column added there is retained as a cache column; the `LandingMarkdownSection` workaround is removed; the `useCorpusMdDescription` hook is unchanged (`CorpusDetailsView` still uses it for the About card).

### Fixed

- **Loading the configured embedder is no longer "super duper slow" on query-time endpoints.** Every embed call instantiated the embedder class fresh (`embedder_class()`), and `PipelineComponentBase.__init__` (`opencontractserver/pipeline/base/base_component.py`) loads component settings from the `PipelineSettings` DB singleton and, whenever any secret is configured, decrypts them with a deliberately expensive PBKDF2 KDF (`PipelineSettings._derive_key`, 480,000 HMAC iterations — `opencontractserver/documents/models.py`). Query-time endpoints paid that DB read + KDF on **every request**: `Corpus.embed_text` behind the public search API (`opencontractserver/discovery/views.py`), the MCP `search_corpus` tool (`opencontractserver/mcp/tools.py`), and global vector search (`opencontractserver/llms/vector_stores/core_vector_stores.py`).

  - **New process-local embedder instance cache** (`opencontractserver/pipeline/utils.py`): `get_embedder_instance()` / `invalidate_embedder_cache()` cache the constructed embedder keyed by `(class_path, PipelineSettings.modified)`, mirroring the existing reranker instance cache. The `modified` (auto_now) component of the key means any settings write through the GraphQL mutations — which all call `save()` — busts the cache process-wide on the next lookup, so config changes still take effect without a worker restart.
  - **Hot paths switched to the cache**: `generate_embeddings_from_text` (`opencontractserver/utils/embeddings.py`) and `CoreAnnotationVectorStore._generate_global_query_vector` (`core_vector_stores.py`). Per-call kwarg overrides are unaffected — the cached instance's `embed_text`/`embed_texts_batch` still accept call-time overrides; only the resolved-from-DB defaults are pinned.
  - **Test isolation**: the autouse `_invalidate_pipeline_settings_singleton_cache` fixture (`opencontractserver/conftest.py`) now also clears the embedder/reranker instance caches before each test, so tests that mock `get_component_by_name` to return different classes for the same path don't collide on a stale cached instance (the migration-seeded `PipelineSettings.modified` is stable across `TestCase` tests).
  - **Tests** (`opencontractserver/tests/test_pipeline_utils.py::TestEmbedderInstanceCache`): cover per-class-path caching, distinct-path isolation, default class-path derivation, settings-write invalidation, explicit cache invalidation, and construction-failure propagation (a probe embedder whose `__init__` raises must propagate the exception and leave the cache unpopulated — locking in the intentional divergence from the reranker cache's `None`-degradation).
  - **Thread-safety contract documented on `get_embedder_instance()`**: the shared-instance requirement (implementations must be thread-safe for read) and the deliberate exception-propagation behaviour now live in the function docstring, not only the module-level comment.

- **Corpus dashboard no longer leaves a sub-pixel body-scroll strip that subtly repositions the viewport.** `NavbarHeightSync` (`frontend/src/components/layout/NavMenu.tsx`) kept the global `--oc-navbar-height` CSS variable in sync with the real navbar height using `offsetHeight`, which is rounded to an integer (rounds **down**). The viewport-bounded corpus layouts — `CorpusViewContainer` / `CleanViewContainer` (`Corpuses.styles.ts`), `CorpusQueryView`'s `COMPOSER_MAX_HEIGHT`, and the chat-tab cap — all derive their height from `calc(100dvh - var(--oc-navbar-height))`. When the navbar's real border-box height was fractional (e.g. 72.5px from `padding: 12px 24px` + flex-wrap), the variable under-reported it by up to ~1px, so each bounded tab reserved slightly too little space and the document gained a permanent ~0.5px scrollable strip beneath the viewport (felt as a subtle, annoying need to reposition the screen). Now measures `Math.ceil(navbar.getBoundingClientRect().height)` so the variable preserves the sub-pixel value and never under-reports the navbar — reserved space always covers the space the navbar actually consumes. Single-line measurement change; the deliberate `min-height`/natural-document-flow design of the power-user tabs (issue #1558) is untouched.

- **Large bulk-import and corpus-export-import ZIPs no longer fail with "NetworkError when attempting to fetch resource."** Both modals were the last upload paths still routing through GraphQL with the full ZIP base64-encoded into a JSON request body (`BulkImportModal.tsx`, `ImportCorpusModal.tsx`). For files past ~100 MB, base64 inflation (+33%) plus Apollo's JSON.stringify of the request body exceeded V8's contiguous-string allocation limit, so the request never reached the network. Migrated both to multipart REST:
  - **New REST endpoint `POST /api/imports/zip-to-corpus/`** (`opencontractserver/document_imports/views.py::ZipToCorpusImportView`) — bulk ZIP with folder-structure preservation. Replaces the deleted `ImportZipToCorpus` GraphQL mutation. Wraps the existing `import_zip_with_folder_structure` celery task. Reuses `DocumentImportThrottle`, bearer JWT auth, `_enforce_size_cap`, and the `BULK_UPLOAD_OWNER_CACHE_PREFIX` IDOR-protection cache.
  - **New REST endpoint `POST /api/imports/corpus/`** (`opencontractserver/document_imports/views.py::CorpusExportImportView`) — OpenContracts corpus-export ZIP import. Replaces the deleted `UploadCorpusImportZip` GraphQL mutation. Creates the placeholder corpus synchronously (same `title="New Import"` contract as the legacy mutation) and queues `import_corpus`.
  - **Shared service functions** (`opencontractserver/document_imports/services.py`) — `import_zip_to_corpus_for_user` and `import_corpus_export_for_user`. Both accept either an `UploadedFile` (REST, streamed through storage) or raw `bytes` (tests/legacy). Both preserve the existing unified IDOR "Corpus not found or you do not have permission to add documents to it" message for inaccessible / non-existent / read-only corpora.
  - **New frontend helpers** (`frontend/src/utils/importHttp.ts`) — `importZipToCorpusMultipart` and `importCorpusExportMultipart`. Stream the `File` directly through `FormData`; the browser hands the byte stream to the network layer without a JS-side allocation of the full payload.
  - **GraphQL surface removals** — `UploadCorpusImportZip` and `ImportZipToCorpus` mutation classes deleted from `config/graphql/document_mutations.py`; `import_open_contracts_zip` and `import_zip_to_corpus` field bindings removed from `config/graphql/mutations.py`. Frontend `START_IMPORT_CORPUS` / `IMPORT_ZIP_TO_CORPUS` mutations and their input/output types removed from `frontend/src/graphql/mutations.ts`. The `me { canImportCorpus }` resolver description updated to point at the REST endpoints.
  - **Tests** — `opencontractserver/tests/test_document_imports_rest.py` adds `ZipToCorpusImportViewTests` and `CorpusExportImportViewTests` covering happy path, IDOR error collapse for inaccessible/read-only/non-existent corpora, cross-corpus folder rejection, non-ZIP rejection, oversize (413), usage-cap gating, and the corpus-export placeholder-corpus contract. `test_graphql_import_export_mutations.py` drops `test_zip_upload` (moved to REST). `frontend/src/utils/__tests__/importHttp.test.ts` adds `importZipToCorpusMultipart` and `importCorpusExportMultipart` coverage. `frontend/tests/bulk-import-modal.ct.tsx` swaps the Apollo MockedProvider mock for a `window.fetch` stub on `/api/imports/zip-to-corpus/`.

### Added

- **In-run history compaction via pydantic-ai `history_processors`.** A new
  `shrink_old_artifacts_processor` in `opencontractserver/llms/history_processors.py`
  is unconditionally installed as a `ProcessHistory` capability on every agent constructed through
  `make_pydantic_ai_agent`. Before each model request inside an
  `Agent.run()` loop, the processor checks whether cumulative tokens
  exceed `CompactionConfig.threshold_ratio * context_window`; when over
  threshold it truncates older `ToolReturnPart.content` to
  `in_run_tool_return_target_chars` (default 4,000 chars) and drops
  older `ThinkingPart` instances entirely, while leaving the last
  `in_run_keep_recent_pairs` (default 4) ModelResponse/ModelRequest
  pairs untouched. Tool call/return correlation is preserved by
  shrinking in place (never dropping `ToolCallPart` or its paired
  `ToolReturnPart`). Empty-`parts` ModelResponses are also guarded —
  if dropping the only ThinkingPart would empty the parts list, the
  message is left intact.
  - `CompactionConfig` gains four new fields with sensible defaults:
    `in_run_enabled`, `in_run_keep_recent_pairs`,
    `in_run_tool_return_target_chars`, `in_run_drop_thinking`. Set
    `in_run_enabled=False` as a kill switch if regression appears.
  - New constants in `opencontractserver/constants/context_guardrails.py`:
    `IN_RUN_KEEP_RECENT_PAIRS`, `IN_RUN_TOOL_RETURN_TARGET_CHARS`,
    `IN_RUN_DROP_THINKING_DEFAULT`.
  - New `compaction: CompactionConfig` field on `PydanticAIDependencies`
    so the processor reads its knobs from the production path (`deps.compaction`).
  - New telemetry: streamed chats emit a `ThoughtEvent` with metadata
    key `in_run_shrink` whenever the processor fires; non-streamed
    chats produce an INFO log line with the same numbers (tokens
    before/after, tool returns shrunk, thinking parts dropped).
  - Closes the in-loop context-pressure gap surfaced by the deep-research
    agent in PR #1814: previously the OpenContracts compaction pipeline
    ran only once before `Agent.run()` started; tool calls/returns
    generated inside the autonomous loop accumulated unbounded until
    pydantic-ai's `UsageLimits.request_tokens_limit` terminated the run.
  - Tests: `opencontractserver/tests/test_history_processors.py` (12
    unit tests covering threshold gate, shrink, ThinkingPart drop,
    tool_call_id preservation, last-message invariant, short
    histories, disabled config, deps=None, callback contract,
    production-path config resolution, empty-parts guard).
    `opencontractserver/tests/test_pydantic_ai_factory.py` gets three
    new tests pinning `ProcessHistory` injection order, legacy-kwarg
    auto-wrapping, and modern `capabilities=` passthrough.

### Fixed

- **Per-conversation `CompactionConfig` overrides reach the in-run
  history processor.** `PydanticAICoreAgent._apply_context_budget`
  was forwarding only `compaction.threshold_ratio` onto
  `agent_deps.compaction_threshold_ratio` and never copying the full
  config object to `agent_deps.compaction` — so any caller-supplied
  `in_run_enabled` / `in_run_keep_recent_pairs` /
  `in_run_tool_return_target_chars` / `in_run_drop_thinking`
  silently fell back to defaults inside the processor. Now
  `agent_deps.compaction = config.compaction` is set on every turn.
  Regression test pinned in
  `opencontractserver/tests/test_context_guardrails.py::TestRefreshContextBudgetFallback::test_compaction_config_is_copied_to_deps`.

- **Loud import-time guard on `dataclasses.replace` assumption.**
  `opencontractserver/llms/history_processors.py` now asserts at
  module import that `ToolReturnPart`, `ModelRequest`, and
  `ModelResponse` remain stdlib `@dataclass` types. A future
  pydantic-ai release migrating any of them to `pydantic.BaseModel`
  would have broken `replace()` inside the processor — and the
  defensive outer `try/except` would have silently swallowed the
  exception, turning the in-run shrink into a no-op with no visible
  signal. Now CI fails loudly on the next pydantic-ai bump that
  invalidates the assumption.

- **Telemetry-callback failures no longer silently disable shrink
  side-effects via the outer try/except.** Pinned a regression test
  (`test_callback_exception_does_not_propagate`) confirming that a
  raising `on_in_run_shrink` callback is caught locally, logged via
  `logger.exception`, and does NOT bubble out of the inner callback
  block — so the actual message-list shrink still survives even when
  a telemetry sink dies mid-run.

- **Deep-research agent — chat-triggered, long-running corpus research with grounded citations** (new app `opencontractserver/research/`, new Celery task `opencontractserver/tasks/research_tasks.py`, new chat tool `opencontractserver/llms/tools/research_tools.py`, GraphQL surface in `config/graphql/research_{types,queries,mutations}.py`). A user in a corpus chat can ask the agent to "research X" and a long-lived (5-30 min) autonomous research job kicks off in the background. The job runs a PydanticAI corpus agent with a read-only retrieval surface plus two job-bound scratchpad tools (`record_finding`, `finalize_report`), produces a markdown report with footnote citations, and notifies the user when complete (new `RESEARCH_REPORT_*` `NotificationTypeChoices` values, broadcast via the existing notification WebSocket) plus drops a system `ChatMessage` back into the originating conversation.

  - **`ResearchReport` model** (sibling of `Analysis`/`Extract`) tracks the full lifecycle: corpus FK, prompt, status (`JobStatus` + new `CANCELLED`), `started_at`/`completed_at`/`last_progress_at`, `cancel_requested` (cooperative cancel polled between tool calls), step budget, rendered `content` markdown, structured `findings`/`citations`/`tool_call_log`/`model_usage`/`warnings` JSON sidecars, and M2M to `source_annotations`/`source_documents` for provenance. Creator-only visibility in v1 (no sharing → no IDOR surface to defend on shared reports). Migration: `opencontractserver/research/migrations/0001_initial.py` + `opencontractserver/notifications/migrations/0005_add_research_report_notification_types.py`.
  - **`ResearchReportService`** (`opencontractserver/research/services/research_reports.py`) — canonical entry point per CLAUDE.md rule 7. Lifecycle helpers (`mark_started` / `mark_progress` / `mark_completed` / `mark_failed` / `mark_cancelled`), scratchpad writes (`append_finding`, `append_tool_call`), terminal `finalize` (runs citation post-processor: parses `<cite ids="...">claim</cite>` placeholder tags into `[^n]` footnote markers + builds the `## Sources` section + populates the `source_annotations` / `source_documents` M2Ms), and a concurrency soft-guard (`start` refuses a second QUEUED/RUNNING report for the same `(user, corpus)` within `DEEP_RESEARCH_CONCURRENCY_GUARD_SECONDS`).
  - **Closed-citation-graph invariant.** The job-bound `record_finding` closure validates every `supporting_source_id` against the live `PydanticAIDependencies.retrieved_annotation_ids` accumulator (`opencontractserver/llms/tools/pydantic_ai_tools.py:263`) — annotation IDs not surfaced by a retrieval tool in this run are rejected with an error string so the model self-corrects, defending against fabricated citations.
  - **Chat kickoff tool** `start_deep_research` is appended to the default corpus-agent tool set for authenticated users (`opencontractserver/llms/agents/pydantic_ai_agents.py` corpus-agent factory). It calls `ResearchReportService.start`, returns a confirmation string, and ends the chat turn. `corpus_id` / `user_id` / `conversation_id` are auto-injected from `PydanticAIDependencies` (not LLM-controlled), closing the prompt-injection vector where a malicious user message could spoof a different conversation.
  - **Plumbing changes to the agent framework** (small, additive):
    - `restrict_tool_names` is now honoured on the corpus path in addition to the existing document path (`agent_factory.py` + `pydantic_ai_agents.py:PydanticAICorpusAgent.create`). Lets the deep-research task strictly limit the agent to read-only retrieval tools.
    - `PydanticAIDependencies.conversation_id` (new field) + `build_inject_params_for_context` inject helper now threads conversation_id into tools that declare it (`pydantic_ai_tools.py`, `tool_factory.py`, `agent_factory.py`). `PydanticAICorpusAgent.create` populates it from the resolved conversation.
  - **GraphQL**: `ResearchReportType` (with `findings`/`citations`/`toolCallLog`/`modelUsage`/`warnings` as `GenericScalar`, `fullSourceAnnotationList`/`fullSourceDocumentList` resolvers, `durationSeconds` derived); `researchReport(id)` + `researchReports(corpusId?, status?)` queries; `startResearchReport(corpusId, prompt, title?, maxSteps?)` + `cancelResearchReport(id)` mutations.
  - **Settings**: `DEEP_RESEARCH_DEFAULT_MAX_STEPS` (60), `DEEP_RESEARCH_MAX_TOKENS_DEFAULT` (400k), `DEEP_RESEARCH_SOFT_TIME_LIMIT` / `DEEP_RESEARCH_HARD_TIME_LIMIT` (30/60 min), `DEEP_RESEARCH_SIMILARITY_TOP_K` (6 — lower than interactive chat to reduce vector-search load during long autonomous runs), `DEEP_RESEARCH_CONCURRENCY_GUARD_SECONDS` (1 hour).
  - **Tests**: `opencontractserver/tests/research/test_research_report_model.py` (model defaults, slug auto-gen, `is_terminal`, creator-only visibility), `test_research_report_service.py` (lifecycle transitions, concurrency soft-guard, cancel propagation, finalize with valid + defense-in-depth filtering of rogue citations + handling of deleted annotations, citation-rendering helper unit tests), `test_research_kickoff_tool.py` (kickoff happy path, unknown corpus, concurrency soft-guard returns a friendly error string).
  - **Out of scope for v1** (tracked for v2): sharing / `is_public` on `ResearchReport`; viewer-aware citation redaction for shared reports; document-subset scoping (currently corpus-wide only); resume-from-partial-findings; per-user notification preferences; `AgentConfiguration` integration for saved research presets; watchdog cleanup task for crashed workers (intentionally deferred — `last_progress_at` is in place for when it's added); frontend `ResearchReportView`.

- **"Run on every document" for agent-based corpus actions.** Closes the gap between auto-fire-on-add and one-doc-at-a-time manual runs: a new Layers-icon button on each agent action card in `CorpusActionsSection` fans the action out to every active doc in the corpus that does not already have a queued / running / completed execution for that action (failed executions are intentionally re-queued so the same button doubles as a retry path).

  - **New `CorpusActionTrigger.MANUAL_BATCH`** value (`opencontractserver/corpuses/models.py:1443`) so batch-run executions are distinguishable from auto-fired `ADD_DOCUMENT` / `EDIT_DOCUMENT` rows in the audit trail. Migration `corpuses/0051_add_manual_batch_trigger.py` extends `CorpusActionExecution.trigger`'s `choices` only — `CorpusAction.trigger` and `CorpusActionTemplate.trigger` now use the auto-fire-only subset (`AUTO_FIRE_TRIGGER_CHOICES`) which intentionally excludes `MANUAL_BATCH`, so they need no DDL change.
  - **New service** `CorpusActionService.batch_run_on_corpus(user, action, request)` in `opencontractserver/corpuses/services/corpus_actions.py`. Gates on corpus `UPDATE`, refuses fieldset/analyzer/disabled actions, computes the eligible-doc set (active corpus docs minus docs with a `QUEUED|RUNNING|COMPLETED` execution for the same action), bulk-queues `CorpusActionExecution` rows via the existing `bulk_queue` classmethod, and dispatches one `run_agent_corpus_action.delay(...)` per row inside `transaction.on_commit`. Capped at `BATCH_RUN_MAX_DOCS = 200` per call (`opencontractserver/constants/corpus_actions.py`) so a single click cannot fan out hundreds of LLM calls.
  - **New GraphQL mutation** `startCorpusActionBatchRun(corpusActionId: ID!)` (`config/graphql/corpus_mutations.py`). Permission gate is corpus `UPDATE` (looser than the superuser-only single-doc `runCorpusAction`, matching the rest of `CorpusActionsSection`). Returns `queuedCount`, `skippedAlreadyRunCount`, `totalActiveDocuments`, and the fresh execution rows (the UI consumes only the summary counts for the toast; the execution rows remain on the response for future callers that want to surface queued documents directly).
  - **Frontend** new `BatchRunCorpusActionModal` (`frontend/src/components/corpuses/BatchRunCorpusActionModal.tsx`) — confirmation modal explaining the skip-already-run semantics. Wired into `CorpusSettings` next to the existing `RunCorpusActionModal`; the new button is `canUpdate`-gated (not `isSuperuser`) and only appears on enabled agent actions. New `START_CORPUS_ACTION_BATCH_RUN` GraphQL mutation in `frontend/src/graphql/mutations.ts`.
  - **Tests** `opencontractserver/tests/test_batch_run_corpus_action.py` covers happy path, non-superuser collaborator with corpus UPDATE, IDOR-safe denial for outsiders, fieldset/analyzer/disabled rejection, FAILED re-queue, RUNNING/QUEUED skip, all-completed → 0 queued, soft-deleted DocumentPath exclusion, and the `BATCH_RUN_MAX_DOCS` cap. `test_constants.py` and `test_corpus_action_model.py` already auto-derive their expected keys from the `CorpusActionTrigger` enum so the new value is picked up by the existing alignment tests.

- **Agentic `create_or_update_text_document` tool — create or version-up a text document in a corpus** (`opencontractserver/llms/tools/core_tools/text_document_import.py`, `opencontractserver/llms/tools/core_tools/__init__.py`, `opencontractserver/llms/tools/tool_registry.py`). New corpus-scoped LLM tool that lets a corpus-level agent author or version-up a text-based document inside the active corpus. Initial scope is **text formats only** (`text/plain` default, `text/markdown`, `application/txt` — the `TEXT_MIMETYPES` set); binary formats like PDF/DOCX still require the parsing pipeline and are intentionally out of scope. The tool derives the corpus filesystem path from the `title` using the same sanitisation as `Corpus.add_document` (`MAX_FILENAME_LENGTH` truncation, non-alphanumeric → `_`, `DEFAULT_DOCUMENT_PATH_PREFIX` prefix) so calling it twice with the same `title` in the same corpus hits the same path and the dual-tree versioning architecture (`opencontractserver/documents/versioning.py::import_document`) creates a new version on top of the existing tree. Returns `{status: "created"|"updated", document_id, corpus_id, path, version_number, file_type, byte_count, message}`. IDOR-safe: user / corpus / folder lookups all return the same "does not exist or is not accessible" message regardless of whether the row is missing or unreachable. Permissions: requires `PermissionTypes.UPDATE` on the corpus (raises `PermissionError` otherwise) and grants `CRUD` on the resulting document to the author. Quota: routes through `DocumentService.check_user_upload_quota` so capped users get a clean `ValueError` instead of a half-created doc. Registered as `ToolCategory.CORPUS` with `requires_corpus=True`, `requires_approval=True`, `requires_write_permission=True` (mirrors `move_document`'s HITL posture). `aupload_text_document` / `upload_text_document` are exported as aliases for callers that prefer the verb-first name. Tests in `opencontractserver/tests/test_text_document_import_tool.py` cover the create path, in-folder placement, second-call version-up (asserts shared `version_tree_id`, `is_current` flip on the old row, single active `DocumentPath`), `text/markdown` acceptance, `application/pdf` rejection, empty-title rejection, `None`-content rejection, nonexistent-user rejection, IDOR rejection on a corpus owned by another user, read-only-corpus rejection (`PermissionError`), folder-in-wrong-corpus rejection, and an `acreate_or_update_text_document` async-wrapper smoke test.

- **`Corpus.description_preview` field + markdown rendering on the corpus home page.** Fixes two related rendering bugs surfaced after the CAML/markdown description rollout: (1) corpus cards showed an unformatted wall of stripped-markdown text from `description`, and (2) when a corpus had a markdown `md_description` but no `Readme.CAML` article, the rich markdown was only visible after drilling into the Details → About tab — the home page rendered nothing of it.

  - **Backend (`opencontractserver/corpuses/models.py`, `opencontractserver/corpuses/migrations/0050_corpus_description_preview.py`, `opencontractserver/constants/truncation.py`)** — new `Corpus.description_preview` `TextField` (editable=False) carries an auto-generated short summary derived from `description`. New `Corpus._summarize_for_preview` static helper grabs the first paragraph, collapses internal whitespace into a single line, and truncates at the nearest word boundary capped by `MAX_CORPUS_DESCRIPTION_PREVIEW_LENGTH` (`= 280`, lives in `constants/truncation.py`); appends `…` when truncation occurred. `Corpus.save()` recomputes `description_preview` on every write so packaging imports, direct ORM writes, and the versioned `update_description` path all stay consistent without each caller having to remember; when a caller passes `save(update_fields=["description"])` the override transparently appends `description_preview` to keep the partial write coherent. Migration `0050` adds the column and includes a `RunPython` data migration that backfills `description_preview` for every existing row via the live `_summarize_for_preview` helper (pure string-manipulation, no ORM access, safe from a data migration).
  - **GraphQL (`config/graphql/corpus_types.py`)** — `description_preview` auto-exposes as `descriptionPreview` via `DjangoObjectType` field auto-discovery; no resolver needed.
  - **Frontend GraphQL (`frontend/src/graphql/queries.ts`, `frontend/src/graphql/landing-queries.ts`, `frontend/src/types/graphql-api.ts`)** — `GET_CORPUSES`, `GET_DISCOVERY_DATA`, `GET_CORPUS_WITH_HISTORY`, and the inline `GET_EDITABLE_CORPUSES` query in `AddToCorpusModal` all fetch `descriptionPreview` alongside `description`. `RawCorpusType` gets the new optional `descriptionPreview` field and `GetTrendingCorpusesOutput` picks it.
  - **Frontend card surfaces (`frontend/src/components/corpuses/CorpusListView.tsx`, `frontend/src/components/landing/FeaturedCollections.tsx`, `frontend/src/components/modals/AddToCorpusModal.tsx`, `frontend/src/components/widgets/modals/UploadModal/components/CorpusSelectorCard.tsx`)** — every corpus-card surface now reads `corpus.descriptionPreview ?? corpus.description` so the truncated preview drives the card body. The fallback to `description` only ever triggers for pre-migration cache hits.
  - **Frontend hero subtitle + new MD section (`frontend/src/components/corpuses/CorpusHome/CorpusLandingView.tsx`, `frontend/src/components/corpuses/CorpusHome/CorpusDetailsView.tsx`)** — the landing hero subtitle now reads `fullCorpus.descriptionPreview ?? fullCorpus.description` (no more first-line `.slice(0, 200)` on the raw markdown). A new `LandingMarkdownSection` renders the full markdown content with `SafeMarkdown` directly below the hero whenever `mdContent` is present; this is the same renderer the About card uses, so the home page now shows the rich description without users having to drill into Details. CAML, when present, short-circuits to `CorpusArticleView` upstream, so the new section never competes with an article. Both views were also DRY-refactored to use the existing `useCorpusMdDescription` hook in `frontend/src/hooks/useCorpusMdDescription.ts` instead of duplicating the `useEffect`-fetch boilerplate; the hook's in-flight dedupe map means toggling between Landing and Details no longer re-downloads the markdown file.
  - **Tests (`opencontractserver/tests/test_corpus_description.py`)** — new `SummarizeForPreviewTest` class covers the empty / short-passthrough / collapse-internal-whitespace / first-paragraph-only / truncation-with-ellipsis / word-boundary-respect cases. `UpdateDescriptionSyncTest` gains `test_syncs_preview_on_update`, `test_preview_truncates_long_descriptions`, `test_preview_synced_on_direct_description_assignment`, and `test_preview_included_when_update_fields_targets_description` to pin the cross-caller sync contract and the `update_fields` cascade.

- **Explicit `mode` knob on annotation vector search — `"vector"` / `"fts"` / `"hybrid"`** (`opencontractserver/llms/vector_stores/core_vector_stores.py`, `opencontractserver/llms/vector_stores/pydantic_ai_vector_stores.py`). Hybrid RRF (pgvector + PostgreSQL `tsvector`) has been the implicit default for `async_search` whenever a text query was present, but callers had no way to ask for vector-only or full-text-only retrieval, and the sync `search()` path was vector-only with no way to ask for hybrid. New `SearchMode = Literal["vector", "fts", "hybrid"]` is a field on `VectorSearchQuery` (default `"hybrid"`) and is plumbed through `CoreAnnotationVectorStore.search` / `.async_search` / `.global_search` / `.async_global_search`, plus `PydanticAIAnnotationVectorStore.search_annotations` and `.search_sync`. The dispatcher (`_resolve_mode`) gracefully degrades `"fts"` and `"hybrid"` to `"vector"` when no `query_text` is supplied, and `"hybrid"` falls back to text-only when embedding generation fails (preserves existing behavior). The sync `search()` body is split into `_run_vector_only_sync` / `_run_fts_only_sync` (with async counterparts `_async_vector_only` / `_async_fts_only`) and `_run_fts_query` is now a `@staticmethod` so `global_search` (a classmethod) can reuse it — `global_search` now honors `mode` end-to-end, including a real FTS arm + RRF fusion, addressing the longstanding `TODO` at the prior `global_search` vector arm. The agent-facing tool closure (`vector_search_tool` in `pydantic_ai_vector_stores.py`) intentionally does NOT expose `mode` — LLMs continue to get hybrid by default; the knob is for library callers (GraphQL resolver, eval harness, advanced API users). New tests in `opencontractserver/tests/test_hybrid_search.py::TestSearchModeDispatch` cover the dispatch (`mode='vector'` skips FTS arm, `mode='fts'` skips embedding generation, both sync and async paths) and pin `_resolve_mode`'s degradation contract. **Round-3 review follow-ups**: (a) extracted `_generate_global_query_vector` so `global_search`'s embedder-availability degradation (`hybrid → fts` when the embedder factory is unavailable or returns `None`) lives in a single, unit-testable helper instead of being open-coded across nested `if/else` blocks that mutated `effective_mode`; (b) demoted per-call `INFO` log lines in `search` / `async_search` to `DEBUG` (per-call tracing belongs at `DEBUG`; the `global_search` result-count line stays `INFO`); (c) added `global_search` dispatch coverage in `TestSearchModeDispatch` for `mode='vector'` (FTS arm not called), `mode='fts'` (embedder factory not called), `mode='fts'` with empty text (degrades to vector — FTS arm skipped), `mode='hybrid'` (FTS arm called), `mode='hybrid'` with embedder failure (degrades to FTS), `mode='vector'` with embedder failure (returns `[]`, FTS arm NOT triggered as a silent fallback), plus unit coverage of `_generate_global_query_vector`; (d) documented the no-op `[:first_stage_top_k]` slice in the `global_search` FTS arm as a defensive guard against future `oversample_k` changes; (e) added a `mode` field to the `PydanticAIVectorSearchRequest` docstring.

- **Corpus upvote / downvote with anonymous-friendly voting and score-based sorting.** Adds a Reddit-style vote signal on the corpus list view so users (and unauthenticated visitors) can surface high-signal corpora.
  - **Model (`opencontractserver/corpuses/models.py`)** — new `CorpusVote` (votes table) + `CorpusVoteType` text-choices + Guardian permission tables. The model supports two voter shapes in one table: authenticated votes are keyed on `creator`, anonymous votes on Django `session_key`. Three new denormalized columns on `Corpus` — `upvote_count`, `downvote_count`, and an indexed `score` (= `upvotes - downvotes`) — are maintained by signal handlers in `opencontractserver/corpuses/signals.py` so the corpus list view can `ORDER BY score` without a JOIN-and-aggregate per page.
  - **Migration `corpuses/0049_corpusvote_corpus_upvote_count_and_more.py`** — adds the three count columns, the `CorpusVote` table + Guardian permission tables, the per-row lookup indexes, and two **partial UNIQUE** constraints (`one_vote_per_user_per_corpus` keyed on `(corpus, creator)` where `creator IS NOT NULL`; `one_anon_vote_per_session_per_corpus` keyed on `(corpus, session_key)` where `creator IS NULL`). The partial conditions are required because Postgres treats every NULL as distinct.
  - **Service layer (`opencontractserver/corpuses/services/votes.py`)** — new `CorpusVoteService(BaseService)` exposes `cast_vote(user, corpus_pk, vote_type, *, session_key, ip_address, request)` and `remove_vote(...)`. Both call paths IDOR-gate via `BaseService.get_or_none` (the unified "Corpus not found or you do not have permission" message), block self-vote on the authenticated branch, and persist the anonymous branch through `session_key` with a salted-SHA256 `ip_hash` for audit only (NOT part of the unique constraint, so co-located voters behind a shared NAT can still vote). Wired into `corpuses/services/__init__.py` and re-exported.
  - **GraphQL mutations (`config/graphql/voting_mutations.py`)** — new `voteCorpus(corpusId, voteType)` and `removeCorpusVote(corpusId)` mutations. **Not decorated with `@login_required`** — anonymous voting is the whole point. The mutation lazily calls `session.save()` on first cast to materialise a `session_key` so subsequent votes from the same browser dedup correctly. Both mutations return the refreshed corpus row (with updated counts + `myVote`).
  - **GraphQL CorpusType (`config/graphql/corpus_types.py`)** — new `myVote` resolver returns `"UPVOTE" | "DOWNVOTE" | null` for the calling viewer, looking up by authenticated user when present and falling back to the request session key for anonymous viewers. The three new denormalized columns auto-expose as `upvoteCount` / `downvoteCount` / `score` via `DjangoObjectType` field auto-discovery.
  - **GraphQL CorpusFilter (`config/graphql/filters.py`)** — new `order_by` `OrderingFilter` exposing `top` / `-top`, `created`, `-modified`, `title` GraphQL values. Top sort excludes `is_personal=True` corpora on the discovery surface (anywhere `mine` is not set) — personal "My Documents" corpora are private singletons that shouldn't cross-rank against shared content. The "My Corpuses" tab (`mine=True`) keeps personal corpora visible under every sort so users see all of their own content.
  - **Frontend (`frontend/src/components/corpuses/CorpusVoteWidget.tsx`)** — new lightweight up/down + score pill overlayed on the top-left of each corpus card. Optimistic update + Apollo `cache.modify` keep every corpus card on screen in sync after a vote without a refetch. Hidden on the user's own personal corpus.
  - **Frontend sort UI (`frontend/src/components/corpuses/CorpusListView.tsx` + `frontend/src/views/Corpuses.tsx`)** — new "Sort by" dropdown next to the search box (Newest / Oldest / Recently updated / Top / Most downvoted / Title) wired to the `orderBy` GraphQL argument. The MCP share button remains top-right; vote widget is top-left.
  - **Frontend mutations + types (`frontend/src/graphql/mutations.ts`, `frontend/src/graphql/queries.ts`, `frontend/src/types/graphql-api.ts`)** — new `UPVOTE_CORPUS` / `DOWNVOTE_CORPUS` / `REMOVE_CORPUS_VOTE` mutations and `upvoteCount` / `downvoteCount` / `score` / `myVote` on `RawCorpusType`. `GET_CORPUSES` extended with `orderBy: String` and the four new fields.
  - **Tests (`opencontractserver/tests/test_corpus_voting.py`)** — `CorpusVoteServiceTests` covers auth + anonymous voting, self-vote block, IDOR-safe denial, vote switching, anonymous session dedup, and the "logged-in user cannot claim an anonymous vote" invariant. `CorpusVoteCountDenormalizationTests` (TransactionTestCase) pins the signal-driven count refresh. `CorpusVoteUniqueConstraintTests` pins both partial UNIQUE indexes. `CorpusVoteGraphQLTests` exercises the mutations end-to-end through the graphene schema (authenticated path, anonymous session-bootstrap path, self-vote block, IDOR-safe denial on malformed corpus id, and the `myVote` resolver), plus a `CaptureQueriesContext` regression test that pins the `my_vote` annotation contract — no per-card `SELECT FROM corpuses_corpusvote` queries are issued for a list page (N+1 fix from PR #1789 review).
  - **PR review fixes** (post-feature, on top of the bullets above): (a) `CorpusType.get_queryset` now annotates `_viewer_vote` via a single `Subquery` per page so `resolve_my_vote` reads the annotation instead of firing one `CorpusVote` query per card; (b) `voteCorpus` / `removeCorpusVote` route the post-vote corpus refresh through `BaseService.get_or_none` instead of a direct `Corpus.objects.get`, keeping CLAUDE.md rule 7 satisfied; the bare `assert result.value is not None` is replaced with a proper error-return branch so `python -O` cannot turn a corner-case service bug into a 500; (c) the default corpus-list sort changed from empty string ("default" = oldest-first per `Corpus.Meta.ordering`) to `-created` so the "Newest first" label is accurate; (d) `CorpusVoteWidget.stop` no longer calls `preventDefault` (the click handler only needs `stopPropagation` to keep the card's navigation onClick from firing); (e) `_client_ip` docstring spells out the `X-Forwarded-For` trust posture (audit-only — no abuse decisions); (f) `CorpusVote` model docstring documents that anonymous rows persist after `clearsessions` reaps the matching `Session` row (intentional, Reddit-style; counts stay accurate because the signal recomputes from scratch). `test_corpus_services_package.py` extended to expect `CorpusVoteService` in the `services/__all__` re-export.
  - **PR #1789 follow-up fixes (issue #1792)** — surgical post-merge cleanup:
    - **Race-tight `remove_vote`** (`opencontractserver/corpuses/services/votes.py`). Replaced the `count()`-then-`delete()` two-query sequence with `qs.delete()`'s returned `(rows_deleted, _)` tuple — closes the tiny window where a concurrent double-click could delete the row between the `count` and the `delete` and have us report a stale `True`.
    - **DRY: shared `is_authenticated_user` helper** (`opencontractserver/utils/auth.py`). The triple-guard auth check (`user is not None and getattr(user, "is_authenticated", False) and not getattr(user, "is_anonymous", True)`) was repeated verbatim in `config/graphql/corpus_types.py`, `config/graphql/voting_mutations.py`, and `opencontractserver/corpuses/services/votes.py`. Extracted into a single `is_authenticated_user(user)` helper that all three modules now import, so the auth/anon branch contract for the voting surface lives in one place.
    - **`stop` added to `useCallback` deps** (`frontend/src/components/corpuses/CorpusVoteWidget.tsx`). `handleUpvote` / `handleDownvote` call `stop(event)` but the prior dep arrays omitted `stop` — `stop` itself is stable (empty deps) so behavior was correct, but `react-hooks/exhaustive-deps` flags this and it misleads future readers.
    - **`RawCorpusType` vote count fields are non-optional** (`frontend/src/types/graphql-api.ts`). `upvoteCount` / `downvoteCount` / `score` always have a backend default of `0` and the columns can never be absent on a `CorpusType` row — they were typed as optional anyway, forcing null-guards everywhere downstream. Tightened to required. `myVote` stays optional (it genuinely is `null` for any viewer who hasn't voted).
    - **`get_user_vote_type` SECURITY docstring** (`opencontractserver/corpuses/services/votes.py`). Spells out the contract that the classmethod takes a `Corpus` object directly and does NOT run a READ gate — callers must already have resolved the corpus through a visibility-checked path. All in-tree call sites do so (the GraphQL `resolve_my_vote` reaches the corpus through `CorpusType.get_queryset` which gates via `BaseService.filter_visible_qs`); the docstring makes the contract explicit so future internal callers can't accidentally leak vote identity for unreadable corpora.
    - **SortSelect Safari prefix** (`frontend/src/components/corpuses/CorpusListView.tsx`). Added `-webkit-appearance: none` alongside the existing `appearance: none` so Safari < 15 actually suppresses the native dropdown affordance rather than rendering its own chevron over our custom one.
    - **Test: `resolve_my_vote` fallback path** (`opencontractserver/tests/test_corpus_voting.py`). New `test_my_vote_resolver_uses_service_fallback_when_not_annotated` exercises the per-row `CorpusVoteService.get_user_vote_type` path that fires when the corpus instance was loaded outside `CorpusType.get_queryset` (so the `_viewer_vote` annotation isn't attached). Complements the existing `CaptureQueriesContext` test that pins the fast path.

### Fixed

- **Corpus folder browser: 10-20 s load time fixed by collapsing the per-folder query fan-out in `corpusFolders`** (`opencontractserver/corpuses/services/folders.py`, `config/graphql/corpus_types.py`, `config/graphql/corpus_queries.py`). Browsing a corpus folder took 10-20 seconds against a corpus with ~100 folders because every field requested by `frontend/src/graphql/queries/folders.ts::GET_CORPUS_FOLDERS` resolved through a per-folder resolver on `CorpusFolderType`:

  - `path` ran `CorpusFolder.get_path()` → `self.ancestors(include_self=True)` — one recursive CTE per folder.
  - `documentCount` ran `CorpusFolder.get_document_count()` — one `DocumentPath.objects.count()` per folder.
  - `descendantDocumentCount` ran `CorpusFolder.get_descendant_document_count()` → `self.descendants(include_self=True)` + `.count()` — one recursive CTE plus one `COUNT` per folder.
  - `myPermissions` fell through `AnnotatePermissionsForReadMixin.resolve_my_permissions` to two `.filter()` queries per folder against the always-empty `corpusfolderuserobjectpermission_set` and `corpusfoldergroupobjectpermission_set` relations (`CorpusFolder` rows never carry guardian permission rows — see model docstring at `opencontractserver/corpuses/models.py`).
  - `isPublished` ran `get_groups_with_perms(folder)` + `.filter().count()` per folder for the same always-empty guardian tables.

  Net: ~6N + 1 SQL round-trips on the folder-list view, half of them recursive CTEs against `corpuses_corpusfolder`. With `fetchPolicy: "cache-and-network"` set on the frontend (`frontend/src/components/documents/CorpusDocumentCards.tsx:159`), every folder switch re-paid the full cost.

  Fix:

  - New `FolderCRUDService.get_visible_folders_with_aggregates(user, corpus_id, *, request)` materialises the folder list and attaches `_path`, `_doc_count`, and `_descendant_doc_count` to each instance using one `GROUP BY folder_id` query for direct document counts (mirrors the existing batched pattern in `get_folder_tree()`), an in-memory `parent_id` walk with memoisation for paths (no ancestor CTE), and a post-order DFS over the in-memory adjacency map for descendant counts (no descendant CTE).
  - `CorpusFolderType.resolve_path` / `resolve_document_count` / `resolve_descendant_document_count` prefer the pre-attached attribute and fall back to the per-folder model methods so single-folder reads (`corpusFolder(id:)`) keep working unchanged.
  - `CorpusFolderType.resolve_my_permissions` is overridden to delegate to the parent corpus's resolver (folders inherit corpus permissions exclusively) and memoise the result per `(corpus_id, user_id)` on `info.context`, then translate `_corpus` permission strings to `_corpusfolder` so the API contract stays consistent.
  - `CorpusFolderType.resolve_is_published` is overridden to return `False` directly, skipping the per-folder `get_groups_with_perms` lookup.
  - `resolve_corpus_folders` (`config/graphql/corpus_queries.py`) now calls `get_visible_folders_with_aggregates`.

  Total query count is now ~5-6 regardless of folder count (folder list + GROUP BY counts + the corpus-level permission lookup), pinned by the new `CorpusFoldersQueryCountTest` in `opencontractserver/tests/test_query_resolvers.py` which seeds two folder trees of very different sizes (6 vs 155 folders) and asserts the query-count delta stays small. A second test pins the aggregate values themselves (path, direct count, descendant count) so the in-memory roll-up stays equivalent to the per-folder model methods.

- **DocumentKnowledgeBase desktop rail: hover tooltip collapsed into a "tiny black dot"** (`frontend/src/components/knowledge_base/document/styled/SidebarTabs.tsx`). Regression from the unified-rail consolidation in #1764. `SidebarTab` declared both `&[data-tooltip]::after` (hover tooltip) and `&::after` (a 3px active-state indicator line) on the same pseudo-element; CSS resolves per-property, so the indicator's `width: 3px; height: 60%;` still applied to the tooltip — rendering it as a 3px-wide dark pill with overflowing text. Fix: dropped the redundant `&::after` indicator rule (active state is already conveyed by the blue gradient background and hover glow).

- **DocumentKnowledgeBase Settings popover floated far above the gear trigger in unified rail** (`frontend/src/components/knowledge_base/document/FloatingDocumentControls.tsx`). Regression from the unified-rail consolidation in #1764. `ControlPanel` was anchored `bottom: calc(56px + 1rem)` from its `ControlsContainer`, which assumed `flex-direction: column-reverse` (Settings at the bottom). The unified `RightEdgeRail` uses `flex-direction: column`, so the anchor was measured past the other action buttons and left the popover ~280px above the Settings trigger. Fix: `ControlPanel` accepts a `$bare` prop and switches to `right: calc(100% + 8px); top: 0;` in bare mode, anchoring the popover to the left of the rail aligned with the Settings button. Regression coverage in `frontend/tests/FloatingDocumentControls.ct.tsx` (popover-position + tooltip-pill assertions); `FloatingDocumentControlsTestWrapper` gained a `bareContainer` prop that mounts the controls inside a viewport-anchored rail so bare-mode geometry is exercised.

- **Corpus vote pill no longer overlaps the avatar icon on the corpus list** (`frontend/src/components/corpuses/CorpusVoteWidget.tsx`, `frontend/src/components/corpuses/CorpusListView.tsx`). The vote pill was an absolutely-positioned overlay at `top: 12px; left: 12px` inside `CardWrapper` and the horizontal `up | score | down` pill (≈70px wide) sat on top of the 48×48 corpus avatar at the top-left of every `CollectionCard` — blocking the icon and any subsequent badge in that corner. Fix: introduced an `orientation: "horizontal" | "vertical"` prop on `CorpusVoteWidget` (default `"horizontal"` preserves the existing flush, inline use); `vertical` flips `Pill` to `flex-direction: column` with `padding: 4px 2px` and gives `Score` `padding: 2px 0` + `line-height: 1` so chevron-up / score / chevron-down stack as a Reddit-style rail. `CardWrapper` in `CorpusListView` now reserves a fixed left rail (`VOTE_RAIL_WIDTH = 36px`, `30px < 480px viewport`) via `padding-left` regardless of whether the row has a vote widget, so personal corpora (where the widget is suppressed because self-vote is disallowed) stay vertically aligned with the rest of the list. `VoteWidgetOverlay` is repositioned to fill that rail (`top: 50%; left: 0; transform: translateY(-50%)`, `width: VOTE_RAIL_WIDTH`, flex-centred) so the vertical pill sits in the rail and never overlaps the avatar. Existing `CorpusVoteWidget.ct.tsx` coverage continues to exercise the default horizontal orientation; the new vertical orientation reuses the same accessible markup (`aria-label="Upvote corpus" / "Downvote corpus" / "Score: N"`).

- **Anonymous corpus voting 403'd every subsequent GraphQL POST with "CSRF verification failed"** (`config/graphql/security.py`). The voting flow added in #1789 calls `_ensure_session_key` (in `config/graphql/voting_mutations.py:77`) which forces `session.modified = True; session.save()` so the first anonymous vote materialises a `sessionid` cookie to dedupe later votes. The existing `conditional_csrf_exempt` gate then treated **any** session cookie as a reason to invoke `CsrfViewMiddleware.process_view`, so the next anonymous POST to `/graphql/` (a second vote, a refetch, or a query firing during the brief Auth0-callback window) was rejected with `CSRF cookie not set` — the SPA has no CSRF token flow because it normally rides on Bearer auth. `errorLink.ts` interpreted that 403 as `Your session has expired`, cleared the auth state, and the cascade then re-fired the `GET_ME` / `GET_CORPUSES` queries against the same cookie, producing the "Unable to fetch corpuses" / "Could not get user details from server" toasts. Fix: `conditional_csrf_exempt` now skips CSRF when the request's session has no `_auth_user_id` (Django's `contrib.auth.SESSION_KEY`) — an empty/anonymous session has no logged-in identity for an attacker to ride on, so the CSRF check has no security value. Session-authenticated requests (admin, the legacy `tokenAuth` mutation that calls `django.login`) still go through the standard CSRF flow. New helper `_session_is_authenticated` encapsulates the check defensively (handles missing `request.session`, exotic backends). Test updates: existing `test_session_auth_without_csrf_rejected` / `test_empty_authorization_with_session_still_enforces_csrf` and the `_post_with_session` helper now attach a session with `_auth_user_id` so they actually exercise the session-authenticated branch; new tests in `opencontractserver/tests/test_security_hardening.py` (`test_anonymous_session_cookie_bypasses_csrf`, `test_session_without_session_attr_bypasses_csrf`) pin the new bypass branch. New end-to-end regression in `opencontractserver/tests/test_corpus_voting.py` (`AnonymousVotingMiddlewareIntegrationTests.test_repeated_anonymous_votes_do_not_403_on_csrf`) drives `/graphql/` through the real middleware stack via `django.test.Client(enforce_csrf_checks=True)` to verify two consecutive anonymous votes from the same browser both succeed.

- **Mobile annotations tray: Content Types dropdown menu clipped by the collapsed "Filter & sort" panel** (`frontend/src/components/knowledge_base/document/unified_feed/SidebarControlBar.tsx`). On the mobile side-tray Annotations view, opening the "Content Types" multi-select after expanding "Filter & sort" showed only the top half of the first option (e.g. the green "Notes ✓" pill) — the rest of the menu was clipped by the parent `CompactExpandPanel`'s `overflow: hidden`. That `overflow: hidden` is required for Framer Motion's `height: 0 → "auto"` collapse animation (otherwise content would leak during the transition), but it also clips the `position: absolute` `DropdownMenu` that drops down below the Content Types trigger inside the panel. The Sort dropdown didn't have this problem because the `@os-legal/ui` `Dropdown` renders into a portal. Fix: track a new `compactPanelSettled` state and drive `overflow` via inline style — `hidden` while the open/close animation runs, `visible` once `onAnimationComplete` fires with `height: "auto"`. `onAnimationStart` flips the flag back to `false` at the start of the exit animation so the collapse still clips cleanly. The `overflow: hidden` rule is removed from the styled `CompactExpandPanel` (now an empty styled wrapper) and managed entirely through the inline style.

- **Tool approval modal overflows the viewport when arguments are large** (`frontend/src/components/chat/ApprovalDialog.tsx`, `frontend/src/components/corpuses/CorpusChat.tsx`, `frontend/src/components/knowledge_base/document/right_tray/ChatTray.tsx`). Both approval popups for pending agent tool calls rendered the tool's JSON `arguments` inside a `<pre style={{ whiteSpace: "pre-wrap" }}>` nested in an inner modal that had only `maxWidth: 500px` — no `maxHeight`, no scroll container. When a tool call carried a multi-thousand-character payload (e.g. `update_corpus_description` with a long `new_content` string), the inner modal grew unbounded, the Approve / Reject buttons were pushed off-screen, and the user couldn't act on the request. Fix: inner modal is now a `display: flex; flex-direction: column` box capped at `maxHeight: calc(100% - 2rem)` (accounts for the backdrop's 1rem padding on each side) with `boxSizing: border-box`; the body section (`flex: 1; minHeight: 0; overflowY: auto`) scrolls; header and footer (`flexShrink: 0`) stay pinned so the action buttons remain visible. The `<pre>` gains `overflowWrap: anywhere` so a single unbroken token can't blow out the width. The dialog now exposes proper `role="dialog"` + `aria-modal="true"` + `aria-labelledby` for screen readers. New regression test `ApprovalDialog › constrains modal height and scrolls body when args are very long` (`frontend/tests/ApprovalDialog.ct.tsx`) mounts the modal with ~10k-char `new_content` arguments and asserts both buttons stay within the viewport's `boundingBox`.

- **Sidebar discussion-view: badge tooltip no longer clipped + reworked top header layout** (`frontend/src/components/badges/MessageBadges.tsx`, `frontend/src/components/threads/ThreadDetail.tsx`). The `Conversationalist`-style badge tooltip in the right-sidebar discussion view was getting clipped by the `overflow: hidden` chain that the sidebar shell enforces (`ContextSidebarContainer`, `DocumentDiscussionsContent` `Container`, parent `FlexColumnPanel`) — the tooltip was a child `position: absolute` element with no way to escape its scroll/clip parent. `MessageBadges.tsx` now renders `TooltipPopup` through `ReactDOM.createPortal(..., document.body)` with `position: fixed` coordinates derived from the badge's `getBoundingClientRect()`, including viewport-edge clamping and a flip-below-the-badge fallback when there isn't room above (avoids the same clip at the top of any scroll container). The arrow stays anchored under the badge centre via a `--arrow-offset` CSS variable so a clamped tooltip still points at the right element. The portal layer uses `POPOVER_Z_INDEX` (100002) so it outranks context menus and modal overlays, hides itself when the underlying badge scrolls fully out of the viewport (no orphan tooltip floating over the rest of the UI), and captures both `top` and `bottom` inside the effect so the JSX never re-reads `window.innerHeight` at render time (no 1-frame misalignment on resize). Bounds come from shared / module-level constants (`POPOVER_GAP`, `CONTEXT_MENU_VIEWPORT_PADDING`, `TOOLTIP_HEIGHT_EST`, `TOOLTIP_MAX_WIDTH`) rather than magic numbers buried inside the effect. While in the same view, `ThreadDetail.tsx`'s compact sidebar header was reworked so the type badge, title, status pills, document context link, and meta row each get their own line instead of one wrap-row that flowed into a cramped jumble; the compact back-arrow is unconditionally rendered in compact mode (the `customOnBack` prop is the back-navigation handler the parent passes in, not a signal that the parent renders its own button — `ThreadDetailWithContext` delegates click handling without showing its own back affordance, so suppressing the button on `customOnBack` would strand the user with no way to return to the list, regressing the `threads-discussions.spec.ts` e2e flow). New Playwright CT coverage in `frontend/tests/threads/ThreadDetailSidebarLayout.ct.tsx` (+ `ThreadDetailSidebarLayoutHarness.tsx`, `BadgeClipHarness.tsx`) pins both the new header layout and the tooltip-escapes-clip behaviour, and emits `discussions--sidebar-thread-detail--layout.png` and `discussions--sidebar-thread-detail--tooltip-portal.png` for docs.

- **Sidebar discussion-view: badge tooltip no longer clipped + reworked top header layout** (`frontend/src/components/badges/MessageBadges.tsx`, `frontend/src/components/threads/ThreadDetail.tsx`). The `Conversationalist`-style badge tooltip in the right-sidebar discussion view was getting clipped by the `overflow: hidden` chain that the sidebar shell enforces (`ContextSidebarContainer`, `DocumentDiscussionsContent` `Container`, parent `FlexColumnPanel`) — the tooltip was a child `position: absolute` element with no way to escape its scroll/clip parent. `MessageBadges.tsx` now renders `TooltipPopup` through `ReactDOM.createPortal(..., document.body)` with `position: fixed` coordinates derived from the badge's `getBoundingClientRect()`, including viewport-edge clamping and a flip-below-the-badge fallback when there isn't room above (avoids the same clip at the top of any scroll container). The arrow stays anchored under the badge centre via a `--arrow-offset` CSS variable so a clamped tooltip still points at the right element. The portal layer uses `POPOVER_Z_INDEX` (100002) so it outranks context menus and modal overlays, hides itself when the underlying badge scrolls fully out of the viewport (no orphan tooltip floating over the rest of the UI), and captures both `top` and `bottom` inside the effect so the JSX never re-reads `window.innerHeight` at render time (no 1-frame misalignment on resize). Bounds come from shared / module-level constants (`POPOVER_GAP`, `CONTEXT_MENU_VIEWPORT_PADDING`, `TOOLTIP_HEIGHT_EST`, `TOOLTIP_MAX_WIDTH`) rather than magic numbers buried inside the effect. While in the same view, `ThreadDetail.tsx`'s compact sidebar header was reworked so the type badge, title, status pills, document context link, and meta row each get their own line instead of one wrap-row that flowed into a cramped jumble; the compact back-arrow is unconditionally rendered in compact mode (the `customOnBack` prop is the back-navigation handler the parent passes in, not a signal that the parent renders its own button — `ThreadDetailWithContext` delegates click handling without showing its own back affordance, so suppressing the button on `customOnBack` would strand the user with no way to return to the list, regressing the `threads-discussions.spec.ts` e2e flow). New Playwright CT coverage in `frontend/tests/threads/ThreadDetailSidebarLayout.ct.tsx` (+ `ThreadDetailSidebarLayoutHarness.tsx`, `BadgeClipHarness.tsx`) pins both the new header layout and the tooltip-escapes-clip behaviour, and emits `discussions--sidebar-thread-detail--layout.png` and `discussions--sidebar-thread-detail--tooltip-portal.png` for docs.

- **Loading splash rebrand + mobile coverage** (follow-up to PR #1781). The `ModernLoadingDisplay` widget (used by `AuthGate` and every route-level loader) still showed the legacy `os_legal_FullColor.png` and the string `Loading OpenContracts`; on mobile it also bled through to the underlying app chrome.
  - **Brand swap.** `frontend/src/components/widgets/ModernLoadingDisplay.tsx` now renders the `CiteMark` SVG (slate brackets + teal node) at 60px in place of the PNG, and the default message is `Loading cite`. The unreachable lucide-icon fallback path (`useOsLogo` was hard-defaulted to `true` with no caller overriding it) and unused `rotate` keyframe were removed. `AuthGate.tsx` now passes `message="Initializing cite"` (test assertion updated in `AuthGate.test.tsx`). Asset `frontend/src/assets/images/os_legal_FullColor.png` deleted.
  - **Mobile coverage bug.** The `Container` was `position: fixed` centered on screen but only painted its paper-tinted backdrop when callers passed `fullScreen={true}` — and only `AuthGate` ever did. Every route loader (and every panel-embedded loader wrapped in a `LoadingContainer`) therefore showed a tiny centered pill with the navigation chrome bleeding through behind it, most visibly on mobile. Reworked the styling so the overlay (`position: fixed; inset: 0`, paper-tinted gradient, blur + opaque `@supports` fallback) is the default; the rare panel-embedded callers opt into a new `inline` mode (`position: relative; width: 100%; no background`). Dropped the `fullScreen` prop entirely.
  - **Caller migration.** `AuthGate.tsx` (removes the now-redundant `fullScreen={true}`); panel-embedded callers `frontend/src/components/threads/ThreadList.tsx`, `ThreadDetail.tsx`, `frontend/src/views/DiscoverSearchResults.tsx`, `GlobalDiscussions.tsx` add `inline`. The dead `frontend/src/components/search/SearchResults.tsx` (no imports anywhere) is untouched and still compiles since it never passed `fullScreen`.
  - **Out of scope.** A broader rebrand sweep of `OpenContracts`/`Open Contracts` strings in product UI, MCP/agent identity, and GraphQL types is still pending (the same scope the v3 surface rebrand flagged as out of scope in PR #1781).

### Changed

- **Approval modal DRY pass: collapsed the two ~90%-duplicated approval popups into a single `ApprovalDialog`** (`frontend/src/components/chat/ApprovalDialog.tsx`). The corpus-chat `ApprovalModal` (`frontend/src/components/corpuses/corpus_chat/ApprovalModal.tsx`) and the document-tray `ApprovalOverlay` (`frontend/src/components/knowledge_base/document/right_tray/ApprovalOverlay.tsx`) were near-identical copy-paste — same markup, same flex column scroll structure, with only cosmetic divergence (header icon `AlertCircle` vs `AlertTriangle`, Reject icon `X` vs `XCircle`, unicode `✕` vs lucide `<X>` for the close button, slightly different prop naming). They have been replaced with one shared `ApprovalDialog` component in `frontend/src/components/chat/ApprovalDialog.tsx` (the chat-domain sibling of the existing `RequestingAgentAttribution`). `CorpusChat` (`frontend/src/components/corpuses/CorpusChat.tsx`) and `ChatTray` (`frontend/src/components/knowledge_base/document/right_tray/ChatTray.tsx`) now render `ApprovalDialog` conditionally inside an `AnimatePresence` (`{showApprovalModal && pendingApproval && <ApprovalDialog … />}`) so framer-motion's `exit` variants actually fire on dismiss — the previous always-mounted form returned `null` internally and `AnimatePresence` never observed an unmount. `ApprovalDialog`'s API consequently dropped the redundant `show` prop and made `pendingApproval` non-nullable. The old `ApprovalModal.tsx` file is deleted; `ApprovalOverlay.tsx` is reduced to just the small `ReopenApprovalButton` that's only relevant to the document tray. Standardized icons (AlertTriangle for the header, XCircle for Reject, lucide `<X>` for close with an `aria-label="Close approval modal"`) and removed the redundant `wordBreak: break-word` (superseded by `overflowWrap: anywhere`) per Claude review on PR #1787. The screenshot tag stays `chat--approval-modal--pending` per CLAUDE.md's "filename IS the contract" rule, so existing docs referencing that path keep resolving. `PendingApproval` re-export pointers in `useChatSendHandlers` / `useChatStreamHandlers` / `useChatAgentMessageHandler` and their unit tests repoint from `./ApprovalOverlay` to the canonical `components/chat/types`. The CT test file is renamed `frontend/tests/ApprovalModal.ct.tsx` → `frontend/tests/ApprovalDialog.ct.tsx`. All 6 CT tests + 58 hook unit tests pass.

- **Breaking — `ModernLoadingDisplay.fullScreen` prop removed.** The `fullScreen?: boolean` prop on `frontend/src/components/widgets/ModernLoadingDisplay.tsx` is gone; the overlay (`position: fixed; inset: 0`, paper-tinted gradient + blur) is now the default. Panel-embedded callers opt out with the new `inline` boolean prop. Downstream forks or plugin builds that pass `fullScreen={true}` will get a TypeScript error — drop the prop. Anything that passed `fullScreen={false}` (none in-tree) should switch to `inline`. Component is also now accessible (`role="status"`, `aria-live="polite"`, `aria-busy="true"`) and reads the mobile shrink breakpoint from `SMALL_MOBILE_BREAKPOINT` in `frontend/src/assets/configurations/constants.ts` instead of an inline `480px` magic number.

- **Social-card / OG worker rebranded to [cite]** (`cloudflare-og-worker/`, `frontend/src/components/seo/MetaTags.tsx`, `docs/architecture/social-media-previews.md`). The Cloudflare Worker that intercepts social-media crawler requests for `/c/*`, `/d/*`, `/e/*` deep-links no longer serves "OpenContracts" branding in the OG/Twitter meta tags it generates. User-visible strings (`og:site_name`, page `<title>`, "View on **_" anchor, "Document on _**" fallback description, and the generic fallback card's title/description) now read `cite` and `cite.opensource.legal`, and the generic-card title + description switch from the bare brand string to the cite tagline + summary that the frontend `index.html` already uses. Internal identifiers also renamed: the Worker's `name` flips from `opencontracts-og` → `cite-og` in `wrangler.toml`, the npm package from `opencontracts-og-worker` → `cite-og-worker`, and the package description rewords to "Cloudflare Worker for cite.opensource.legal social media link previews"; this deploys as a _new_ worker on the next `npm run deploy:production`, so the legacy `opencontracts-og` worker will have to be deleted from the Cloudflare dashboard once the new one is serving. **Routes extended to cover both domains** (`wrangler.toml`): three new `cite.opensource.legal/{c,d,e}/*` patterns were added alongside the existing `contracts.opensource.legal/{c,d,e}/*` patterns, and `SITE_URL` / `API_URL` / `OG_IMAGE_BASE` were repointed from `https://contracts.opensource.legal` to `https://cite.opensource.legal` (the new canonical) — without this the Worker would have missed every crawler request hitting the new canonical domain that `frontend/index.html` already advertises in its `og:url`. The legacy `contracts.opensource.legal` routes stay live so deep-links shared during the 90-day grace period still get rich previews. `frontend/src/components/seo/MetaTags.tsx`'s `pageTitle`/`pageDescription` fallbacks (only reached when an entity supplies no title/description) swap from "OpenContracts" / "Legal document analysis platform" to the same cite tagline + summary. Worker unit tests (`cloudflare-og-worker/src/html.test.ts`) and the architecture doc updated to pin the new strings; all 93 worker tests pass and `tsc --noEmit` is clean.

- **Discovery endpoints rebranded to [cite]** (`opencontractserver/discovery/views.py`). The agent/crawler-facing static surfaces (`/llms.txt`, `/llms-full.txt`, `/.well-known/mcp.json`) now identify as `cite` with the citation-graph framing from the README, instead of describing the platform as "OpenContracts … document analytics platform". `Source code` / `Project home` links repoint from `Open-Source-Legal/OpenContracts` + `contracts.opensource.legal` to `Open-Source-Legal/cite` + `cite.opensource.legal`. In `/.well-known/mcp.json` the global server key is renamed from `opencontracts` to `cite` and scoped entries from `opencontracts-{slug}` to `cite-{slug}` — cosmetic for clients (they pick their own local label) but aligns the discovery document with the new brand. Each text endpoint carries a one-line rebrand note ("Released as OpenContracts since 2019; rebranded as cite for the v3 release line") so crawlers indexing the old name still get a match. `robots.txt`, `sitemap.xml`, `/.well-known/oauth-protected-resource`, and `/api/search/` were already brand-free and untouched. Tests in `opencontractserver/discovery/tests/test_discovery_views.py` updated to pin the new strings.

### Added

- **Native PDF link annotations are now clickable in the viewer** (PR #1788). New `NativeLinkLayer` (`frontend/src/components/annotator/renderers/pdf/NativeLinkLayer.tsx`) reads `page.getAnnotations()` and renders a transparent `<a target="_blank" rel="noopener noreferrer">` per safe-URL Link annotation, stacked above `SelectionLayer` via the new `Z_INDEX.PDF_NATIVE_LINK_LAYER` constant so external URLs embedded in PDFs are reachable by click, right-click, and keyboard focus. Protocol allowlist is `http: / https: / mailto:`; internal `dest`-based PDF navigation is not yet wired. Coverage in `NativeLinkLayer.test.tsx`.

- **v3 surface rebrand: OpenContracts → [cite]** (PR #1781). Ships the public-surface rebrand from OpenContracts to [cite] as part of the v3 milestone. Scope is intentionally narrow: brand assets + landing + /about + chrome (NavMenu, Footer, Login, CookieConsent). The deeper product UI strings, repo name, README references, MCP/agent identity strings, GraphQL types, and the loading splash still use the OpenContracts name and are explicitly out of scope.
  - **Brand components.** New `frontend/src/components/brand/CiteMark.tsx` (the bracketed teal node icon, with scaled stroke widths and auto-aria-hidden when used decoratively) and `CiteWordmark.tsx` (the `[cite]` wordmark as inline SVG with a Source Serif fallback chain). Wired into `Login`, `NavMenu`, `Footer`, `CallToAction`, `GetStarted`, `About`, and `CookieConsent`.
  - **/about page.** New `frontend/src/views/About.tsx` + route, plus a JSON-driven landing-content variant system (`frontend/src/config/landingContent/`) with `default` and `public-record` packs selectable via `REACT_APP_LANDING_VARIANT`. Hero, GetStarted, CallToAction, About copy all source from the active variant.
  - **Chrome + meta.** `index.html` (title, description, og tags, JSON-LD), `manifest.json` (cite name + theme color `#FAFAF7`), favicons (SVG), Source Serif 4 + Inter from Google Fonts. NavMenu's brand slot, Footer lockup, and Login card now show the cite wordmark/lockup.
  - **PWA + OG assets.** New `frontend/public/cite-192.png`, `cite-512.png`, and `cite-maskable.png` (with the 80% safe-area inset on the brand background) wired into `manifest.json` with the correct `purpose: any` / `purpose: maskable` split. `apple-touch-icon` now points to `cite-192.png`. Open Graph / Twitter card (`OpenContractsScreenshot.png` — legacy filename retained so the existing `<meta og:image>` reference resolves) regenerated as a cite-branded card. All four are produced by `frontend/scripts/generate-brand-pngs.js`, which renders the icon-mark SVG at the target sizes through headless Chromium so we don't pull in a librsvg / sharp / inkscape dependency.
  - **Tests + screenshots.** New Playwright CT tests for `About`, `Login` (cite branding), `Footer`, `NavMenu` branding, `CookieConsent` rebrand, plus updated `landing-components.ct.tsx` and a Vitest suite (`landingContent.test.ts`) that pins the variant registry contract. Doc + release screenshots refreshed.
  - **Out of scope.** A codebase-wide string scrub of `OpenContracts`/`Open Contracts` (repo name, README, MCP/agent identity strings, GraphQL types, in-product loading splash, etc.).

### Fixed

- **Phase 5 follow-ups — correctness + lint hygiene on the PR #1770 service-layer migration** (issue #1779). Five small post-merge items surfaced by code review on PR #1770. No behaviour change for happy paths; only error-handling polish and mypy/lint hygiene.
  - **`config/graphql/search_queries.py`** (`resolve_search_agents_for_mention`): the new inline `corpus_pk = int(from_global_id(corpus_id)[1]) if corpus_id else None` was unguarded — a malformed `corpus_id` (bad base64 / non-numeric body) raised an unhandled `ValueError` and surfaced as a 500. Now wrapped in `try/except (ValueError, TypeError, UnicodeDecodeError)` with `corpus_pk` falling back to `None`, mirroring the IDOR-safe decode pattern already used by `resolve_search_notes_for_mention`. New regression test `AgentMentionCorpusScopingTestCase.test_agent_search_with_malformed_corpus_id_degrades_gracefully` (`opencontractserver/tests/test_mention_permissions.py`) pins the contract for three flavours of malformed id.
  - **`opencontractserver/analyzer/services/analysis_lifecycle_service.py`** (`AnalysisLifecycleService.delete_analysis`): the initial visibility lookup called the module-level `get_for_user_or_none(Analysis, pk, user)` helper instead of `cls.get_or_none(Analysis, pk, user, request=request)`. The bare helper has no `request=` kwarg, silently bypassing the Tier-2 permission cache for the lookup half of the two-step check while the subsequent `user_can(..., request=request)` call did engage it. Now routes through `BaseService.get_or_none` so both halves share the same cached permission entry. The now-unused `get_for_user_or_none` import is removed. New regression test `TestAnalysisLifecycleServiceBehavioral.test_delete_analysis_threads_request_into_initial_lookup` (`opencontractserver/tests/test_service_layer_phase5_behavioral.py`) patches `BaseService`'s local `get_for_user_or_none` reference and asserts the `request=` kwarg is threaded through.
  - **`config/graphql/worker_mutations.py` + `worker_queries.py`**: replaced the four `assert result.value is not None  # narrowed by result.ok invariant` lines with `typing.cast` narrowing. `assert` is stripped under `python -O`; `cast` is mypy-only and survives optimised builds. `worker_queries.py`'s `resolve_corpus_access_tokens` uses a bare `cast("QuerySet", result.value)` (not parameterised by row type) because the service dynamically annotates `_pending` / `_completed` / `_failed` upload counts on each row — those are not real fields on `CorpusAccessToken`, so a typed `QuerySet[CorpusAccessToken]` cast would make the annotated-attribute access fail mypy.
  - **`opencontractserver/feedback/services/user_feedback_service.py`** (`UserFeedbackService._set_feedback`): added a docstring note pinning the intentional `comment="" → preserve old comment` semantics inherited verbatim from the pre-relocation `ApproveAnnotation` / `RejectAnnotation` mutations. No code change — purely documents the intent so a future caller doesn't reach for `comment=""` expecting it to clear the field.

### Changed

- **Service-layer centralization Phase 6 — follow-up fixes from PR #1780 code review** (issue #1782). Five targeted corrections on top of #1720's mechanical refactor; the migration itself is unchanged.

  - **New `BaseService.filter_visible_qs(queryset, user, \*, request=None, **kwargs)`** (`opencontractserver/shared/services/base.py`). The Phase-6 `get_queryset`migrations re-fetched the full visible set via`BaseService.filter_visible(Model, user).values("pk")`and intersected with the incoming queryset via`queryset.filter(pk\_\_in=<subquery>)`— a correlated subquery over the full model table. The new helper chains the queryset's own`visible_to_user`instead (one`WHERE`expression tree, no subquery). Refactored`get_queryset`/ field-resolver call sites in`config/graphql/annotation_types.py` (`AnnotationType`, `NoteType`, `LabelSetType.resolve_corpus_count`), `corpus_types.py` (`CorpusFolderType.resolve_children`/`get_queryset`, `CorpusType.resolve_folders`/`get_queryset`), `document_types.py` (`DocumentType.get_queryset`), `conversation_types.py` (`ConversationType.get_queryset`), and `user_types.py` (`UserFeedbackType.get_queryset`— prefetch-cache pass-through preserved). New regression tests in`opencontractserver/tests/test_base_service.py` (`test_filter_visible_qs_chains_on_incoming_queryset`etc.) pin the single-pass-SQL contract by asserting the compiled query contains no`pk IN (SELECT`/`id IN (SELECT` subqueries against the full table.
  - **Missing `request=info.context` on four corpus UPDATE gates touched by PR #1780.** `UpdateDocumentSummary` (`config/graphql/document_mutations.py:326`) and three sibling `BaseService.require_permission` calls in `config/graphql/badge_mutations.py` (CreateBadge / UpdateBadge / DeleteBadge corpus permission gates) now thread `request=info.context` so the Tier-2 permission cache is engaged. Functionally identical (the permission check itself was already correct); this only fixes the cache miss.
  - **Strengthened the architecture-invariant tests** (`opencontractserver/tests/architecture/test_graphql_service_layer.py`). `test_allowlist_is_documented` was vacuously true with `ALLOWED_FILES = frozenset()` — replaced the silent-pass loop with a positive `assert ALLOWED_FILES == frozenset()` so a stray addition without documentation trips the test deliberately. `test_django_system_check_is_registered` previously only asserted that _some_ check existed under the `"architecture"` tag; now pins `check_graphql_service_layer` specifically (via `django.core.checks.registry.get_checks(tags=["architecture"])`) so deleting our check while leaving the tag in place cannot silently pass.
  - **Documented the AST-scan startup cost** in the docstring of `check_graphql_service_layer` (`opencontractserver/shared/checks.py`) — sub-50 ms over ~41 files, intentionally not fast-pathed or `DEBUG`-gated because the "fail on first management command" semantic is the whole point of dual-enforcement.
  - **Documented the `require_permission` truthiness inversion** in its docstring — returns `""` on grant and an error string on denial, the inverse of the legacy `if user_can(...)` idiom. Cross-readers transcribing `if user_can(...)` to `if require_permission(...)` directly would flip the gate; the docstring now calls this out alongside the recommended `BaseService.user_has` (legacy direction) for UI-state fields.
  - **Synced the CHANGELOG + architecture-invariants doc + query-permission-patterns doc** to match the live `ALLOWED_FILES = frozenset()` state (the original PR #1780 description claimed an allowlist of `filters.py` — accurate to an interim commit, not to the merged code; comments-only references mean `filters.py` no longer needs an entry).

- **Service-layer centralization Phase 6 — resolver cleanup + enforcement** (issue #1720; depends on Phases 1-5). Final phase of the roadmap in `docs/refactor_plans/2026-05-19-service-layer-centralization-design.md`. Every GraphQL resolver/mutation in `config/graphql/` now reaches models through `opencontractserver/<app>/services/` instead of inline `visible_to_user` / `user_can` / `user_has_permission_for_obj` composition.
  - **Dual enforcement.** Two independent layers point at the same scanner (`opencontractserver/shared/architecture_audit.py`) so violations cannot slip through either: (1) a Django system check (`opencontractserver/shared/checks.py`, `opencontracts.E001`, registered from `users/apps.py:ready()`) that fails any `manage.py` command — `runserver`, `migrate`, `shell`, `test`, `check --deploy` — with a non-zero exit code on the first inline Tier-0 use; (2) an AST-based pytest invariant (`opencontractserver/tests/architecture/test_graphql_service_layer.py`) that fails CI on the same condition, plus a regression test pinning the Django check to its registered state. The allowlist is empty: every `config/graphql/` module is scanned (including `filters.py`, whose only remaining references to forbidden identifiers are inside comments — and comments are ignored by the AST scanner, so no allowlist entry is needed).
  - **Self-teaching failure messages.** `architecture_audit.format_violation` builds a per-identifier copy-pasteable recipe (gate vs. boolean for `user_can`; queryset vs. IDOR-safe fetch for `visible_to_user`) and threads it through both surfaces — the Django `Error` `hint=` and the pytest `pytest.fail` blob — so a dev hitting the rule for the first time has the exact replacement code in front of them without needing to chase a doc link. New `docs/development/architecture_invariants.md` is the browsable invariant index + "how to add another invariant" template; `docs/architecture/query_permission_patterns.md` gained a "Migration Recipes" section with the same content in Markdown form.
  - **New BaseService helper.** `BaseService.user_has(instance, user, permission, *, request=None) -> bool` (`opencontractserver/shared/services/base.py`) — boolean companion to `require_permission` for resolvers that need yes/no state (UI flags like `can_edit_summary`) rather than a fail-fast gate. `BaseService.filter_visible` also gained `**kwargs` pass-through so per-model perf knobs (e.g. `Document(..., lightweight=True)`) keep working.
  - **Migrated files (31 total).** `config/graphql/base.py` (DRFMutation / DRFDeletion / OpenContractsNode framework code), `agent_mutations.py`, `annotation_mutations.py`, `annotation_queries.py`, `annotation_types.py`, `badge_mutations.py`, `conversation_mutations.py`, `conversation_queries.py`, `conversation_types.py`, `corpus_folder_mutations.py`, `corpus_mutations.py`, `corpus_queries.py`, `corpus_types.py`, `document_mutations.py`, `document_queries.py`, `document_relationship_mutations.py`, `document_types.py`, `extract_mutations.py`, `extract_queries.py`, `extract_types.py`, `label_mutations.py`, `search_queries.py`, `slug_queries.py`, `smart_label_mutations.py`, `social_queries.py`, `social_types.py`, `user_queries.py`, `user_types.py`, `voting_mutations.py`, `action_queries.py`. Pure relocation — no behaviour, permission semantics, transaction boundary, or unified IDOR-safe error string changes. Verified by `test_authorization_invariants.py` + `test_security_hardening.py`.
  - **Docs refresh.** CLAUDE.md rule 7 collapsed from a multi-bullet paragraph to a single canonical sentence. `docs/architecture/query_permission_patterns.md` rewritten around the three-tier mental model with the full per-app service inventory. `docs/permissioning/consolidated_permissioning_guide.md` updated to point at `UserService` / `BadgeService` instead of the now-internal optimizer classes, with a new Phase-6 callout explaining the service-layer rule.

### Fixed

- **Corpus / folder document-list badge query: replace the connection-shaped `docAnnotations` field with a flat-list `docTypeLabels` field, collapsing per-document SQL to zero.** Loading 24 documents in a folder used to fire ~240 SQL statements (~20 s on a remote RDS + S3 deployment) for the per-card DOC_TYPE_LABEL badge. Two root causes stacked: (1) the query was shaped as a Relay connection (`docAnnotations(annotationLabel_LabelType: DOC_TYPE_LABEL)`) when the frontend only ever read `edge.node.annotationLabel.{labelType,text}` — `DjangoFilterConnectionField`'s wrapper insists on a real `QuerySet` and fires one `COUNT(*)` + one `SELECT` per parent doc no matter what the resolver returns; (2) a name mismatch between `config/graphql/custom_resolvers.SUPPORTED_FILTER_KEYS` (camelCase GraphQL arg names) and the snake-case Django ORM lookup names that `DjangoFilterConnectionField` actually passes as kwargs at runtime sent every request into the resolver's "extra-key" escape hatch, defeating the focused `_prefetched_doc_annotations` prefetch and forcing per-row FK descriptor lookups — each `annotation.corpus` access then ran a recursive `WITH __rank_table` CTE because `Corpus` is `TreeNode(with_tree_fields=True)`.

  - **New `DocumentType.docTypeLabels: [AnnotationLabelType!]!`** (`config/graphql/document_types.py`): a flat list of distinct labels backing the badge UI. Consumes `_prefetched_doc_annotations` directly when set (zero per-doc SQL) and falls back to one targeted `SELECT … DISTINCT` per document when not. No Relay connection, no filterset wrapper, no cursor pagination — the field shape now matches what the UI actually needs.
  - **`config/graphql/custom_resolvers.py`**: `requests_doc_label_annotations` was renamed to `requests_doc_type_labels` and now triggers the focused prefetch on either the new `docTypeLabels` field OR the legacy `docAnnotations(annotationLabel_LabelType: DOC_TYPE_LABEL)` shape (the old name is preserved as an alias so existing imports keep working). `SUPPORTED_FILTER_KEYS` now lists the snake-case Django lookup names so the legacy connection-shaped field stops escape-hatching; every `kwargs.get(...)` inside `resolve_doc_annotations_optimized` was switched to match. New `_enum_value(...)` helper unwraps graphene's `Enum` kwargs so `kwargs["annotation_label__label_type"]` (a Python `Enum` member at runtime) compares equal to the underlying string. The Python-filtered branch returns a real `Annotation.objects.filter(pk__in=...).select_related(...)` queryset (lists break `django_filters/filterset.py:196`'s `queryset.model` read).
  - **`config/graphql/annotation_types.py`**: `AnnotationType.get_queryset` now applies `select_related("annotation_label", "corpus")` on **both** branches (the optimizer-annotated short-circuit and the `visible_to_user` fallback). Visibility filtering semantics unchanged; this is purely additional JOINs.
  - **`config/graphql/corpus_types.py`**: new `CorpusType.get_node(info, id)` adds (a) per-request memoisation on `info.context._corpus_node_cache` so repeat FK lookups of the same pk inside one GraphQL request hit the DB once, and (b) routes the underlying fetch through `Corpus.objects.visible_to_user(info.context.user)`. Graphene-django's default `DjangoObjectType.get_node` does an unprotected `.get(pk=id)`, so this also brings the FK / relay-Node access path into agreement with `OpenContractsNode.get_node_from_global_id` (which already filters by visibility for the root `node(id:)` query). See `docs/permissioning/consolidated_permissioning_guide.md` §1394 (the optimizer pattern) and §2727 (Tier-2 request cache).
  - **`config/graphql/permissioning/permission_annotator/mixins.py`**: extracted `_get_anonymous_user_id(info)` so the django-guardian anonymous-user pk (which `AnnotatePermissionsForReadMixin.resolve_my_permissions` / `resolve_object_shared_with` read on every node) is memoised on `info.context._anon_user_id`. Without it, a connection of N documents that includes `myPermissions` previously issued N `SELECT users_user WHERE username = 'AnonymousUser'` round-trips because django-guardian's `get_anonymous_user` is deliberately uncached.
  - **Frontend `GET_DOCUMENTS` query** (`frontend/src/graphql/queries.ts`): swapped `doc_label_annotations: docAnnotations(...)` for `docTypeLabels { labelType text }`. Dropped the unused `corpus { title icon preferredEmbedder }` selection (the document-card components never read it — pure overfetch). `DocumentItem.tsx` and `ModernDocumentItem.tsx` now consume `item.docTypeLabels` directly, falling back to the legacy `docLabelAnnotations.edges` shape if the backend hasn't been updated yet (staged-rollout / older-backend tolerance).
  - **Regression test** (`opencontractserver/tests/test_doc_annotations_prefetch_n_plus_one.py`): five invariants pinned independently — (1) every `AnnotationFilter.base_filters` key is classified by `SUPPORTED_FILTER_KEYS` or `UNSUPPORTED_FILTER_KEYS` (drift detection); (2) `docTypeLabels` SQL count is constant in the document count; (3) the per-row recursive CTE on `corpuses_corpus` does not scale with the document count; (4) `myPermissions` resolution issues at most one anonymous-user lookup per request; (5) `_get_anonymous_user_id` memoises after the first call. A sixth non-asserting "slap measurement" test prints query-count + wall-clock for both shapes so the speedup stays visible.
  - **Measured impact** (4-document fixture corpus, `test.yml` local Postgres, 5 iterations after warmup): new `docTypeLabels` shape = **5 queries / 40 ms median**; legacy `docAnnotations` connection shape (with all the same backend fixes applied) = 15 queries / 402 ms median. The new field's query count is **constant in the document count**, so a 24-doc folder still fires ~5 queries. On a remote RDS-style deployment with 50-100 ms per round-trip, the user-reported 20 s badge query drops to **~250-500 ms**.

- **`PipelineSettings` singleton cache leaks across tests on the same worker** (refs the `TransactionTestCase` rethink from issue #1769; root-cause investigation that supersedes the per-class `update_or_create` workaround in PR #1773). `PipelineSettings.get_instance()` (`opencontractserver/documents/models.py`) caches the singleton row in `LocMemCache` with a 5-minute TTL. The cache is per-worker process and survives both `TestCase` rollback and `TransactionTestCase` truncation — neither fires the model signal that would invalidate the cache. Tests like `TestPipelineComponentBaseSettings` in `test_pipeline_component_base.py` explicitly do `PipelineSettings.objects.all().delete(); PipelineSettings.objects.create(id=1)` (empty `default_embedder`) inside their transaction; when the transaction rolls back, the DB row reverts to migration-seeded state but the polluted Python instance stays in the cache. A later test on the same worker that calls `get_instance()` then hit the stale cache and saw `default_embedder=""`, producing the recurring flaky `ValueError: get_embedder() resolved no embedder_path` failures in `TestPydanticAIConversationAdaptersAsync` (PR #1773), `TestDuplicateToolRegistration`, and `TestApprovalFlow`.
  - **Fix** (`opencontractserver/conftest.py`): autouse function-scoped fixture `_invalidate_pipeline_settings_singleton_cache` calls `PipelineSettings._invalidate_cache()` before every test, so each test starts with a fresh DB read. Cost is one `cache.delete()` per test — penny of overhead per ~7,400-test run.
  - **Reverted the per-class workaround** in `test_pydantic_ai_conversation_adapters.py::TestPydanticAIConversationAdaptersAsync.setUp` (PR #1773) — the structural fixture supersedes it, so the band-aid `update_or_create` block is gone.
  - **Regression test** (`opencontractserver/tests/test_pipeline_settings_cache_isolation.py`): two-test sequence proves the autouse fixture works. `test_a` deletes + creates with empty defaults and forces a cache populate; `test_b` then asserts `get_instance().default_embedder == settings.DEFAULT_EMBEDDER` — which can only pass if the autouse fixture cleared the cache between them. Without the fixture this test would fail; with it, both pass.
  - **Audit confirmed** that no other `_invalidate_cache`-using service (`conversation_service.py`, `versions.py`, `relationships.py`) has the same exposure profile — they all use per-request/per-user cache keys, not a process-wide singleton key.

### Changed

- **`DocumentKnowledgeBase` desktop initial PDF zoom — reserve a horizontal margin so the fit-to-width result never overflows on narrower laptop viewports** (issue #1736; `frontend/src/components/annotator/renderers/pdf/PDFPage.tsx`, `frontend/src/utils/pdfZoom.ts` (new), `frontend/src/components/knowledge_base/document/layouts/mobile/useMobileFitToWidth.ts`, `frontend/src/assets/configurations/constants.ts`). Verified the desktop initial PDF zoom is already a fit-to-width calculation derived from the measured document-layer width (not a fixed seed), but the calculation reserved no horizontal margin — the rendered page touched both edges of the container, leaving no safety budget for sub-pixel rounding or container chrome and risking a horizontal scrollbar on 1280 px / 1024 px laptops. The desktop calculation now flows through the same shared `computeFitToWidthZoom(naturalPageWidth, containerWidth)` helper used by the mobile fit-to-width hook (issue #1735's mobile work introduced `FIT_WIDTH_MARGIN`), reserving `FIT_WIDTH_MARGIN` (16 px) off the container width and clamping to the canonical `ZOOM_MIN`/`ZOOM_MAX` (50 % / 400 %) — previously the desktop path clamped to the magic numbers `0.3` / `4.0`, drifting from the shared constants. The mobile hook is refactored to call the shared helper so both surfaces stay in lockstep going forward. The helper lives under `frontend/src/utils/` (pure number-in/number-out, depends only on `constants.ts`) to keep the `annotator` and `knowledge_base` component subtrees free of a back-edge import cycle. New unit test `frontend/src/utils/__tests__/pdfZoom.test.ts` pins the helper's contract; new CT test group `Default PDF zoom fit-to-width — {viewport} (#1736)` in `frontend/tests/DocumentKnowledgeBase.ct.tsx` mounts the full DKB at 1440×900, 1280×720 and 1024×768 and asserts that the rendered PDF canvas width is strictly less than the `#pdf-container` width at every viewport, capturing one documentation screenshot per viewport (`knowledge-base--pdf-default-zoom--1440x900.png`, `--1280x720.png`, `--1024x768.png`).

- **Service-layer centralization Phase 5 — fill the coverage gaps** (issue #1719; depends on Phase 1 #1715). Phase 5 of the roadmap in `docs/refactor_plans/2026-05-19-service-layer-centralization-design.md`. The five models that still hand-rolled permission composition inline now have per-app `services/` packages, each service inheriting `BaseService`. Pure relocation + encapsulation — no behaviour, permission semantics, transaction boundary, or unified IDOR-safe error string changes.

  - **New service packages.** `opencontractserver/agents/services/` (`AgentConfigurationService` — agent CRUD + visibility; `AgentActionResultService` — action-result visibility). `opencontractserver/analyzer/services/` (extended with `AnalysisLifecycleService` — `make_public`, `start_document_analysis`, `delete_analysis` — alongside the Phase-3 read-only `AnalysisService`). `opencontractserver/notifications/services/` (`NotificationService` — simple-ownership notification queries + mutations; does NOT use `BaseService`'s `user_can`-backed helpers because `Notification` has no `user_can` manager — instead provides its own `recipient=user` IDOR-safe lookup). `opencontractserver/feedback/services/` (`UserFeedbackService` — `approve_annotation` / `reject_annotation` flows). `opencontractserver/worker_uploads/services/` (`WorkerAccountService`, `CorpusAccessTokenService`, `WorkerDocumentUploadService`).
  - **Migrated GraphQL call sites.** `config/graphql/agent_mutations.py` (Create / Update / Delete), `config/graphql/social_queries.py` (agent + notification resolvers), `config/graphql/action_queries.py` (`resolve_agent_action_results`), `config/graphql/analysis_mutations.py` (`MakeAnalysisPublic`, `StartDocumentAnalysisMutation`, `DeleteAnalysisMutation`), `config/graphql/notification_mutations.py` (all four mutations), `config/graphql/annotation_mutations.py` (`ApproveAnnotation`, `RejectAnnotation`), `config/graphql/worker_queries.py` (three resolvers), `config/graphql/worker_mutations.py` (four mutations). Every public service method threads `request=info.context` so the Tier-2 permission cache is engaged.
  - **Tests.** New `opencontractserver/tests/test_service_layer_phase5.py` pins the structural contract: each new service is importable from its package root, inherits `BaseService`, exposes public methods that accept `request` as keyword-only, and (for `NotificationService`) implements its own simple-ownership IDOR-safe lookup rather than inheriting `get_or_none` (which would fail without `user_can`). Behavioural regression coverage stays in the existing GraphQL / model test suites for these flows.
  - **Plan.** `docs/refactor_plans/2026-05-23-service-layer-phase5-gap-models-plan.md`.

- **Backend test suite speedup — Phase A3b: portable buildx cache, shared across CI + local dev + prod** (issue #1769). Swaps the GitHub-Actions-only buildx cache backend (`type=gha`, landed in Phase A3 / PR #1774) for a portable registry-backed cache at `ghcr.io/open-source-legal/opencontracts/django-{local,production}-cache:main`. GHA cache only works inside GitHub Actions, so it could not help contributors building from a fresh checkout. The registry cache is pullable from any docker buildx invocation that can reach ghcr.

  - **CI (`.github/workflows/backend.yml`)**: replaces `--cache-from/--cache-to type=gha,scope=django-test` with `type=registry,ref=ghcr.io/.../django-local-cache:main`. Adds `permissions: packages: write` on the pytest job (needed for the `cache-to` push on main merges), a `docker/login-action@v3` step (using the existing `GITHUB_TOKEN` — no new secrets), and gates `--cache-to` to `push` events only so PR runs read the cache but don't write to it. Drops the inline `actions/github-script` env-var step (registry cache doesn't need `ACTIONS_RUNTIME_TOKEN`).
  - **Local dev (`local.yml`)**: adds `cache_from: [type=registry,ref=ghcr.io/.../django-local-cache:main]` to the `django` service's build block. A fresh `docker compose -f local.yml build` should drop from ~5-10 min to ~30-60 s on cache hit. Contributors need to `docker login ghcr.io` (one-time, via `gh auth token | docker login ghcr.io -u USERNAME --password-stdin`); the cache image is public so unauthenticated pulls fall back gracefully (cache miss, normal build) if login is skipped.
  - **Prod (`production.yml`)**: wires `cache_from: [type=registry,ref=ghcr.io/.../django-production-cache:main]` into both the `django` and `migrate` services (they share the same image). No-op until a follow-up PR adds the CI job that publishes the production cache (`compose/production/django/Dockerfile` is distinct from the local/test Dockerfile and needs its own scope). Wiring it now means that follow-up is a single workflow file rather than a workflow + compose-file change.

  _Visibility note:_ the ghcr cache packages inherit the repo's visibility on first publish (public, in this repo) — if either package is created private by mistake, change visibility in the Packages settings once.

- **Typing: graduated the `tests.test_agent*` 9-module agent domain chunk out of the mypy baseline** (issue #1738, continuing the #1331 → #1335 → #1447 cadence; the `agent` chunk from the issue's `tests.*` leaf-files batching plan, following the `corpus` chunk in PR #1866). Removed all 9 `[mypy-opencontractserver.tests.test_agent*]` `ignore_errors` blocks from `mypy.ini` (`test_agent_action_result`, `test_agent_action_result_admin`, `test_agent_api`, `test_agent_factory`, `test_agent_framework_api`, `test_agent_memory`, `test_agent_tasks`, `test_agentic_highlighter_task`, `test_agents`), pruned the corresponding 364 lines from `docs/typing/mypy_baseline.txt` (4012 → 3648), and fixed the 338 errors that surfaced under the currently-pinned mypy / django-stubs (`mypy==2.0.0`, `django-stubs==6.0.5`, per `.pre-commit-config.yaml`; also verified clean under CI's `mypy==2.1.0`). Pre-commit `mypy` continues to pass on the full project surface (`mypy --config-file mypy.ini opencontractserver config` → "Success: no issues found in 1247 source files"). The fixes are the established mechanical patterns from prior chunks: class-level annotations for `setUpTestData`/`setUpClass` attributes (the recommended fix from #1479, the bulk of the 338), `User = get_user_model()` → direct `from opencontractserver.users.models import User` import, the graphene `self.client` → `self.graphene_client` rename, `assert … is not None` narrowing of Optional returns, and a small number of error-code-scoped `# type: ignore`s. All changes are type-level and behaviour-neutral or behaviour-equivalent. Grouped highlights:

  - **Graphene-client rename** (`test_agents.py`): the `graphene.test.Client` assigned to `self.client` shadows the inherited `django.test.TestCase.client`, which has no `.execute()`. (`test_agent_action_result_admin.py` was deliberately left untouched — it uses the real `django.test.Client` for admin GET requests.)
  - **Class-attribute annotations + direct `User` import** (most of the 338): every `test_agent*` `TestCase` whose `setUpTestData` populated `cls.` model fixtures invisible to mypy — `test_agent_framework_api.py` (one shared `TestAPISetup` base, 11 fixtures, inherited by 3 subclasses), `test_agents.py`, `test_agent_tasks.py`, `test_agent_factory.py` (5 classes), `test_agent_action_result_admin.py`, `test_agent_memory.py` (4 classes).
  - **List-invariance annotations** (`test_agent_api.py`, `test_agent_factory.py`): `list[str]` / mixed tool literals passed to `_resolve_tools` / `create_document_agent` (both take `list[ToolType]`) annotated as `list[ToolType]` so invariance is satisfied.
  - **`Optional` narrowing** (`test_agent_api.py`, `test_agent_memory.py`, `test_agent_action_result.py`): `assertIsNotNone(x)` → `assert x is not None` (preserves the assertion while narrowing) for `ToolRegistryEntry`/`CoreTool` registry returns, `_FakeConfig.system_prompt` (`Optional[str]`), and `AgentActionResult.duration_seconds` (`float | None`).
  - **Scoped `# type: ignore`s** (no blanket ignores; `warn_unused_ignores=True` keeps them honest): a deliberately-`None` `AdminSite` (`test_agent_action_result_admin.py`), the deliberately-invalid `framework="invalid_framework_name"` string that exercises the `ValueError` path (`test_agent_factory.py`), a deliberately-invalid `["summarize", 123, None]` tool list that exercises the skip-and-warn path (`test_agent_api.py`), the per-instance override of the base `ClassVar` `corpus`/`docs` fixtures (`test_agentic_highlighter_task.py`), and the legacy `stream_chat` wrapper (see below).

  Two test-quality findings surfaced behind the mocks (the audit the issue invites):

  - **`test_agent_api.py::test_contextual_embedding_generation` passed a non-existent `embedder_path=` kwarg to `embeddings.generate`** — the public `EmbeddingAPI.generate` parameter is `embedder` (no `embedder_path`, no `**kwargs`). The call only "worked" because `generate` was mocked, so the wrong kwarg name was hidden. Corrected the call (and its `assert_called_once_with`) to `embedder=`, which keeps the test green and self-consistent while aligning it with the real public API.
  - **`test_agent_api.py::test_vector_store_with_all_options` retains `embedder_path=`** — unlike the agent/embedding APIs, the public `VectorStoreAPI.create` parameter is genuinely named `embedder_path` (it forwards straight through to `UnifiedVectorStoreFactory.create_vector_store(embedder_path=...)`). The same blanket rename applied elsewhere had been carried into this test, but `create` has a `**kwargs` catch-all, so `embedder="custom-embedder"` was silently absorbed while `embedder_path` defaulted to `None` — making the `assert_called_once_with(embedder="custom-embedder")` mismatch the actual `create_vector_store(embedder_path=None, …, embedder="custom-embedder")` call. Kept this test on the real `embedder_path=` parameter name.
  - **`CoreAgent.stream_chat` is a legacy compatibility wrapper that lives on the concrete `CoreAgentBase` but is deliberately absent from the modern `CoreAgent` protocol** (which exposes `stream`). It is never called in production — only `test_agent_api.py` and `test_agent_framework_api.py` exercise it, via mocks that replace it wholesale. Left the protocol as-is (the omission looks intentional) and scoped the two test calls with `# type: ignore[attr-defined]`; a follow-up could either fold it into the protocol or remove the now-effectively-dead wrapper.

- **Typing: graduated the `tests.test_corpus_*` 23-module corpus domain chunk out of the mypy baseline** (issue #1738, continuing the #1331 → #1335 → #1447 cadence; the `corpus` chunk from the issue's `tests.*` leaf-files batching plan, following the `document` chunk in PR #1851). Removed all 23 `[mypy-opencontractserver.tests.test_corpus_*]` `ignore_errors` blocks from `mypy.ini`, pruned the corresponding 798 lines from `docs/typing/mypy_baseline.txt` (4810 → 4012), and fixed the 898 errors that surfaced under the currently-pinned mypy / django-stubs (`mypy==2.0.0`, `django-stubs==6.0.5`, per `.pre-commit-config.yaml`; also verified clean under CI's `mypy==2.1.0`). Pre-commit `mypy` continues to pass on the full project surface (`mypy --config-file mypy.ini opencontractserver config` → "Success: no issues found in 1245 source files"). The fixes are the established mechanical patterns from prior chunks: class-level annotations for `setUpTestData`/`setUpClass`/`setUp` attributes (the recommended fix from #1479), `User = get_user_model()` → direct `from opencontractserver.users.models import User` import, the graphene `self.client` → `self.graphene_client` rename, `assert … is not None` narrowing of Optional service/ORM returns, and `cast(...)` at call sites that pass intentionally-incomplete export TypedDict fixtures (the repo's established idiom). Grouped highlights:

  - **Graphene-client rename** (a `graphene.test.Client` assigned to `self.client` shadows the inherited `django.test.TestCase.client`, which has no `.execute()`): `test_corpus_action_graphql.py`, `test_corpus_category.py`, `test_corpus_license.py`, `test_corpus_query_optimization.py`. (`test_corpus_action_execution_admin.py` was deliberately left untouched here — it uses the real `django.test.Client`, not graphene.)
  - **Class-attribute annotations + direct `User` import** (the bulk of the 898): every `test_corpus_*` `TestCase`/`TransactionTestCase` whose `setUpTestData`/`setUpClass`/`setUp` populated `cls.`/`self.` model fixtures invisible to mypy — e.g. `test_corpus_annotations_query.py` (130), `test_corpus_category.py` (126), `test_corpus_query_optimization.py` (120), `test_corpus_objs_service.py` (98), `test_corpus_description_cache.py` (51), `test_corpus_engagement_metrics.py`, `test_corpus_collector.py`, …
  - **Optional-return narrowing**: `assert … is not None` after `FolderCRUDService.{create,update,delete,move}_folder` / `add_document_to_corpus` (`test_corpus_objs_service.py`), `_execute_fork(...)` and `.first()`/`.get()` results (`test_corpus_fork_round_trip.py`, `test_corpus_forking.py`, `test_corpus_import.py`, `test_corpus_document_actions.py`, `test_corpus_description_cache.py`), each extracted to a local so the existing `self.assertEqual(...)` check is preserved verbatim.
  - **TypedDict fixtures via `cast`** (`test_corpus_export_import_v2.py`, `test_corpus_fork_round_trip.py`): the v2 import helpers (`import_structural_annotation_set`, `import_corpus_folders`, `import_agent_config`, `_reconstruct_document_paths`, …) take full export TypedDicts, but the tests pass deliberately-partial literals; wrapped at the call site with `cast(...)` (matching the graduated enricher tests) rather than annotating the literal, which would raise `typeddict-item` "missing key".
  - **Scoped, error-coded `# type: ignore`s** (no blanket ignores; `warn_unused_ignores=True` keeps them honest): `.with_stats()` on a `Manager.filter(...)` chain django-stubs can't see the custom queryset through (`test_corpus_action_execution.py`); the `corpuses` M2M reverse accessor (`related_name`) django-stubs can't resolve (`test_corpus_category.py`); `SpooledTemporaryFile._rolled` (absent from typeshed) and a deliberately-`None` `AdminSite` test arg (`test_corpus_export_import_v2.py`, `test_corpus_action_execution_admin.py`); a faked psycopg2 `pgcode` attribute on a bare `Exception` (`test_corpus_objs_service.py`).

  **Latent production bug surfaced (flagged for a follow-up, intentionally NOT fixed in this test-only chunk):** `opencontractserver/utils/etl.py::build_label_lookups` declares `annotation_filter_mode: AnnotationFilterMode` (a plain, non-`str` `Enum`) but compares the parameter **inconsistently** — against raw string literals at `etl.py:79,92,110,123` (`== "ANALYSES_ONLY"`, …) yet against enum members at `etl.py:162-165` (`in (AnnotationFilterMode.CORPUS_LABELSET_ONLY, …)`). A `str` argument (what every caller and the docstring actually use) therefore satisfies the string branches but silently fails the enum-membership branch, skipping the corpus-labelset-inclusion block; an enum-member argument would fail the string branches. The corpus tests pass the runtime-correct string with a behavior-neutral `cast`/scoped ignore. Recommended fix (separate PR, needs the full test suite): normalize `build_label_lookups`/`build_document_export` to compare consistently — make `AnnotationFilterMode` a `StrEnum`, or coerce to `.value` and compare against strings throughout.

- **Typing: graduated the `tests.test_document_*` 12-module document domain chunk out of the mypy baseline** (issue #1738, continuing the #1331 → #1335 → #1447 cadence; the `document` chunk from the issue's `tests.*` leaf-files batching plan, following the `annotation` chunk in PR #1776). Removed all 12 `[mypy-opencontractserver.tests.test_document_*]` `ignore_errors` blocks from `mypy.ini`, pruned the corresponding 331 lines from `docs/typing/mypy_baseline.txt` (5141 → 4810), and fixed the 243 errors that surfaced under the currently-pinned mypy / django-stubs (`mypy==2.0.0`, `django-stubs==6.0.5`, per `.pre-commit-config.yaml`). The current count is below the 331-line baseline snapshot because the newer stubs already resolve several historical errors (notably the `set_permissions_for_obj_to_user` `arg-type` family — see below). Pre-commit `mypy` continues to pass on the full project surface (`mypy --config-file mypy.ini opencontractserver config` → "Success: no issues found in 1242 source files"). Per-file fixes:

  - `test_document_index_tool.py` (132 baseline errors) — added class-level annotations (`user: User`, `corpus: Corpus`, `doc: Document`) on all four `TestCreateDocumentIndex{PDF,Text}` / `TestUpdateAnnotationLongDescription` / `TestLongDescriptionExportImportRoundTrip` classes covering the `cls.<attr> = ...` assignments in `setUpClass`; switched `User = get_user_model()` to a direct `from opencontractserver.users.models import User` import; annotated the nine `entries: list[IndexEntryItem]` literals and the two `exported: OpenContractsAnnotationPythonType` literals so the `create_document_index` / `import_annotations` call sites type-check against their real contracts (the inline-literal call sites already type-checked via bidirectional inference). **Latent type gap revealed by typing `self.doc`:** once `doc` was annotated `Document`, the four `setUp` methods surfaced `FieldFile.name` as `str | None` being passed to `Storage.exists()` / `FieldFile.save()` (previously masked because `self.doc` resolved to `Any` behind the `attr-defined` error) — narrowed via a local `name = self.doc.<field>.name` + `assert name is not None` (the file is always saved in `setUpClass`).
  - `test_document_versioning_graphql.py` (69 baseline errors) — renamed the `self.client` attribute to `self.graphene_client` across all nine test classes (41 `.execute()` call sites) so the assignment of a `graphene.test.Client` no longer shadows the inherited `django.test.TestCase.client`, which has no `.execute()` (behaviour-equivalent — the tests only ever use the GraphQL client — and matches the convention from PR #1776 / `permissioning/test_permissioning.py`); replaced three `assertIsNotNone(x)` with `assert x is not None` narrowings so the subsequent `.is_deleted` / `.title` / `.version_number` / `.parent` accesses type-check.
  - `test_document_admin.py` (34 baseline errors) — added class-level annotations on `TestDocumentAdmin` (`superuser`, `corpus`, `document`, `document2`, `embedding_384`, `embedding2_384`, `embedding2_768`) covering the `setUpTestData` assignments; same `get_user_model()` → direct import switch. The legitimate Django `self.client = Client()` in `setUp` is left untouched (that file uses the real test client, not graphene).
  - `test_document_relationship_mutations.py` (34 baseline errors, 2 still reproduced) — replaced `result.first().relationship_type` with `result[0].relationship_type` after the pre-existing `result.count() == 1` guard, and one `assertIsNotNone(result)` with `assert result is not None`. The 32 historical `set_permissions_for_obj_to_user` `arg-type` errors no longer reproduce under the current pins.
  - `test_document_relationship_permissions.py` (23 baseline errors) and `test_document_analysis_row.py` (8 baseline errors) — no code change needed; the historical `set_permissions_for_obj_to_user` `arg-type` errors no longer reproduce under the current pins, so only the `[mypy-…]` blocks were removed.
  - `test_document_versioning.py` (6 baseline errors) — six `assert … is not None` narrowings before passing `import_document`'s `Document | None` result to `get_content_history` and before `FieldFile.name` (`str | None`) `.endswith(...)` / `assertIn(...)` calls.
  - `test_document_path_migration.py` (2 baseline errors) — declared the dynamically-assigned `user` / `context` attributes on the local `MockContext` mock class so the `ctx.user = …` / `info.context = …` assignments type-check; same `get_user_model()` → direct import switch.
  - `test_document_mutations.py`, `test_document_relationships.py`, `test_document_uploads.py`, `test_document_queries.py` (11/5/7/4 baseline errors) — the same `self.client` → `self.graphene_client` graphene-client rename (no other changes).

- **Typing: graduated the `tests.test_annotation_*` 6-module annotation domain chunk out of the mypy baseline** (issue #1738, continuing the #1331 → #1335 → #1447 cadence; the `annotation` chunk from the issue's `tests.*` leaf-files batching plan, following the `tests.permissioning.*` subpackage in PR #1768). Removed all 6 `[mypy-opencontractserver.tests.test_annotation_*]` `ignore_errors` blocks from `mypy.ini`, pruned the corresponding 138 lines from `docs/typing/mypy_baseline.txt` (5279 → 5141), and fixed the errors that surfaced under the current mypy / django-stubs pins (`mypy==2.1.0`, `django-stubs==6.0.4`). Pre-commit `mypy` continues to pass on the full project surface (`mypy --config-file mypy.ini opencontractserver config` → "Success: no issues found in 1157 source files"). Per-file fixes:

  - `test_annotation_admin.py` (43 baseline errors) — added class-level annotations on `TestAnnotationAdmin` covering every `cls.<attr> = ...` assignment inside `setUpTestData` (`superuser`, `corpus`, `document`, `annotation_label`, `annotation`, `annotation2`, `embedding_384`, `embedding2_384a`, `embedding2_384b`); switched `User = get_user_model()` to a direct `from opencontractserver.users.models import User` import so the name is usable as a type annotation (`get_user_model()` returns `type[AbstractBaseUser]`, which mypy cannot use as a type).
  - `test_annotation_images_api.py` (34 baseline errors) — added class-level annotations on `AnnotationImagesAPITestCase` (`user`, `other_user`, `label_set`, `annotation_label`, `corpus`); same `get_user_model()` → direct import switch (also resolves the `valid-type` error on the `owner: User` parameter at `_create_test_document_with_images`).
  - `test_annotation_permission_mutations.py` (12 baseline errors) — no code change needed; the `set_permissions_for_obj_to_user` `arg-type` errors no longer reproduce under the current `mypy==2.1.0` / `django-stubs==6.0.4` pins.
  - `test_annotation_privacy.py` (6 baseline errors) — same as above; the historical `arg-type` errors no longer reproduce.
  - `test_annotation_tree.py` (8 baseline errors) — renamed the `self.client` attribute on `AnnotationTreeTestCase` to `self.graphene_client` so the assignment of a `graphene.test.Client` no longer shadows the inherited `django.test.TestCase.client` attribute (mypy was resolving the call through Django's `Client`, which has no `.execute()` method; the existing tests only ever use the GraphQL client, so the rename is behaviour-equivalent and matches the project convention in `permissioning/test_permissioning.py`); replaced one `assertIsNotNone(root_node)` with an `assert root_node is not None` narrowing so the subsequent `root_node["raw_text"]` indexing type-checks; same `get_user_model()` → direct import switch.
  - `test_annotation_utils.py` (35 baseline errors) — added class-level annotations on `TestComputeContentModalitiesWithDocument` (`user`, `corpus`) and `TestUpdateAnnotationModalities` (`user`, `corpus`, `label`); scoped two `# type: ignore[arg-type]` comments on the two `compute_content_modalities` calls that deliberately pass mixed/invalid items (`["invalid_string", None, 123, {...}]` and `["not_a_dict", None, {...}]`) to exercise the function's defensive skipping paths; same `get_user_model()` → direct import switch.

- **Backend test suite speedup — Phase A3: cache the django image build between CI runs** (issue #1769). The `Build the Stack` step rebuilt the django image from layer one on every PR run (~2:40 wall) because the runner ships from scratch. Switches to `docker buildx build` with GitHub Actions layer cache (`--cache-from type=gha,scope=django-test --cache-to type=gha,mode=max,scope=django-test`): on cache hit, most layers (apt-get, pip wheel, spaCy model fetch) restore in seconds rather than rebuilding. The image is tagged to match the `image:` directive on the `django` service in `test.yml`, so the subsequent `docker compose build` only compiles the small `postgres` image. Cache invalidates automatically whenever any of the layer inputs change (Dockerfile, requirements, COPY'd source). No new secrets or external registry required — GHA cache uses the project's existing `actions/cache` quota.

- **Backend test suite speedup — drop `collectstatic` from backend CI** (issue #1769). The `Collect Static Files` workflow step ran `manage.py collectstatic` against the test image (~1:40 wall on a cold django container start). The test suite does not serve static files — `config/settings/test.py` deliberately uses `StaticFilesStorage` (the manifest-less variant) so admin tests resolve `{% static %}` URLs at template render time without needing the post-collectstatic manifest file. The step produced no test-relevant artefacts; the wall time was dominated by the django container cold start, not by the actual collection work. Stacks on top of Phase A2 (drop redundant migrate step, PR #1772) for ~2:30 cumulative pre-test wall reduction.

- **Backend test suite speedup — Phase A2 + B2: slim `test_data.json` and drop the redundant `Run DB Migrations` CI step** (issue #1769; plan §4-§5 in `docs/refactor_plans/2026-05-23-test-suite-speedup-A1-B1.md`). Two changes that compound with Phase A1 (PR #1767):

  - **`opencontractserver/tests/fixtures/test_data.json` from 16.4 MB / 4,284 objects → 0.123 MB / 331 objects** (133× smaller, 92% object reduction). Removed: 1,344 `annotations.annotation` rows (every one had `document=None, corpus=None` — orphan rows unreferenced by any test in the suite); 1,344 `annotations.embedding` rows (all FK-pointed at those orphan annotations); 472 `annotations.relationship` rows (same pattern); 117 `contenttypes.contenttype` + 676 `auth.permission` rows (Django recreates these via `post_migrate` on every test-DB creation). Kept: 4 documents, 2 users, 1 group, 40 annotation labels, 4 structural annotation sets, 280 guardian-user-object permissions (all 280 grant on annotationlabel — required for tests that resolve label READ permission via `visible_to_user`). Audit of all 19 fixture-using test files confirmed zero of them read the dropped bulk rows. Local smoke: the 3 hottest websocket/extract/structured-response files plus the 2 fixture-using files that _do_ reference `Annotation.objects`/`AnnotationLabel.objects` query patterns (`test_note_tree.py`, `test_agentic_highlighter_task.py`) pass cleanly in 2:52 wall — per-test fixture-load setup dropped from ~25 s → ~2 s (the targeted 10× reduction). All 331 retained objects are byte-for-byte identical to their pre-slim counterparts; only deletions, no edits.
  - **Removed the `Run DB Migrations` step from `.github/workflows/backend.yml`** (was ~3 min wall per CI run). The step ran `manage.py migrate` against the `opencontractserver` default database, which is never read by the test suite — pytest-django creates per-worker test databases (`test_opencontractserver_gw0`, `_gw1`, …) on first invocation and runs migrations against those, and `DATABASES['default']['TEST']` does not declare a `TEMPLATE`, so the default DB is not used as a `createdb` template either. `pytest --reuse-db` in `pytest.ini` handles both cold creation and warm reuse of the test DBs on its own.

- **Backend test suite speedup — Phase A1: Python 3.12 + `sys.monitoring` coverage** (plan in `docs/refactor_plans/2026-05-23-test-suite-speedup-A1-B1.md`). Targets the highest-ROI lever from a CI runtime audit of `gh run view 26334920376` (47-min wall, 41-min pytest step): coverage on Python 3.11 measured at **+156% over the no-cov baseline** on the hot WebSocket / extract / structured-response files. Measured CI delta after this change alone: backend pytest step ~41 min → ~32 min (~22% reduction; full audit of remaining bottlenecks captured in the plan doc for follow-up). A companion `--dist worksteal` change was trialled in PR #1767 and reverted — it exposed pre-existing test-isolation bugs across ~39 plain `TestCase` subclasses that were latent under loadscope's class-pinning, and needs a dedicated test-isolation sweep before it can be re-attempted.

  - **Python 3.12.7 in the test/runtime image** (`compose/local/django/Dockerfile`, `compose/production/django/Dockerfile`). Bumps `ARG PYTHON_VERSION` from `3.11.15-slim-bookworm` to `3.12.7-slim-bookworm`. CI's linter job has been running on Python 3.12 for weeks, so dependency compatibility is already proven; this brings the django image in line.
  - **`COVERAGE_CORE=sysmon` in the test env** (`.envs/.test/.django`). Switches coverage.py to `sys.monitoring`-backed instrumentation (the `SysMonitor` tracer class), ~5-10× cheaper than the legacy C trace function used on Python 3.11. Codecov receives the same `coverage.xml` content — only the collection path changes.
  - **Dropped `django_coverage_plugin`** (`setup.cfg`). Coverage 7.x silently falls back to the legacy C trace path whenever any file-tracer plugin is configured, defeating `COVERAGE_CORE=sysmon`. The project owns 7 Django templates total (4xx/5xx pages plus a couple of admin views), so template coverage was a trivial slice of total coverage but the plugin was forcing the ~150% wall-time tax. Restore if/when coverage.py supports sysmon-compatible plugin tracers.

- **`DocumentKnowledgeBase` desktop polish — consolidate the two right-edge control systems into one unified rail** (issue #1734; `frontend/src/components/knowledge_base/document/styled/SidebarTabs.tsx`, `frontend/src/components/knowledge_base/document/document_kb/SidebarTabs.tsx`, `frontend/src/components/knowledge_base/document/FloatingDocumentControls.tsx`, `frontend/src/components/knowledge_base/document/layouts/DesktopDocumentLayout.tsx`). At a desktop viewport the document layer's right edge previously rendered two competing vertical stacks: a rotated-text tab rail (`INDEX`/`CHAT`/`FEED`/`DISCUSSIONS`) and, just to its left, a separate floating action-button cluster (settings gear + colored circular FABs for extracts / analyses / create-analysis). The audit flagged this as visual clutter that made it ambiguous which control does what, and called out the rotated text labels as hard to scan.

  - **Visual unification.** When the right panel is closed, both control groups now render inside a single new `RightEdgeRail` wrapper (`DesktopDocumentLayout.tsx`), vertically centered against the viewport's right edge. The navigation tabs sit on top, a 1-pixel `RailDivider` separates the two control intents, and the document tool pills (settings, extracts, analyses, create-analysis) sit below — one coherent column with consistent pill shape, sizing, and active-state treatment.
  - **No more rotated text.** `SidebarTab` (styled) drops its `writing-mode: vertical-rl` label rendering. The `.tab-label` span stays in the DOM as a screen-reader-only string (visually clipped to 1×1px) so assistive tech still announces it; the visible affordance is the icon plus a CSS-only hover tooltip driven by the new optional `data-tooltip` attribute. The tooltip flips to the opposite side of the pill based on `$panelOpen` so it never overflows the viewport edge.
  - **Accessibility.** Every tab and every action button now carries an `aria-label` (most action buttons previously had only a `title` attribute and no accessible name). View/extract tabs announce their purpose ("Document index", "Annotation feed", "View extracts", "Start new analysis"); the discussions tab pluralizes the thread count ("Document discussions, 3 threads"); toggle FABs (extracts / analyses / settings / panel-width) advertise their open/expanded state via `aria-pressed` / `aria-expanded`.
  - **Shared styled primitive.** The action-button styled component (`ActionButton` in `FloatingDocumentControls.tsx`) now extends `SidebarTab` instead of defining its own circular FAB style. Both surfaces share the same 44×44 pill geometry, white background with subtle gradient overlay, and blue active treatment; per-action color cues (purple for extracts, amber for analyses, green for create) are preserved as a lightweight `$accent` icon tint instead of the heavy colored-circle background.
  - **Bare-container plumbing.** `DesktopSidebarTabs` gains an optional `bareContainer` prop and `FloatingDocumentControls` gains an optional `bareContainer` prop. When true, each component renders without its own fixed/absolute positioning so the parent `RightEdgeRail` can stack them in a single flex column. The standalone (non-bare) positioning is preserved for the existing component test harnesses and for the panel-open layout (where the navigation tabs anchor to the left edge of the open right panel and the action cluster keeps its bottom-right placement so the panel-width FAB rides alongside the open panel).
  - **Tests.** `SidebarTabs.ct.tsx` gains an `each tab carries an accessible name and tooltip data` test that pins every tab's `aria-label` + `data-tooltip` mapping (regression guard against a re-introduced rotated rail), a `tab labels are visually hidden but in the DOM for screen readers` test that asserts the `.tab-label` span has the expected text but a sub-2px bounding box, and a new `RightEdgeRail (issue #1734)` test group with three screenshot tests covering the unified rail (`knowledge-base--right-edge-rail--unified`, `--with-extract-analysis`, `--read-only`). The existing `FloatingDocumentControls.ct.tsx` (20 tests), `DesktopSidebarTabs` (4 tests), and `DocumentKnowledgeBase.ct.tsx` (70 tests) suites continue to pass unchanged — `data-testid`s, click semantics, panel toggle behavior, panel-width controls, exclusivity logic, permissions gating, and `hideDocumentTools` are preserved.
  - **Removed: `onSummaryClick` callback on `FloatingDocumentControls`.** The old FAB cluster offered a redundant "open summary in knowledge layer" entry point alongside the `FloatingSummaryPreview`'s own "View Full Screen" / version-chip path. With the rail consolidation, this redundant FAB is gone — the knowledge-layer switch remains fully reachable via the summary preview's `onSwitchToKnowledge` wiring in `DesktopDocumentLayout.tsx`. Net effect: one canonical entry point instead of two.

- **`DocumentKnowledgeBase` mobile UX cleanup — three "get the overlay out of the way" fixes** (`frontend/src/components/knowledge_base/document/`). Mobile review surfaces no longer trap the user on the wrong surface after a selection; once they pick something, the layout switches to the surface where the selection is meaningful.
  - **Annotations feed → auto-switch to Document tab on row tap** (`document_kb/RightPanelContent.tsx`, `layouts/MobileDocumentLayout.tsx`). Tapping an annotation row in the Annotations surface now also switches the active tab to `Document` in addition to opening the existing detail sheet. Dismissing the sheet reveals the annotation in the viewer instead of dropping the user back on the feed. Implemented as a new optional `onFeedItemSelected?: () => void` prop on `RightPanelContent` that the mobile layout wires to `setActiveTab("document")`; desktop omits the callback and keeps byte-identical behavior.
  - **Mobile Ask bar — type-to-launch new chat + separate history affordance** (`layouts/mobile/MobileAskBar.tsx`, `right_tray/ChatTray.tsx`, `layouts/MobileDocumentLayout.tsx`). Focusing the bar no longer opens the chat sheet — users can compose their message inline on the main view and submit only when ready. Submitting opens the chat sheet directly in new-chat mode so the typed text launches a fresh conversation immediately, instead of being dropped on the conversation list. A new history icon button on the bar reaches the conversation list explicitly (replaces the old focus-opens-sheet path). The `ChatTray` gains an `autoStartNewChat?: boolean` prop that OR's into its `isNewChat` initialization; the mobile layout passes it when there is a pending message.
  - **Mobile Find sheet — tappable results list** (`layouts/mobile/MobileFindSheet.tsx`, `layouts/MobileDocumentLayout.tsx`). The find sheet now renders every match as a tappable row (snippet, page reference, match index) below the existing prev/next chevrons. Tapping a row selects that match and closes the sheet so the viewer is immediately visible with the match scrolled in. Reopening the sheet restores the same query, list, and selection from the existing atoms.
  - **Tests.** `frontend/tests/MobileAskBar.ct.tsx` — focus is a no-op, submit fires `onSubmit`, the new history button is gated on `onOpenHistory` and fires correctly. `frontend/tests/MobileFindSheet.ct.tsx` + `.harness.tsx` — seed real-shape token/span matches, assert the results list renders with per-row meta, taps update the selection and fire `onClose`. `frontend/tests/MobileDocumentKnowledgeBase.ct.tsx` — split the old "Ask bar opens Chat sheet" test into three (focus is a no-op, submit lands in new-chat mode, history button opens the list); the annotation-feed test now also asserts the Document tab becomes selected after a row tap.

### Added

- **Dedicated mobile layout for `DocumentKnowledgeBase`** (`frontend/src/components/knowledge_base/document/`; design and plan docs in `docs/frontend/mobile-document-knowledge-base-design.md` and `docs/frontend/mobile-document-knowledge-base-plan.md`). Adds `MobileDocumentLayout`, a mobile-first navigation model built on a tab bar (Document / Summary / Annotations / More) plus a persistent Ask bar that opens the chat. New mobile chrome primitives live under `frontend/src/components/knowledge_base/document/layouts/mobile/`: `MobileSheet` (bottom-sheet container), `MobileTabBar`, `MobileAskBar`, and `MobileDocToolbar`. The Document surface defaults to fit-to-width zoom on mobile. Mobile is a consumption/review surface only — there is no annotation authoring on mobile, and Analyses/Extracts are not rendered on mobile. Includes a new mobile integration test suite (`frontend/tests/MobileDocumentKnowledgeBase.ct.tsx`) plus `MobileDocumentLayout` unit tests (`frontend/tests/MobileDocumentLayout.ct.tsx`).

- **Coverage-fixture import-discipline guard** (issue #1746, follow-up to PR #1744; `frontend/src/__tests__/coverageFixtureDiscipline.test.ts`). Adds a Vitest regression test that walks `frontend/tests/` and fails if any component test (`tests/**/*.ct.tsx`) or e2e spec (`tests/e2e/**/*.spec.ts`) imports the `test`/`expect` runtime from a raw Playwright package (`@playwright/experimental-ct-react` / `@playwright/test`) instead of its Istanbul-coverage fixture (`tests/utils/coverage.ts` / `tests/e2e/fixtures.ts`). Only the extended `page` fixture in those wrapper modules reads `window.__coverage__`; a test that uses the raw import compiles, runs and passes normally while its coverage is silently discarded — an invisible failure that keeps CI green but artificially depresses `codecov/patch` (on PR #1744 seven mobile component tests reported ~30% line coverage despite fully passing suites, falsely blocking the PR). The guard mirrors `frontend/src/routing/__tests__/centralRouteDiscipline.test.ts`: it parses each file's named imports (multi-line aware), ignores type-only imports such as `import type { Page } from "@playwright/test"` so legitimate Playwright type usage is not flagged, and reports both the forbidden-package case and the missing-fixture-import case with a file-and-reason list. Runs in the existing `unit-test` CI job (`yarn test:coverage:unit`); no ESLint rule was added because the frontend does not run ESLint (lint = Prettier only). Re-audit of remaining test categories: `tests/*.story.tsx` files are render-only fixtures with no Playwright `test` import (no gap); all 172 `*.ct.tsx` and 8 `tests/e2e/*.spec.ts` files already import their fixture correctly.

- **Ingest-time enrichment stage — new `BaseEnricher` pipeline component type + `PdfOutlineEnricher`** (`opencontractserver/pipeline/base/enricher.py`, `opencontractserver/pipeline/enrichers/`, `opencontractserver/pipeline/registry.py`, `opencontractserver/pipeline/utils.py`, `opencontractserver/pipeline/base/parser.py`, `opencontractserver/documents/models.py`, `opencontractserver/constants/annotations.py`, `config/settings/base.py`, migration `0039_add_preferred_enrichers_to_pipeline_settings`). Adds a fifth pipeline component family, `BaseEnricher` — a chainable `OpenContractDocExport -> OpenContractDocExport` transform run inside `BaseParser.process_document()` between `parse_document()` and `save_parsed_data()`. Enrichers are the ingest-time analogue of the export-time `BasePostProcessor` chain: one or more run in sequence, each receiving the previous enricher's output. They are configured per MIME type as an ordered list via the new `PipelineSettings.preferred_enrichers` JSONField (default empty, so enrichment is opt-in and there is no behaviour change for existing installs); the chain runner `run_enrichers` (`pipeline/utils.py`) isolates per-enricher failures so a misbehaving enricher can never fail document ingestion. The component registry gains `ComponentType.ENRICHER`, auto-discovery of the `opencontractserver/pipeline/enrichers/` package, `get_enrichers_for_filetype`, and `get_all_enrichers_cached`; the `migrate_pipeline_settings` management command lists enrichers and syncs `PREFERRED_ENRICHERS`. Ships one enricher, `PdfOutlineEnricher`, which reads a PDF's embedded `/Outlines` bookmarks via `pypdf` and appends hierarchical `OC_SECTION` token annotations anchored to the parser's PAWLs tokens — turning embedded bookmarks into a navigable in-document table of contents. Each bookmark title is matched to its destination page's tokens (exact, then whitespace/case-normalized, then fuzzy via `difflib`); a title that cannot be anchored is dropped and its children are re-parented to the nearest matched ancestor. New constants `PDF_OUTLINE_FUZZY_MATCH_THRESHOLD`, `PDF_OUTLINE_MAX_ENTRIES`, `PDF_OUTLINE_MAX_DEPTH` in `opencontractserver/constants/annotations.py`. Tests: `opencontractserver/tests/test_base_enricher.py`, `opencontractserver/tests/test_pdf_outline_enricher.py`, `opencontractserver/tests/test_enricher_pipeline.py`, plus enricher coverage added to `test_pipeline_registry.py` and `test_pipeline_settings.py`.

- **Service-layer foundation — `BaseService`, `ServiceResult`, and IDOR-safe `get_for_user_or_none`** (`opencontractserver/shared/services/__init__.py`, `opencontractserver/shared/services/base.py`, `opencontractserver/shared/services/conventions.py`, `opencontractserver/tests/test_base_service.py`). Phase 1 of the service-layer centralization roadmap (`docs/refactor_plans/2026-05-19-service-layer-centralization-design.md`). Introduces the shared machinery every per-app service package will inherit in later phases: `ServiceResult` — a frozen generic write-result envelope (`success` / `failure` constructors, derived `ok`, `(value, error)` tuple-unpacking for backward compatibility with the legacy `(obj, error)` convention); `get_for_user_or_none(model, pk, user, permission=READ, *, request=None)` — an IDOR-safe single-object lookup that returns `None` indistinguishably for not-found, permission-denied, and malformed-pk; and `BaseService`, a staticmethod-based base class exposing `get_or_none` (IDOR-safe lookup), `filter_visible` (permission-filtered queryset via `visible_to_user`), `require_permission` (uniform write-operation gate returning `""` or a denial string), and `log_action` (structured who-did-what logging). No existing callers are migrated in this phase — it ships the primitives only, covered by 23 unit tests. `get_for_user_or_none` takes a single-query fast path for the high-frequency READ case (via `visible_to_user`), falling back to fetch-then-`user_can` only for non-READ permissions. Permission checks delegate to the existing `Model.objects.user_can` / `visible_to_user` manager API; no permission semantics change.

- **Phase F — authorization-invariant gap-fill + permissioning docs refresh** (issue #1660; final phase of the #1637 permission-centralization plan). Closes the remaining gaps in the `user_can` invariant suite and re-points the permissioning docs at the new API.
  - **Invariant suite** (`opencontractserver/tests/permissioning/test_authorization_invariants.py`). Added a top-of-file fixture-matrix audit comment and these focused tests: group-shared READ equivalence for `Corpus` (`CorpusAuthorizationInvariantsTestCase.test_group_shared_read_equivalence`); `is_public` write-asymmetry for `Note` / `Annotation` / `Conversation` / `UserFeedback` (`test_is_public_does_not_grant_writes` — a non-creator/non-shared user gets READ but never UPDATE/DELETE on a public instance, matching the pre-existing `Corpus` / `Document` pins); a `created_by_extract` annotation added to `AnnotationAuthorizationInvariantsTestCase`'s matrix so the equivalence mixin covers the extract privacy-recursion branch in-class; and `ChatMessageAuthorizationInvariantsTestCase.test_chat_message_does_not_inherit_corpus_context` pinning CHAT/THREAD bifurcation at the message layer.
  - **Audit finding — group-perm filter/check drift.** The audit's group-shared coverage surfaced a genuine per-model drift: `Manager.user_can` honours group object-permissions (`_default_user_can` resolves with `include_group_permissions=True`) but `QuerySet.visible_to_user` in `opencontractserver/shared/QuerySets.py` only consults the _user_ object-permission table, never the `*groupobjectpermission` table. `Corpus` is unaffected (its queryset handles groups); `Document`, `Annotation`, and `Note` drift. The current behaviour is pinned by `DocumentAuthorizationInvariantsTestCase.test_group_shared_known_drift_user_can_vs_visible_to_user` so it cannot change silently. Closing the drift is a query-path behaviour change left out of this test/docs-only sweep.

### Removed

- **Deleted the deprecated `user_has_permission_for_obj` shim** (issue #1661 — final cleanup of the #1637 permission-centralization plan; `opencontractserver/utils/permissioning.py`). The helper was deprecated in Phase A (#1655) and kept for one release as a `DeprecationWarning`-emitting shim that delegated to `Manager.user_can`, protecting any plugin / external code that imported the symbol. With the grace period elapsed and every in-repo caller migrated in Phases B/C/D/E (#1656, #1657, #1658, #1659), the shim and its per-call-site warning-dedup machinery are now gone — `Model.objects.user_can(user, obj, permission)` / `obj.user_can(user, permission)` is the only single-object authorization API. The ~20 test modules that still imported the symbol for parity testing were migrated to call `obj.user_can(...)` directly; the two shim-only tests (`ShimDeprecationWarningTestCase` in `test_authorization_invariants.py`, `ShimTypeErrorGuardTestCase` in `test_authorization_invariants_coverage.py`) were deleted. Doc references in `CLAUDE.md`, `docs/permissioning/consolidated_permissioning_guide.md`, and the architecture docs were re-pointed at the `user_can` API.

### Security

- **Phase D — IDOR-safe `get_for_user_or_none(model_cls, pk, user)` helper + mutation migration** (`opencontractserver/utils/permissioning.py`, `opencontractserver/tests/permissioning/test_get_for_user_or_none.py`, `opencontractserver/users/models.py`, plus the affected `config/graphql/*_mutations.py` files; closes #1658). Adds a generic non-corpus-scoped lookup helper that returns `None` whenever the pk doesn't exist OR the caller can't READ the row, so mutation call sites render a single unified `"<resource> not found or you don't have permission..."` message and stop leaking object existence via differential error text (CLAUDE.md "IDOR Prevention" — formerly `Model.objects.get(pk=X)` raising `DoesNotExist` vs a separate `user_has_permission_for_obj` permission denial returned distinguishable error states). Helper is generic over the model TypeVar so callers keep concrete `.fieldset` / `.is_public` / etc. attribute typing across the gate, and catches `ValueError`/`TypeError` from garbage pk inputs so untrusted GraphQL ids can't bubble up as 500s.
  - **READ is now a precondition for any other permission op.** Granting UPDATE / DELETE / PERMISSION on an object without an explicit READ codename used to "work" on a handful of mutations because they fetched the row with a bare `Model.objects.get(pk=...)` and then checked the higher permission directly. After Phase D every migrated mutation routes the lookup through `visible_to_user` first, so the caller must be able to see the row before any non-READ check fires. Two pinning tests were updated to grant `[READ, UPDATE]` / `[READ, PERMISSION]` instead of the higher permission alone (`test_user_with_permission_perm_can_change_visibility`, `test_non_owner_with_explicit_update_permission_can_create`, `test_remove_labels_allows_non_owner_with_explicit_update_permission`). `test_invalid_id_format_returns_error` was updated to assert the unified `ok=False` + `"not found or you don't have permission"` response now that the helper catches malformed-pk `ValueError`s instead of letting them bubble as GraphQL `errors`.
  - **Superuser bypass added to `UserProfileManager.visible_to_user`** (`opencontractserver/users/models.py:51`). Mirrors `BaseVisibilityManager`'s superuser branch so admin / moderation surfaces (e.g. badge awarding) can route the recipient lookup through `get_for_user_or_none` and still reach private-profile users — without dropping the IDOR-safe pattern for non-superuser callers (who still only see their own profile + public profiles).
  - **Migrated** (non-corpus-scoped sites; `get_for_user_or_none`): `extract_mutations.py` — `UpdateExtractMutation` (Extract / Corpus / Fieldset lookups) and `CreateExtractIteration` (source Extract). `analysis_mutations.py` — `DeleteAnalysisMutation` (previously raised three different exception types depending on existence / read-permission / delete-permission). `agent_mutations.py` — `CreateAgentConfigurationMutation`, `UpdateAgentConfigurationMutation`, `DeleteAgentConfigurationMutation`. `corpus_mutations.py` — `SetCorpusVisibility`, `UpdateCorpusMutation` preferred_embedder pre-check (previously leaked "this corpus has documents" via the early-exit), `UpdateCorpusDescription`, `DeleteCorpusMutation` `is_personal` pre-check, `StartCorpusFork`, `ReEmbedCorpus`, `CreateCorpusAction`, `AddTemplateToCorpus`, `ToggleCorpusMemory`, `AddDocumentsToCorpus`, `RemoveDocumentsFromCorpus`. `badge_mutations.py` — `AwardBadgeMutation` Badge / Corpus / User-recipient lookups. `label_mutations.py` — `DeleteMultipleLabelMutation` (creator-based permissions on `AnnotationLabel`), `CreateLabelForLabelsetMutation`, `RemoveLabelsFromLabelsetMutation`. `conversation_mutations.py` — `CreateThreadMutation` (Corpus, Document), `CreateThreadMessageMutation`, `ReplyToMessageMutation`, `DeleteConversationMutation`, `DeleteMessageMutation`. `document_mutations.py` — `RetryDocumentProcessing`. `document_relationship_mutations.py` — `CreateDocumentRelationship` (Corpus + source/target Document), `UpdateDocumentRelationship` (Corpus).
  - **Corpus-scoped document lookups** continue to flow through `CorpusObjsService.get_corpus_document_by_id` / `is_document_in_corpus` per PR #1685; the helper's docstring cross-references the service so the two IDOR-safe surfaces don't drift.
  - **`from_global_id` decode pre-guarded at every migrated site.** `get_for_user_or_none` catches malformed-pk `ValueError`/`TypeError` _after_ the ORM call, but cannot help if `from_global_id` itself raises (a `binascii.Error` on malformed base64) before the helper is reached — in that case the outer mutation `except Exception` returned a generic message instead of the unified IDOR-safe envelope. The `from_global_id` calls in `agent_mutations.py` (`Create`/`Update`/`Delete AgentConfiguration`) and `conversation_mutations.py` (`CreateThreadMutation` corpus + document arms, `CreateThreadMessageMutation`, `ReplyToMessageMutation`, `DeleteConversationMutation`, `DeleteMessageMutation`) are now wrapped so a malformed id returns the same `"not found / no permission"` response as a genuinely missing or inaccessible resource. `CreateAgentConfigurationMutation`'s malformed-id and not-found branches were unified to emit the identical string. `AddDocumentsToCorpus` / `RemoveDocumentsFromCorpus` now decode the corpus id and the document ids in separate guards so a malformed _document_ id no longer returns a misleading corpus-scoped error message.

### Changed

- **DocumentKnowledgeBase desktop layout — consolidate the three scattered bottom-edge floating controls** (issue #1735; `frontend/src/components/knowledge_base/document/layouts/DesktopDocumentLayout.tsx`, `frontend/src/components/knowledge_base/document/document_kb/styles.ts`, `frontend/src/components/knowledge_base/document/floating_summary_preview/FloatingSummaryPreview.tsx`, `frontend/src/components/annotator/labels/EnhancedLabelSelector.tsx`, `frontend/tests/DocumentKnowledgeBase.ct.tsx`). The desktop document layer used to carry three independently `position: fixed`/absolute floaters along its bottom edge — a circular SUMMARY button (`FloatingSummaryPreview`) bottom-left, a search/chat mini-pill (`FloatingDocumentInput`) bottom-center, and the active-label `OC_SECTION ×` pill (`EnhancedLabelSelector`) bottom-right — with no shared container, reading as visual noise. They are now docked inside a single anchored container, `DocumentBottomBar`, that pins all three to the same baseline at the bottom of `MainContentArea` with consistent spacing (CSS Grid `1fr auto 1fr` columns, `align-items: end`). `FloatingSummaryPreview` and `EnhancedLabelSelector` gained a `fixed?: boolean` prop (default `true`) so existing standalone usages — and the standalone test wrappers — keep their legacy `position: fixed` behaviour; `DesktopDocumentLayout` opts all three into `fixed={false}` so the new bar owns positioning. The legacy `FloatingInputWrapper` styled component is deleted (no-dead-code rule). Mobile (`MobileDocumentLayout`) and the standalone component test wrappers are unaffected. New component test `Desktop bottom toolbar consolidation (#1735)` asserts the bar is the common ancestor of all three controls, that they share the same vertical baseline within tolerance, and that the left → centre → right ordering holds; captures three documentation screenshots (`knowledge-base--document-bottom-bar--default.png`, `--consolidated.png`, `--input-expanded.png`).

- **Service-layer centralization Phase 4 — migrate the remaining query optimizers into `services/` packages** (issue #1718; depends on Phase 1 #1715). Phase 4 of the roadmap in `docs/refactor_plans/2026-05-19-service-layer-centralization-design.md`. The four remaining "query optimizer" modules are relocated into per-app `services/` packages, each service inheriting the `BaseService` foundation shipped in Phase 1; the standalone optimizer modules are deleted (no-dead-code rule) and request threading is unified onto the single `BaseService` convention.

  - **New service packages.** `opencontractserver/documents/services/` (`actions.py` → `DocumentActionsService`, `relationships.py` → `DocumentRelationshipService`, `versions.py` → `DocumentVersionService`), `opencontractserver/conversations/services/` (`conversation_service.py` → `ConversationService`), `opencontractserver/users/services/` (`user_service.py` → `UserService`), `opencontractserver/badges/services/` (`badge_service.py` → `BadgeService`). Each package `__init__.py` re-exports its public service class(es) so callers import from the package, not an internal module path.
  - **Deleted modules.** `opencontractserver/documents/query_optimizer.py`, `opencontractserver/conversations/query_optimizer.py`, `opencontractserver/users/query_optimizer.py`, `opencontractserver/badges/query_optimizer.py`. The renamed classes drop the `QueryOptimizer` suffix for `Service` (`DocumentActionsQueryOptimizer` → `DocumentActionsService`, etc.) — the term "query optimizer" is retired as a public API concept for these apps.
  - **Unified request threading.** Every public service method now takes an optional keyword-only `request` parameter (the `BaseService` convention), replacing the optimizers' three legacy styles: the `documents` optimizers' positional `context=` kwarg becomes `request=`; the instance-based `ConversationQueryOptimizer(user)` + module-level `get_request_optimizer(request)` are retired in favour of classmethod-based `ConversationService` whose request-scoped visible-id cache lives on the `request` object (matching the `documents`/`versions` caching pattern). `UserService`/`BadgeService` accept `request` for API consistency (their visibility is pure `Q`-object filtering, mirroring `BaseService.filter_visible`'s documented stance). Resolver/mutation call sites in `config/graphql/` now thread `request=info.context` so the Tier-2 permission cache is engaged.
  - **Behaviour preserved.** This is a relocation + encapsulation, not a rewrite — prefetch builders, bulk effective-permission annotation, and request caching move byte-for-byte. The two genuinely dead pieces are dropped: `DocumentRelationshipQueryOptimizer._check_corpus_permission` (defined, never called) and `ConversationQueryOptimizer.invalidate_caches` (an instance-cache method with no meaning in the classmethod/request-cache model).
  - **Call sites migrated.** `config/graphql/action_queries.py`, `corpus_queries.py`, `document_types.py`, `document_relationship_mutations.py`, `document_queries.py`, `user_queries.py`, `search_queries.py`, `social_queries.py`. Stale class-name references in code comments (`opencontractserver/documents/models.py`, `conversations/models.py`, `constants/permissioning.py`, `utils/permission_optimizer.py`, `config/graphql/extract_types.py`) and the `docs/architecture/query_permission_patterns.md` optimizer table are updated to the new service names/paths.
  - **Tests.** New structural module `opencontractserver/tests/test_service_layer_phase4.py` (package importability, `BaseService` inheritance, retired-module guard, request-keyword-only signature check). The existing optimizer regression suites — `test_document_actions_permissions.py`, `test_document_relationship_permissions.py`, `test_document_relationship_mutations.py`, `test_document_relationship_count_optimizer.py`, `test_document_version_count_optimizer.py`, `test_conversation_permissions.py`, `test_user_visibility.py`, `test_badge_visibility.py` — are repointed at the service classes; `test_conversation_permissions.py`'s optimizer test class is rewritten for the classmethod API and now covers request-level caching plus the no-request recompute path.

- **Service-layer centralization — Phase 2C: caller migration + `corpus_objs_service.py` shim removal** (issue #1716; ~40 files across `config/graphql/`, `opencontractserver/mcp/`, `opencontractserver/llms/`, `opencontractserver/tasks/`, `opencontractserver/discovery/`, `opencontractserver/utils/`, and the test suite). Every caller of the deprecated `CorpusObjsService` facade is migrated onto the segmented services it shipped behind (`from opencontractserver.corpuses.services import CorpusDocumentService` / `FolderCRUDService` / `FolderDocumentService` / `DocumentLifecycleService` / `CorpusPathService`), and `opencontractserver/corpuses/corpus_objs_service.py` — the thin re-export shim that Phase 2A kept for one release — is deleted along with the `CorpusObjsService` facade class (CLAUDE.md no-dead-code rule). Each `CorpusObjsService.<method>` call is rewritten to its single owning service; the mapping is deterministic because the segmented services have disjoint method names. `test_corpus_objs_service.py` (the behavioural regression suite, ~290 tests) keeps every scenario and assertion unchanged — only the `CorpusObjsService.<method>` call targets are repointed at the owning services. `test_corpus_services_package.py` drops the now-obsolete shim/facade structural tests (the facade no longer exists) and gains `CorpusService` in its package-structure coverage; `test_document_service.py`'s service-split structural tests now assert against the segmented services. `CLAUDE.md` rule 7, the consolidated permissioning guide, and the affected docstrings are repointed at the `opencontractserver.corpuses.services` package. Pure relocation of import paths and call targets — no permission semantics, transaction boundary, or query shape changes.

- **Service-layer centralization — Phase 2B: `CorpusService` for Corpus-row CRUD** (issue #1716; `opencontractserver/corpuses/services/corpus_service.py`, `opencontractserver/corpuses/services/__init__.py`, `config/graphql/corpus_mutations.py`, new `opencontractserver/tests/test_corpus_service.py`; plan in `docs/refactor_plans/2026-05-22-service-layer-phase2bc-corpus-service-and-caller-migration.md`). Adds the sixth segmented corpus service — `CorpusService(BaseService)` — owning the write surface of the `Corpus` row itself, closing the design-doc §3 Problem 3 gap (corpus _contents_ had a service; the corpus _row_ did not). `CorpusService` provides `delete_corpus` (personal-corpus / user-lock / DELETE-permission gates), `set_visibility` (PERMISSION-gated public/private switch + `make_corpus_public_task` cascade), `update_description` (creator-only markdown-description versioning), `assert_embedder_change_allowed` (issue #437 embedder guard), and `grant_creator_permissions` (CRUD + PUBLISH + PERMISSION). The five corpus-row GraphQL mutations now delegate their corpus-specific logic to `CorpusService`: `SetCorpusVisibility`, `UpdateCorpusDescription`, and `DeleteCorpusMutation` become thin wrappers; `CreateCorpusMutation` / `UpdateCorpusMutation` keep their shared `DRFMutation` create/update machinery but call `CorpusService` for the creator-permission grant and the embedder guard. No GraphQL contract change. Minor behaviour note: `set_visibility` passes the corpus's integer pk to `make_corpus_public_task` (the former inline mutation passed the `from_global_id` string — the task accepts either); the `@patch` target for that task in `test_corpus_visibility.py` moved to `opencontractserver.tasks.permissioning_tasks` accordingly.

- **`DocumentKnowledgeBase` renders a mobile layout below 768px** (`frontend/src/components/knowledge_base/document/`). `DocumentKnowledgeBase` now renders `MobileDocumentLayout` below a 768px viewport width and the extracted `DesktopDocumentLayout` at or above it; desktop behavior is unchanged. The desktop floating components are now desktop-only — their dead `isMobile` branches were removed since the mobile layout no longer mounts them.

- **Typing: graduated the `tests.permissioning.*` 18-module subpackage out of the mypy baseline** (issue #1738, continuing the #1331 → #1335 → #1447 cadence; the next batch after the `config.*` tail). Removed all 18 `[mypy-opencontractserver.tests.permissioning.*]` `ignore_errors` blocks from `mypy.ini`, pruned the corresponding 313 lines from `docs/typing/mypy_baseline.txt` (5592 → 5279), and fixed the 162 errors that surfaced. Pre-commit `mypy` continues to pass on the full project surface (`mypy --config-file mypy.ini opencontractserver config` → "Success: no issues found in 1157 source files"). Most errors were the standard Django `setUpTestData(cls)` class-attribute pattern flagged by mypy (recommended fix from #1479: annotate the assigned names as class-level attributes). One real production-code gap surfaced:

  - **`CoreAgent` Protocol was missing its public `config: AgentConfig` attribute** (`opencontractserver/llms/agents/core_agents.py`). Five tests read `agent.config.tools` / `agent.config._approval_bypass_allowed` directly off agents handed back by `llm_agents.for_corpus(...)` / `for_document(...)`. The factory returns the `CoreAgent` Protocol; the concrete `CoreAgentBase` initializes `self.config = config` in `__init__`, but the Protocol never declared it. Tests worked at runtime (the attribute exists on every concrete agent) but failed type-checking. Added `config: AgentConfig` to the Protocol body — captures the existing public API contract without a behaviour change.
  - **Per-file fixes (test side).** `test_query_optimizer_methods.py` (108 errors) — added class-level annotations to `AnalysisServicePermissionTestCase` and `DocumentPermissionFilterQueryCountTestCase` covering every `cls.<attr> = ...` assignment inside `setUpTestData`, switched `User = get_user_model()` to a direct `from opencontractserver.users.models import User` import so the name is usable as a type annotation, and replaced three `qs.first().<attr>` accesses with `qs[0].<attr>` after the pre-existing `qs.count() == 1` guard. `test_metadata_query_optimizer.py` (15 errors) — added `assert status is not None` narrowings before indexing into the `dict | None` return of `MetadataService.get_metadata_completion_status`, and a `column is not None` narrowing in `test_validate_column_valid`. `test_permissioning.py` (10 errors) — same `setUpTestData` annotation pattern on `SetPermissionsIsNewTests`, same `get_user_model()` → direct import switch. `test_permissioned_querysets.py` (8 errors) — narrowed `qs.query.select_related` (`dict | bool`) via `assert isinstance(..., dict)` before three `assertIn` calls, and gave two `PermissionQuerySet` constructions explicit type annotations so the generic queryset type is captured. `test_public_and_permissions.py` (7 errors) — removed six unused `# type: ignore[attr-defined]` comments on `Model.objects.acount()` calls (django-stubs now ships proper async-manager typing). `test_version_aware_query_optimizer.py` (7 errors) — replaced six `result.first().id` accesses with `result[0].id` after the pre-existing `result.count() == 1` guard. `test_custom_permission_filters.py` (4 errors) — replaced two `assertIsNotNone` calls with `assert` narrowings so mypy could see the subsequent `corpusN_edge["node"]...` was non-`None`. `test_creator_based_permissions.py` (2 errors) — narrow `# type: ignore[arg-type]` on the two `get_users_permissions_for_obj(instance=mock_instance, ...)` calls (the test deliberately passes a non-`Model` mock to exercise the no-guardian-tables fallback). `test_comment_permission.py` (1 error) — replaced one `assertIsNotNone` with `assert` for narrowing.

- **Typing: graduated the `config.*` tail out of the mypy baseline** (issue #1738, continuing the #1331 → #1335 → #1447 cadence). Removed the five remaining `config.*` `[mypy-…]` `ignore_errors` blocks from `mypy.ini` — `config.settings.{base,local,test_integration}` and `config.websocket.consumers.{thread_updates,unified_agent_conversation}` — and pruned the corresponding 17 lines from `docs/typing/mypy_baseline.txt` (5611 → 5594). The entire `config.*` surface is now type-checked from scratch; only the `opencontractserver.*` test-suite modules, `opencontractserver.mcp.tests.test_mcp`, and the three intentional `django-manager-missing` model blocks remain on the baseline. Per-file fixes:

  - `config/settings/base.py`: annotated `SECURE_PERMISSIONS_POLICY: dict[str, list[str]]` and `PIPELINE_SETTINGS: dict[str, dict[str, Any]]` — both are literals (all-empty-list values / an empty dict) whose element types mypy cannot infer (`var-annotated`).
  - `config/settings/local.py`: moved `DEBUG = env.bool("DJANGO_DEBUG", False)` above the `if DEBUG and USE_SILK:` Silk-profiler gate. Besides clearing the `used-before-def` error this fixes a latent ordering bug — the gate previously read `base.py`'s star-imported `DEBUG` rather than the env-resolved value assigned a few lines below it. No behaviour change today (`USE_SILK` is hard-coded `False`), but the gate now consults the correct `DEBUG`.
  - `config/settings/test_integration.py`: typed the django-redis `CACHES` override through an annotated `_caches: dict[str, dict[str, Any]]` intermediate. A direct `CACHES: … = {…}` annotation is a `no-redef` because `CACHES` arrives via `from .test import *`; routing through the intermediate gives the nested `OPTIONS` dict the inference context it needs without re-declaring the name.
  - `config/websocket/consumers/thread_updates.py`: `_check_conversation_access` gains an explicit `if self.conversation_id is None: return False` guard — the `Conversation.objects.get(pk=…)` lookup rejected the attribute's `int | None` type. Behaviour is unchanged (a `None` id previously fell through to `DoesNotExist` → `False`).
  - `config/websocket/consumers/unified_agent_conversation.py`: `settings.LLMS_DEFAULT_AGENT_FRAMEWORK` (a `str`) is now converted to the `AgentFramework` enum before being passed to `agents.for_document` / `for_corpus` (whose `framework` parameter is `AgentFramework | Literal["pydantic_ai"] | None`); the conversion also validates the setting value eagerly. `_stream_agent_response` gains an `if self.agent is None` guard mirroring the existing one in `_handle_approval_decision`.
  - `opencontractserver/conversations/models.py`: expanded the `StreamingMessageType` `Literal` from 4 to 10 members. It omitted `ASYNC_THOUGHT`, `ASYNC_SOURCES`, `ASYNC_APPROVAL_NEEDED`, `ASYNC_APPROVAL_RESULT`, `ASYNC_RESUME`, and `ASYNC_ERROR` — all of which `unified_agent_conversation.py` already passes to `_send_safe` / `send_standard_message`. The incomplete alias was a latent type-correctness gap (the type claimed only 4 message kinds were valid while the consumer emits 10); it is consumed only by that consumer.
  - Pre-commit `mypy` continues to pass on the full project surface (`mypy --config-file mypy.ini opencontractserver config` → "Success: no issues found in 1134 source files").

- **Service-layer centralization — Phase 2A: split the `CorpusObjsService` monolith** (issue #1716; `opencontractserver/corpuses/services/`, `opencontractserver/corpuses/corpus_objs_service.py`, `opencontractserver/tests/test_corpus_objs_service.py`, new `opencontractserver/tests/test_corpus_services_package.py`; plan in `docs/refactor_plans/2026-05-21-service-layer-phase2-corpus-services-plan.md`). The ~2,920-line `corpus_objs_service.py` monolith is split into the segmented `corpuses/services/` package — `paths.py` (`CorpusPathService`), `corpus_documents.py` (`CorpusDocumentService`), `lifecycle.py` (`DocumentLifecycleService`), `folders.py` (`FolderCRUDService` — folder CRUD, tree, search), and `folder_documents.py` (`FolderDocumentService` — document-in-folder placement and queries) — one cohesive module per responsibility, each inheriting `BaseService` (Phase 1, #1715) and each within the ~800-line module guideline. Pure relocation, no behaviour change: all 40 methods move byte-for-byte modulo 14 documented cross-module call rewrites. `corpus_objs_service.py` becomes a deprecated facade shim that multiply-inherits the five services, so existing `CorpusObjsService` imports and call sites keep working unchanged; caller migration and shim deletion are deferred to Phase 2C. Regression-covered by the unchanged `test_corpus_objs_service.py` behavioural suite (run through the facade) plus new `test_corpus_services_package.py` (structural tests: package layout, `BaseService` inheritance, facade MRO, standalone operation of each service, cross-service delegation).

- **Service-layer centralization — Phase 3: split the `annotations/query_optimizer.py` monolith into per-app `services/` packages** (issue #1717; depends on the Phase 1 `BaseService` foundation #1715). Phase 3 of the roadmap in `docs/refactor_plans/2026-05-19-service-layer-centralization-design.md` (§6). The 1,507-line `opencontractserver/annotations/query_optimizer.py` hosted four classes — two of which (`Analysis`, `Extract`) were not annotation concerns and were misfiled in the `annotations` app. Each class is relocated into its correct app's `services/` package, renamed from `*QueryOptimizer` to `*Service`, and now inherits the Phase 1 `BaseService`:
  - `AnnotationQueryOptimizer` → `AnnotationService` (`opencontractserver/annotations/services/annotation_service.py`)
  - `RelationshipQueryOptimizer` → `RelationshipService` (`opencontractserver/annotations/services/relationship_service.py`)
  - `AnalysisQueryOptimizer` → `AnalysisService` (`opencontractserver/analyzer/services/analysis_service.py` — relocated to its correct app)
  - `ExtractQueryOptimizer` → `ExtractService` (`opencontractserver/extracts/services/extract_service.py` — relocated to its correct app)
  - `MetadataQueryOptimizer` → `MetadataService` (`opencontractserver/extracts/services/metadata.py`, relocated from the now-deleted `opencontractserver/extracts/query_optimizer.py`)
  - **Pure, behaviour-preserving relocation.** Every method body moves byte-for-byte — permission checks, prefetch shapes, the request-scoped permission/instance caches, and the `MIN(document, corpus)` effective-permission model are unchanged. The only per-method edit is in `RelationshipService`, whose three references to the shared `_compute_effective_permissions` / `_get_document_for_request` helpers are repointed from `AnnotationQueryOptimizer` to `AnnotationService`. The `context=` request-threading kwarg is preserved (standardizing it to `request=` is a later phase). No permission semantics change.
  - **No re-export shim.** Both `opencontractserver/annotations/query_optimizer.py` and `opencontractserver/extracts/query_optimizer.py` are deleted and every caller is migrated onto the new import paths — 12 production files (`config/graphql/{extract_queries,extract_mutations,corpus_queries,extract_types,annotation_queries,corpus_types,custom_resolvers,document_types}.py`, `opencontractserver/mcp/{resources,tools}.py`, `opencontractserver/shared/Managers.py`, `opencontractserver/documents/query_optimizer.py`) plus 12 test modules and assorted comment/docstring references. Per the CLAUDE.md no-dead-code rule and design-doc §9, "query optimizer" is no longer a public API concept for these five classes.
  - **Tests.** The existing optimizer test modules remain the behavioural regression gate and run unchanged except for the mechanical import/class-name repoint (zero scenario/assertion changes). New `opencontractserver/tests/test_service_layer_phase3.py` pins the structural contract — package exports, `BaseService` inheritance, the public-method surface, and the absence of the deleted modules. `docs/architecture/query_permission_patterns.md` (Layer-2 table) is repointed at the new class names/paths and the two advisory `docs/typing/mypy_baseline.txt` entries are renamed. Implementation plan: `docs/refactor_plans/2026-05-21-phase3-split-annotations-query-optimizer.md`.
- **Complete the `DocumentFolderService` removal — regenerate `schema.graphql`** (issue #1683; `schema.graphql`). Grace-period follow-up to #1680 (PR #1701), which deleted `opencontractserver/corpuses/folder_service.py`, removed the `DocumentFolderService` back-compat alias class, migrated every Python call site and docstring to `CorpusObjsService`, and dropped the CLAUDE.md "alias subclass" note — but did not regenerate the committed GraphQL SDL. `schema.graphql` (last regenerated at #1646) therefore still carried 18 stale `Delegates to DocumentFolderService.X()` description lines covering nine corpus/folder mutations (`restoreDeletedDocument`, `addDocumentsToCorpus`, `removeDocumentsFromCorpus`, `createCorpusFolder`, `updateCorpusFolder`, `moveCorpusFolder`, `deleteCorpusFolder`, `moveDocumentToFolder`, `moveDocumentsToFolder`) — each described once on the `Mutation` field and once on its payload type. All 18 now read `Delegates to CorpusObjsService.X()`, matching the resolver docstrings #1680 already updated in `config/graphql/{corpus_folder_mutations,corpus_mutations,document_mutations}.py`; the substitution is exactly what `manage.py graphql_schema` regeneration produces for those lines. No `DocumentFolderService` reference now remains in the repository outside `CHANGELOG.md` historical entries and `docs/releases/v3.0.0.b3.md` — the latter an immutable point-in-time release snapshot, deliberately left untouched per the decision documented in #1680's changelog entry.

- **Permissioning documentation refresh — `user_can` is the canonical authorization API** (issue #1660; `docs/permissioning/consolidated_permissioning_guide.md`, `CLAUDE.md`, `mkdocs.yml`). Documents the multi-phase permission-centralization outcome:
  - **`Manager.user_can` / `obj.user_can` API** (shipped by PR #1663). The canonical single-object authorization check, paired with the `visible_to_user` queryset filter and pinned to agree by `test_authorization_invariants.py`. The consolidated guide's structural-enforcement banner, "single source of truth" section, inheritance references, overview tables, and Agent/LLM section now point at `AnnotationManager.user_can` / `RelationshipManager.user_can` (`opencontractserver/shared/Managers.py`) instead of `user_has_permission_for_obj` at stale `permissioning.py` line numbers. New "The `user_can` Authorization API" section added with a checklist for extending `user_can` to a new visibility-managed model.
  - **Two-tier permission cache** (PR #1665). New guide subsection: Tier-1 per-instance memoization (always on, used by Celery/agent/internal code) vs. Tier-2 request-scoped optimizer (opt-in via `request=info.context`). Convention: pass `request=` in GraphQL resolvers/mutations, omit it everywhere else.
  - **Service-layer split** (PR #1685). New guide subsection pointing corpus-scoped reads at `CorpusObjsService` and document-level operations at `DocumentService`, with the IDOR rationale for not hand-fusing `corpus.get_documents()` with `Document.objects.visible_to_user(user)`.
  - **Deprecated shim.** `user_has_permission_for_obj` is documented as a `DeprecationWarning`-emitting shim that delegates to `Manager.user_can`; removal is tracked in issue #1661 (one release after Phase E).
  - **Creator-status semantic widening** (Phase A). `user_can` honours creator access uniformly — a creator who never received an explicit guardian grant now passes the check, matching the read-side `visible_to_user` filter. Affected paths: annotation recursive-privacy checks (`AnnotationManager.user_can` honours the creator of the source `Analysis`/`Extract`), non-Corpus mutations that previously called bare `user_has_permission_for_obj`, the CAML article edit gate (`caml_article.py`), and the image tools (`image_tools.py`); the guide's Agent/LLM tool-author guidance now tells tool authors not to re-add bare `creator_id == user.pk` OR-branches.
  - **`docs/permissioning/REMEDIATION_GUIDE.md` deleted** along with its `mkdocs.yml` nav entry — a point-in-time remediation tracker whose `user_has_permission_for_obj`-era findings are fully superseded by the centralized API.

### Fixed

- **Fix "Error loading agents" on the Global Agents admin view — pass `scope` as a GraphQL enum literal** (issue #1750; `frontend/src/components/admin/global_agent_management.graphql.ts`, `frontend/tests/admin-components.ct.tsx`, `frontend/src/components/admin/__tests__/global_agent_management.graphql.test.ts`). The `GET_GLOBAL_AGENTS` query passed `agentConfigurations(scope: "GLOBAL")` with `"GLOBAL"` as a quoted string. graphene-django derives the `scope` filter argument from the `AgentConfiguration.scope` model choices, so it is the `AgentsAgentConfigurationScopeChoices` GraphQL **enum** — a quoted string fails server-side query validation with `Argument 'scope' has invalid value "GLOBAL"`, and the admin view rendered an "Error loading agents" panel instead of the agent list. Fix: write the argument as the bare enum literal `agentConfigurations(scope: GLOBAL)`. The mirrored copy of the query in `admin-components.ct.tsx` (kept in sync so Apollo `MockedProvider` matches the production document) was updated identically, and `global_agent_management.graphql.test.ts` gained a regression test asserting the `scope` argument AST node is an `EnumValue`, not a `StringValue`.

- **Make the Admin Settings subviews usable on mobile** (issue #1749; `frontend/src/components/layout/SharedSegments.ts`, `frontend/src/components/badges/BadgeManagement.tsx`, `frontend/src/components/admin/GlobalAgentManagement.tsx`, `frontend/src/components/admin/WorkerAccountManagement.tsx`). The Badge Management, Global Agents, and Worker Accounts views each render a wide multi-column `<Table>` with no horizontal-scroll container, so on a phone-width viewport the browser crushed the columns until cell content wrapped character-by-character (the badge pills in Badge Management collapsed to roughly one letter per line). New shared `ScrollableTableWrapper` styled component (`SharedSegments.ts`) wraps each table in an `overflow-x: auto` container and pins the inner `<table>` to a readable `min-width` (600/720/760px respectively), so wide tables now scroll horizontally on small screens while keeping their normal full-width layout on desktop. Each view's page header (a `display: flex` row with `justify-content: space-between`) gained `flex-wrap: wrap` plus a `gap` so the title and action button stack instead of colliding, and each view's `Container` padding drops from `2rem`/`2em` to `1rem` below the `MOBILE_VIEW_BREAKPOINT` (600px). The shared `PageHeader` styled component now lives in `SharedSegments.ts` (the three admin views previously each defined a near-identical local copy); per-view variants extend it via `styled(PageHeader)` for `align-items` / `margin-bottom` overrides. New mobile-viewport Playwright CT tests in `BadgeManagement.ct.tsx`, `global-agent-management.ct.tsx`, and `WorkerAccountManagement.ct.tsx` assert each table stays inside a horizontally scrollable wrapper.

- **`CorpusService.set_visibility` now dispatches the public-cascade task on commit, not before** (issue #1761, follow-up to #1745; `opencontractserver/corpuses/services/corpus_service.py:150-171`, `opencontractserver/tests/test_corpus_service.py`, `opencontractserver/tests/permissioning/test_corpus_visibility.py`). The Phase 2B refactor (#1745) carried over the pre-existing inline pattern `make_corpus_public_task.si(corpus_id=corpus.pk).apply_async()` byte-for-byte from `SetCorpusVisibility.mutate`. Under `ATOMIC_REQUESTS = True` (the default for both `config.settings.base` and `config.settings.production`) the GraphQL mutation runs in a request-wrapping transaction, so the unwrapped `apply_async` could (a) race a Celery worker against uncommitted state — the worker reads `corpus.is_public` before the mutation has committed it — and (b) fire the cascade after the request transaction rolls back on a mid-handler exception, publicising every child object (documents, annotations, analyses, ...) of a corpus whose own `is_public` flag never persisted. The dispatch is now wrapped in `transaction.on_commit(lambda: make_corpus_public_task.si(corpus_id=corpus_pk).apply_async())`, matching every other Celery dispatch site in `config/graphql/corpus_mutations.py` (`StartCorpusFork` → `dispatch_fork_task`, `ReEmbedCorpus` → `reembed_corpus`, etc.). The `corpus.pk` is read into a local before the closure so the lambda body does not depend on the `corpus` instance still being attached at commit time. New regression test `test_make_public_task_not_dispatched_on_rollback` (`opencontractserver/tests/test_corpus_service.py`) pins the safety invariant by running `set_visibility` inside a forced-rollback `transaction.atomic()` and asserting `make_corpus_public_task.si` is never called; the existing positive test `test_make_public_dispatches_cascade_task` is wrapped in `self.captureOnCommitCallbacks(execute=True)` to fire the deferred callback synchronously under `TestCase`, and the GraphQL-level `test_owner_can_make_corpus_public` (`opencontractserver/tests/permissioning/test_corpus_visibility.py`) gets the same `captureOnCommitCallbacks` wrapper so its `mock_task.si.assert_called_once_with(corpus_id=self.corpus.id)` assertion still pins task arguments. Pure correctness fix — no GraphQL contract change, no permission semantics change.

- **Imported folder-organized corpora appeared empty — make `documents` folder filtering descendant-aware** (`config/graphql/filters.py`, `DocumentFilter.in_folder`; `frontend/src/components/documents/CorpusDocumentCards.tsx`, `frontend/src/graphql/queries.ts`; `opencontractserver/tests/test_document_queries.py`). Importing a V2 corpus export whose documents are all nested inside leaf folders (e.g. an `edgar-fetch` corpus with a `filer → form-type → filing` folder tree) produced a corpus that looked empty: the corpus documents view showed nothing at the root and nothing in the upper folders, even though every document imported correctly. Root cause was purely in the read path — the V2 import itself places all documents and folders correctly. `DocumentFilter.in_folder` used _exact_ folder membership: the default view passes `inFolderId="__root__"` → `DocumentPath.filter(folder__isnull=True)` → 0 rows when every document is foldered, and selecting a filer/form-type folder that contains only sub-folders matched 0 documents because the documents live in deeper leaf folders. The filter is now descendant-aware: `"__root__"` (no folder selected) applies no folder filter so the corpus-scoping `in_corpus` filter alone defines the set (every document in the corpus), and selecting a specific folder returns documents in that folder _and all of its descendant folders_ via `CorpusFolder.get_descendant_folders()`. Both symptoms ("empty index", "no docs in any folder") are resolved with no data change. New `DocumentFolderFilterQueryTestCase` pins the descendant-aware behaviour (parent folder surfaces nested docs; leaf folder returns its docs; corpus root returns every document).

- **Discover search box now returns matches for case-mismatched and partial-word queries** (`config/graphql/filters.py`, `config/graphql/corpus_queries.py`, `config/graphql/search_queries.py`, `opencontractserver/annotations/models.py`, migration `annotations/0074_annotation_raw_text_trigram_index`). The cross-content discover search (`frontend/src/views/DiscoverSearchResults.tsx` — Discussions / Annotations / Collections / Notes) rarely returned results even for terms known to exist. Two root causes:

  - **Case-sensitive corpus and discussion search.** `CorpusFilter.text_search_method` and `ConversationFilter`'s `title` filter matched with Django's `__contains`, which compiles to SQL `LIKE` — case-sensitive on PostgreSQL. Corpus and thread titles are almost always Title-Cased ("Merger Agreement"), so a lowercase query (`merger`) matched nothing. Both now use `__icontains` (`ILIKE`). The conversation title filter is declared explicitly as `title__contains` (keeping the `title_Contains` GraphQL argument unchanged) so every `GET_CONVERSATIONS` caller — discover, thread lists, global discussions — inherits the case-insensitive behaviour. `resolve_corpus_filter_counts` was switched to `__icontains` in lockstep so the Corpuses-view tab badge counts stay consistent with the now case-insensitive result set.
  - **Annotation search missed partial words.** `resolve_search_annotations_for_mention` matched annotation text only via PostgreSQL full-text search on `search_vector`, which matches whole, stemmed lexemes — never substrings — so typing a fragment or prefix (`indemn`) returned nothing until a complete word was entered. The resolver now also ORs in `Q(raw_text__icontains=...)`; full-text search is kept alongside it so stemmed cross-form matches (`running` → `ran`) still work. A new pg_trgm GIN index on `Annotation.raw_text` (`annotation_raw_text_trgm_gin`, migration `0074`, which also enables the `pg_trgm` extension) keeps the `ILIKE` substring lookup index-backed instead of degrading to a sequential scan as the table grows. Tests: `test_corpus_list_filters.py` (case-insensitive corpus search + counts), `test_conversation_query.py` (case-insensitive thread title search), `test_query_resolvers.py` (annotation mid-word fragment match).

- **`BaseVisibilityManager.visible_to_user` now honours group object-permissions** (issue #1731 — the #1714 follow-up; `opencontractserver/shared/Managers.py`, `opencontractserver/tests/test_corpus_action_execution.py`). Issue #1714 closed the same filter/check drift in the `shared/QuerySets.py` bodies but left the structurally identical user-table-only pattern in `BaseVisibilityManager.visible_to_user` for a follow-up, since the only model reaching that body un-overridden is `CorpusActionExecution` (`CorpusActionExecutionManager.visible_to_user` merely re-wraps `super().visible_to_user(...)` in a custom QuerySet). `Manager.user_can` resolves group object-permissions — `_default_user_can` runs with `include_group_permissions=True` — but the manager body only joined the `*userobjectpermission` table, so a user whose only READ grant on a `CorpusActionExecution` was via a **group** passed `obj.user_can(user, READ)` yet was excluded from `CorpusActionExecution.objects.visible_to_user(user)` — authorized for the row but never able to see it in a list query. Fix: the body now also resolves the `*groupobjectpermission` table (filtered by the requesting user's group ids) and ORs `Q(id__in=<group subquery>)` into the visibility filter, mirroring the #1714 `QuerySets.py` fix. The user- and group-table lookups sit in separate `try`/`except LookupError` blocks so a model missing the group table never discards already-resolved user grants; group ids resolve as a lazy `values_list` subquery, keeping the lookup a single round-trip. New `CorpusActionExecutionGroupVisibilityTest` pins filter/check equivalence for a group-shared user, the read/write asymmetry (a group READ grant never widens to UPDATE/DELETE), non-member exclusion, and graceful degradation when the guardian tables can't be resolved.

- **Remove white horizontal stripes from multi-line annotation highlights** (`frontend/src/components/annotator/display/components/SelectionBoundary.tsx`). Annotation highlights are drawn as two layers: a single `SelectionBoundary` rectangle covering the whole bounding box, and per-token `SelectionBox` rectangles that only cover each word's glyph extent. With the default UI (`showAnnotationBoundingBoxes` is `false`), `SelectionBoundary` rendered nothing — its background opacity was forced to `0` whenever `showBoundingBox` was off — so a multi-line annotation showed only the per-token boxes with the unhighlighted line-leading gap between text rows visible as a white horizontal stripe through every line. The gap scales with zoom while the gap-bridging `TOKEN_EXPANSION_PX` (1px) and `TOKEN_SHADOW_BLUR` (4px) are fixed pixels, so the stripes appeared at the higher zoom levels typical of mobile viewing and were most obvious on bold/wrapped section-header annotations. The fix decouples the boundary's translucent fill from the `showBoundingBox` toggle: the fill (`BOUNDARY_OPACITY_SELECTED` / `BOUNDARY_OPACITY_UNSELECTED`) now renders whenever the annotation is not `hidden`, backing the per-token highlights so inter-line gaps no longer show through as white. Only the box-shadow halo remains gated by `showBoundingBox`, so the bounding-box toggle still controls the halo as before. The identical decoupling is applied to `ResultBoundary.tsx` (used for search results and chat sources), which shared the same two-layer rendering pattern and the same latent stripe artifact.

- **`QuerySet.visible_to_user` now honours group object-permissions for Document / Annotation / Note** (issue #1714; `opencontractserver/shared/QuerySets.py`, `opencontractserver/tests/permissioning/test_permissioned_querysets.py`). The Phase F authorization-invariant audit (#1660) surfaced a genuine per-model filter/check drift. `Manager.user_can` resolves group object-permissions — `_default_user_can` runs with `include_group_permissions=True` — but `PermissionQuerySet.visible_to_user`, `DocumentQuerySet.visible_to_user`, and `AnnotationQuerySet.visible_to_user` only joined the _user_ object-permission table (`*userobjectpermission`), never `*groupobjectpermission`. A user whose only READ grant on a Document / Annotation / Note was via a **group** therefore passed `obj.user_can(user, READ)` yet was excluded from `Model.objects.visible_to_user(user)` — authorized for a single object but never able to see it in any list query. `Corpus` was unaffected (its `PermissionedTreeQuerySet` resolves group perms via guardian's `get_objects_for_user`). Fix: each affected queryset now ORs in a subquery against the model's `*groupobjectpermission` table filtered by the requesting user's group ids — `DocumentQuerySet` / `PermissionQuerySet` add one `Q(id__in=<group subquery>)`, `AnnotationQuerySet` adds group subqueries to both its document- and corpus-visibility predicates. `NoteQuerySet` needs no code change: it composes `Document.objects.visible_to_user` / `Corpus.objects.visible_to_user`, so it inherits the fix transitively. Group ids are resolved as a lazy `values_list` subquery, so the filter stays a single round-trip — query count is independent of how many groups the user belongs to (pinned by `test_group_lookup_uses_subquery_not_per_group_round_trips`). New `GroupObjectPermissionVisibilityTest` pins filter/check equivalence for a group-shared user across Document, Annotation, Note, and the generic `PermissionQuerySet` body, plus the read/write asymmetry (a group READ grant never widens to UPDATE/DELETE) and non-member exclusion.

- **Stop `EnabledComponentsMutationTestCase` poisoning the `PipelineSettings` singleton cache for later test classes** (`opencontractserver/tests/test_pipeline_settings.py`). The class's `setUp` sets `default_embedder=""`; a rejection-path mutation under test then calls `PipelineSettings.get_instance(use_cache=True)`, which caches the empty-embedder instance in the process-global, non-transaction-aware `LocMemCache`. The DB row rolls back on teardown but the cached Python object does not, so the next test class on the same xdist worker inherited the poisoned singleton and agent / vector-store tests failed with "resolved no embedder_path". Added a `tearDown` that clears `PipelineSettings.CACHE_KEY` after every test in the class, scoping the fix to the one offending class rather than a global autouse fixture.

- **Restore MIN-permission semantic on the GraphQL `CorpusType.documents` resolver — stop private documents leaking through readable corpora** (issue #1682, security; `opencontractserver/corpuses/corpus_objs_service.py`, `config/graphql/corpus_types.py`, `CLAUDE.md`, `opencontractserver/tests/test_corpus_objs_service.py`, `opencontractserver/tests/test_caml_pipeline_coverage.py`). PR #1699's CAML migration silently flipped `corpus_types.py:resolve_documents` from the MIN-permission semantic (a `Document.objects.visible_to_user(user)` intersection — the resolver's documented behaviour) to the corpus-as-gate `CorpusObjsService.get_corpus_documents`. The flip meant any document with an active path in a corpus the requester could merely READ — including a private document inside a shared or public corpus that the requester had no document-level access to — surfaced through the GraphQL `documents` field, contradicting the `MIN(document_permission, corpus_permission)` model documented in `CLAUDE.md` and `docs/permissioning/consolidated_permissioning_guide.md`. The fix follows issue #1682's recommended Option B: corpus-as-gate stays the documented service default for pipeline-facing callers (MCP, discovery, badge/analysis tasks), and a new explicit `CorpusObjsService.get_corpus_documents_visible_to_user(user, corpus, *, include_deleted=False, include_caml=False, request=None)` carries the MIN-permission semantic — it gates corpus READ, then intersects the corpus document set with `Document.objects.visible_to_user(user)`. `resolve_documents` now delegates to the new method, restoring its pre-#1699 behaviour. The two semantics are separately named so no caller can silently swap one for the other; the decision is documented in `CLAUDE.md`'s Permission System section. New `TestGetCorpusDocumentsVisibleToUser` covers the service method (owner / collaborator / stranger / anonymous / superuser, plus `include_caml` / `include_deleted` toggle composition and a corpus-as-gate contrast assertion); new `CorpusResolveDocumentsRespectsDocumentVisibilityTest` is a GraphQL-level regression guard pinning that a collaborator cannot see a private document through `corpus.documents`.

- **`handleDeleteCorpus` now honours the `ok=False` envelope** (`frontend/src/views/Corpuses.tsx`). Phase D converted `DeleteCorpusMutation` from `DRFDeletion` to a `graphene.Mutation` that returns `ok=False` + `message` (instead of raising a `GraphQLError`) when a delete is rejected — e.g. attempting to delete a personal "My Documents" corpus. The view's `.then()` handler unconditionally fired `toast.success("SUCCESS! Deleted corpus.")` on any resolved promise, so a rejected delete resolved successfully and showed a false success toast. The handler now branches on `data.deleteCorpus.ok` and surfaces `data.deleteCorpus.message` on failure.

- **Mobile layout of discussion thread header — stop description overflow and shrink the breadcrumb pill** (`frontend/src/components/threads/ThreadDetail.tsx`, `frontend/src/components/routes/CorpusThreadRoute.tsx`). The `TitleRow` in `ThreadDetail` was a single non-wrapping flex row holding the QUESTION badge, the serif title, and the description, with the description set to `flex: 1; white-space: nowrap; text-overflow: ellipsis`. On a ~540 px phone viewport the description's single-line ellipsis collapsed to zero width but still rendered its first words past the right edge of the white card, producing the cut-off "Discussing purpose of this[overflow]" seen in the bug screenshot. Added `flex-wrap: wrap; row-gap: 0.375rem` to `TitleRow`, made `Title` `flex-basis: 100%` with `white-space: normal; word-break: break-word; font-size: 1.0625rem` under `@media (max-width: 768px)` so the title gets its own line, and switched `Description` to a 2-line `-webkit-line-clamp` block (`flex: 1 1 100%; overflow-wrap: anywhere`) so subtitles wrap below the title instead of overflowing. `MetaRow` gains `flex-wrap: wrap; row-gap: 0.25rem` so the "author · time · message count" row can break cleanly on narrow widths. `Header` gets `min-width: 0` to honour the wrap, and slightly more vertical padding on mobile (`0.75rem 1rem`). In `CorpusThreadRoute` the breadcrumb's `CorpusLink` could grow to 2–3 wrapped lines holding a long case title; wrapped the title in a new `CorpusLinkLabel` span with `overflow: hidden; text-overflow: ellipsis; white-space: nowrap`, gave `CorpusLink` `min-width: 0; max-width: 100%` plus `flex: 1 1 auto` on mobile, and pinned `flex-shrink: 0` on the folder icon — so the pill now truncates to a single line with ellipsis on narrow screens (full title still available via the existing `title=` tooltip).

- **Restore `.with_tree_fields()` for superusers on `PermissionedTreeQuerySet.visible_to_user`** (issue #1668; `opencontractserver/shared/QuerySets.py`, `opencontractserver/tests/test_resolvers.py`, `opencontractserver/tests/test_visibility_managers.py`, `opencontractserver/tests/permissioning/test_authorization_invariants_coverage.py`). Phase A follow-up. The superuser branch previously returned `self.all().order_by("created")`, which silently dropped `.with_tree_fields()` from the queryset — superusers were missing the `tree_depth` / `tree_path` annotations that every other branch supplied via `with_tree_fields()`. Now returns `self.all().with_tree_fields()`, matching the non-superuser branches. As a hygiene side-effect, the asymmetric `order_by("created")` is also removed from the superuser branch of `BaseVisibilityManager.visible_to_user`; ordering is owned by the resolver / caller, consistent with the rest of the codebase. Two superuser `order_by` assertions in `test_resolvers.py:71` and `test_visibility_managers.py:83` updated to assert no manager-imposed ordering. Also deletes the now-redundant `DocumentManager.user_can` passthrough (MRO falls through to `BaseVisibilityManager.user_can` → `UserCanMixin.user_can` → `_default_user_can` for identical behaviour) and tightens `Any` annotations on `user_can` signatures across six managers and the mixin.

- **Eliminate N+1 permission loops in `get_analysis_annotations` / `get_extract_datacells`** (issue #1692; `opencontractserver/annotations/query_optimizer.py:1259-1271`, `:1443-1455`; `opencontractserver/tests/permissioning/test_query_optimizer_methods.py`). Phase C cleanup follow-up. Both methods previously iterated `analysis.analyzed_documents.all()` / `extract.documents.all()` in Python, calling `doc.user_can(user, PermissionTypes.READ)` per document — O(N) permission checks with full-row deserialisation on each loop step. Replaced both loops with a single `Document.objects.visible_to_user(user).filter(id__in=<m2m subquery>.values("id")).values_list("id", flat=True)` queryset intersection, which already encodes the full visibility model (creator + is_public + guardian + corpus membership) and matches the pattern in use elsewhere in the same file (`query_optimizer.py:745-752`). Behaviour preserved: same readable-doc-id set, same early `Annotation.objects.none()` / `Datacell.objects.none()` exit when nothing is readable. New `DocumentPermissionFilterQueryCountTestCase` pins behaviour parity against a mix of readable / non-readable documents and asserts a bounded query count for the visibility filter regardless of N.

### Changed

- **Test suite optimization Phase 3 — stop `BaseFixtureTestCase` reloading the 18 MB fixture on every test** (issue #1711; `opencontractserver/tests/base.py`, `opencontractserver/tests/fixtures/test_data.json`, `opencontractserver/tests/test_fixture_base_classes.py`, `docs/test_scripts/regenerate_structural_test_data_fixture.md`, and 6 dependent test files). `BaseFixtureTestCase` declared `fixtures = ["…/test_data.json"]` **and** extended `TransactionTestCase`, so Django's `loaddata` re-loaded the 7,584-object / 18 MB fixture before _every_ test method instead of once per class, and `setUp()` re-ran a structural-annotation migration loop plus per-document file/summary work each time.

  - **`base.py` split into a `TestCase` default + a `TransactionTestCase` variant.** Shared setup moved into a `_FixtureSetupMixin`. `BaseFixtureTestCase` is now `TestCase`-backed: the fixture and the derived corpus/document state (`add_document` corpus isolation, guardian grants) build **once per class** in `setUpTestData`, inside the class transaction; each test still runs in its own savepoint. The new `TransactionFixtureTestCase` is `TransactionTestCase`-backed for tests that need committed-data semantics; `WebsocketFixtureBaseTestCase` and `CeleryEagerModeFixtureTestCase` now extend it. Only the fixture-file copy into `MEDIA_ROOT` stays per-test — the autouse `media_storage` fixture repoints `MEDIA_ROOT` at a fresh tmpdir for every test, so the files must be re-materialised into it.
  - **`test_data.json` regenerated in the current schema and slimmed** (7,584 → 4,284 objects; procedure documented in `docs/test_scripts/regenerate_structural_test_data_fixture.md`). The 1,344 structural annotations and 472 structural relationships were migrated from the legacy `Annotation.document` schema to `StructuralAnnotationSet` (4 sets added, one per document); `Document.md_summary_file` was populated with the existing `files/md_summaries/{pk}_summary.md` paths; and the 3,304 `guardian.userobjectpermission` rows for `annotations.relationship` were dropped — relationships carry no individual object permissions in the current model (they inherit from document + corpus). This makes the per-test structural-migration loop and the convention-based `md_summary` discovery / placeholder-regeneration block in `setUp()` dead code; both were deleted.
  - **Variant assignment driven by what each file's tests actually need, verified by running them.** `async def` agent tests fail under `TestCase` with `psycopg2.InterfaceError: connection already closed`, and `transaction.on_commit`-dispatched analyzer tasks produce no `DocumentAnalysisRow` rows when the test transaction never commits — so those files keep `TransactionTestCase` semantics. Moved to `TransactionFixtureTestCase`: `test_extract_analyzer_tools.py` (async `sync_to_async(thread_sensitive=False)` tests), `test_structured_response_api.py` / `test_structured_response_simple.py` / `permissioning/test_public_and_permissions.py` (async agent tests), `test_individual_extract_tasks.py` (`celery_task_with_async_to_sync` task), `test_agentic_highlighter_task.py` (analysis dispatched via `transaction.on_commit`). The synchronous GraphQL/ORM files `test_note_tree.py` and `test_export_mutations.py` stay on the `TestCase`-backed `BaseFixtureTestCase`.
  - **Every dependent file benefits, including those kept on `TransactionFixtureTestCase`:** the slimmer fixture (44 % fewer objects → fewer `loaddata` INSERTs per test) and the deleted per-test `setUp()` work (no 1,344-annotation + 472-relationship migration loop, no per-document summary-file generation).
  - **Measured delta:** for `test_note_tree.py` the per-test `setup` under the old `TransactionTestCase` base was 129 s / 28 s (the smoke run's two test-class first methods, fixture reloaded before every test); under the new `TestCase` base the fixture loads once per class (~18 s, attributed to each class's first test) after which per-test `setup` is negligible.
  - **AC pinned by `test_fixture_base_classes.py`** — a `setUpTestData` load counter asserts the fixture/corpus build runs once per class for `TestCase`-backed subclasses, alongside base-class wiring assertions (`BaseFixtureTestCase` → `TestCase`; `TransactionFixtureTestCase` / `WebsocketFixtureBaseTestCase` → `TransactionTestCase`).

- **Test suite optimization Phase 2 — convert 160 needlessly-`TransactionTestCase` classes to `TestCase`** (issue #1710; 28 files under `opencontractserver/tests/` and `opencontractserver/tests/permissioning/`). `TransactionTestCase` `TRUNCATE`s every table and re-emits the `post_migrate` signal (`create_permissions`, `create_contenttypes`, django-guardian setup) between _every_ test method; `TestCase` instead wraps each test in a transaction and rolls it back — far cheaper per-test overhead, and the highest-leverage structural lever for bringing the ~1 hr backend suite under 20 minutes. All 235 classes that extend `TransactionTestCase` directly were audited against the patterns that genuinely require transaction semantics, and the 160 pure-synchronous ORM / GraphQL / permission / management-command classes were reclassified to `django.test.TestCase`.

  - **Highest-yield files:** `test_corpus_objs_service.py` (63 classes, via 3 converted base classes plus direct extenders), `test_corpus_export_import_v2.py` (15), `permissioning/test_authorization_invariants.py` (10), `test_v3_migration.py` (10), `permissioning/test_authorization_invariants_coverage.py` (9), `permissioning/test_permission_optimizer.py` (8), `test_corpus_fork_round_trip.py` (5), plus `test_badges.py`, `test_notifications.py`, `test_worker_uploads.py`, `test_corpus_forking.py`, `test_personal_corpus.py`, `test_analyzer_app_coverage.py`, and ~16 other files.
  - **Kept on `TransactionTestCase` (75 classes):** every class with an `async def` test method / `await` body, OS-thread races (`Thread(` / `ThreadPoolExecutor` — `EmbeddingManagerConcurrentTest`, `Tier1CacheThreadSafetyTestCase`, `TestConcurrentPersonalCorpusCreation`), and genuine `transaction.on_commit` tests (`InstallGremlinOnCreationTests`, `TestThreadCorpusActionSignals`, `CorpusForkOrphanedBlobGCTest`, `TestMoveDocumentAsync`). `test_blob_retention.py` is left entirely untouched — its blob garbage-collection assertions depend on real `on_commit` deletion callbacks firing, which `TestCase` (no commit inside the test) silently skips.
  - **`async_to_sync` failure mode pinned by running the suite.** A first conversion pass also flipped the `async_to_sync`-of-Django-ORM classes; running them under `pytest -n 4 --dist loadscope` surfaced `psycopg2.InterfaceError: connection already closed` — `async_to_sync` of ORM code recycles the DB connection mid-test, destroying the long-lived transaction `TestCase` depends on (`TransactionTestCase`'s autocommit-per-statement model tolerates it). This is broader than the issue's original grep checklist, which only named `async def` / `WebsocketCommunicator` / `database_sync_to_async`. Those classes were reverted to `TransactionTestCase`: the memory-CRUD tests in `test_agent_memory.py` (which call `async_to_sync` directly) and the whole of `test_extract_tasks.py`, `test_individual_extract_tasks.py`, `test_pii_scanner_analyzer_task.py`, and the async-wrapper tests in `test_caml_review_tools.py` (whose code under test runs `async_to_sync` internally). `test_multimodal_integration.py` was likewise left as-is — it could not be runtime-verified in this environment (real Docling I/O).
  - **No behaviour or coverage change** — `TestCase` vs `TransactionTestCase` only changes DB isolation. `transaction.atomic()` blocks inside the code under test become savepoints, so the existing rollback / mocked-`IntegrityError` regression tests (`TestErrorPaths_*AtomicRollback`, `RollbackAndEdgeCaseTests`, `TestMoveDocumentIntegrityRecovery`, …) execute identical paths and assert identical outcomes.
  - **Measured delta:** an A/B run of `test_corpus_objs_service.py` (the largest converted file, 63 classes / ~200 tests) under `pytest -n 4 --dist loadscope --reuse-db` drops from **244 s on `TransactionTestCase` to 29 s on `TestCase`** — an ~8× reduction, all tests passing both ways. Under `TransactionTestCase` even a trivial `assertRaises(ValueError)` test costs ~5 s because of the inter-test `TRUNCATE` + `post_migrate` re-emit; under `TestCase` the same test is ~0.01 s.
  - **`conftest.py::make_create_permissions_xdist_safe` retained.** That fixture papers over the cross-worker `auth_permission` `IntegrityError` race introduced by `TransactionTestCase._fixture_teardown`'s `flush` → `emit_post_migrate_signal`; it can only be deleted once _no_ `TransactionTestCase` classes remain, and 75 still do.

- **Skip coverage instrumentation on pull-request CI runs** (`.github/workflows/backend.yml`). The Backend CI `pytest` invocation now passes `--cov --cov-report=xml` only on `push` events (merges to a protected branch); pull-request runs execute the suite without coverage. coverage.py's trace function — the C tracer, since the test image runs Python 3.11 and `sys.monitoring` / `COVERAGE_CORE=sysmon` requires 3.12+ — adds noticeable wall time to the full ~6.4k-test suite, paid on every PR push. Coverage is still measured on every merge, so Codecov's tracked metric on `main` is unchanged; only the per-PR coverage delta is no longer computed. The `Verify Coverage File Exists` and `Upload Coverage Reports to Codecov` steps are gated to `push` accordingly. **Branch-protection note:** if a Codecov `patch` / `project` status for the `backend` flag is configured as a _required_ status check, it must be made non-required (or `informational`) before this merges — `.codecov.yml` currently leaves `coverage.status.patch.default` non-informational. First step of a broader effort to bring the backend suite under 20 minutes.

- **Sweep stale `Corpus.get_documents()` docstrings and test fixtures** (issue #1684; `opencontractserver/shared/QuerySets.py`, `opencontractserver/tests/test_document_path_migration.py`, `opencontractserver/tests/test_corpus_fork_round_trip.py`, `opencontractserver/tests/test_corpus_export.py`, `opencontractserver/tests/test_corpus_forking.py`, `opencontractserver/tests/test_bulk_document_upload.py`, `opencontractserver/tests/test_versioning_migration.py`, `opencontractserver/tests/test_agentic_highlighter_task.py`, `opencontractserver/tests/test_extract_analyzer_tools.py`). Migrated 32 fixture-style `corpus.get_documents()` call sites across 8 test files to `corpus._get_active_documents()` — the blessed internal alias the deprecated wrapper already delegates to (identical semantics, same active-path predicate). Eliminates `DeprecationWarning` noise from setup helpers without touching the per-user `CorpusObjsService` path reserved for request-context code. Also updated two docstrings in `QuerySets.py` (lines 34 and 355) that named the predicate via the deprecated public method. `test_corpus_get_documents_deprecation.py` is intentionally left unchanged — it exercises the deprecation warning. Out of scope: the `DocumentFolderService` back-compat shim and the production GraphQL / agent-tool call sites that still call `DocumentFolderService.X()` / `corpus.get_documents()` directly; those move when the alias is removed.

- **Thread GraphQL context through optimizer helpers for Tier-2 cache hits** (issue #1691; `opencontractserver/annotations/query_optimizer.py`, `opencontractserver/documents/query_optimizer.py`, `config/graphql/{action_queries,annotation_queries,corpus_queries,extract_queries,extract_types}.py`). Follow-up to Phase C (#1689). Five `AnnotationQueryOptimizer` / `AnalysisQueryOptimizer` / `ExtractQueryOptimizer` helpers — `get_corpus_annotations`, `check_analysis_permission`, `get_visible_analyses`, `check_extract_permission`, `get_visible_extracts` — plus the intermediate `DocumentActionsQueryOptimizer.get_document_actions` now accept an optional `context: Optional[Any] = None` kwarg and thread it into every `user_can(...)` call as `request=context`. The Tier-2 request-scoped permission cache introduced in PR #1665 was dormant on these helpers because their GraphQL callers had `info.context` in scope but did not propagate it. After this PR every GraphQL caller (`annotation_queries`, `corpus_queries`, `extract_queries`, `extract_types` get_node, `action_queries`) passes `info.context` through. Backward-compatible: `None` default keeps test callers (`test_query_optimizer_methods.py`, `test_corpus_annotations_query.py`) and any Celery / internal sync code on the Tier-1 instance cache. No behaviour change beyond cache-hit rate.

- **Retire `DocumentFolderService` back-compat shim — migrate all importers to `DocumentService` / `CorpusObjsService`** (issue #1680). PR #1685 split the monolithic `DocumentFolderService` along its natural seam and kept the original class as a `class DocumentFolderService(CorpusObjsService, DocumentService)` shim in `opencontractserver/corpuses/folder_service.py` so the ~18 existing importers could migrate later. This change finishes that migration:

  - **Production call sites (9 files):** `config/graphql/{corpus_folder_mutations,corpus_mutations,corpus_queries,document_mutations,document_types}.py`, `opencontractserver/tasks/import_tasks.py`, `opencontractserver/llms/tools/core_tools/documents.py`, `opencontractserver/mcp/server.py`, `opencontractserver/utils/import_v2.py` — every `DocumentFolderService.X(...)` rewritten to `CorpusObjsService.X(...)` based on the partition documented in `test_service_split_method_partition` (`opencontractserver/tests/test_corpus_objs_service.py:46`). None of these call sites hit a `DocumentService`-routed method.
  - **Test call sites (3 files):** `opencontractserver/tests/test_permanent_deletion.py`, `opencontractserver/tests/test_zip_import_integration.py`, `opencontractserver/tests/permissioning/test_permission_optimizer.py` — same surgical rewrite to `CorpusObjsService`.
  - **Test suite merge:** `opencontractserver/tests/test_document_folder_service.py` (4423 lines) and the existing `opencontractserver/tests/test_corpus_objs_service.py` (523 lines) collapsed into a single `opencontractserver/tests/test_corpus_objs_service.py` (4925 lines). The two test classes that exercise `DocumentService` methods (`TestEdgeCases_UploadQuota`, `TestValidateFileType`) now call `DocumentService.check_user_upload_quota` / `DocumentService.validate_file_type` directly and inherit from `TransactionTestCase` rather than the folder-test base. The folder-test base itself was renamed from `DocumentFolderServiceTestBase` to the underscore-prefixed `_CorpusObjsServiceFolderTestBase` so it no longer clashes with the existing public `CorpusObjsServiceTestBase` fixture for the convenience-method tests.
  - **Shim removed:** `opencontractserver/corpuses/folder_service.py` deleted along with the `TestServiceSplit_BackwardsCompatAlias` class in `opencontractserver/tests/test_document_service.py` and the surrounding docstring bullet that explained the alias. The disjoint-partition test (`TestServiceSplit_MethodPartition`) survives — it still pins the invariant that `DocumentService.__dict__` and `CorpusObjsService.__dict__` share no method names, which is exactly what made the two-parent inheritance unambiguous in the first place and remains the load-bearing assertion behind the split.
  - **Stale docstring sweep:** comment / docstring mentions of "DocumentFolderService" / "folder_service" in `opencontractserver/corpuses/{models.py,corpus_objs_service.py}`, `opencontractserver/mcp/resources.py`, `opencontractserver/tests/{test_document_stats.py,test_import_utils.py,test_move_document_tool.py}`, the `CLAUDE.md` follow-up bullet at line 245, and the active architecture / permissioning docs (`docs/architecture/{bulk-import.md,data_model/documents_and_annotations.md}`, `docs/permissioning/REMEDIATION_GUIDE.md`, `docs/plans/documentation_alignment_review.md`) reworded to name the new services. The three historical standalone docs (`docs/architecture/document_folder_service{,_plan,_migration_inventory}.md`) and the mkdocs nav entry pointing at the first of those deleted along with the shim — they document a class that no longer exists. `docs/releases/v3.0.0.b3.md` is intentionally untouched (release notes are an immutable snapshot of the release that shipped); `docs/typing/mypy_baseline.txt` carries stale entries that point at the renamed test file but will be regenerated by the next mypy run.

- **CAML-aware corpus-objs service method + caml_article tooling migration** (issue #1681; `opencontractserver/corpuses/corpus_objs_service.py`, `opencontractserver/llms/tools/core_tools/caml_article.py`, `config/graphql/corpus_types.py`, `opencontractserver/tests/test_corpus_objs_service.py`). Follow-up to PR #1685's service split: the two CAML-aware production paths that were deliberately left calling `Corpus._get_active_documents(include_caml=True)` directly now route through `CorpusObjsService`. New `CorpusObjsService.get_corpus_caml_articles(user, corpus, *, request=None)` returns the CAML article documents (`title="Readme.CAML"`, `file_type="text/markdown"`) under the same corpus-as-gate semantic as the rest of the service surface — empty queryset on permission denial, IDOR-safe. Plural shape mirrors `get_corpus_documents` so future multi-article designs do not require a signature change. `caml_article.py:_load_caml_document_for_user` now delegates to the new method, dropping the manual `Corpus.objects.visible_to_user(user)` + `_get_active_documents(include_caml=True)` + `Document.objects.visible_to_user(user)` fusion (and the now-unused `MARKDOWN_MIME_TYPE` import); the IDOR-safe identical-`ValueError` contract is preserved. `corpus_types.py:resolve_documents` similarly delegates to `CorpusObjsService.get_corpus_documents(user, self, include_caml=True, request=info.context)`, retiring its own copy of the fusion and threading the Tier-2 request-scoped permission cache through. Net result: every CAML-aware corpus read passes through one entry point, the `Corpus.get_documents()` deprecation surface stays cold on these paths, and the manual-fusion IDOR pattern CLAUDE.md warns against is gone from the codebase.

- **Phase E permission centralization — migrate field resolvers, agent tools, Celery tasks, websocket consumer, and import services to `obj.user_can(...)`** (issue #1659; `config/graphql/document_types.py`, `config/graphql/extract_types.py`, `config/websocket/consumers/unified_agent_conversation.py`, `opencontractserver/llms/agents/agent_factory.py`, `opencontractserver/llms/tools/image_tools.py`, `opencontractserver/llms/tools/core_tools/caml_article.py`, `opencontractserver/llms/tools/core_tools/extracts_and_analyzers.py`, `opencontractserver/llms/tools/core_tools/memory.py`, `opencontractserver/tasks/doc_tasks.py`, `opencontractserver/tasks/extract_orchestrator_tasks.py`, `opencontractserver/document_imports/services.py`, `opencontractserver/tests/test_caml_review_tools.py`). The remaining 20 `user_has_permission_for_obj` call sites that lived outside GraphQL mutations now route through `obj.user_can(...)`. After Phase E the in-repo call count of `user_has_permission_for_obj` drops to zero; the shim survives one release cycle for plugin/external code (deletion tracked in #1661). Net effect:

  - **Cache plumbing.** Six GraphQL field resolvers (`DocumentType.resolve_can_restore` x2, `DocumentType.resolve_can_view_history`, `DocumentType.resolve_can_retry`, `ExtractType.resolve_document_count`, `ExtractType.resolve_full_document_list`) now pass `request=info.context` so the Tier-2 request-scoped cache from PR #1665 fires on the per-row permission checks that previously ran cold. Agent tools, Celery tasks, the websocket consumer, and import services have no GraphQL request in scope and omit the kwarg — they degrade gracefully to Tier-1 instance caching.
  - **Creator-honoring uniform across agent tools.** The `_apply_caml_article_edit` site drops the legacy `locked_doc.creator_id == user.pk` OR-branch because `user_can` now honours creator access uniformly per Phase A. The `image_tools.py`, `extracts_and_analyzers.py`, and `memory.py` sites similarly switch from "do I have READ/UPDATE/CRUD on this resource" to "can I (per the new uniform creator-honoring path) act"; a creator who never received an explicit guardian grant now passes the gate, matching the read-side `visible_to_user` filters. Pinned by `test_creator_without_explicit_perm_can_edit` in `test_caml_review_tools.py`.
  - **Group-permission widening.** Five call sites (`unified_agent_conversation.py` x2, `agent_factory.py`, `memory.py`, `caml_article.py`, `services.py` x2) previously relied on the shim's legacy `include_group_permissions=False` default. The new `obj.user_can(...)` defaults to `True`, matching `_default_user_can` and every other `Manager.user_can` / `obj.user_can` surface. Users who hold the relevant guardian permission via a group on Corpuses / Documents / Extracts now pass these websocket/agent/import gates; this matches the documented per-model policy.
  - **No remaining `user_has_permission_for_obj` calls in any of the migrated files** — verified by grep. The handful of surviving references in `opencontractserver/utils/permissioning.py`, `opencontractserver/shared/`, `opencontractserver/annotations/query_optimizer.py`, etc. are docstring / comment mentions describing the shim and its Phase A / B / C / E migration history.

- **xdist-safe `auth_permission` teardown — kill the cross-worker `IntegrityError` cascade** (`conftest.py:15-67`). Every `TransactionTestCase._fixture_teardown` calls Django's `flush` which fires `emit_post_migrate_signal` → `django.contrib.auth.management.create_permissions` → `Permission.objects.bulk_create(perms)`. Under `pytest -n 4 --dist loadscope`, workers race on the shared `auth_permission` table (postgres content_type ids overlap across the per-worker test DBs because they're assigned serially during migrate), and the `(content_type_id, codename)` unique constraint produces `UniqueViolation` for whichever worker loses. The loser's teardown then dies un-rolled-back, the next test's `setUp` finds an empty fixture, and a cascade of `User.DoesNotExist: User matching query does not exist` setUp failures follows — surfacing as ~20+ apparently-random failures in `test_agent_permission_escalation.py`, `test_unified_agent_consumer.py`, etc. Phase E itself didn't cause the race; it concentrated enough fixture-heavy classes into one dispatch to make it routinely fire. Fix is a session-scoped `autouse=True` fixture in the root `conftest.py` that re-registers the `post_migrate` handler with a thin wrapper: `with transaction.atomic(using=using): create_permissions(...)` plus `except IntegrityError: pass`. The conflicting rows are exactly the permissions we'd have inserted anyway (winning worker already wrote them), so the swallow is a true no-op and the atomic block guarantees no half-inserted state leaks past the rollback. The fixture restores the original handler at session teardown, so no production import path is affected.

- **Phase C permission centralization — migrate optimizer call sites to `obj.user_can(...)`** (issue #1657; `opencontractserver/annotations/query_optimizer.py`, `opencontractserver/extracts/query_optimizer.py`). The 24 remaining `user_has_permission_for_obj` call sites across `AnnotationQueryOptimizer` (analysis/extract/datacell/corpus-annotation/document-relationship paths) and `MetadataQueryOptimizer` (`_compute_effective_permissions`, `get_corpus_metadata_columns`, `get_documents_metadata_batch`, `check_metadata_mutation_permission`) now route through `obj.user_can(...)` — completing the substitution PR #1663 started in `AnnotationQueryOptimizer._compute_effective_permissions`. Net effect:

  - **Cache plumbing.** Three of the migrated sites (`AnnotationQueryOptimizer.get_document_annotations` analysis check, `RelationshipQueryOptimizer.get_document_relationships` analysis check, `MetadataQueryOptimizer.get_documents_metadata_batch` corpus check) now pass `request=context` so the Tier-2 request-scoped cache (PR #1665) fires for the optimizer's existing analysis/corpus dereferences. The 21 other migrated sites live in helpers without a GraphQL request in scope (`check_analysis_permission`, `get_visible_extracts`, `_compute_effective_permissions` on the extract path, etc.) and degrade gracefully to Tier-1 instance caching. The annotation optimizer's bespoke `context._effective_perms_cache` (Phase A) is untouched — it continues to sit on top of `permission_cache` / `permission_optimizer`.
  - **Behavioral preservation.** The MIN(doc, corpus) bodies in both `_compute_effective_permissions` methods are unchanged; only the per-call shim invocations move. BACON-mode (`corpus.allow_comments`) branch ordering in the annotation optimizer is preserved verbatim per the issue's pin. The legacy `include_group_permissions=True` argument drops because `obj.user_can(...)` defaults to `True` (matching every other `Manager.user_can` / `obj.user_can` surface, and matching what these call sites were already passing explicitly).
  - **No remaining `user_has_permission_for_obj` calls in either optimizer file** — verified by grep. The two surviving references in `annotations/query_optimizer.py` are docstring mentions describing the Phase A migration.

- **Bulk permission collapse in `MetadataQueryOptimizer._compute_effective_permissions`** (issue #1690; `opencontractserver/extracts/query_optimizer.py:34-126`, `opencontractserver/tests/permissioning/test_metadata_query_optimizer.py`). The Phase C migration left the method issuing 8 sequential `user_can` calls (4 per object × 2 objects); since `get_document_metadata` and `get_metadata_completion_status` don't thread a GraphQL `request`, the Tier-2 `PermissionQueryOptimizer` never fires and each call paid full cold-path overhead. The method now fetches the guardian codename set once per object via `get_users_permissions_for_obj` (Tier-1 cached) and resolves the 4 permission types via set membership (`"read_document" in granted`, etc.). Creator short-circuit is preserved verbatim to match `_default_user_can` semantics on instances without explicit guardian grants. Two new `assertNumQueries` regression tests (`test_compute_permissions_query_count_superuser_is_zero` and `test_compute_permissions_query_count_collaborator_is_bounded`) lock the budget so any future reintroduction of per-permission fan-out fails CI. Tier-2 batching across both objects is deferred to sister issue #1691, which threads `request` into this code path.

- **Phase B permission centralization — replace compound `user_has_permission_for_obj` checks with `obj.user_can(...)`** (issue #1656; `config/graphql/{base,agent_mutations,analysis_mutations,annotation_mutations,badge_mutations,conversation_mutations,corpus_mutations,document_mutations,document_relationship_mutations,extract_mutations,ingestion_source_mutations,label_mutations,smart_label_mutations,voting_mutations}.py`, `opencontractserver/documents/query_optimizer.py`). Every mutation site that previously composed `user.is_superuser or obj.creator_id == user.id or user_has_permission_for_obj(user, obj, perm, include_group_permissions=True)` now calls the single `obj.user_can(user, perm, request=info.context)` surface introduced in PRs #1663 and #1665. Net effect:

  - **Cache plumbing.** Each migrated check passes `request=info.context` so the Tier-2 request-scoped permission cache from PR #1665 actually fires; previously the cache stayed dormant across mutations. Every `set_permissions_for_obj_to_user(...)` call in the mutation layer also now passes `request=info.context` (plus `is_new=True` on fresh-creation paths) so Tier-1 / Tier-2 cache invalidation lands on grant / revoke.
  - **Creator-status widening across non-Corpus mutations.** A handful of legacy call sites passed `include_group_permissions=False` (the shim's default). The new `obj.user_can(...)` defaults to `True`, matching `_default_user_can` and every other `Manager.user_can` / `obj.user_can` surface. Users who hold the relevant guardian permission via a group on Documents / Extracts / Annotations / Notes / Conversations / Labels / Badges now pass the relevant mutation gate; this matches the documented per-model policy and the read-side `visible_to_user` queryset filters.
  - **`DocumentActionsQueryOptimizer._check_document_permission` / `_check_corpus_permission` deleted** (`opencontractserver/documents/query_optimizer.py`); five call sites in `get_document_actions`, `get_corpus_actions_for_corpus`, `get_extracts_for_document`, `get_analysis_rows_for_document`, and `DocumentRelationshipQueryOptimizer.get_relationships_for_document` now call `document.user_can(user, READ)` / `corpus.user_can(user, READ)` directly. `DocumentRelationshipQueryOptimizer.user_has_permission` similarly drops its three compound checks. The two callsites with no GraphQL request in scope omit `request=` and degrade gracefully to Tier-1 instance caching.
  - **`_resolve_annotation_parents` and `_clone_fieldset_for_iteration` gained `request=None` kwargs** so their callers thread `info.context` through, keeping the cache lit on the helper paths that were previously dark.
  - **No behaviour change for the corpus-level checks** that already used `include_group_permissions=True` — those collapse to a single call against the same default-True API.
  - Out of scope (deferred to Phase D / E per issue #1658 / #1659): IDOR-unified-error refactors, field-resolver / agent-tool / Celery / import-service migrations, and the websocket consumer in `unified_agent_conversation.py`.

- **Permission cache hardening — review follow-ups on PR #1665** (`opencontractserver/utils/{permissioning,permission_optimizer}.py`, `opencontractserver/shared/user_can_mixin.py`, `opencontractserver/constants/permissioning.py`, `opencontractserver/conversations/models.py`, `opencontractserver/tests/permissioning/test_permission_optimizer.py`, `opencontractserver/shared/Managers.py`).

  - Aligned `get_users_permissions_for_obj`'s `include_group_permissions` default to `True`, matching `PermissionQueryOptimizer.get_granted`, `_default_user_can`, and every `Manager.user_can` / `obj.user_can` surface. One default, one answer — eliminates the latent footgun where a caller mixing the Tier 1 helper and the Tier 2 optimizer would get different group-permission semantics without passing the flag. The legacy `user_has_permission_for_obj` shim deliberately keeps its `False` default (its ~170 callers were written against that behaviour).
  - Added a `threading.Lock()` around `PermissionQueryOptimizer._cache` reads/writes/iterates. Cheap insurance against future shapes that violate the one-request-one-thread assumption (multi-user delegation, background threads spun off mid-request, service-account sharing). The DB-bound `get_users_permissions_for_obj` call still runs outside the lock so guardian lookups never serialise on it.
  - Replaced the raw Tier 1 `dict` with `_InstancePermsCache(dict)` — a thread-safe subclass exposing `drop_for_user(user_id)` that holds an internal `Lock` for the compound iterate-then-delete sweep in `set_permissions_for_obj_to_user`. Individual `cache[key]` reads/writes still ride CPython's GIL atomicity; the lock exists for the compound op only. `_get_or_create_instance_perms_cache` uses `instance.__dict__.setdefault` so two threads first-touching the same instance converge on a single cache rather than racing via `getattr`+`setattr`. Closes the Tier 1 ASGI / async-view exposure flagged in PR review. New `Tier1CacheThreadSafetyTestCase` pins the contract under N=4 reader-thread contention against repeated invalidate sweeps.
  - Backwards-compat fallback in `set_permissions_for_obj_to_user`: if an instance carries a plain-`dict` cache (e.g. unpickled from an older worker or attached by a test fixture), the helper falls back to a snapshot-then-`.pop()` invalidation pattern so `RuntimeError` cannot escape even when the thread-safe wrapper is absent.
  - Added explicit `request: Any = None` annotation on `ConversationManager.user_can` and `ChatMessageManager.user_can` (previously inferred as `None` by type checkers, which would reject real request objects at the type level).
  - `InstanceUserCanMixin.__getstate__` strips `INSTANCE_PERMS_CACHE_ATTR` before pickling. Any `BaseOCModel` (or `Corpus` / `CorpusFolder`) passed verbatim to a Celery task — the well-known anti-pattern — can no longer carry a stale Tier 1 frozenset to the worker; the footgun is impossible rather than merely documented. Producer-side instance keeps its cache; only the serialised state is scrubbed.
  - Documented in code that `ContentType.objects.get_for_model` is backed by `ContentType._cache`, so `_resolve_content_type_id` is effectively a dict read after the first call per process — pinning the reliance for future readers.
  - Documented that creator-based (non-guardian) models on the Tier 1 cache path are always cache-safe: `set_permissions_for_obj_to_user` only mutates guardian tables, which don't exist for these models, so the cached frozenset can't drift from the standard mutation path.
  - Threaded the `request=None` kwarg through `AnnotationManager.user_can` → `_check_annotation_privacy_recursion` → `Analysis.objects.user_can` / `Extract.objects.user_can` and through `_compute_annotation_effective_permission` → `AnnotationQueryOptimizer._compute_effective_permissions(context=request)` so a paginated annotation list resolver that calls `obj.user_can` per row shares one request-scoped Tier 2 cache across the privacy-recursion source-object checks AND the MIN(doc, corpus) Document/Corpus lookups, instead of re-querying guardian per row.

- **UserFeedback visibility now follows the commented annotation's full visibility model** (`opencontractserver/shared/Managers.py:353-401`, `opencontractserver/shared/QuerySets.py:169-217`, `opencontractserver/tests/test_feedback.py:334-510`, `opencontractserver/tests/permissioning/test_authorization_invariants.py:1211-1310`, `opencontractserver/tests/permissioning/test_permissioning.py:783-893`). The Phase A `UserFeedbackManager.user_can` and `UserFeedbackQuerySet.visible_to_user` previously granted READ when `commented_annotation.is_public=True` rather than when the annotation itself was visible to the requesting user. This produced two divergences from the intended "feedback inherits permissioning from the annotation it comments on" policy: (1) anonymous users could read feedback on a non-structural `is_public=True` annotation that `Annotation.objects.visible_to_user(AnonymousUser())` does not surface, and (2) authenticated users with a guardian READ grant on a private annotation could read the annotation but not the feedback on it. Both surfaces now route through `Annotation.objects.visible_to_user(user).filter(pk=commented_annotation_id).exists()` (and the equivalent `commented_annotation_id__in=...` subquery on the queryset path), keeping the manager check and the queryset filter aligned with each other and with the annotation's own visibility model. Inherited grant remains READ-only; non-READ permissions continue to require creator status or an explicit guardian grant on the feedback row itself.

- **Documents.tsx decomposition — coverage follow-up on PR #1677** (`frontend/src/utils/__tests__/files.test.ts`, `frontend/tests/Documents.ct.tsx`). The decomposition PR's three new presentational components (`DocumentsGridView`, `DocumentsListView`, `DocumentsCompactView`) and the extracted `getDocumentTypeBadge` helper landed under codecov's frontend patch target (61.15% vs target 69.89%). The pre-existing component tests only ever drove the list and compact view branches (and even then, the list test switched to list view immediately, leaving grid-view selection/context-menu untested). Two test additions push patch coverage above target:
  - **Unit tests for `frontend/src/utils/files.ts`.** Exhaustive `getDocumentTypeBadge` branch coverage (canonical fileTypes, mixed case, legacy `doc` alias, generic uppercase fallback, title-extension fallback, and the final `"PDF"` safety net), plus the previously-untested `isDocxFileType` / `isSpanBasedFileType` / `formatFileSize` exports.
  - **`tests/Documents.ct.tsx > should exercise grid view selection and context menu`.** Stays in the default grid view, clicks the per-card checkbox (drives `DocumentsGridView.onSelect` and the `isSelected` branch on the `CardCheckbox` visibility state), then clicks the kebab menu (drives the second `onContextMenu` trigger path — the row right-click is the first).
- **`AnnotationManager.user_can` refactored into helpers** (`opencontractserver/shared/Managers.py:630-846`). The 183-line method is now decomposed into `_anonymous_read_via_visible_to_user` (line 728), `_check_annotation_privacy_recursion` (line 744), and `_compute_annotation_effective_permission` (line 807) — load-bearing branch order on the top-level method is unchanged, behavior is identical, but the main body now reads as a scannable sequence of branches without scrolling. Phase B activation note added to the docstring re: per-call Analysis/Extract dereferences in the privacy-recursion branch (issue #1655 follow-up).
- **`_user_has_permission_for_obj_warned` dedup set documented as bounded** (`opencontractserver/utils/permissioning.py:546-558`). Existing comment expanded to make the bounded-by-call-sites invariant explicit, plus a pointer to `ShimDeprecationWarningTestCase.setUp` as the documented test-isolation hook.
- **`ShimDeprecationWarningTestCase` clears the warning dedup set before each test** (`opencontractserver/tests/permissioning/test_authorization_invariants.py:1313-1342`). Without this, when a previous test in the same `pytest --keepdb` run triggered a warning from the same `(filename, lineno)` site, `catch_warnings` would record nothing and the assertion would fail silently.
- **`RelationshipNonOwnerCreatorTestCase` docstring clarifies intended behavior** (`opencontractserver/tests/permissioning/test_authorization_invariants_coverage.py:627-742`). Reframed the class docstring as a POLICY note rather than "addresses the latent invariant violation flagged on PR #1663" — relationship creators retain READ/UPDATE/DELETE via the creator branch after explicit doc/corpus guardian grants are revoked **by design**, mirroring `RelationshipQuerySet.visible_to_user`'s `Q(creator=user)` predicate to keep manager and queryset surfaces aligned.
- **`permission_cache.py` module docstring notes nested-scope inheritance for Phase B** (`opencontractserver/shared/permission_cache.py:33-51`). The current allocation-on-entry behavior means nested scopes do not inherit parent entries; a "join existing scope" variant may better suit Phase B's wire-up pattern. Documented as a follow-up consideration without behavior change.

- **Extract creation chokepoint — analyzer-style auto-CRUD for Extracts** (`opencontractserver/utils/extract.py` (new), `opencontractserver/utils/analysis.py`, `opencontractserver/llms/tools/core_tools/extracts_and_analyzers.py`, `opencontractserver/tasks/corpus_tasks.py`, `opencontractserver/tests/test_extract_utils.py` (new), `opencontractserver/tests/test_analysis_utils.py`). The analyzer framework already had `create_and_setup_analysis` as the canonical Analysis-creation chokepoint that grants the creator guardian CRUD; the Extract side had no equivalent — every implementer (`start_extract` agent tool, `process_corpus_action`, the GraphQL mutations) had to remember to call `set_permissions_for_obj_to_user` separately, and `process_corpus_action` had silently been omitting the grant. New `create_and_setup_extract(user_id, *, corpus, fieldset, name=None, document_ids=None, corpus_action=None, mark_started=False)` mirrors `create_and_setup_analysis`: atomic create, creator-CRUD grant via `set_permissions_for_obj_to_user(..., is_new=True)`, optional document linking, optional started-timestamp marking. `start_extract` now routes through the helper instead of hand-rolling the create + grant; `process_corpus_action`'s `get_or_create` path now inlines the grant on first creation with a comment pointing at the framework helper. The docstring on `create_and_setup_analysis` was strengthened to pin the CRUD-grant contract as part of the framework guarantee so a future refactor can't quietly drop it. New `test_extract_utils.py` covers creation, the CRUD-grant invariant, document linking, started-timestamp marking, corpus-action lineage, and name defaulting; `test_analysis_utils.py` gains a matching CRUD-grant invariant test for the analysis helper.
- **Extract & analyzer agent tool hardening** (`opencontractserver/llms/tools/core_tools/extracts_and_analyzers.py`, `opencontractserver/llms/tools/tool_registry.py`, `opencontractserver/constants/tools.py`, `opencontractserver/tests/test_extract_analyzer_tools.py`).
  - **Status vocabulary unified across `start_analysis` and `list_recent_analyses`.** `start_analysis` previously returned `EXTRACT_STATUS_QUEUED` (`"queued"`) while `list_recent_analyses` passed the raw model status through (`JobStatus.QUEUED.value` → `"QUEUED"`). An LLM polling the listing right after dispatch saw the status appear to change without any state transition. New `_normalize_analysis_status` lowercases every status going through the listing tool; `start_analysis` now returns `_ANALYSIS_STATUS_QUEUED` (derived from `JobStatus.QUEUED.value`) so the two surfaces agree on a single lowercase vocabulary that matches the extract tools. `EXTRACT_STATUS_QUEUED` is no longer used outside the extract path.
  - **`analysis_input_data` privilege-escalation hardening.** New `ANALYSIS_INPUT_DATA_RESERVED_KEYS` enumerates the kwargs the analyzer task framework owns (`analysis_id`, `user_id`, scoping IDs, etc.); `start_analysis` strips them from the agent-supplied payload before dispatch and logs a warning. The task remains the canonical validation boundary; this is defense-in-depth on the uniquely-exposed agent path (LLM payloads can be adversary-controlled via prompt injection through document content).
  - **`list_analyzers` truncates oversized `input_schema` payloads.** `Analyzer.input_schema` is an unbounded `JSONField`. New `ANALYZER_INPUT_SCHEMA_MAX_INLINE_CHARS = 4_000` cap replaces schemas whose JSON dump exceeds the limit with a placeholder pointer; `start_analysis` itself does not need the schema (the task validates) and the LLM still discovers the analyzer. `list_analyzers` also now orders by `-created` rather than the CharField `id` so the listing is meaningful (newest first) rather than lexicographic across arbitrary dotted IDs.
  - **`list_fieldsets` defaults to slim payloads + `extractable` flag.** Returns column names only by default (`column_names: [...]`), with the full long-form column definitions (`query`, `match_text`, `instructions`, `output_type`) available behind a new `include_columns=True` flag. Bounds the context-window cost on a corpus admin with many fieldsets of many long-form columns. The listing also surfaces an `extractable` flag so the LLM knows up-front that manual-only or empty fieldsets will be rejected at `start_extract`. Listing is ordered by `-created` (not `-modified`) to match `list_recent_extracts` and avoid silent position shifts when a fieldset is edited mid-session.
  - **`_resolve_target_document_ids` pushes the intersection into SQL.** Previously materialised the entire corpus document ID set into a Python `set` on every dispatch. Now a `Document` queryset filter on the corpus's active document set when `requested_ids` are supplied, or an `exists()` membership check for a document-agent's `document_id`. The corpus-agent default path still walks the full ID list (necessary for the response payload) but at least streams via `values_list(flat=True)` rather than re-collecting from a set.
  - **Document-agent scope mismatch now fails loudly.** A document agent whose injected `document_id` is not a member of the working corpus previously fell back to the full corpus scope with only a logged warning. Now raises `ValueError` so a single-doc agent cannot silently expand into a corpus-wide dispatch. The `start_extract` / `start_analysis` tool descriptions also warn that omitting `document_ids` on a corpus agent dispatches against the entire corpus, so admins working over corpora with thousands of documents pass an explicit subset.
  - New tests pin each contract: status-normalisation in `list_recent_analyses` (including all four lifecycle states for the extract surface), reserved-key strip + warning emission in `start_analysis`, untouched legitimate payload in `start_analysis`, oversized-schema truncation in `list_analyzers`, the `extractable` flag in the manual-only-fieldset listing-vs-dispatch invariant, the slim-vs-full `list_fieldsets` payload toggle, the `limit` pagination clamp end-to-end, and the new outside-corpus `ValueError` for document agents.

### Added

- **Extract & Analyzer agent tools — let LLM agents dispatch configured extracts/analyses** (`opencontractserver/llms/tools/core_tools/extracts_and_analyzers.py` (new), `opencontractserver/llms/tools/core_tools/__init__.py`, `opencontractserver/llms/tools/tool_registry.py`, `opencontractserver/tests/test_extract_analyzer_tools.py` (new), `docs/architecture/llms/README.md`, `docs/corpus_actions/intro_to_corpus_actions.md`, `docs/extract_and_retrieval/data_extraction.md`). Six new tools slot into the existing registry alongside notes/summaries/annotations: `list_fieldsets`, `start_extract`, `list_recent_extracts`, `list_analyzers`, `start_analysis`, `list_recent_analyses`. Discovery tools are read-only and filter via the model manager's `visible_to_user`; `start_extract` and `start_analysis` are `requires_approval=True` + `requires_write_permission=True` so they pass through the standard approval gate (`ToolConfirmationRequired` → `ApprovalNeededEvent` → `resume_with_approval`) and the agent-factory's write-permission filter. New `ToolCategory.EXTRACTS` and `ToolCategory.ANALYZERS` keep them browsable in the available-tools GraphQL surface. **The agent picks from existing Fieldsets/Analyzers** — there is no schema-creation surface; this matches the user's requirement that the tool not invent column definitions from scratch. **Document scope** follows the agent context: corpus agents default to the full corpus (via `corpus.get_documents()`, which already excludes CAML and soft-deleted docs); document agents default to the current document; any `document_ids` the LLM passes are intersected with the corpus's active document set so the agent cannot reach documents outside its scope. **Dispatch reuses existing code paths** — `start_extract` creates the `Extract` row, adds documents, grants creator CRUD via `set_permissions_for_obj_to_user`, and queues `run_extract.s(extract_id, user_id).apply_async()` on `transaction.on_commit`; `start_analysis` delegates entirely to `process_analyzer` so chat-agent runs and auto-CorpusAction runs produce identical `Analysis` records. Both tools accept `corpus_action_id` (auto-injected by `build_inject_params_for_context`) to keep lineage tracking intact when an agent corpus action delegates onward. **Opt-in for CorpusActions**: deliberately _not_ added to `DEFAULT_DOCUMENT_ACTION_TOOLS` — admins must list them on a CorpusAction's `pre_authorized_tools` to skip approval in the automation path. Test coverage in `test_extract_analyzer_tools.py` exercises registry integration & flag wiring (`TestExtractAnalyzerRegistryIntegration`), visibility filtering for discovery tools (`TestListFieldsets`, `TestListAnalyzers`, `TestListRecentExtracts`, `TestListRecentAnalyses`), document-scope resolution and permission gating for the write tools (`TestStartExtract`, `TestStartAnalysis`), and the approval gate firing on the `PydanticAIToolWrapper` (`TestApprovalGate`). Docs updated in three places: a new "Extract & Analyzer Tools" subsection in `docs/architecture/llms/README.md` (with a usage example and the opt-in CorpusAction pattern), a callout in `docs/corpus_actions/intro_to_corpus_actions.md` pointing at the delegation path, and a cross-link from `docs/extract_and_retrieval/data_extraction.md` so people browsing the extract docs find the agent route.
- **Service-layer split — review follow-ups on PR #1685** (`opencontractserver/corpuses/corpus_objs_service.py`, `opencontractserver/users/types.py`, `opencontractserver/llms/agents/core_agents.py`, `opencontractserver/tests/test_corpus_objs_service.py`, `opencontractserver/tests/test_document_folder_service.py`, `opencontractserver/tests/test_llms_typing_coverage.py`, `opencontractserver/tests/test_user_types_resolver.py` (new)).

  - **Unified `get_corpus_documents(include_deleted=True/False)` paths.** Pre-fix, the True branch hand-rolled a `DocumentPath` query that silently bypassed CAML filtering altogether; the False branch delegated to `Corpus._get_active_documents()` which excluded CAML. New `_build_corpus_documents_queryset` helper is the single source of truth — both branches now compose `include_deleted` and `include_caml` toggles uniformly so CAML can never sneak through the trash-view path. Added `include_caml=False` to the public method signature so callers can opt in explicitly. New `TestGetCorpusDocuments_CamlAndDeleted` fixture pins all four flag combinations including the regression guard `test_include_deleted_still_excludes_caml`.
  - **Moved `_resolve_user_or_anon` from `core_agents.py` to shared `users/types.py`** as the public `resolve_user_or_anon(user_id)` helper. Centralising avoids duplicate `User.DoesNotExist` handling and gives the docstring (re: must be called inside the same `sync_to_async` block) one place to live. `core_agents.py` retains both call sites, imported from the shared module. New `test_user_types_resolver.py` pins the three branches (existing id, `None` → `AnonymousUser`, unknown id → `DoesNotExist` rather than silent demotion).
  - **Pytest fixes:** `test_document_folder_service.py` mocks of `post_save` were patching `opencontractserver.corpuses.folder_service.post_save` which no longer exists in the shim module after the split — repointed to the actual home in `corpus_objs_service`. `test_llms_typing_coverage.py::TestCorpusAgentContextInitialize` mocked `Corpus.get_documents` but `CorpusAgentContext.initialize` now routes through `CorpusObjsService.get_corpus_documents` — updated the patch target so the test continues to drive both the "empty list triggers fetch" and "non-empty list short-circuits" branches.

- **PR #1685 third-pass review polish** (`opencontractserver/corpuses/corpus_objs_service.py`, `config/graphql/corpus_types.py`, `opencontractserver/llms/agents/core_agents.py`, `opencontractserver/llms/tools/core_tools/caml_article.py`). Addresses the third Claude review batch on the service-layer split:

  - **`move_documents_to_folder` N+1 eliminated**. The validation loop that ran one `DocumentPath.objects.filter(...).exists()` per document via `check_document_in_corpus` is gone. The atomic block's existing `select_for_update` on `DocumentPath` (filtered by `document_id__in=document_ids, corpus=..., is_current=True, is_deleted=False`) already proves membership, so the missing-set is now computed as `set(document_ids) - {p.document_id for p in current_paths}` inside the locked transaction. Removes O(N) round-trips and folds the membership check into the TOCTOU-coherent atomic step. The error message now enumerates every missing id rather than only the first.
  - **`check_document_in_corpus` → `_check_document_in_corpus`** to match its "NO PERMISSION CHECK" contract. The method has three internal callers (all of which already enforce corpus READ upstream) and zero external callers; the underscore makes the footgun less reachable from autocomplete and signals service-internal use.
  - **CAML production callsites migrated to `Corpus._get_active_documents(include_caml=True)`** (`opencontractserver/llms/tools/core_tools/caml_article.py:180`, `config/graphql/corpus_types.py:resolve_documents`, `config/graphql/corpus_types.py:resolve_annotations`). Each site was already gated by an upstream corpus READ check (`Corpus.objects.visible_to_user(user).get(...)` in caml_article, parent-query visibility for the CorpusType sub-resolvers), so calling the deprecated user-facing `get_documents()` alias was emitting a `DeprecationWarning` on every live CAML resolve / corpus document render. The internal helper is the correct entry point for already-gated sites.
  - **Hoisted the two function-local `CorpusObjsService` imports in `opencontractserver/llms/agents/core_agents.py`** to module level, completing the second-pass hoist work that left these two sites behind. No circular import exists (the service is a pure consumer of model classes).

- **PR #1685 second-pass review polish** (`config/graphql/{og_metadata_queries,document_relationship_mutations,extract_mutations}.py`, `opencontractserver/corpuses/{corpus_objs_service,models}.py`, `opencontractserver/users/types.py`, `opencontractserver/tests/test_document_service.py`). Addresses the latest Claude review on the service-layer split:
  - **Hoisted three function-local `from … corpus_objs_service import CorpusObjsService` imports to module level** in `og_metadata_queries.py`, `document_relationship_mutations.py`, and `extract_mutations.py`. Verified no circular dependency exists at module-load time — the inline form was cargo-cult, not load-bearing.
  - **`resolve_user_or_anon` stale-id contract is now explicit in the docstring** — `User.DoesNotExist` propagates by design; callers that need a "treat deleted creator as anonymous" fallback must catch at their own boundary. Silent demotion would broaden access (the deleted creator's private content would suddenly resolve through AnonymousUser visibility).
  - **`Corpus._get_active_documents` API note** explains why the underscore-prefixed name is the blessed cross-module helper for non-user-context callers (badge / corpus / analyzer tasks, signal handlers, LLM tool batches) despite Python convention pointing the other way. The underscore distinguishes the bypass-permission path from the deprecated user-facing `get_documents()` alias.
  - **`CorpusObjsService.check_document_in_corpus` docstring now leads with "NO PERMISSION CHECK"** — the method is a raw `DocumentPath` membership probe; callers must gate corpus READ before invoking. Suggests `get_corpus_document_by_id` as the higher-level alternative for new resolvers.
  - **`og_metadata_queries.resolve_og_corpus_document` now documents that the `is_public=True` filter on the corpus lookup is load-bearing** — without it, an `AnonymousUser` call to `CorpusObjsService.get_corpus_documents` would resolve documents for any public corpus. The service's anonymous READ path is permissive by design; the gate must live in the resolver's `Corpus.objects.get(... is_public=True)`.
  - **`_build_corpus_documents_queryset` `.distinct()` defensiveness is now documented**: `DocumentPath` has no DB constraint preventing multiple `is_current=True` rows per `(corpus, document)`, only the `(corpus, path)` uniqueness — so an aliased current path or an `include_deleted=True` query that catches both a soft-deleted and a non-deleted current row could surface the same `document_id` twice. `.distinct()` collapses that defensively.
  - **`get_folder_tree` `O(N)` query loop is now flagged with a `TODO(#1685 follow-up)`** documenting the path-from-parent-map / annotate-Count optimisation. Pre-existing behaviour; not changing in this PR to keep the service-split surface stable.
  - **`DocumentService` codecov coverage lifted** from 42.68% to ~95% by adding behavioural test cases for each method's branches (quota over/under, validate_file_type plaintext/binary/disallowed-MIME, create_document PDF/TXT/unsupported/quota-blocked/validation-failed/atomic-rollback, get_document_by_id IDOR matrix, set_document_permissions owner / PERMISSION / denied / exception fall-through). The previous test file only verified structural assertions (method partition, backwards-compat alias surface).
  - **Known follow-up — `corpus_objs_service.py` SRP**: the file is ~2,700 lines and now monolith-scale on its own. Splitting along the sub-seams (folder CRUD, document lifecycle, corpus-document membership) is tracked for a future PR; not scope-creep on this split.

### Added

- **Service-layer split: `DocumentService` + `CorpusObjsService` as the canonical entry points** (`opencontractserver/documents/document_service.py` (new), `opencontractserver/corpuses/corpus_objs_service.py` (new), `opencontractserver/corpuses/folder_service.py` (converted to back-compat shim), `opencontractserver/corpuses/models.py` (added `_get_active_documents` + `DeprecationWarning` wrapper on `get_documents`), `CLAUDE.md`, plus migrations across `opencontractserver/mcp/tools.py` (5 sites), `opencontractserver/mcp/resources.py` (2 sites), `opencontractserver/llms/agents/core_agents.py` (2 sites), `opencontractserver/discovery/views.py` (3 sites in `_search_within_corpus`), `config/graphql/og_metadata_queries.py`, `config/graphql/extract_mutations.py`, `config/graphql/document_relationship_mutations.py`, `opencontractserver/tasks/badge_tasks.py`, `opencontractserver/tasks/corpus_tasks.py`, `opencontractserver/utils/sharing.py`, `opencontractserver/utils/analyzer.py`, `opencontractserver/llms/tools/core_tools/{pii.py,annotations.py,document_indexing.py}`). The single-class `DocumentFolderService` has been split along the natural seam revealed by the PR review of `opencontractserver/mcp/tools.py`: methods where the **document** is the noun and corpus context is incidental (`create_document`, `check_user_upload_quota`, `validate_file_type`, `get_document_by_id`, `set_document_permissions`) now live on `DocumentService`; methods of the form _"give me X inside corpus Y for user Z"_ (every folder operation, the corpus-scoped doc lifecycle helpers `soft_delete_document` / `restore_document` / `permanently_delete_document` / `empty_trash`, the upload/add/remove corpus mutations, and `get_corpus_documents`) now live on `CorpusObjsService`. The new corpus-objs service is designed to grow to other corpus-linked object types — notes, annotations, relationships, extracts, analyses — and is the canonical extension point for future "corpus-scoped X" patterns. `DocumentFolderService` survives as a multiple-inheritance subclass of both new services so the ~18 existing importers continue to work unchanged (cleanup tracked in follow-up issue). New convenience methods on `CorpusObjsService` close the buggy fusion pattern that motivated the split: `get_corpus_document_by_slug(user, corpus, slug)`, `get_corpus_document_by_id(user, corpus, doc_id)`, and `is_document_in_corpus(user, corpus, doc_id)` — all corpus-as-gate, all IDOR-safe (same `Document.DoesNotExist` for not-found, not-in-corpus, and no-permission). All 15 user-context call sites that previously composed `corpus_doc_ids = corpus.get_documents().values_list("id", flat=True)` + `Document.objects.visible_to_user(user).get(id__in=corpus_doc_ids, slug=...)` now flow through the service. The previous pattern silently mixed two different permission semantics (corpus-membership AND document `visible_to_user`) that the canonical service had standardised on as corpus-as-gate; the fusion produced inconsistent behaviour between MCP, discovery, and the GraphQL CAML resolver. The single service entry point now guarantees every caller asks the same question with the same answer. CLAUDE.md item 7 declares the service layer as the canonical entry point for any user-context code touching documents. `Corpus.get_documents()` is now a thin `DeprecationWarning`-emitting wrapper around the new internal helper `Corpus._get_active_documents(include_caml=...)`; internal/Celery callers (`badge_tasks.py:234`, `corpus_tasks.py:72`, `utils/sharing.py:make_corpus_public`, `utils/analyzer.py:run_analysis`, the framework-private LLM helpers in `pii.py` / `annotations.py` / `document_indexing.py`) opt out by calling `_get_active_documents()` directly. Production CAML article paths (`opencontractserver/llms/tools/core_tools/caml_article.py:180`, `config/graphql/corpus_types.py:resolve_caml_articles`) are deliberately left on `corpus.get_documents(include_caml=True)` and will emit the warning at runtime — they are tracked under follow-up issues (CAML-aware service method; MIN-semantic convergence). **Phase-B caching follow-up — `request=` kwarg + N+1 fix:** the new `CorpusObjsService` and `DocumentService` methods that perform `corpus.user_can(...)` / `document.user_can(...)` permission gates now accept and forward a kw-only `request=None` kwarg (mirroring the `DocumentFolderService` signature from PR #1665). Every GraphQL caller (`config/graphql/extract_mutations.py`, `config/graphql/document_relationship_mutations.py`, `config/graphql/og_metadata_queries.py`) now passes `request=info.context` so the two-tier `PermissionQueryOptimizer` cache spans the full corpus-scoped lookup path, eliminating the N+1 regression that the service split would otherwise have introduced. The `get_folder_tree` `O(N)` query loop flagged with `TODO(#1685 follow-up)` is also fixed: descendant counts are now resolved with a single `.annotate(direct_children_count=Count("children", filter=Q(deleted_at__isnull=True), distinct=True))` pass instead of a `.count()` per node.

- **Two-tier permission caching for `Manager.user_can` (closes #1640)** (`opencontractserver/constants/permissioning.py` (new), `opencontractserver/utils/permission_optimizer.py` (new), `opencontractserver/utils/permissioning.py`, `opencontractserver/shared/user_can_mixin.py`, `opencontractserver/corpuses/models.py`, `opencontractserver/corpuses/folder_service.py`, `config/graphql/{corpus_queries,corpus_mutations,corpus_folder_mutations,document_mutations}.py`, `opencontractserver/tests/permissioning/test_permission_optimizer.py` (new), `docs/architecture/query_permission_patterns.md`). Follow-up to PR #1637 which centralized authorization in `_default_user_can`. The cold path (non-superuser, non-creator, non-public-READ) used to hit the DB on every `user_can` call — fine for Phase 0 (Corpus only, one check per folder-service entry point) but linear in the row count once paginated list resolvers start calling `user_can` per row.

  **Tier 1 — per-instance memoization.** `get_users_permissions_for_obj` now caches its result on the instance as `_oc_granted_perms_cache: dict[(user_id, include_group_permissions_bool), frozenset[str]]`. Transparent to every caller (including the deprecated `user_has_permission_for_obj`). Anonymous / unauthenticated users bypass the cache. Cache lifetime equals the instance lifetime; `refresh_from_db` does NOT clear it — long-lived Celery instances must `del instance._oc_granted_perms_cache` after out-of-band perm changes.

  **Tier 2 — request-scoped `PermissionQueryOptimizer`.** Mirrors `ConversationQueryOptimizer` (`opencontractserver/conversations/query_optimizer.py`). Lazily attached to the request as `request._permission_query_optimizer` via `get_request_optimizer(request)`. Keyed by `(user_id, content_type_id, instance_pk, include_group_permissions)` so multiple instances of the same model in one request share a cache. Opt-in via the new `request=` kwarg threaded through `Manager.user_can` / `obj.user_can` / `_default_user_can`. `get_request_optimizer(None)` returns a one-shot optimizer for Celery / management-command contexts.

  **Composition with PR #1663's `permission_cache_scope`.** PR #1663 introduced a dormant `ContextVar`-backed boolean cache wrapping `_default_user_can` (Phase A — opt-in via `permission_cache_scope()`, not yet wired in production). The two layers compose cleanly: scope cache wraps the cold-path lookup at the boolean level, Tier 2 optimizer/Tier 1 instance memo wrap the underlying granted-set computation. Outside a scope (today's default), Tier 1 and Tier 2 do all the deduplication; once Phase B activates `permission_cache_scope()` via middleware, both layers continue to compose without invalidation hazards (the scope is read-only and Tier 2's `set_permissions_for_obj_to_user` invalidation only touches Tier 1/Tier 2, not the scope dict).

  **Wiring.** Plumbed `request=info.context` through:

  - `DocumentFolderService` classmethods (25+ entry points in `corpuses/folder_service.py`) and their callers in `config/graphql/{corpus_queries,corpus_mutations,corpus_folder_mutations,document_mutations}.py`.
  - `CorpusFolder.user_can` override (`corpuses/models.py:1945`) so folder permission checks share the corpus's cache entry.
  - `UserCanMixin` and `InstanceUserCanMixin` in `shared/user_can_mixin.py`.

  **Mutation invalidation.** `set_permissions_for_obj_to_user` gained a `request=None` kwarg and now self-invalidates both tiers for the affected `(user, instance)` pair when called inside the HTTP lifecycle. The Phase-0 `Corpus` mutation call sites (`corpus_mutations.py:180,562,1007,1025,1056,1468`; `document_mutations.py:655,1024,1508`) pass `request=info.context`. Out-of-band call sites (Celery tasks, fixtures, importing utilities) continue to call without a request — Tier 2 is skipped, Tier 1 is still scrubbed for the target user so reused instances see the new grant.

  **Coverage.** New `tests/permissioning/test_permission_optimizer.py` covers per-instance memoization (zero queries on repeat, no leakage across `include_group_permissions`, fast-path short-circuits don't populate cache, defensive copy contract), the request-scoped optimizer (lazy attach, `None`-safe, distinct users keyed independently, per-user / per-instance / total invalidation, anonymous skip), the mutation invalidation contract (cache invalidates mid-request when `request=` is supplied; cache is scrubbed for the target user even when `request=None`), and `Manager.user_can` / `obj.user_can` request passthrough.

- **Decompose `Documents.tsx` into thinner composer + sibling files** (`frontend/src/views/Documents.tsx`, plus new siblings `Documents.styles.ts`, `DocumentsGridView.tsx`, `DocumentsListView.tsx`, `DocumentsCompactView.tsx`, and an extension to `frontend/src/utils/files.ts`). Final slice of the multi-PR decomposition tracked by #1446 (after #1459 / `ModernDocumentItem`, #1578 / `Corpuses`, #1586 / `ChatMessage`, #1580 / `DocumentKnowledgeBase` partial, #1639 / `ChatTray`). `frontend/src/views/Documents.tsx` had grown to **1,725 lines**. Five extractions land it at **915 lines (-47%)**:

  - **`Documents.styles.ts`** _(new, 581 lines)_: All 50 styled-components moved to a sibling file as named exports. Cross-component selectors (`${DocumentCardWrapper}:hover &` referenced by `CardCheckbox` and `CardMenuButton`; `${ListItem}:hover &` referenced by `ListItemActions`) require same-module scope, so all styles stay in one file (same rule that kept `ModernDocumentItem.styles.ts` monolithic in PR #1459). Renamed three styled-components to free up names taken by the new React components — `DocumentsListContainer` (outer section wrapper) → `DocumentsSection`; `DocumentsListView` (inner list table wrapper) → `DocumentsListContainer`; `DocumentsCompactView` (inner compact list wrapper) → `DocumentsCompactContainer`. Dropped the unused `MetaSeparator` styled-component (declared but never rendered in any view branch — CLAUDE.md "no dead code" rule).
  - **`DocumentsGridView.tsx`** _(new, 145 lines)_: The grid-view branch — `DocumentCardWrapper` cards with checkbox / preview / body / footer. Props: `documents`, `selectedIds`, `activeContextMenuDocId`, `onDocumentClick`, `onSelect`, `onContextMenu`.
  - **`DocumentsListView.tsx`** _(new, 121 lines)_: The list-view branch — table-shaped grid including `ListHeader` with select-all checkbox. Adds `onSelectAll` and pre-computed `allSelected: boolean` to the shared prop shape.
  - **`DocumentsCompactView.tsx`** _(new, 91 lines)_: The compact-view branch — flat row layout. Same props as `DocumentsGridView`.
  - **`frontend/src/utils/files.ts`** _(extended, +27 lines)_: Adds `getDocumentTypeBadge(fileType, title)`, the file-extension/type normalizer previously inlined in `Documents.tsx`. Lives in the existing shared `files.ts` utility module per CLAUDE.md's "utility functions belong in utility files" rule. Consumed by `DocumentsGridView` and `DocumentsListView`.
  - Behavior is preserved verbatim — same handler bodies, same DOM, same Apollo reactive-var reads (all 11 stay at the composer; props drilled down to children, matching `ExtractsTabContent.tsx` precedent which receives data via props rather than reading reactive vars directly). Public API (`Documents` named export) is unchanged; importers (`frontend/src/App.tsx:49`, `frontend/tests/Documents.ct.tsx:8`, `frontend/tests/DocumentsTestWrapper.tsx:7`) need no changes. The hero section, search box, filter tabs row, advanced-filters popup, stats grid, section header with view-mode toggle, fetch-more sentinel, empty-state + error branches, context menu, modals, and the `DocumentIcon` SVG fallback stay inline in `Documents.tsx`. All 10 component tests in `frontend/tests/Documents.ct.tsx` and the 4 cases in `frontend/src/views/Documents.refetch-shape.test.ts` pass without modification.

  Closes #1446.

- **Surface OC_SUBTREE_GROUP relationships in semantic search + doc-viewer deep-link** (closes #1645). End-to-end wiring that turns the materialised `OC_SUBTREE_GROUP` rows from #1646 into a first-class semantic-search surface and lets the document viewer jump directly to a surfaced block.

  - **Block-context augmentation on annotation hits** (`opencontractserver/llms/vector_stores/core_vector_stores.py`, `opencontractserver/constants/annotations.py`, `opencontractserver/llms/vector_stores/pydantic_ai_vector_stores.py`). New `BlockContext` dataclass on `VectorSearchResult` carries the smallest-enclosing `OC_SUBTREE_GROUP` relationship for each hit: `relationship_id`, `source_annotation_id`, `source_text`, `target_annotation_ids`, plus a newline-joined `block_text` capped at `SUBTREE_GROUP_BLOCK_TEXT_MAX_CHARS` (16k chars). `_attach_block_context_sync` runs as a single `Relationship.objects.filter(target_annotations__in=hit_ids, relationship_label__text="OC_SUBTREE_GROUP", structural=True).annotate(descendant_count=Count(...))` join after reranking, picks the smallest enclosing group per hit (ties broken on lowest relationship pk for determinism), and constructs `block_text` from one cached `IN`-query over the referenced annotations. Wired into `search()`, `async_search()`, `hybrid_search()`, `async_hybrid_search()`, and `global_search()` so every public retrieval entry-point benefits. `PydanticAIVectorSearchResponse.from_core_results` / `.async_from_core_results` pass `block_context` through as a plain dict on the agent tool surface.

  - **Relationship as a first-class vector target** (`opencontractserver/annotations/models.py`, `opencontractserver/annotations/migrations/0073_embedding_relationship.py`, `opencontractserver/shared/Managers.py`, `opencontractserver/tasks/embeddings_task.py`, `opencontractserver/utils/subtree_groups.py`, `opencontractserver/llms/vector_stores/core_relationship_vector_store.py`). Added polymorphic `Embedding.relationship` FK alongside the existing document/annotation/note/conversation/message slots, with a partial-unique constraint on `(embedder_path, relationship)`. `Relationship` now mixes in `HasEmbeddingMixin` (`get_embedding_reference_kwargs -> {"relationship_id": self.pk}`); `EmbeddingManager.store_embedding` accepts `relationship_id`. New Celery task `calculate_embeddings_for_relationship_batch` mirrors the per-annotation dual-embedding strategy (default embedder + corpus-preferred embedder when distinct), keyed on `synthesize_relationship_block_text(relationship)` — the same bounded source+targets concatenation the GraphQL surface returns, so the embedded payload and the snippet clients see never drift. `build_subtree_groups_for_document` dispatches the task after each successful materialisation (`_dispatch_relationship_embeddings`), fanning out per-corpus when a document belongs to multiple corpora with distinct preferred embedders. New `CoreRelationshipVectorStore` runs the same cosine-distance pass against `Embedding` filtered by `relationship__isnull=False`, scoped by `Relationship.objects.visible_to_user(user)` + corpus/document-aware Q-OR (mirrors the annotation store's structural-set inclusion logic).

  - **GraphQL surface** (`config/graphql/social_types.py`, `config/graphql/graphene_types.py`, `config/graphql/search_queries.py`). New `BlockContextType` exposed as a `block_context` field on `SemanticSearchResultType`. New `semanticSearchRelationships` resolver returns `SemanticSearchRelationshipResultType` (relationship pk, similarity, label, source/target annotation pks, block_text, document/corpus pks). Permission flow mirrors `resolve_semantic_search` — empty-list response for missing-or-denied document/corpus prevents IDOR enumeration; corpus-scoped queries derive the embedder from `Corpus.preferred_embedder` so relationship vectors stay in the corpus's frozen embedding space.

  - **Frontend deep-link + jump-to-relationship** (`frontend/src/graphql/{queries,cache}.ts`, `frontend/src/utils/{navigationUtils,__tests__/navigationUtils.test}.ts`, `frontend/src/routing/CentralRouteManager.tsx`, `frontend/src/components/knowledge_base/document/document_kb/useJumpToRelationship.ts`, `frontend/src/components/knowledge_base/document/DocumentKnowledgeBase.tsx`, `frontend/src/components/annotations/CorpusAnnotationCards.tsx`). New `SEMANTIC_SEARCH_RELATIONSHIPS` query and `blockContext { … }` field added to the existing `SEMANTIC_SEARCH_ANNOTATIONS` selection. New URL parameter `?rel=<relationship_pk>` (raw Django PK, not a Relay global ID) consumed by `CentralRouteManager` and stored on a new `selectedRelationshipId` reactive var. New `useJumpToRelationship` hook, mounted unconditionally in `DocumentKnowledgeBase`, watches `selectedRelationshipId` + `allRelationsAtom` (combined user-editable + structural relations) and, on a match, sets `selectedRelationsAtom` and scroll-into-views the source annotation — applied exactly once per id change via a ref so subsequent local edits aren't fought. `CorpusAnnotationCards` annotation click now includes `relationshipId` in the navigation query params when the hit carries a `block_context`, so users land on the doc with both the leaf annotation selected (existing `?ann=` flow) AND the containing block's relation line rendered. `buildQueryParams` learns to emit the new `rel` segment; navigation-utils tests cover the round-trip.

  - **Tests** (`opencontractserver/tests/test_subtree_group_vector_search.py`, `frontend/src/utils/__tests__/navigationUtils.test.ts`). Backend suite covers: smallest-enclosing-group selection on a 4-level structural tree, root-level hit returns `None`, empty input is a pass-through, `block_text` truncation at `SUBTREE_GROUP_BLOCK_TEXT_MAX_CHARS`, end-to-end relationship vector retrieval with the same embedding the helper synthesised, and IDOR denial returning `[]` for a user lacking corpus visibility. Helper-text synthesis tests cover deterministic id-ordering of source+targets, cap-enforcement, and empty-component skipping. Frontend test pins the `rel=<pk>` URL-builder contract.

- **PII scanner task-based analyzer + analyzer-app coverage lift** (`opencontractserver/tasks/doc_analysis_tasks.py`, `opencontractserver/tests/test_pii_scanner_analyzer_task.py` (new), `opencontractserver/tests/test_analyzer_app_coverage.py` (new), `docs/test_scripts/pii_scanner_analyzer_smoke.md` (new)). A new `@async_doc_analyzer_task`-decorated function, `pii_scanner_privacy_filter`, wraps the existing `adetect_pii` privacy-filter client and exposes PII scanning as a first-class analyzer that can be invoked via `startAnalysisOnDoc` (i.e. the same UI/GraphQL path as any other analyzer), rather than only through the chat-sidebar agent tool `scan_and_annotate_pii`. The task chunks via the underlying client (40 KB chunks with 500-char overlap and `(start, end, group)` de-dup) and produces SPAN_LABEL annotations for text/markdown and TOKEN_LABEL annotations for PDFs via the standard `async_doc_analyzer_task` post-processing. Labels are pre-created with the canonical text/colour/icon from `opencontractserver.llms.tools.core_tools.pii.ENTITY_GROUP_LABELS` (e.g. `"PII: Email"` / `#1f77b4` / `mail`) so visual parity with the agent-tool path is preserved. A single configurable knob `min_score` (default 0.5) is surfaced through `input_schema`. The analyzer is auto-registered by `auto_create_doc_analyzers` (migration `0009_auto_load_doc_analyzers` plus the startup hook in `opencontractserver/analyzer/startup.py`); no new migration. Companion coverage work targets the lowest-coverage backend app: `test_analyzer_app_coverage.py` adds tests for `signals.install_gremlin_on_creation` (chain assembly under `transaction.on_commit`), `signals.handle_analysis_completion` (with a documented mismatch where the handler checks the literal `"COMPLETE"` while `JobStatus.COMPLETED.value == "COMPLETED"`), `checks.check_unsynced_analyzers` (warning emitted / silent / import-error swallow), `startup.sync_analyzers_on_startup` (happy + exception path), `admin_views.AnalyzerSyncView` (GET helper + POST with and without `analyzer.add_analyzer` perm), `utils.get_gremlin_manifests` (HTTP 500, malformed manifest item, unknown gremlin id), `utils.auto_create_doc_analyzers` edge cases (no users, celery unavailable, update_existing=False), and the `sync_doc_analyzers` management command (`--dry-run` + normal run). `views.AnalysisCallbackView` and `views._create_analysis_notification` get their first happy-path tests (valid token + valid body → COMPLETED + `import_analysis` scheduled + ANALYSIS_COMPLETE notification; invalid body → FAILED + ANALYSIS_FAILED notification; create raising RuntimeError swallowed with warning).

- **Phase A review follow-ups on PR #1663** (`opencontractserver/shared/user_can_mixin.py`, `opencontractserver/shared/Managers.py`, `opencontractserver/shared/QuerySets.py`, `opencontractserver/utils/permissioning.py`, `opencontractserver/tests/permissioning/test_authorization_invariants_coverage.py`). Extracted the duplicated int/str → `User` resolution from `AnnotationManager.user_can`, `NoteManager.user_can`, `RelationshipManager.user_can` and `_default_user_can` into a single `resolve_user_for_user_can` helper in `shared/user_can_mixin.py` (Claude review #1 — ~40 lines of duplicated boilerplate removed). `user_has_permission_for_obj` shim now raises `TypeError` with an actionable message when the instance's `_default_manager` doesn't implement `user_can` (Claude review #2). Previously the call would surface as a confusing bare `AttributeError` deep inside a resolver during the incremental Phase A → Phase B migration of the ~170 legacy call sites. Removed the unexplained `.order_by("created")` from `PermissionQuerySet.visible_to_user` superuser branch (Claude review #7) — not every model that uses `PermissionManager` has a `created` column, so the new clause was a latent `FieldError`. Added `test_authorization_invariants_coverage.py` (46 new test cases) covering the codecov-flagged patch lines and the Claude-flagged behavioural gaps: direct unit tests of the `permission_cache_scope` / `cached_user_can` / `store_user_can` machinery (cache went from 73.91% → 100% patch coverage); the access-widening case where a stranger with explicit doc+corpus READ now sees the relationship (Claude review #3); the `created_by_extract` privacy-recursion branch on annotations (Claude review #4 — the partner of the `created_by_analysis` test already in the file); the int/str user-id resolver paths for every per-model override; the anonymous note-visibility leaf branches under public/private corpus combinations; the `PermissionQuerySet` fallback body (superuser/anonymous/authenticated/guardian/LookupError); the `UserFeedbackQuerySet` guardian path; the shim's `DoesNotExist` deny and the `TypeError` guard for unmigrated managers.

- **Phase A of permission centralization: per-model `Manager.user_can` overrides + request-scoped permission cache** (`opencontractserver/shared/{Managers,QuerySets,permission_cache}.py`, `opencontractserver/conversations/models.py`, `opencontractserver/annotations/query_optimizer.py`, `opencontractserver/utils/permissioning.py`, `opencontractserver/tests/permissioning/test_authorization_invariants.py`). Issue #1655 — the follow-up to PR #1637 (Step 0 vertical slice). Every visibility-managed model now exposes a `Model.objects.user_can(user, instance, permission)` override aligned with its `visible_to_user` filter so the two surfaces answer the same question for the same `(user, instance)`: **`DocumentManager`** (thin delegate; default rules suffice), **`AnnotationManager`** (load-bearing order — superuser → structural-write-deny → privacy recursion via `Analysis.objects.user_can` / `Extract.objects.user_can` → MIN(doc, corpus) via `AnnotationQueryOptimizer._compute_effective_permissions`), **`RelationshipManager`** (structural-write-deny + MIN; deliberately preserves the legacy decision to NOT consult `created_by_analysis`/`created_by_extract`), **`NoteManager`** (creator OR MIN(doc, corpus), composed via `Document.objects.user_can` and `Corpus.objects.user_can`), **`UserFeedbackManager`** (adds the `commented_annotation.is_public` READ grant on top of the default rules), **`ConversationManager`** (READ uses `visible_to_user(...).filter(pk=).exists()` to preserve the CHAT/THREAD bifurcation; non-READ uses `_default_user_can` so is_public does NOT grant writes), **`ChatMessageManager`** (READ uses the same exists trick to preserve the moderator branch — corpus owner sees messages in threads on their corpus). New `opencontractserver/shared/permission_cache.py` provides a `ContextVar`-backed request-scoped read-only cache (`permission_cache_scope()`) keyed by `(user_id, app_label, model_name, pk, permission, include_group_permissions)`; dormant outside scope, wired into `_default_user_can` at the guardian-lookup boundary so any future request/Celery middleware can flip it on by entering the scope. Invariant test suite (`test_authorization_invariants.py`) expanded with 7 new `TransactionTestCase` classes (Document/Annotation/Relationship/Note/Conversation/ChatMessage/UserFeedback) plus a shared `_UserCanInvariantsMixin` that pins `Model.objects.user_can(u, obj, READ) == Model.objects.visible_to_user(u).filter(pk=obj.pk).exists()` across the fixture matrix, manager/instance-surface parity, and superuser bypass. Plus per-model invariants: is_public-grants-only-READ on Document, structural-write-deny on Annotation and Relationship, the new privacy-recursion bug-fix posture, MIN visibility for Notes, CHAT-no-context-inheritance and THREAD-inherits-corpus for Conversation, moderator-via-corpus-ownership for ChatMessage, and the `commented_annotation.is_public` READ grant (with no bleed into writes) for UserFeedback.

- **Materialised `OC_SUBTREE_GROUP` relationships at ingestion** (`opencontractserver/constants/annotations.py`, `opencontractserver/utils/subtree_groups.py`, `opencontractserver/pipeline/base/parser.py`, `opencontractserver/tests/test_subtree_group_materialization.py`). After a parser imports a document's structural annotations and explicit relationships, `BaseParser.save_parsed_data` now runs `build_subtree_groups_for_document` between the relationship-import step and the structural-set migration. The new utility walks the unified parent-child hierarchy (from `Annotation.parent` self-FKs plus any `OC_PARENT_CHILD_LABEL_NAME` structural Relationship rows) and creates one materialised `Relationship` row per non-leaf with the ancestor as the sole source annotation and the ancestor's full transitive descendant set as targets. Retrieval can then surface the larger "block" of an annotation hit with a single join instead of a recursive CTE per result. Guardrails: groups whose descendant set exceeds `SUBTREE_GROUP_MAX_DESCENDANTS` (default 500) are skipped with a warning; the walker prunes branches deeper than `SUBTREE_GROUP_MAX_DEPTH` (default 32) and detects cycles defensively via on-stack tracking. The built-in `OC_SUBTREE_GROUP` `AnnotationLabel` is resolved by filter-first / create-as-fallback (ignoring `creator` in the lookup) so the system-wide label never silently forks per-user in multi-user installations. Re-parse safe: prior `OC_SUBTREE_GROUP` rows are deleted from both `document`-scoped and `structural_set`-scoped tables, and freshly created rows are attached directly to the existing `StructuralAnnotationSet` when one is already linked to the document (avoiding orphaned `document`-scoped rows that would otherwise dangle because `_create_structural_annotation_set` short-circuits on a re-parse). Test coverage: 11 cases covering one-row-per-non-leaf, transitive content correctness, idempotent re-runs, structural-set migration through the parser, size cap, depth cap, cycle detection (with an `on_stack`-reachable cycle exercising the warning), empty document, parent-child relationship edges, label-non-proliferation, and the re-parse-after-structural-set path. **Post-review polish (closes #1662)**: hoisted the inline `_PRUNED_SAMPLE_CAP` magic number to `SUBTREE_GROUP_PRUNED_SAMPLE_CAP` in `opencontractserver/constants/annotations.py` (CLAUDE.md compliance); refactored the three `qs1 | qs2` union querysets in `subtree_groups.py` (annotations, parent-child relationships, idempotency delete) to single `Q`-OR filters so `.delete()` runs against an explicit WHERE clause rather than relying on Django queryset combining; made root traversal order deterministic (`sorted`) to mirror the existing `non_leaves` sort; clarified the comment above the per-row `Relationship.objects.create()` loop to note that `Relationship.save()` is overridden to call `clean()` (Django's default `Model.save()` does not, so `bulk_create` would silently bypass the document-XOR-structural_set constraint); added `test_depth_cap_warning_logged` mirroring `test_size_cap_warning_logged` to lock in the depth-cap warning contract.

- **OC_URL link annotations — clickable hyperlinks anchored to highlighted text** (`opencontractserver/annotations/models.py`, `opencontractserver/annotations/migrations/0072_annotation_link_url.py`, `opencontractserver/constants/annotations.py`, `config/graphql/{annotation_mutations,serializers,mutations}.py`, `frontend/src/assets/configurations/constants.ts`, `frontend/src/types/graphql-api.ts`, `frontend/src/components/annotator/types/annotations.ts`, `frontend/src/components/annotator/utils/urlAnnotation.ts` (new), `frontend/src/graphql/{queries,mutations}.ts`, `frontend/src/components/annotator/hooks/AnnotationHooks.tsx`, `frontend/src/components/annotator/display/components/{Selection,SelectionBoundary}.tsx`, `frontend/src/components/annotator/renderers/pdf/{PDF,PDFPage,SelectionLayer}.tsx`, `frontend/src/components/annotator/renderers/txt/TxtAnnotator.tsx`, `frontend/src/components/annotator/components/wrappers/TxtAnnotatorWrapper.tsx`, `frontend/src/components/annotator/components/modals/CreateUrlAnnotationModal.tsx` (new), `frontend/src/components/knowledge_base/document/DocumentKnowledgeBase.tsx`, `frontend/src/components/knowledge_base/document/document_kb/DocumentViewer.tsx`, `frontend/src/utils/transform.tsx`). Annotations carrying the new `OC_URL` label render as hyperlinks (underline + external-link icon, pointer cursor) and open their `link_url` on click in both the PDF viewer and the text/markdown viewer. The `Annotation.link_url` URL field (max 2048, nullable) carries the target; an "Add link…" item in the selection action menu prompts for a URL and calls a new `add_url_annotation` GraphQL mutation that auto-creates the `OC_URL` label per-corpus via `Corpus.ensure_label_and_labelset` (mirroring `OC_SECTION`). The pencil icon on an existing `OC_URL` annotation opens a URL-edit modal instead of the label modal. Held `Shift`/`Ctrl`/`Cmd` while clicking a link annotation falls back to the normal "toggle selection" behaviour so authors can still pick a link to delete or re-edit it. **Security**: `link_url` is validated both in `Annotation.save()` (always runs, regardless of `VALIDATE_ANNOTATION_JSON`) and in the GraphQL mutation/serializer layer via a shared `validate_link_url(...)` helper — only `http://`, `https://`, and site-relative `/...` paths are accepted, so `javascript:`, `data:`, and other dangerous schemes are rejected before persistence and never reach `window.open`. External targets open in a new tab with `noopener,noreferrer`; site-relative paths navigate in the current tab so the SPA router can resolve them.

### Security

- **Auth0 permissioning audit — privilege escalation, IDOR, and credential storage hardening** (`config/settings/base.py`, `config/graphql_auth0_auth/utils.py`, `config/graphql/annotation_mutations.py`, `config/graphql/extract_mutations.py`, `opencontractserver/analyzer/{models.py,views.py,migrations/0021_hash_analysis_callback_token.py}`, `opencontractserver/utils/analyzer.py`, `opencontractserver/users/{apps.py,checks.py,models.py}`, `opencontractserver/constants/auth.py`, `opencontractserver/tests/{test_analyzers.py,test_security_hardening.py}`, `docs/configuration/authentication.md`). Outcome of an end-to-end audit of every authorization pathway that touches Auth0. Verified findings only — agent-reported "CRITICAL" claims that turned out to be false positives (intentional IDOR-safe `SetCorpusVisibility`, `AnnotationImagesView`, moderation mutations, `conversation_queries.moderation_metrics`, etc.) were dropped after self-verification. Twelve real fixes shipped across three layers:

  - **F1 — Defense-in-depth allowlist for `is_superuser` JWT claim sync.** Added `AUTH0_SUPERUSER_SUB_ALLOWLIST` (env, default `[]`). `sync_admin_claims_from_payload` now refuses to flip `User.is_superuser` to `True` unless the user's Auth0 `sub` (Django `username`) is in the allowlist, regardless of what the verified token says. This blocks the structural privilege-escalation path where a misconfigured Auth0 tenant Action sources `is_superuser` from user-writable `user_metadata` instead of admin-only `app_metadata`. New `users.W001` system check warns at startup when `USE_AUTH0=True` but the allowlist is empty (the safe default that demotes existing superusers on next sync until operators populate the list).
  - **F2 — Tightened admin-claim cache window.** `ADMIN_CLAIMS_CACHE_TTL` reduced from `300s` to `30s` (`opencontractserver/constants/auth.py`). Bounds the privilege-retention gap when Auth0 demotes a user — was up to 5 minutes, now ≤30s on API requests. Admin login was already cache-bypassed and unchanged.
  - **F3 + F4 — `AddRelationship` IDOR oracle eliminated** (`config/graphql/annotation_mutations.py:466-560`). `Annotation.objects.filter(id__in=...)` replaced with `visible_to_user(user).filter(...)` plus a count comparison; `Corpus.objects.get(pk=...)` replaced with `visible_to_user(user).get(...)`. All not-found and no-permission branches now return a single `"Relationship target(s) not found or you do not have permission to create a relationship here."` message — no more annotation-ID echoing, no more `str(DoesNotExist)` leaks. Broad `except` no longer surfaces ORM text. **Follow-up:** the `document_id` argument is now visibility-checked too via `Document.objects.visible_to_user(user).filter(pk=document_pk).exists()`, closing the residual IDOR oracle where a caller with CREATE on a corpus could still land a relationship row on a `document_id` they could not see. Failure path collapses into the same not-found message. Coverage in `TestAddRelationshipDocumentIDOR` (`opencontractserver/tests/test_security_hardening.py`).
  - **F5 — `CreateMetadataColumn` corpus existence oracle eliminated** (`config/graphql/extract_mutations.py:170-280`). Same `visible_to_user`+unified-message pattern.
  - **F6 — `UpdateMetadataColumn` column existence oracle eliminated** (`config/graphql/extract_mutations.py:281-352`). `Column` inherits `BaseVisibilityManager` from `BaseOCModel`, so `Column.objects.visible_to_user(user).get(...)` already works.
  - **F7 — Datacell mutation broad-except message leaks** (`config/graphql/extract_mutations.py:51-138`). `Datacell.DoesNotExist` now caught explicitly with a generic message; broad `except` logs server-side and returns a fixed string instead of `f"...{e}"`.
  - **F9 — Bounded JWKS stale-cache fallback** (`config/graphql_auth0_auth/utils.py:_get_cached_jwks`). Added `_JWKS_STALE_GRACE = 3600s`. Stale JWKS continues to be served only within `_JWKS_CACHE_TTL + _JWKS_STALE_GRACE` of the last successful fetch; beyond that window, decode fails closed. Closes the indefinite-stale-key window where an Auth0 outage after key rotation would let the old (potentially compromised) key keep verifying tokens.
  - **F10 — Hash `Analysis.callback_token` at rest, plaintext only at submit time** (`opencontractserver/analyzer/{models.py,views.py,migrations/0021_hash_analysis_callback_token.py}`, `opencontractserver/utils/analyzer.py`). Replaced the `UUIDField` with `callback_token_hash` (`CharField(max_length=64)` storing SHA-256 hex). Helper methods `Analysis.rotate_callback_token() -> plaintext` and `Analysis.verify_callback_token(candidate) -> bool` (constant-time) live on the model. `submit_corpus_documents_to_analyzer` mints a fresh plaintext at submit time and sends it to the gremlin worker; the DB never stores the plaintext. The submit-time logging line is now redacted. Migration `0021_hash_analysis_callback_token` backfills `callback_token_hash` by hashing existing UUID `callback_token` values, so any in-flight analyzer that still holds a previously-issued plaintext continues to verify successfully. Disabled-test `__test_receive_callback_with_gremlin_results` and `test_correct_token_uuid_type_accepted` updated to use `rotate_callback_token`.
  - **F8 + F11 — Documentation hardening for AUTH0_CREATE_NEW_USERS and `User.email` non-uniqueness.** No code/schema change. `docs/configuration/authentication.md` now includes a Step 7 walkthrough for `AUTH0_SUPERUSER_SUB_ALLOWLIST`, env-var rows for the allowlist and `AUTH0_CREATE_NEW_USERS`, a `!!! danger` callout pinning admin claims to `app_metadata` (never `user_metadata`), and a note that `User.email` is informational only — identity is the Auth0 `sub` in `User.username`. Inline comment added at the email field declaration.

  **Verified false positives (NOT fixed — agents were wrong):** `SetCorpusVisibility` (`corpus_mutations.py:82-104`), `AnnotationImagesView`, `bulk_document_upload_status` cache-miss, GraphQL introspection in DEBUG, moderation mutations (`moderation_mutations.py:413/492/556`), `conversation_queries.moderation_metrics:553`, `UpdateCorpusMutation` preferred_embedder oracle, JIT email-collision claim. Each was re-read in source before being dismissed.

  **Operator action required before deploy:** populate `AUTH0_SUPERUSER_SUB_ALLOWLIST` with the Auth0 `sub` of every user who must retain Django `is_superuser`. Existing superusers whose subs are not in the allowlist will be demoted on their next claim sync (within ~30s of their next API call). See `docs/configuration/authentication.md` Step 7 and `users.W001`.

  **Review follow-ups** (`config/settings/base.py`, `config/settings/test.py`, `config/graphql_auth0_auth/utils.py`, `opencontractserver/analyzer/migrations/0021_hash_analysis_callback_token.py`, `opencontractserver/tests/{test_admin_auth.py,test_auth0_jwks_cache.py,test_security_hardening.py,test_metadata_columns_graphql.py}`):

  - Made `ADMIN_CLAIMS_CACHE_TTL` configurable via `AUTH0_ADMIN_CLAIMS_CACHE_TTL` so operators can tune the revocation-SLA vs. write-pressure trade-off without a code change.
  - Wired `AUTH0_CREATE_NEW_USERS` env var into `AUTH0_JWT` settings so `get_auth0_user_from_token` actually honours operator-disabled provisioning.
  - Migration `0021` reverse path uses `QuerySet.update()` instead of `model.save()` (no historical-model signals on downgrade) and `iterator(chunk_size=500)` to bound memory.
  - `_can_serve_stale` now asserts it is holding `_jwks_cache_lock` and carries a prominent docstring warning.
  - `SILENCED_SYSTEM_CHECKS = ["users.W001"]` added to test settings so empty-allowlist tests don't print noise.
  - New direct unit tests cover `TestAuth0SuperuserAllowlist` (allowlist gate), `TestAuth0SuperuserAllowlistSystemCheck` (`users.W001`), `TestAnalysisCallbackTokenHelpers` (rotate/verify edges), `TestCanServeStale` (stale-grace helper), and JWKS fail-closed paths beyond the stale-grace window. Updated the existing failing tests to populate `AUTH0_SUPERUSER_SUB_ALLOWLIST` for elevation paths and match the new IDOR-safe `"Corpus not found or you do not have permission"` message.

### Fixed

- **Import/export V2 follow-ups from PR #1635 review (closes #1649)** (`opencontractserver/constants/zip_export.py` (new), `opencontractserver/tasks/export_tasks_v2.py`, `opencontractserver/tasks/export_tasks.py`, `opencontractserver/tasks/import_tasks_v2.py`, `opencontractserver/tasks/fork_tasks.py`).

  - **Memory regression in V2 corpus export → fork OOM risk on large corpora** (`opencontractserver/tasks/export_tasks_v2.py`, `opencontractserver/tasks/export_tasks.py`, new `opencontractserver/constants/zip_export.py`). `build_corpus_v2_zip` materialised the entire export — including every active document's binary PDF content — into an `io.BytesIO` buffer before the caller (fork or `finalize_export`) consumed it. A 200 × 5 MB-PDF corpus produced a ~1 GB in-process allocation; on a worker with a 2 GB cap, fork would silently OOM. Replaced `io.BytesIO()` with a `tempfile.SpooledTemporaryFile` whose `max_size` is the new `EXPORT_SPOOL_MAX_SIZE_BYTES` constant (default 100 MB, settings-overridable). Small archives stay entirely in heap; large archives transparently roll over to a temp file the OS can page out, bounding peak heap to ~100 MB regardless of corpus size. Companion fix in `finalize_export`: switched the persistent write from `ContentFile(output_bytes.read())` (which re-buffered the _entire_ archive in memory before writing) to `django.core.files.File(output_bytes, name=filename)`, so `default_storage.save` streams chunks straight off the spool. Parameter type widened from `io.BytesIO` to `IO[bytes]` to match. The Celery wrapper `package_corpus_export_v2` and the fork path now use the spooled buffer as a context manager so the on-disk temp file releases even if the import raises.
  - **Redundant exception summary in import V2 logging** (`opencontractserver/tasks/import_tasks_v2.py`). `logger.error("import_corpus_v2_from_bytes() - Exception: %s", e, exc_info=True)` duplicated the exception summary — `exc_info=True` already attaches the formatted traceback to the log record, so structured-log aggregators showed the exception text twice (once in the message body, once in the stack trace). Simplified to `logger.error("import_corpus_v2_from_bytes() failed", exc_info=True)` and added an inline comment explaining the convention so the duplication does not creep back.
  - **Partial-rollback contract documented in `_import_corpus` and `fork_corpus`** (`opencontractserver/tasks/import_tasks_v2.py`, `opencontractserver/tasks/fork_tasks.py`). `_import_corpus` opens nested `transaction.atomic()` blocks for ingestion sources, metadata schema, etc. Django promotes these to savepoints (not autonomous transactions), and the function's broad-except returns `None` after rolling back only the failing savepoint — earlier savepoints that already released remain pending against whatever atomic the caller wraps the call in. Added a docstring paragraph to `_import_corpus` spelling out the savepoint semantics and the caller's obligation to wrap in an outer atomic if it wants all-or-nothing semantics on failure. Added a matching block-level comment in `fork_corpus` at the `with transaction.atomic():` site explaining that the `raise RuntimeError(...)` on `imported_id is None` is what triggers the outer rollback, and that `transaction.on_commit` callbacks registered inside (the orphaned-blob GC) only fire on a successful outer commit so rollback is leak-safe.

- **Flaky `TestApprovalFlow` / `TestNestedApprovalGates` / `TestPydanticAIAgents*` / `TestRunAgentThreadActionAsync` / `TestLongConversationAPI` from stale `@override_settings(DATABASES=...)` decorator** (`opencontractserver/tests/test_tool_approval_gate.py:114`, `test_nested_approval_gates.py:132`, `test_pydantic_ai_agents.py:116,721`, `test_thread_corpus_actions.py:1190`, `test_long_conversation_api.py:35`). The four `test_tool_approval_gate.py` failures observed on the main CI/CD run (`test_event_serialisation`, `test_multiple_gates_interleaved`, `test_rejection_branch`, `test_single_gate_pause_resume`) — and intermittent failures across the four sibling files using the same harness — traced back to a class-level `@override_settings(DATABASES={"default": {"CONN_MAX_AGE": 0}})` decorator. The intent was "force fresh DB connections for async ORM," but the override fully _replaces_ `settings.DATABASES["default"]` with a one-key dict missing `ENGINE` / `NAME` / `USER` / etc. Django itself flags `DATABASES` as a `COMPLEX_OVERRIDE_SETTINGS` (`django/test/signals.py`) and emits `UserWarning: Overriding setting DATABASES can lead to unexpected behavior.` for exactly this reason. Existing `BaseDatabaseWrapper` instances survive (they captured `settings_dict` at creation), but any _new_ connection created while the override is active reads the broken settings. For `TransactionTestCase` + `@pytest.mark.serial` runs, `conftest.py:258-264` aggressively calls `db.connections.close_all()` in teardown, forcing reconnection during the override window for the next test; `TransactionTestCase` also flushes `django_content_type` + `auth_permission` between tests, so the next `setUp` racing with the stale ContentType cache hit a `duplicate key value violates unique constraint "auth_permission_content_type_id_codename_..."` (or `..."django_content_type_app_label_model_..."`) `IntegrityError`. Symptom on the failing-tests path was either `Document.DoesNotExist` (data not visible through the freshly-rebuilt connection) or the permission `IntegrityError` in `setUp`. `CONN_MAX_AGE=0` does nothing for tests anyway — it is only consulted by `close_old_connections`, wired exclusively to `request_started`/`request_finished` signals (`django/db/__init__.py:61-62`). Fix: dropped the decorator from all six sites (and the unused `override_settings` import where it became dead), and removed the now-misleading "@override_settings to prevent connection pooling issues" line from the `TestRunAgentThreadActionAsync` docstring in `test_thread_corpus_actions.py`. Local repro before fix: 2/8 failures on the file alone, 3/5 with all five files (one run hitting 34 failed tests). After fix: 5/5 single-file, 3/3 multi-file, plus a clean parallel `-n 4 --dist loadscope` run across all five files.

- **Auth0 admin login race condition — SDK loaded asynchronously while `initAuth0` ran synchronously** (`opencontractserver/templates/admin/auth0_login.html`, `opencontractserver/tests/test_admin_auth.py`). The Auth0 SPA SDK is loaded via `document.createElement('script')`, which is implicitly `async=true` per the HTML spec. The template had two separate inline `<script>` blocks: the first injected the SDK element with only an `error` listener, the second called `initAuth0()` synchronously. Because the SDK script was still fetching when `initAuth0` ran, `typeof auth0 === 'undefined'` was true and the misleading message `Failed to load Auth0 SDK - SRI check may have failed` was logged while the admin login UI showed "Auth0 unavailable (security check failed)" — even though the SRI hash matched the live CDN file and `cdn.jsdelivr.net` responded HTTP 200. Fix: merged the two scripts into one nonce'd block and wired `sdkScript.addEventListener('load', initAuth0)` (alongside the existing `error` listener) _before_ `insertBefore`, so a cached SDK cannot fire `load` before the listener is attached. Removed the now-dead `typeof auth0 === 'undefined'` defensive check inside `initAuth0` (the `load` event guarantees the global is present) and the synchronous `initAuth0()` call that was the race trigger. Tightened the error message in `handleScriptError` from `"Failed to load Auth0 SDK - SRI check may have failed"` to `"Auth0 SDK failed to load (network or SRI check)"` since after the refactor the function only fires from the script element's `error` event, which triggers on any real load failure (network, CSP block, or SRI mismatch). Auth0 callback handling (`?code=…&state=…`), CSP nonce / integrity / crossOrigin attributes, and the rate-limit-aware password fallback form (separate script tag) are all preserved. New regression test `TestAdminLoginView.test_auth0_sdk_load_listener_wired` asserts that both `load` and `error` listeners attach before `insertBefore` and that no synchronous `initAuth0()` call remains in the rendered template, so the race cannot silently come back via a future refactor.

### Added

- **Sidecar + upversion intersection test** (`opencontractserver/tests/test_sidecar_import.py` — new `TestSidecarUpversioning` class). Pins the contract that re-importing a zip with a sidecar at an already-occupied corpus path (default pipeline route, i.e. `skip_pipeline` absent / falsy) (a) creates a new `Document` in the same `version_tree_id` with `parent` pointing at the prior version and `DocumentPath.version_number` incremented; (b) attaches the new sidecar's text annotations / doc-level labels to the **new** `Document`; (c) leaves the prior version's annotations attached to the now-non-current `Document` (no migration, no deletion); and (d) the current-path query (`DocumentPath.is_current=True, is_deleted=False`) resolves only to the new version's annotations. Sidecar title / description are intentionally not asserted here because they only land on the Document row via the `skip_pipeline=True` route (already covered by `test_skip_pipeline_creates_document_from_export_data`). Existing upversion tests in `test_zip_import_integration.py::TestDocumentUpversioning` covered the path/version-tree mechanics; this test pins the sidecar-payload-meets-collision case that they didn't exercise. Also factors out a shared `_SidecarImportTestMixin` to DRY the in-memory zip / `TemporaryFileHandle` helpers across the sidecar-import test classes in the file.

### Changed

- **Decompose `ChatTray.tsx` into thinner composer + sibling hooks** (`frontend/src/components/knowledge_base/document/right_tray/ChatTray.tsx`, plus new siblings `useChatStreamHandlers.ts`, `useChatAgentMessageHandler.ts`, `useChatSendHandlers.ts`, and `chatUtils.ts`). Next slice of the multi-PR decomposition tracked by #1446 (after #1459 / `ModernDocumentItem`, #1578 / `Corpuses`, #1586 / `ChatMessage`, #1580 / `DocumentKnowledgeBase` partial). `ChatTray.tsx` had grown to **1,815 lines**, but unlike prior decompositions it owned no inline styled-components or React sub-components — its weight was concentrated in stateful handler functions. Three hook extractions land it at **1,160 lines (-36%)**:

  - **`useChatStreamHandlers.ts`** _(new)_: Bundles the six chat-state mutators that ingest WebSocket frames and persist complete messages into `ChatSourceAtom` — `updateMessageApprovalStatus`, `appendStreamingTokenToChat`, `appendThoughtToMessage`, `finalizeStreamingResponse`, `handleCompleteMessage`, `mergeSourcesIntoMessage`. Each handler is wrapped in `useCallback` with stable setter deps; the bundle is `useMemo`-stabilized so callers can pass it as a single param without rebinding. The hook's param surface is deliberately kept free of `documentId` / `corpusId` so `CorpusChat.tsx` (which has the same five stream functions verbatim) can adopt it in a follow-up PR.
  - **`useChatAgentMessageHandler.ts`** _(new)_: Wraps the 188-line `handleAgentMessage` switch (10+ message types: ASYNC_START / ASYNC_CONTENT / ASYNC_THOUGHT / ASYNC_SOURCES / ASYNC_APPROVAL_NEEDED / ASYNC_APPROVAL_RESULT / ASYNC_RESUME / ASYNC_FINISH / ASYNC_ERROR / SYNC_CONTENT). Receives `pendingApprovalRef` (the existing stale-closure escape hatch from the WebSocket subscription) plus the stream-handlers bundle, and returns the stable `onMessage` callback consumed by `useWebSocketAuth`. Carries main's rich-mention `requesting_agent` plumbing through `ASYNC_APPROVAL_NEEDED` and the `persistence_failed` console warning on `ASYNC_FINISH` so sub-agent delegation behaviour from PR #1651 is preserved on this decomposition.
  - **`useChatSendHandlers.ts`** _(new)_: Bundles the three user-facing send actions — `sendMessageOverSocket` (debounced via `sendingLockRef`), `sendApprovalDecision`, and `sendTextImmediately` (programmatic send for the `initialMessage` prop). Takes `updateMessageApprovalStatus` as an input from the stream hook for optimistic approval UI.
  - **`chatUtils.ts`** _(extended)_: Adds `adjustTextareaHeight(textarea, maxHeight = 200)` — a pure DOM helper extracted from the composer. The composer wraps it in a `useCallback` that reads from `textareaRef.current`.
  - Behavior is preserved verbatim — same handler bodies, same `pendingApprovalRef` read pattern, same inline auto-scroll DOM side-effect on streaming token append, same empty-sources guard on `mergeSourcesIntoMessage`. Public API (`ChatTray` named export and the `WebSocketSources` / `MessageData` type re-exports) is unchanged; importers (`RightPanelContent.tsx`, `ChatTray.ct.tsx`, `ChatTrayTestWrapper.tsx`, `ContextMeter.ct.tsx`) need no changes. Rich-mention agent picker wiring (`useChatMentionPicker`, the `mentionPopover` portal, and the textarea's `handleMentionValueChange` integration) introduced upstream by PR #1651 is retained.
  - Render-body splitting is **deferred** to a follow-up PR: the natural sub-views (header / messages list / compaction banner / context meter / input footer / approval modal / conversation-list alternate branch) each share a large overlapping prop surface that has no atom-friendly seam without more preparatory work. Mixing render-body extraction with hook extraction in one slice would blow the diff and make review hard to bisect.

- **PII scan tool — review follow-ups from PR #1624 (closes #1638)** (`opencontractserver/llms/tools/core_tools/pii.py`, `opencontractserver/llms/tools/core_tools/_privacy_filter_client.py`, `opencontractserver/agents/{apps,checks}.py`, `opencontractserver/tests/test_pii_scan_tool.py`, `opencontractserver/tests/test_privacy_filter_client.py`, `docs/sample_env_files/backend/production/django.env`).

  - **`_persist_annotations_sync` switched from per-row `Annotation.save()` to `Annotation.objects.bulk_create(batch_size=200)`** (`pii.py`). A document with 200+ PII hits no longer pays 200+ round-trips inside the atomic block. The signal handler that normally queues `calculate_embedding_for_annotation_text` (`opencontractserver/annotations/signals.py:process_annot_on_create_atomic`) is bypassed by `bulk_create`, so we now register `transaction.on_commit` Celery enqueues directly inside the atomic block — mirroring the pattern in `annotations/models.py` for duplicated annotation sets — and keep the `embed-annot-{pk}` task-id-based dedup. PDF TOKEN_LABEL JSON is also explicitly `compact_annotation_json()`'d before `bulk_create` because `Annotation.save()`'s auto-compaction is skipped on the bulk path.

  - **Hoisted `plasmapdf.models.types` imports to module scope** (`pii.py`). `SpanAnnotation` and `TextSpan` previously imported inside an `if file_type == "application/pdf":` branch but referenced inside an inner loop branch — Python scoping made it work but static analyzers flagged both names as possibly-undefined.

  - **Dropped `CHUNK_SIZE` / `CHUNK_OVERLAP` aliases from `_privacy_filter_client.py`** (`_privacy_filter_client.py`, `test_privacy_filter_client.py`). The aliases looked like a public re-export of the canonical constants in `opencontractserver/constants/document_processing.py`; tests now import `PRIVACY_FILTER_CHUNK_SIZE` / `PRIVACY_FILTER_CHUNK_OVERLAP` directly from the constants module, and the client uses the long names internally.

  - **New `agents.W001` Django system check** (`opencontractserver/agents/checks.py`, `opencontractserver/agents/apps.py`). Surfaces the `PRIVACY_FILTER_URL set + PRIVACY_FILTER_API_KEY empty` misconfiguration on every `manage.py` invocation (in addition to the existing one-shot `logger.warning` emitted by `_privacy_filter_client.py` on first request). Production env template (`docs/sample_env_files/backend/production/django.env`) now has a `Privacy Filter` section that calls out the same risk explicitly.

  - **Regression test `test_persist_annotations_sync_duplicate_label_race`** (`test_pii_scan_tool.py`). Documents the accepted-duplicate behavior of `Corpus.ensure_label_and_labelset` under PostgreSQL READ COMMITTED — two concurrent scans on the same fresh corpus can both create labels with the same `(text, label_type)` because there is no DB-level uniqueness constraint. The test simulates the post-race outcome by patching `ensure_label_and_labelset` and verifies (a) both `_persist_annotations_sync` calls succeed, (b) two label rows exist, (c) each annotation points at its scan's own race-created label. If somebody later adds a `UniqueConstraint(fields=["text", "label_type"], ...)` on `AnnotationLabel`, this test fails and forces a deliberate revisit of the trade-off.

  - **`_persist_annotations_sync` defensive unknown-group filter pinned** (`opencontractserver/llms/tools/core_tools/pii.py`, `opencontractserver/tests/test_pii_scan_tool.py`). The set comprehension that builds `needed_groups` already filters detections by membership in `ENTITY_GROUP_LABELS`, and the per-detection loop also skips any group not in `ENTITY_GROUP_LABELS`. New `PersistAnnotationsUnknownGroupTests.test_unknown_entity_group_is_skipped_silently` pins both filters together: a batch mixing one known group with one unknown group yields exactly one persisted annotation and exactly one new label (for the known group). This guards against a future refactor that drops either filter and lets unknown groups leak into `ensure_label_and_labelset` (which would crash or, worse, succeed and create orphan labels with bogus colors / icons).

  - **`_persist_annotations_sync` lambda lifted to named factory for mypy** (`opencontractserver/llms/tools/core_tools/pii.py`). The inline `transaction.on_commit(lambda pk=ann.pk, cid=corpus_pk: ...)` registration broke `mypy` with `Cannot infer type of lambda`. Replaced with `_queue_embed(pk, cid) -> Callable[[], None]` so the return type is explicit and the closure binding stays correct. Behaviour is unchanged.

- **Corpus Home document-tree loads ~order-of-magnitude faster** (`config/graphql/filters.py`, `frontend/src/graphql/queries.ts`, `frontend/src/components/corpuses/DocumentTableOfContents.tsx`). The "Loading document structure…" wait on Corpus Home was driven by two over-fetched GraphQL queries. (1) `GET_DOCUMENT_RELATIONSHIPS` (capped at `first: 500`) selected the full `sourceDocument`/`targetDocument` objects (title, description, fileType, **icon**, slug, creator), plus `corpus { …creator }`, `creator { …username }`, `annotationLabel { id, text, color, icon }`, `data`, `created`, `modified`, and `myPermissions`. `DocumentTableOfContents.tsx:535-611` then consumed only four of those fields (`relationshipType`, `annotationLabel.text`, `sourceDocument.id`, `targetDocument.id`) — every `icon` selection still triggered `build_absolute_uri()` via `create_file_resolver` and the per-row document payloads were thrown away because `GET_CORPUS_DOCUMENTS_FOR_TOC` already provided the same fields. Worse, all relationships came back regardless of type/label and were filtered client-side to `relationshipType === "RELATIONSHIP"` AND `annotationLabel.text.toLowerCase() === "parent"`. (2) `GET_CORPUS_DOCUMENTS_FOR_TOC` selected `icon` and `creator { id, slug }`, neither of which the TOC renders (`renderNode` derives the on-screen icon from `fileType` via `getFileIcon`, and creator metadata is never displayed). Fix: a new `annotation_label_text` `iexact` filter on `DocumentRelationshipFilter` (paired with the existing `relationship_type` filter) so the parent-only restriction now runs on the server; a new ultra-lean `GET_CORPUS_DOCUMENT_TOC_EDGES` query that returns just `{ id, sourceDocument { id }, targetDocument { id } }` and lives alongside (not in place of) the original `GET_DOCUMENT_RELATIONSHIPS` because `CorpusDocumentRelationships.tsx:314` and `DocumentRelationshipModal.tsx:611` still need the rich payload; `GET_CORPUS_DOCUMENTS_FOR_TOC` slimmed to `{ id, title, description, slug, fileType }`. `RunCorpusActionModal.tsx` (the only other consumer of the doc query) reads only `node.id` / `node.title`, so the field removal is safe. End result: for a 76-document corpus, the relationships payload drops from "76+ source-and-target document blobs" to "76 ID-only edge rows," up to 152 `build_absolute_uri()` icon-resolver calls and 76 description blobs are eliminated, and any non-parent relationships (notes, future link types) never leave the server.

- **Fork ≡ export+import: V2 parity refactor + roundtrip-loss fixes** (`opencontractserver/tasks/fork_tasks.py`, `opencontractserver/tasks/export_tasks_v2.py`, `opencontractserver/utils/{export_v2,import_v2,etl}.py`, `opencontractserver/tests/{test_corpus_export_import_v2.py,test_corpus_forking.py,test_ingestion_source.py}`, and shared fixture helpers `opencontractserver/tests/{_corpus_fixture,_corpus_snapshot}.py`). Replaces the legacy bespoke `fork_corpus` machinery with a thin shell that drives the V2 export → V2 import pipeline so the two code paths can no longer drift. `build_corpus_v2_zip` is the pure builder that `package_corpus_export_v2` (Celery task) now wraps; `import_corpus_v2_from_bytes` is the in-process entry point that `fork_corpus` invokes after `build_corpus_v2_zip`. Restores fields that previously dropped on round-trip (manual-metadata `Fieldset` / `Column` / `Datacell` rows, `IngestionSource` rows, `DocumentPath` and `CorpusFolder` snapshots, structural-set membership) and adds a three-roundtrip invariant test (`TestV2ThreeRoundTripDataIntegrity`) plus error-handling coverage (`CorpusForkErrorHandlingTest`).

  - **Behavioural change:** `fork_corpus` no longer respects selective `doc_ids` / `annotation_ids` arguments. Any caller passing those now gets a _full_ fork (with a `logger.warning`). No live caller in `opencontractserver/`, `config/`, or tests still passes selective args; legacy queued Celery tasks would still run safely (full fork instead of partial). A short note here flags the contract change for downstream forks of this repo.
  - **Import-side correctness:** `import_metadata_schema` now clears its `column_map` on rollback so callers can't accidentally re-link freshly imported rows to pks that no longer exist.
  - **Observability:** `package_metadata_schema` gains an explicit `-> MetadataSchemaExport | None` return annotation so the optional behaviour is part of the type contract.
  - **Storage GC for orphaned V2-import blobs** (`opencontractserver/tasks/fork_tasks.py`): when fork re-points a forked document's `pdf_file` / `pawls_parse_file` / `txt_extract_file` / `icon` / `md_summary_file` at the source corpus's storage, the freshly-written blob the V2 import just allocated is collected and deleted via `transaction.on_commit(default_storage.delete)`. Using `on_commit` (rather than synchronous `delete()` inside the atomic block) ensures the GC only fires when the fork commits — a rolled-back fork leaves the V2-import blobs intact because they are still the live references on whatever survives the rollback. Per-blob `try / except` in the callback makes the GC pass best-effort: a storage failure on one path logs at WARNING and continues with the rest instead of aborting and leaking the remainder. Two new tests in `CorpusForkOrphanedBlobGCTest` (`opencontractserver/tests/test_corpus_forking.py`) cover the happy path (orphan paths are deleted and never collide with live references on the forked corpus) and the failure-tolerant path (`default_storage.delete` raising on every path still attempts each one). Closes the silent storage leak flagged against the prior implementation.

- **Backend dependency audit — aggressive trim of unused / vestigial packages** (`requirements/base.txt`, `requirements/local.txt`, `requirements/production.txt`, `.pre-commit-config.yaml`, `compose/production/django/Dockerfile`, deleted `requirements/filetypes/docx.txt`, `requirements/ingestors/nlm_ingest.txt`, `requirements/processors/gliner.txt`, `.pylintrc`). Grep-audited every direct dependency across `opencontractserver/`, `config/`, scripts, compose, Dockerfiles, settings, and CI workflows.
  - **Removed from `base.txt`** (zero verifiable references in code): `pandas`, `python-slugify`, `django-model-utils`, `tiktoken`, `jsonschema`.
  - **Removed from `local.txt`**: `Werkzeug[watchdog]`, `ipdb`, `pytest-sugar`, `pylint-django`, `pylint-celery`.
  - **Removed from `production.txt`**: `django-anymail[mailgun]` (settings block in `config/settings/production.py` was fully commented out; a breadcrumb comment now points future operators at re-adding the package if they want to enable Mailgun).
  - **Deleted three orphan optional-feature requirement files**: `requirements/filetypes/docx.txt` (`mammoth` for an inactive docx pipeline), `requirements/ingestors/nlm_ingest.txt` (body was literally `# pass`), `requirements/processors/gliner.txt` (body was only commented placeholders).
  - **Deleted `.pylintrc`**: pylint is not invoked by pre-commit, CI, or any script.
  - **Deduplicated `django-extensions==4.1`**: was redundantly listed in both `base.txt` and `local.txt`; kept only in `local.txt` since it is only added to `INSTALLED_APPS` in `config/settings/local.py`.
  - **Moved `pytesseract` from `base.txt` to `local.txt`**: only used by `opencontractserver/tests/test_pdf_redaction.py` (which now also gains `@skipUnless(...)` guards on its two OCR-backed tests so they degrade gracefully when the binary is absent) to mirror docling-parser microservice OCR. Production OCR runs in the docling-parser container, so neither the package nor the `tesseract-ocr` system packages are needed in the production image (`tesseract-ocr` + `tesseract-ocr-eng` apt installs dropped from `compose/production/django/Dockerfile`).
  - **Pre-commit mypy hook kept in lockstep with `requirements/`**: the `additional_dependencies` list drops the same packages that `base.txt` / `local.txt` no longer install.
  - **Snyk pins** duplicated across all three requirements files are intentionally left in place per the existing comment in `production.txt` (Snyk does not understand `-r` inheritance).
  - **Net result**: ~10 direct deps removed from `base.txt`, ~5 from `local.txt`, 1 from `production.txt`, 4 files deleted, smaller production image (no Rust-built tiktoken wheel, no pandas, no tesseract apt packages).

### Added

- **Rich-mention agent delegation in chat** (`opencontractserver/llms/agents/mention_extractor.py`, `opencontractserver/llms/tools/delegation_tools.py`, `opencontractserver/llms/agents/unified_agent_conversation.py`, `opencontractserver/llms/agents/unified_agent_consumer.py`, `config/graphql/graphene_types.py`, `config/graphql/queries.py`, `frontend/src/components/chat/{AgentMentionPopover.tsx,RequestingAgentAttribution.tsx,agentChipStyles.ts,types.ts}`, `frontend/src/hooks/{useAgentMentionTrigger.ts,useChatMentionPicker.ts}`, `frontend/src/components/knowledge_base/document/right_tray/{ChatTray.tsx,ApprovalOverlay.tsx}`, `frontend/src/components/corpuses/CorpusChat.tsx`, `frontend/src/components/corpuses/corpus_chat/ApprovalModal.tsx`, `frontend/src/components/widgets/chat/{ChatMessage.tsx,ChatMessageTimeline.tsx,timelineEntryFactory.ts}`, `frontend/src/components/markdown/MarkdownMessageRenderer.tsx`, `frontend/src/assets/configurations/osLegalStyles.ts`). Users can `@`-mention an agent in `ChatTray` (document chat) and `CorpusChat` (corpus chat) to delegate that turn's work to a sub-agent via the conductor's tool surface. Per-turn allowlist (only @-mentioned agents become delegation tools, rebuilt by `UnifiedAgentConsumer` before each turn), scoped sub-agent context (no inherited conversation history; document/corpus tools attached based on the sub-agent's scope), and pinned-or-timeline attribution: `pin=true` creates a distinct `ChatMessage` row with `agentConfiguration` set so the sub-agent's reply gets its own bubble + attribution chip in the bubble header; `pin=false` keeps the sub-agent's work as a `tool_call`/`tool_result` pair inside the conductor's timeline with an `@<slug>` chip in place of the raw `delegate_to_<slug>` tool name. Approval bubbling: when a sub-agent's tool call requires confirmation, the `ASYNC_APPROVAL_NEEDED` frame carries `requesting_agent` so the approval modal attributes the request to `@<slug>` rather than the conductor. Backend pieces: shared mention extractor at `opencontractserver/llms/agents/mention_extractor.py` (single source of truth for the markdown URL grammar, used by `ChatMessageType.mentioned_resources` + the consumer's per-turn extraction), delegation tool factory + `StreamRelay` dataclass at `opencontractserver/llms/tools/delegation_tools.py`, `filter_by_scope` helper for resolving GLOBAL + corpus-scoped agent visibility, per-turn conductor rebuild in `UnifiedAgentConsumer.handle_query`. Frontend pieces: lightweight `@`-popover overlay (`useAgentMentionTrigger` hook + `AgentMentionPopover` component, debounced via `useUnifiedMentionSearch`) on the existing chat textarea, `MarkdownMessageRenderer` now wired into `ChatMessage` so inline `[@slug](/agents/slug)` markdown links render as styled chips, shared attribution chips on pinned sub-agent bubbles (`ChatMessage.styles.ts`) + timeline entries (`ChatMessageTimeline.styles.ts`) + approval modals (`RequestingAgentAttribution.tsx`). Pinned sub-agent rows expose `agentConfiguration` on `GET_CHAT_MESSAGES` so re-hydrated conversations restore the bubble + chip. Design spec: `docs/superpowers/specs/2026-05-13-rich-mention-agent-delegation-design.md`. Implementation plan: `docs/superpowers/plans/2026-05-13-rich-mention-agent-delegation.md`. Backend test coverage: `test_mention_extractor.py`, `test_delegation_tools.py`, `test_unified_agent_consumer_delegation.py`, `test_chat_message_mentioned_resources.py`, `test_mentions.py`. Frontend CT coverage: `AgentMentionPopover.ct.tsx` and new delegation scenarios in `ChatTray.ct.tsx` + `CorpusChat.ct.tsx`. Documentation screenshots: `chat--agent-mention--{popover-open,pinned-bubble,timeline-entry,approval-with-attribution}`.

  **Review follow-ups** (`opencontractserver/llms/tools/delegation_tools.py`, `opencontractserver/llms/agents/mention_extractor.py`, `config/websocket/consumers/unified_agent_conversation.py`, `frontend/src/components/chat/{AgentMentionPopover.tsx,RequestingAgentAttribution.tsx,agentChipStyles.ts}`, `frontend/src/components/widgets/chat/{ChatMessage.styles.ts,ChatMessageTimeline.styles.ts}`, `frontend/src/components/knowledge_base/document/right_tray/ChatTray.tsx`, `frontend/src/assets/configurations/osLegalStyles.ts`, `opencontractserver/tests/test_delegation_tools.py`, `frontend/tests/AgentMentionPopover.ct.tsx`):

  - `filter_by_scope` replaces an `assert document_id is not None` with an explicit `if document_id is None: return qs.filter(scope="GLOBAL")` so the path can't disappear under `python -O`.
  - `build_delegation_tool` / `_build_stream_relay_factory` tighten the `relay_factory` return type from `StreamRelay | None` to `StreamRelay` — the production factory always builds a relay, and the previous union widened the contract for no benefit. Test helper `_make_noop_relay` added so non-relay tests construct a real no-op relay instead of returning `None`.
  - `extract_mentions` swaps the legacy-pattern `any(m.type == ... for m in out)` dedupe scan (O(n²)) for a `(type, slug, corpus_slug)` set lookup (O(1) per match). `corpus_slug` is included in the key so a corpus-scoped document and a standalone document with the same slug stay distinct entries.
  - Agent attribution palette extracted to a single shared CSS fragment (`agentChipPaletteCss` in `frontend/src/components/chat/agentChipStyles.ts`, sourced from new `OS_LEGAL_COLORS.agentChipViolet` / `agentChipText` tokens). The three duplicated `#8b5cf6` / `#6366f1` / `#7c3aed` chip definitions in `ChatMessage.styles.ts`, `ChatMessageTimeline.styles.ts`, and `RequestingAgentAttribution.tsx` now compose this fragment, so the bubble-header, timeline, and approval-modal chips can never drift apart visually.
  - `AgentMentionPopover` gains keyboard navigation: ArrowDown / ArrowUp move the highlight with wrap-around, Enter selects the active option, the active row carries `aria-selected="true"`, and `scrollIntoView({ block: "nearest" })` keeps it visible inside the popover's scroll container. The handler is installed on `document` in the capture phase so the textarea (which retains focus) doesn't see Enter/Arrow keys while the picker is open. Three new CT cases in `AgentMentionPopover.ct.tsx` pin the wrap-around and Enter-select behaviour.
  - `ChatTray.tsx` inline `requesting_agent` cast replaced with `PendingApproval["requestingAgent"]` so the persisted-message and live-WebSocket frame shapes share one type definition.

  **Second round of review follow-ups** (`opencontractserver/llms/tools/delegation_tools.py`, `opencontractserver/llms/agents/mention_extractor.py`, `config/websocket/consumers/unified_agent_conversation.py`, `config/graphql/conversation_types.py`, `frontend/src/components/knowledge_base/document/right_tray/ChatTray.tsx`, `frontend/src/components/corpuses/CorpusChat.tsx`, `frontend/src/utils/agentMentionLink.ts`, `opencontractserver/tests/test_core_agents.py`):

  - **Dead `if relay is None` branches removed** in `delegation_tools._body`. The relay is non-Optional per the tightened factory contract; the previous defensive checks were unreachable. Each `if pin and relay is not None` collapses to `if pin`.
  - **Monkey-patched `_parent_message_id_box` eliminated.** `_build_stream_relay_factory` no longer stamps a private attribute onto its returned callable. `handle_query` now holds the shared one-slot box directly and passes it into `_stream_agent_response(..., parent_message_id_box=...)` as an explicit kwarg. Removes the `type: ignore` and the `getattr(relay_factory, "_parent_message_id_box", None)` lookup; the contract between the per-turn relay factory and the conductor stream consumer is now type-checked.
  - **`resolve_mentions_for_user` N+1 eliminated.** The resolver now does one batched `slug__in` / `id__in` query per mention type (corpus, document, annotation, agent) plus two batched `DocumentPath` queries (corpus-scope confirmation + best-effort context for standalone `@document` mentions) before the per-mention loop. The loop is now pure dict lookups against the pre-fetched maps — a 50-message conversation with 10 mentions per message drops from `O(n × m)` queries to a fixed `O(n_types)`.
  - **Explicit `_error` on `on_approval` auto-reject**: when the sub-agent fires its approval before the conductor has emitted its first event, the relay now returns `_error` in the decision payload so the delegation body's existing error guard surfaces it as a tool-call failure (`Sub-agent error: ...`) instead of feeding the conductor an empty string.
  - **Cross-user IDOR test for `_aget_conversation_visible_to_user`**: pins that user A receives `None` when probing a conversation owned by user B, plus owner/anonymous/bogus-id/bogus-user-id edges (5 new tests in `TestAgetConversationVisibleToUser`).
  - **`persistence_failed` flag surfaced via `console.warn`** in both `ChatTray.tsx` and `CorpusChat.tsx` ASYNC_FINISH handlers — when the backend reports that a pinned sub-agent ChatMessage couldn't be persisted, the warning makes the discrepancy visible in dev tools. The bubble still renders for the live session; a non-blocking toast is a follow-up.
  - **`filter_by_scope` docstring** now notes the sync-context requirement (DocumentPath lookup) and references the lone async caller that already wraps it correctly.
  - **`_LEGACY_CORPUS_RE` Python version guard**: top-of-module `sys.version_info < (3, 11)` check raises a loud RuntimeError instead of letting `re.compile` fail with a confusing `multiple repeat` error on older interpreters.
  - **`agentMentionLink.ts` stale comment fix**: lockstep reference points to `opencontractserver.llms.agents.mention_extractor._classify_url` (the real source) rather than the renamed `opencontractserver.utils.mentions`.
  - **Mechanical guard on `on_approval` `_pending_approvals` key collision** (`config/websocket/consumers/unified_agent_conversation.py`): the "one sub-agent per turn per `(parent_message_id, agent.pk)`" invariant is now enforced at runtime — a duplicate registration cancels the prior future and logs an error instead of silently leaking it.

- **`CorpusFolder.user_can` overridden to delegate to the parent corpus** (`opencontractserver/corpuses/models.py`, `opencontractserver/tests/permissioning/test_authorization_invariants.py`). Closes the footgun where `folder.user_can(user, perm)` would run against the folder row's (non-existent) guardian object-permission rows and silently return `False` for shared readers. Permissions for folders are inherited from the parent corpus, and every legacy call site in `DocumentFolderService` already routes through `folder.corpus.user_can(...)`; the override makes that delegation the only path so a future contributor can't get a wrong answer by accident. `CorpusFolderDelegatesUserCanToCorpusTestCase` (7 new tests) pins creator / shared-reader / stranger / anonymous / superuser semantics, the `is_public`-on-corpus toggle, and the full delegation matrix across every `PermissionTypes` member.

- **Centralized single-object authorization API: `Manager.user_can(user, instance, permission)` and `obj.user_can(user, permission)`** (`opencontractserver/utils/permissioning.py`, `opencontractserver/shared/{Managers,QuerySets,Models,user_can_mixin}.py`, `opencontractserver/corpuses/models.py`, `opencontractserver/tests/permissioning/test_authorization_invariants.py` (new)). Step 0 of the permission-centralization effort. Manager- and QuerySet-side `user_can` bodies share a new `UserCanMixin` (lives in `shared/user_can_mixin.py` to keep `utils.permissioning`'s eager-import chain out of the early startup graph). Instance-side `user_can(user, perm)` is provided by a sibling `InstanceUserCanMixin` applied to both `BaseOCModel` and the `TreeNode`-rooted `Corpus` / `CorpusFolder`, so every visibility-managed model exposes the same ergonomic surface (without it, `corpus.user_can(...)` calls in `folder_service.py` would raise `AttributeError` at runtime — the missing mixin on `Corpus`/`CorpusFolder` was the original CI failure mode for this PR). The codebase had two parallel authorization systems: `Manager.visible_to_user(user)` (filter side, well-centralized with per-model overrides for Annotation/Document/Conversation/etc.) and `user_has_permission_for_obj(user, instance, perm)` (single-object side, called from 104 non-test sites — mutations, optimizers, services, agent tools, tasks, imports). The two disagreed for the same `(user, instance)` because the function-side intentionally ignored creator status (see `permissioning.py:366-384`), so ~22 mutations and three service classes (`FolderService.check_corpus_*`, `DocumentQueryOptimizer._check_*_permission`, `user_can_modify_corpus`) repeated the compound `is_superuser or creator or has_perm` pattern as a workaround. This PR adds `Manager.user_can(...)` as the single source of truth that mirrors `visible_to_user` semantics: for READ, `Model.objects.user_can(u, obj, READ) == Model.objects.visible_to_user(u).filter(pk=obj.pk).exists()`; for non-READ, `user_can` is the only correct check (no queryset analogue). The default body lives in `_default_user_can` in `permissioning.py` and is shared by `BaseVisibilityManager.user_can` and `PermissionedTreeQuerySet.user_can`. `BaseOCModel.user_can(user, perm)` is a one-line ergonomic delegate so callers can write `corpus.user_can(user, PermissionTypes.UPDATE)`. New invariant test pins the read-equivalence, manager/instance-surface parity, superuser bypass, creator default access, the `is_public`-grants-only-READ asymmetry (a security-critical invariant the deleted FolderService methods enforced), and anonymous-only-reads-public for Corpus. Subsequent migration phases extend `user_can` overrides to Annotation/Relationship/Conversation/etc. and migrate the remaining 100+ call sites.

- **Authenticated MCP sessions and `create_thread_message` write tool** (`opencontractserver/mcp/server.py`, `opencontractserver/mcp/tools.py`, `opencontractserver/constants/mcp.py`, `opencontractserver/mcp/tests/test_mcp.py`).

  - MCP ASGI layer extracts `Authorization: Bearer <jwt>`, validates via `config.jwt_utils.get_user_from_jwt_token`, and stores the caller in a `ContextVar` (`_mcp_user`) that all tool handlers consume. Invalid/expired tokens → 401; absent tokens preserve anonymous semantics.
  - First write-capable MCP tool, `create_thread_message`, lets authenticated users post into any thread they can already see (`visible_to_user` on corpus, thread, parent message). Blank/oversized content raises `ValidationError`; the dispatcher surfaces both `PermissionDenied` and `ValidationError` as structured `{"error": ...}` tool results so the LLM can react instead of hitting a transport error.
  - Existing read tools (`list_documents`, `search_corpus`, `list_threads`, etc.) gain an optional `user` parameter and call `visible_to_user(user)` so authenticated callers see private resources they own / are shared on; anonymous callers still see only public, published resources.
  - Both dispatchers strip any client-supplied `user` argument before invoking the handler so a hostile client can't trigger a `TypeError: got multiple values for keyword argument 'user'` that would escape as a transport error.
  - Scoped dispatcher restructured to mirror the non-scoped one: corpus-visibility and unknown-tool checks moved outside the `try` block so each error path records telemetry exactly once (no double-records).
  - Tests cover the valid-JWT and invalid-JWT 401 paths, `ValidationError` → structured result on both dispatchers, the scoped write happy path with creator persistence, IDOR / anonymous rejection, the user-keyword-collision guard, and a new in-thread reply happy path that exercises a valid `parent_message_id`.
  - **Operator note — write access on public threads.** `create_thread_message` permission is `visible_to_user` (read) on the parent thread, not a separate write-specific gate. Threads attached to a public corpus are therefore writable by any authenticated token-holder — including tokens issued by the same Auth0 instance for users outside the deploying tenant. This is a deliberate policy: a public corpus is "open to discussion," not just "open to read." Operators who want read-only public discoverability should keep the corpus public but the thread private, or layer their own write gate at the proxy. The `test_create_thread_message_unrelated_user_can_post_to_public_thread` test in `opencontractserver/mcp/tests/test_mcp.py` pins this contract so a future visibility change cannot silently flip it.

- **MCP OAuth discovery endpoints for interactive sign-in** (`opencontractserver/mcp/server.py`, `opencontractserver/discovery/{views,urls}.py`, `opencontractserver/discovery/tests/test_discovery_views.py`, `opencontractserver/mcp/tests/test_mcp.py`). New `/.well-known/oauth-protected-resource` endpoint (RFC 9728) advertises the Auth0 tenant as the authorization server, and `/.well-known/mcp.json` now surfaces the metadata URL. The 401 response from the MCP ASGI layer emits `WWW-Authenticate: Bearer realm="opencontracts", resource_metadata="..."` so interactive MCP clients (Claude Desktop, Cursor) can discover Auth0 and bounce users through hosted login without preconfigured tokens. Auth0-mode deployments only; `USE_AUTH0=False` returns 404 from the protected-resource endpoint and omits the `resource_metadata` hint, with interactive sign-in unavailable until a local OAuth 2.1 authorization server is added.

- **Markdown-customizable user profile fields** (`opencontractserver/users/models.py`, `opencontractserver/users/migrations/0030_user_profile_about_markdown_user_profile_headline_and_more.py`, `config/graphql/{serializers,user_mutations,user_types}.py`, `frontend/src/components/modals/UserSettingsModal.tsx`, `frontend/src/views/UserProfile.tsx`, `frontend/src/graphql/{mutations,queries}.ts`). New User fields `profile_headline` (200 chars), `profile_about_markdown` (5,000 chars), and `profile_links_markdown` (5,000 chars) editable from the settings modal and rendered via `SafeMarkdown` on the profile view. Size limits are enforced server-side: `UpdateMe.mutate` delegates to `UserUpdateSerializer` (DRF `ModelSerializer`), which auto-applies each model field's `max_length` validator, so an oversized payload posted directly via GraphQL is rejected with a serializer error rather than silently persisted. The `max_length` on the two `TextField`s is **validator-only** (Postgres `TEXT` has no DB-level length cap) so any future code path that bypasses the serializer must validate length explicitly — documented inline at the model field declarations. Visibility piggybacks on the existing `User.objects.visible_to_user()` (private profiles stay hidden whole-record), and owners can still read their own fields on a private profile. `SafeMarkdown` now pins an explicit `urlTransform` allowlist (`https://`, `http://`, `mailto:`, `tel:`, relative / fragment) so `javascript:` / `data:` link targets are stripped even if `react-markdown`'s default ever loosens; protocol-relative URLs like `//phishing.example` (which browsers resolve to the page protocol) are explicitly rejected so a profile author can't disguise an external link as an in-app relative path. The `MarkdownTextarea` focus ring uses a 2px `accentAlpha(0.35)` `box-shadow` in addition to the border-color shift so it stays visible in high-contrast / forced-colors mode (WCAG 2.4.7). Backend test `test_owner_can_read_their_own_private_profile_markdown_fields` plus the `renders markdown headline, about, and links via SafeMarkdown` CT case in `user-profile.ct.tsx` cover markdown rendering plus the four URL-allowlist branches (`https://` pass, `javascript:` strip, protocol-relative strip, `data:` strip).

- **NavMenu "More" overflow keeps footer-essential links reachable on long-scroll views** (`frontend/src/components/layout/{NavOverflowMenu.tsx,overflowMenuItems.ts}` (new), `frontend/src/components/layout/{NavMenu,MobileNavMenu}.tsx`, `frontend/tests/NavMenu.ct.tsx`). Closes #1609. After the natural-flow refactor that restored scrolling on corpus Annotations / Analyses / Extracts tabs, the in-flow `<Footer>` is rendered at the end of the document — effectively unreachable on a corpus with thousands of annotations. Audited `Footer.tsx` and identified the small set of links that genuinely need to be reachable from any scroll position: Privacy Policy (`/privacy`), Terms of Service (`/terms_of_service`), and the GitHub repo link. Site Map (→ `/`) is redundant with the brand logo; Contact Us (→ `/contact`) was intentionally excluded because the route is **not** registered in `App.tsx` and would 404. Added a new `NavOverflowMenu` desktop component (kebab `MoreHorizontal` trigger in the `NavBar`'s `actions` slot, white surface dropdown with ARIA `role="menu"` + arrow-key navigation, ESC / outside-click close, focus restoration to the trigger per WCAG 2.4.3) and extended `MobileNavMenu` with a parallel "More" section in its sheet so the same links are one tap away on mobile. The in-flow `Footer` is unchanged on landing / corpus list / settings views — the overflow is purely additive. Eleven CT tests in `tests/NavMenu.ct.tsx` cover trigger presence, dropdown open, link contents, external-link `target`/`rel`, ARIA attributes, and ESC / outside-click close in both anonymous and authenticated states, on desktop and mobile.

### Added

- **`InlineChatBar` (CorpusHome landing + article-floating chat bars) now opens the agent @mention picker on `@`** (`frontend/src/components/corpuses/CorpusHero/InlineChatBar.tsx`, `frontend/src/components/corpuses/CorpusHome/CorpusLandingView.tsx`, `frontend/src/components/corpuses/CorpusHome.tsx`, `frontend/tests/CorpusHome.ct.tsx`). The bar accepts a new optional `corpusId` prop and, when provided, wires its existing `inputRef` through `useChatMentionPicker` — same hook as ChatTray and CorpusChat — so the rich-mention agent picker is reachable from the corpus landing chat input (`data-testid="corpus-home-landing-chat-input"`) and the article-floating controls (`data-testid="corpus-article-chat-input"`). Both `CorpusHome` and `CorpusLandingView` now plumb `corpus.id` to the bar. Pinned by `tests/CorpusHome.ct.tsx:typing @ in landing chat input opens the agent picker`, which mounts the full landing view, types a bare `@`, and asserts the popover surfaces with the mocked agent visible.

### Fixed

- **@mention picker rendered invisibly behind the DocumentKnowledgeBase fullscreen modal in ChatTray** (`frontend/src/hooks/useChatMentionPicker.tsx`, `frontend/src/assets/configurations/constants.ts`, `frontend/tests/ChatTray.ct.tsx`). The picker is portalled to `document.body`, which makes it a sibling — not a descendant — of the DKB's `.fullscreen-modal-overlay` (`Z_INDEX.APP_MODAL` = 3000). With the historical `zIndex: 1000` on the popover anchor, the modal overlay painted on top and the picker rendered correctly but invisibly behind it. The bug surfaced ONLY in document context (ChatTray) because CorpusChat is not mounted inside the DKB modal — that's why both surfaces shared the same hook yet behaved differently. Bumped the popover anchor to `Z_INDEX.APP_MODAL_CHILD` (3100), the existing semantic slot for popovers nested inside the DKB modal. Pinned by a regression assertion in `tests/ChatTray.ct.tsx:typing bare @ in ChatTray opens picker with full agent list` that reads the computed `z-index` off the anchor and asserts it is `> 3000`, so a future refactor that drops it back down breaks the test instead of the picker.

- **@mention agent picker never appeared on a bare `@` or 1-char fragment** (`frontend/src/components/threads/hooks/useUnifiedMentionSearch.ts`, `frontend/src/hooks/useChatMentionPicker.tsx`, `frontend/tests/ChatTray.ct.tsx`). `useUnifiedMentionSearch` gated all 5 mention-type queries (users / corpuses / documents / annotations / agents) behind `MENTION_SEARCH_MIN_CHARS` (=2), and `useChatMentionPicker` hid the popover until `agentItems.length > 0`. The combination meant typing `@` alone — or `@a` — produced no UI feedback at all, even though the trigger detector had opened. The agent list is intentionally tiny (a handful of templates plus a few per corpus), so the picker should appear the instant `@` is hit, the same way Slack / GitHub / Discord mention pickers do. Fix: (1) split out the agent search to fire on every debounced fragment regardless of length while leaving the other four open-ended resolvers gated by `minChars` (they're expensive `icontains` scans over open sets — keeping them gated preserves the performance reason the gate existed); (2) drop the `agentItems.length > 0` precondition from the popover render so `AgentMentionPopover`'s own "No matching agents." state is reachable from the live UI, not just from CT tests. Pinned by new `tests/ChatTray.ct.tsx:typing bare @ in ChatTray opens picker with full agent list (no minChars gate)`.

- **Migration-seeded agents had `slug=NULL` and crashed the @mention picker** (`opencontractserver/corpuses/template_seeds.py`, `opencontractserver/agents/models.py`, `opencontractserver/agents/migrations/0013_backfill_and_tighten_agent_slug.py`, `opencontractserver/tests/test_corpus_action_template.py`, `frontend/src/components/chat/AgentMentionPopover.tsx:103`). `create_default_action_templates` resolves the model via `apps.get_model("agents", "AgentConfiguration")` so the same function is reusable from both the data migration and the `seed_action_templates` management command. In migration context that registry returns a _historical_ model class, which does NOT carry the live `save()` override that auto-generates a slug from the name — so the six template-owned agents created by migrations `0010` and `0011` (`Document Description Updater Agent`, `Corpus Description Updater Agent`, `Document Summary Generator Agent`, `Key Terms Annotator Agent`, `Document Notes Generator Agent`, `CAML Article Writer Agent`) landed in the DB with `slug=NULL`. `searchAgentsForMention` happily surfaced them, the frontend `AgentItem` typed `slug: string` lied about the data shape, and `AgentMentionPopover.matches` exploded with `can't access property "toLowerCase", a.slug is null` the moment the user typed `@`. Fix: (1) `template_seeds._build_unique_agent_slug` re-implements the `save()` slug algorithm and is now called explicitly when creating each `AgentConfiguration` so migration-context writes can't regress; (2) new migration `0013_backfill_and_tighten_agent_slug` first backfills any existing NULL slugs (mirroring the `0005` algorithm so prior deployments are healed in place) and then `AlterField`s the column to `null=False` — the DB is now the final guard, so any future seeder that forgets to set `slug=` fails loudly at write time instead of silently corrupting search results; (3) the model field comment now spells out why `blank=True` is kept (the live `save()` override fills the value before super-save) but `null=False` is non-negotiable; (4) `test_seeded_agent_configs_have_non_null_slugs` pins the seeder contract so a future seeder regression breaks the test instead of the picker.

### Changed

- **Deleted `FolderService.check_corpus_{read,write,delete}_permission` (3 sister wrappers, ~130 LOC) in favor of `corpus.user_can(user, perm)`** (`opencontractserver/corpuses/folder_service.py`, `opencontractserver/tests/test_document_folder_service.py`). Vertical slice for the centralization. The three wrappers each encoded the same `is_superuser or creator or guardian-perm` compound (the read variant additionally honored `is_public`; the write/delete variants explicitly rejected `is_public` to preserve the read/write asymmetry). All 27 internal callers in `folder_service.py` and 20 test sites in `test_document_folder_service.py` migrated to the new `corpus.user_can(user, PermissionTypes.{READ,UPDATE,DELETE})` API. The asymmetry is preserved by `_default_user_can`'s explicit "is_public branch only fires for READ" check and pinned by the new invariant test `test_is_public_grants_only_read_not_writes`.

- **Phase A: `PermissionQuerySet.visible_to_user` is now guardian-aware** (`opencontractserver/shared/QuerySets.py:193-247`). Previously the base `PermissionQuerySet` only filtered on `creator` and `is_public` and explicitly deferred guardian to per-model overrides (`DocumentQuerySet`, `AnnotationQuerySet`, `NoteQuerySet`). That gap created drift: a model whose default manager fell back to the raw `PermissionQuerySet` would have `visible_to_user(u).filter(pk=x).exists()` return `False` while `_default_user_can` (guardian-aware) returned `True` for an explicitly-shared object. Issue #1655 closes the drift by porting the same `apps.get_model(app_label, "{model}userobjectpermission")` lookup used by `BaseVisibilityManager.visible_to_user` into `PermissionQuerySet.visible_to_user`. Concrete subclasses (`DocumentQuerySet`, `AnnotationQuerySet`, `NoteQuerySet`) already override and are unaffected; the change kicks in only for any direct `PermissionManager` consumer.

- **Phase A: `user_has_permission_for_obj` is now a `DeprecationWarning`-emitting shim** (`opencontractserver/utils/permissioning.py:541-604`). The ~300 LOC body (annotation/relationship branches + standard codename intersection) was absorbed into per-model `Manager.user_can` overrides; the function now delegates to `type(instance)._default_manager.user_can(...)`. Every call emits `DeprecationWarning` with `stacklevel=2` so the originating call site shows up in the warning. The `include_group_permissions=False` default is preserved verbatim so existing callers keep getting the same answer when they omit the kwarg (note: `_default_user_can` defaults to `True` — the difference is intentional). Migration of the ~170 existing call sites is tracked in Phases B/C of issue #1655.

- **Phase A: Deleted `user_can_modify_corpus`** (`opencontractserver/utils/permissioning.py:815-879` removed; 4 production callsites migrated in `config/graphql/document_mutations.py:327` and `config/graphql/badge_mutations.py:92,212,329`; `opencontractserver/tests/test_user_can_modify_corpus.py` converted to assert the same matrix against the new API). The helper consolidated the inline `is_superuser or creator or guardian-perm` compound — exactly the cases that `corpus.user_can(user, PermissionTypes.UPDATE)` now handles via `_default_user_can`'s standard branch (superuser short-circuit + creator short-circuit + guardian codename intersection). One call site, one source of truth.

- **Cookie consent modal visual polish on mobile** (`frontend/src/components/cookies/CookieConsent.tsx`). The mobile presentation of the consent modal was cramped and felt off-brand compared with the rest of the OS Legal surfaces. Reworked mobile styling without touching desktop: `.oc-modal` now uses `display: flex; flex-direction: column` with a `flex-shrink: 0` header, a `flex: 1 1 auto; overflow-y: auto` body, and a `flex-shrink: 0` footer so the Accept button stays anchored above the safe-area inset with a soft top shadow instead of relying purely on `position: sticky`. The body's mobile background switches to `OS_LEGAL_COLORS.background` so the white data cards lift off the surface with their new `OS_LEGAL_SHADOWS.card` shadow. Data list items adopt `accentSurface` + `textPrimary` on mobile (replacing the heavier slate-chip look) to feel brand-cohesive. The demo banner gains a 3px warning-text left bar for clearer hierarchy; the header icon badge gains an inner accent ring; section labels, lead copy, and disclaimer text get tighter mobile sizing. The Accept & Continue footer button is 48px tall with `font-weight: 600` for a comfortable thumb target and visual weight.

### Fixed

- **Phase A: Annotation privacy recursion now honors creator status on the source `Analysis` / `Extract`** (`opencontractserver/shared/Managers.py` — new `AnnotationManager.user_can`; `opencontractserver/annotations/query_optimizer.py:99-149` — `_compute_effective_permissions` retargeted to `Document.objects.user_can` / `Corpus.objects.user_can`). Before Phase A, the recursive privacy check for an `Annotation` whose `created_by_analysis_id` (or `created_by_extract_id`) was set went through `user_has_permission_for_obj(user, source_analysis, permission)` (legacy `permissioning.py:612` / `:625`). That helper intentionally ignored creator status — its docstring at line 521 still warns "It does NOT consider: Creator status". So a user who owned both an Analysis AND a private annotation it created could be denied READ on their own annotation because the recursive call returned `False` for the creator with no explicit guardian grant. Phase A rewires both call sites to `Analysis.objects.user_can(...)` and `Extract.objects.user_can(...)`, which route through `_default_user_can` and DO honor creator (it's the canonical short-circuit at `permissioning.py:444-449`). The same recursion in `AnnotationQueryOptimizer._compute_effective_permissions` (which the new `AnnotationManager.user_can` calls into for the MIN(doc, corpus) computation) was updated for the same reason — keeping it on the deprecated helper would have fired `DeprecationWarning` on every annotation visibility check. Other sites that consumed `user_has_permission_for_obj` indirectly (mutations, optimizers, services) are migrated in Phases B/C. Pinned by `test_privacy_recursion_honors_creator_on_source_analysis` in `test_authorization_invariants.py`.

- **Orphan annotations/relationships visible in global search after a doc was soft-deleted from a corpus** (`opencontractserver/shared/QuerySets.py`, `opencontractserver/shared/Managers.py`, `opencontractserver/annotations/models.py`, `opencontractserver/documents/versioning.py`, `opencontractserver/tests/test_soft_delete_visibility.py`). When `RemoveDocumentsFromCorpus` soft-deleted a document, the resulting `DocumentPath(is_current=True, is_deleted=True)` tombstone hid the doc from `corpus.get_documents()` but left its annotations and relationships intact. Global annotation queries used `Annotation.objects.visible_to_user()` / `Relationship.objects.visible_to_user()` which had no DocumentPath awareness, so the rows surfaced in global searches pointing at documents the user could no longer navigate to ("annotations linked to unknown document"). Fix: (1) new shared helper `_exclude_soft_deleted_doc_orphans` in `QuerySets.py` excludes rows whose `(document_id, corpus_id)` pair has DocumentPath rows but none with `is_current=True, is_deleted=False` — same predicate used by `Corpus.get_documents()` and `AnnotationQueryOptimizer.get_corpus_annotations()`. Standalone docs (no paths at all) and structural rows (no document FK) are intentionally untouched so test fixtures and pre-corpus-isolation data keep working. (2) `AnnotationQuerySet.visible_to_user()` applies the helper before its existing privacy/visibility logic; superusers retain the full-bypass branch. (3) New `RelationshipManager` (subclass of `BaseVisibilityManager`) layered onto `Relationship.objects` applies the same filter on top of the inherited creator/public/explicit-permission base case. (4) Data is preserved — only display is hidden — so `RestoreDeletedDocument` (the trash-restore mutation) brings the annotations and relationships back into visibility without resurrection logic. (5) `permanently_delete_document` hardened to also delete corpus-scoped non-structural relationships explicitly tagged `document=doc` (previously only relationships referenced via the deleted annotations' source/target M2M were cleaned; orphan relationships with empty source/target survived). The structural-set GC continues to fire via the existing `_gc_orphan_structural_set` post_delete signal — its `Document.objects.filter(structural_annotation_set_id=set_id).exists()` check is exactly "any other corpus-isolated doc copy with the same content_hash". New `test_soft_delete_visibility.py` covers: visibility before/after soft-delete, non-owner visibility, superuser bypass, restore round-trip, standalone-doc regression guard, relationship cleanup of doc-tagged orphans on permanent delete, and the structural set lifecycle across two corpus-isolated copies (preserved when one copy remains, GC'd when the last copy is permanently deleted).

- **`isProfilePublic` toggle silently dropped from UpdateMe mutation** (`frontend/src/graphql/mutations.ts`). The Profile Visibility switch in `UserSettingsModal` looked like it worked locally (optimistic state flipped), but the field was missing from both `UpdateMeInputs` (TypeScript) and the `UPDATE_ME` mutation variable list, so the toggle never reached the server. The field is now part of the input type and the mutation variables, and a backend regression test (`test_update_me_persists_is_profile_public_toggle`) pins the round-trip.

- **PR #1600 review follow-ups (closes #1608)** (`opencontractserver/users/api/serializers.py`, `config/graphql/permissioning/permission_annotator/mixins.py`, `frontend/src/graphql/mutations.ts`, `opencontractserver/utils/export_v2.py`, `CHANGELOG.md`). Surgical fixes for the five items raised in #1608 against the slug-only privacy gate landed in PR #1600:

  - **`UserSerializer` ↔ `UserViewSet` `lookup_field` contract.** The DRF `UserSerializer` is currently unwired (no `UserViewSet` exists in the codebase), and the existing docstring already noted the constraint, but the warning was easy to miss. The class docstring is upgraded from `.. note::` to `.. warning::` with the exact viewset skeleton future contributors must use, and the `extra_kwargs[url][lookup_field]` line gains a paired inline comment. Greppable for `UserViewSet` so anyone wiring this serializer surfaces the privacy contract on the first search.
  - **`object_shared_with` GenericScalar rationale.** Added a block comment at the field declaration in `AnnotatePermissionsForReadMixin` explaining why the field is intentionally typed as `GenericScalar` rather than a named output type — external callers reading `objectSharedWith[].email` / `.username` will see those keys disappear at runtime rather than triggering a schema error, and the untyped scalar lets us evolve the inner shape without a schema-breaking rename.
  - **`isUsageCapped` nullable break in the hand-written login interface.** PR #1600 made `UserType.isUsageCapped` and `UserType.canImportCorpus` nullable to enforce the self-only gate, and the codegen'd `graphql-api.ts` picked up the change. The hand-written `LoginOutputs.tokenAuth.user.isUsageCapped` interface in `frontend/src/graphql/mutations.ts` still typed the field as a non-nullable `boolean`, so a future consumer of that exact field would receive `null` silently for any non-self code path. Switched to `Maybe<boolean>` (the schema contract) and documented why the value is always defined at this specific call site (login → self-view) so the type-narrowing makes sense to future readers.
  - **Frontend audit of `isUsageCapped` / `canImportCorpus` consumers (no further fixes needed).** Ran the grep from #1608 across `frontend/src`: the three production consumers (`Corpuses.tsx:944` `backendUser?.canImportCorpus`, `Corpuses.tsx:1414` `Boolean(backendUser?.canImportCorpus)`, `Documents.tsx:1315` `backend_user && !backend_user.isUsageCapped`) all use null-safe access (optional chaining, explicit `Boolean()` coercion, truthy short-circuit). No non-null assertions (`!`) exist on either field in the repo. Verified.
  - **`export_v2.py` PII leak — `# TODO(#1608)` markers added.** PR #1600 deliberately left the V2 corpus export format unchanged because rewriting `author_email` / `creator_email` to slug-based keys is a breaking change to the export schema. Added `# TODO(#1608)` comments at the four leak sites (one in `package_md_description_revisions` covering revision authors; one umbrella TODO in `package_conversations` covering the conversation / message / vote loops) and updated the existing CHANGELOG bullet under PR #1600's "Out of scope" line to reference #1608 explicitly so the gap stays linked when the next export-version bump happens.

- **Corpus icons missing on the landing "Featured Collections" list (anonymous + authenticated)** (`frontend/src/components/landing/FeaturedCollections.tsx`). The `GET_DISCOVERY_DATA` query selects `corpus.icon` and the backend `resolve_icon` returns an unauthenticated absolute media URL (no permission gate — icon visibility is implicit from the corpus being returned by `Corpus.objects.visible_to_user`, which already includes public corpora for anonymous users), but the `CollectionCard` render call in `FeaturedCollections.tsx` never forwarded `image`/`imageAlt` props. The result: every featured collection rendered the type-based placeholder glyph instead of its uploaded icon, so anonymous landing-page visitors saw the same generic icon for all collections even though the icon URL was sitting in the GraphQL response. Now passes `image={corpus.icon || undefined}` and `imageAlt={corpus.title || "Corpus icon"}` — matching how `CorpusListView.tsx:620-621` already wires the same `@os-legal/ui` `CollectionCard`.

- **Mobile layout overflow across CAML editor, DocumentKnowledgeBase, floating annotation/document controls, and corpus TOC** (`frontend/src/components/corpuses/CamlArticleEditor.tsx`, `frontend/src/components/corpuses/caml/CamlArticleFrame.tsx`, `frontend/src/components/corpuses/CorpusHome.tsx`, `frontend/src/components/corpuses/DocumentTableOfContents.tsx`, `frontend/src/components/knowledge_base/document/{FloatingDocumentControls,LayoutComponents,NewNoteModal,NoteEditor}.tsx`, `frontend/src/components/knowledge_base/document/document_kb/{DocumentModals,HeaderBar,styles}.{tsx,ts}`, `frontend/src/components/knowledge_base/document/styled/HeaderAndLayout.tsx`, `frontend/src/components/annotator/labels/EnhancedLabelSelector.tsx`, `frontend/src/components/layout/{CardLayout,MobileNavMenu}.tsx`, `frontend/src/styles/appShellLayout.ts`, `frontend/src/utils/{layout,react}.ts`, `frontend/src/index.css`). Several mobile-only layout regressions are addressed in one sweep: (1) the DocumentKnowledgeBase fullscreen modal now propagates `--oc-dkb-visible-viewport-height` / `--oc-dkb-visible-viewport-offset-top` so the floating controls stay above the on-screen keyboard; (2) the CAML article editor fullscreen modal gets a class-targeted `createGlobalStyle` and a new `CamlArticleFrame` viewport guard to contain the third-party CAML library's content; (3) mobile annotation tools and the document FAB are rendered via `createPortal` to escape stacking contexts; (4) all stacking-order constants (mobile annotation tools 3048, floating controls 3050, annotation label modal 3100, app modal 3000/3100) now live in `Z_INDEX` in `frontend/src/assets/configurations/constants.ts` per the CLAUDE.md "no magic numbers" rule, instead of being defined file-local; (5) `FullScreenModal` uses a module-level open-counter so nested fullscreen modals don't release the body scroll lock prematurely; (6) `HeaderBar` is refactored from inline-style soup into named styled components with an icon-only mobile pattern; (7) `DocumentTableOfContents` flex/height contract is reworked so a single-document corpus renders correctly inside the new modal viewport (intentional UX trade-off: the prior `isSingleTopLevelLeaf` shortcut that auto-skipped the document header is removed in favor of a uniform layout — users now expand the document node). New CT regression tests at iPhone-13-sized viewports (390x844) pin the worst regressions. `hasRenderableNode` and `visualViewportAwareBottom` utility helpers live in `frontend/src/utils/{react,layout}.ts` with unit tests.

- **Mobile-nav follow-up: dialog/landmark semantic conflict, missing type imports, and iOS scroll-lock bleed-through** (`frontend/src/components/layout/MobileNavMenu.tsx`, `frontend/src/utils/initials.ts`, `frontend/tests/NavMenu.ct.tsx`). Closes #1610 — surgical follow-up to PR #1606. Code-review feedback on the new floating-sheet mobile nav surfaced four real defects (#1, #6, #7, plus three latent code-clarity issues #2/#3/#4) and confirmed one non-issue (#5):

  1. **`role="dialog"` on a `<nav>` erased the navigation landmark.** `Sheet` was `styled(motion.nav)` _and_ carried `role="dialog"`, which per ARIA overrides the implicit `<nav>` landmark — assistive tech announced it as a dialog but the landmark vanished from landmark navigation. Split the two roles: `Sheet` is now `styled(motion.div)` with `role="dialog"` + `aria-label="Navigation menu"`, and the actual navigation landmark moves to a new inner `SheetNav` (`styled.nav`) with `aria-label="Site navigation"` wrapping only the browse items + Account actions (the AuthFooter stays outside the landmark since it's auth state, not navigation).
  2. **`React.ReactNode` / `React.FC` referenced an unimported namespace.** With the new JSX transform (`"jsx": "react-jsx"`) the file doesn't import `React`, so `React.*` type references relied on the implicit global — fragile and confusing. Switched to `import type { FC, ReactNode } from "react"` and use `FC<…>` / `ReactNode` directly.
  3. **iOS Safari rubber-band scroll bled through the modal.** `document.body.style.overflow = "hidden"` doesn't stop overscroll on iOS — only `document.documentElement.style.overflow` does. The component now locks/restores both `<body>` _and_ `<html>` overflow on open/close, and both the unmount cleanup and `onExitComplete` restore the saved values.
  4. **Latent: toggle outside focus trap, `aria-controls` on portalled target, `initialsFor` whitespace-only word boundary.** Added inline comments documenting why each is intentional / acceptable rather than changing behavior. The toggle's z-index 1100 placement above the sheet is required so it stays tap-targetable while the dialog is open; the trap's "focus leaked outside" branch already pulls focus back. `aria-controls` resolves correctly once the portalled sheet mounts and screen readers tolerate the dangling reference when closed, so the attribute stays static. `initialsFor` is tuned for Latin/CJK whitespace-separated names (Auth0 display names, local usernames) and degrades gracefully for Arabic/Thai/etc. — switch to `Intl.Segmenter` if richer handling is ever needed.
  5. **Tests updated to the new DOM structure.** `NavMenu.ct.tsx` previously located the sheet via `[aria-label="Site navigation"]` — that label now lives on the inner `<nav>`, so 15 sheet-locator sites switched to the stable `#mobile-nav-sheet` id (already wired up via `aria-controls`). Updated the aria-modal/aria-label test to assert the new `"Navigation menu"` value on the dialog, and added a regression `should expose the navigation landmark as a separate <nav> inside the dialog` that pins the split: the dialog is a `<div>`, the inner `<nav aria-label="Site navigation">` is a sibling landmark inside it, and the nav landmark wraps the browse items. All 33 NavMenu CT tests pass.

- **Dark mode contrast in CAML article renderer** (`frontend/package.json`). Upgraded `@os-legal/caml` and `@os-legal/caml-react` from 0.1.0 to 0.1.2 to pull in a rendering fix that restores text/background contrast in dark-mode CAML article views. Also updated the `resolutions` entry so both the top-level app and `caml-react`'s own peer resolve to the same 0.1.2 version (previously the stale `^0.1.0` resolution caused `caml-react` to bundle caml 0.1.0 internally despite the dependency version bump).

- **OAuth IDs leaked publicly on leaderboard and user profiles** (`opencontractserver/users/models.py`, `config/graphql_auth0_auth/utils.py`, `config/graphql/search_queries.py`, new migration `0029_backfill_oauth_slugs.py`, new management command `regenerate_user_slugs`). When social/OAuth users first logged in, `User.save()` generated their URL slug from `self.username` — which for Auth0 users is the raw provider sub like `google-oauth2|114688257717759010643`. `sanitize_slug()` strips the `|` separator but keeps everything else, so slugs like `google-oauth2114688257717759010643` were committed to the DB. These slugs then surfaced verbatim on the community leaderboard (via `entry.user.slug` → `getCreatorDisplay()`) and as navigation targets. Secondary issue: `resolve_search_users_for_mention` searched the `username` and `email` DB columns, allowing any authenticated user to confirm whether a specific OAuth sub or email address maps to an existing account (even though those fields return `null` in the response). Three-part fix: (1) `User.save()` now generates handle **before** slug so the handle is available as the slug base; for social/OAuth users (`"|" in username` or `is_social_user=True`), `handle` is used as `base_value` instead of `username`. (2) `configure_user()` in the Auth0 middleware now sets `is_social_user=True` so newly created Auth0 users are correctly flagged. (3) `resolve_search_users_for_mention` now searches only `slug__icontains` and `handle__icontains`. Migration `0029` backfills all existing users whose slug equals `sanitize_slug(username)` — i.e. was OAuth-derived — replacing it with a handle-based slug. The `regenerate_user_slugs` management command re-runs the same repair on demand.

- **Corpus tabs (Annotations / Analyses / Extracts) lost scrolling** (`frontend/src/components/layout/CardLayout.tsx`, `frontend/src/views/Corpuses.styles.ts`, `frontend/src/views/Corpuses.tsx`, `frontend/src/components/annotations/{AnnotationsPanel,CorpusAnnotationCards}.tsx`, `frontend/src/components/extracts/{ExtractCards,CorpusExtractCards}.tsx`, `frontend/src/components/analyses/{AnalysesCards,CorpusAnalysesCards}.tsx`). The corpus view chain was a viewport-stuck flex tower: every layer (`CardContainer`, `ScrollableSegment`, `CorpusViewContainer`, `MainContentArea`, the per-tab wrapper divs, plus each panel's own `Container` / `AnnotationsListContainer`) declared `height: 100%; max-height: 100vh; overflow: hidden`, with the inner panel using `flex: 1; overflow-y: auto` to scroll. After the SPA-shell refactor (#1561) made the outer wrapper use `min-height: 100vh` instead of `height: 100vh`, none of those `height: 100%` declarations could resolve to a definite value (per CSS spec percentage heights compute to `auto` when the containing block height is not specified explicitly), so the inner `overflow-y: auto` containers silently grew to their content height instead of scrolling. Three wrappers (`CorpusAnnotationCards`, `CorpusAnalysesCards`, `CorpusExtractCards`) were also passing `style={{ minHeight: "70vh", overflowY: "unset" }}` to their panels — leftover from a much earlier outer-scroll era — actively cancelling whatever inner scroll did try to engage. Replace the whole tower with natural-flow: drop the `height: 100%; max-height; overflow: hidden` clamps from `CardContainer`, `ScrollableSegment`, `CorpusViewContainer`, `MainContentArea`, the per-tab wrapper divs, and the panels' own `Container` / `AnnotationsListContainer` / `ExtractCards Container` / `AnalysesCards` div; drop the legacy inline `style` overrides on the three corpus wrappers. Cards stack, the document scrolls naturally, and the App-shell's `min-height: 100vh` outer keeps the sticky-footer behaviour intact. `CorpusViewContainer` keeps a `min-height: 100dvh` so the corpus view fills at least the viewport when content is short.

- **Cookie consent modal scroll + off-screen on mobile** (`frontend/src/components/cookies/CookieConsent.tsx`). `.oc-modal` used `max-height: 100vh` on mobile, which doesn't shrink with browser chrome (URL bar / toolbars), pushing the bottom of the modal — including the Accept button — below the visible viewport. The body also redefined `max-height: 70vh` / `calc(100vh - 180px)` and `overflow-y: auto`, fighting the base `@os-legal/ui` `.oc-modal-body { flex: 1; min-height: 0; overflow-y: auto }` flex setup. Switch the modal's mobile cap to `100dvh`, drop the redundant body `max-height` overrides (let base flex handle it), and zero out the overlay's mobile `padding` so the fullscreen mobile modal isn't inset by 16px on each side.

- **Extracts view (`/extracts`): broken pagination + per-row N+1 + auth-change refetch storm** (`frontend/src/views/Extracts.tsx`, `frontend/src/graphql/queries.ts`, `config/graphql/extract_types.py`, `frontend/src/components/extracts/ExtractListCard.tsx`). Audit follow-up to the Documents view perf work (PRs #1553, #1556, #1554, #1566). Three cooperating issues:

  1. **Broken pagination.** `GET_EXTRACTS` declared no `$cursor` / `$limit` parameters and did not pass `first` / `after` to the `extracts(...)` connection arguments, so the server quietly clamped every request to `max_limit=15` and `fetchMore`'s cursor variable was sent over the wire but never honoured. Infinite scroll never advanced beyond the first 15 rows.
  2. **Per-row N+1 in `resolve_full_document_list`.** The list query selected `fullDocumentList { id }` purely to read `.length` on the frontend, which on the backend iterates `self.documents.all()` and calls `user_has_permission_for_obj` for every document on every extract — 15 extracts × N documents permission checks per page. Fieldset's `fullColumnList { id }` similarly shipped full Column rows when only a count was needed.
  3. **`useEffect(refetch on currentUser)` storm.** Every `userObj` reactive-var settle fired a refetch on top of the implicit `useQuery` mount fetch — the same shape Documents.tsx removed in PR #1553.
     Fix: new slim `GET_EXTRACTS_FOR_LIST` query (`frontend/src/graphql/queries.ts`) wires `first` / `after` properly and selects only fields the list view actually renders. `ExtractType` gains a `documentCount` resolver (and `FieldsetType` a `columnCount` resolver) that read from the existing `prefetch_related("documents", "fieldset__columns")` cache populated by `ExtractQueryOptimizer.get_visible_extracts` — no extra COUNT queries per row, no per-doc permission fan-out. `ExtractListCard.formatStats` prefers the new count fields and falls back to `fullDocumentList?.length` so consumers still on the heavy `GET_EXTRACTS` (`ExtractItem`, `CorpusExtractCards`, `CamlArticleEditor`, `CreateExtractModal`) keep working unchanged. Variables are `useMemo`'d on the search-term primitive, the auth-change effect is removed, `fetchMore` passes both `limit` and `cursor`, and `EXTRACTS_PAGE_SIZE = 20` makes first paint pay for one focused page instead of the previous max_limit. Test wrapper (`frontend/tests/ExtractsTestWrapper.tsx`) projects fixtures to the slim shape so the existing CT suite runs unchanged. Backend regression `test_extracts_list_count_fields_match_lists` (`opencontractserver/tests/test_extract_queries.py`) pins `documentCount === len(fullDocumentList)` and `columnCount === len(fullColumnList)` so the slim/legacy queries can never drift. Frontend regression `frontend/src/views/Extracts.refetch-shape.test.ts` pins the three fixes at the source level (Apollo's request dedup makes the storm invisible to MockedProvider).

- **Annotations view (`/annotations`): refetch storm + non-memoised query variables** (`frontend/src/views/Annotations.tsx`). The view had two `useEffect` blocks that called `refetch_annotations()`/`refetch_corpus()` on filter and corpus changes, doubling every refetch on top of Apollo's implicit one (the variables already drove the refetch). On top of that, `annotation_variables` was a fresh `let`-bound object literal each render, forcing Apollo to deep-compare the variables on every parent re-render before deciding _not_ to refetch. Fix: (a) memoise `annotation_variables` on the underlying primitives (search term, label/labelset/corpus filters, structural-annotation toggle); (b) drop both filter-change and `opened_corpus` refetch effects — `useQuery` already refetches when its variables change, and `AuthGate.resetOnAuthChange` already clears the cache on login/logout, so the explicit auth-token refetch is also redundant; (c) remove the now-unused `refetch_annotations` / `refetch_corpus` destructures and the `auth_token` import. New regression `frontend/src/views/Annotations.refetch-shape.test.ts` pins the structural fix.

- **Strip per-request `.count()` debug logs from `resolve_annotations`** (`config/graphql/annotation_queries.py`). The resolver had seven `logger.info` statements that embedded `{queryset.count()}` in their format strings — every annotations request issued ~5 extra `SELECT COUNT(*)` queries (pre-corpus filter, post-corpus filter, pre-label-type filter, post-label-type filter, pre-analysis-isnull filter, post-analysis-isnull filter, plus the `visible_to_user` fallback log) regardless of log level, since f-string interpolation runs unconditionally. The chatty filter-application log statements (`"Filtering by labelset_id: ..."`) were also removed — they did not embed counts but added noise to every request without helping debug a real production issue. The `structural_set__documents` prefetch is intentionally retained because `AnnotationType.resolve_document` walks it for structural annotations.

- **Anonymous users could see public structural annotations but not their attached images** (`opencontractserver/annotations/views.py`, `opencontractserver/llms/tools/image_tools.py:472`, `opencontractserver/tests/test_annotation_images_api.py`, `docs/permissioning/consolidated_permissioning_guide.md`). `AnnotationImagesView` (`/api/annotations/<id>/images/`) was hardcoded to `IsAuthenticated`, so the frontend hook silently rendered "No thumbnail" for anonymous viewers even when the annotation was returned by the GraphQL feed under public-read rules. The endpoint now uses `AllowAny`; `get_annotation_images_with_permission` delegates directly to `AnnotationQuerySet.visible_to_user` so image visibility is **identical** to annotation visibility (no forked rule set to drift). Throttling is split into authenticated (`AnnotationImagesThrottle`) and anonymous (`AnnotationImagesAnonThrottle`) buckets, each gated on the matching user type so anonymous requests no longer consume two cache slots in lockstep. IDOR protection unchanged — empty 200 for missing/unauthorized.

### Changed

- **Redesign mobile navigation as a floating sheet that overlays content** (`frontend/src/components/layout/MobileNavMenu.tsx` (new), `frontend/src/components/layout/NavMenu.tsx`, `frontend/tests/NavMenu.ct.tsx`). The previous mobile menu used `@os-legal/ui`'s built-in inline drawer (`.oc-navbar__mobile-menu`), which expanded as a full-width opaque dark panel that pushed page content downward and clashed visually with the os-legal "transparent infrastructure" aesthetic.

  - At `< 1100px`, `NavMenu` now renders a custom `MobileNavMenu` (compact dark header + floating, rounded white sheet) instead of the `@os-legal/ui` `NavBar`. Desktop (≥ 1100px) is unchanged.
  - The new sheet is portalled to `document.body`, sits 8px below the header with a 12px side gutter, has a 16px border-radius, soft layered shadow, and a `backdrop-filter: blur(8px)` semi-transparent backdrop dimming the page beneath. Spring-animated entry/exit via `framer-motion` (already a dependency).
  - Active nav items use the design-system teal (`OS_LEGAL_COLORS.accent` on `accentSurface`) with a 3px left bar, matching the hover/selection language used elsewhere. Auth footer renders a teal "Sign in" CTA when signed out and a user chip with avatar initials + "Signed in" label when authenticated.
  - Closes on backdrop tap, ESC key, route change, and any nav-item / user-action selection. Locks `body` scroll while open and exposes accessible `aria-modal="true"`, `aria-label="Site navigation"`, and `aria-expanded` semantics.
  - Updated the two responsive tests in `frontend/tests/NavMenu.ct.tsx` to target the new component via stable `aria-label` selectors (`"Open navigation"`, `"Close navigation"`, `"Site navigation"`) instead of the now-removed `.oc-navbar__mobile-toggle` / `.oc-navbar__mobile-link` library classes. Test intent is unchanged; only selectors moved to the new implementation.
  - Review follow-ups: `handleLogin` wrapped in `useCallback` for consistency with `runAndClose`; `logoNode`, `mobileNavItems`, and `mobileUserActions` memoized in `NavMenu`; section labels (`"Browse"` / `"Account"`) lifted to named constants; the Sheet's programmatic-focus fallback suppresses its `:focus-visible` outline; an unmount-cleanup `useEffect` restores `body.style.overflow` if the component is torn down mid-exit-animation (so `AnimatePresence#onExitComplete` not firing can't leave the page scroll-locked). Eleven additional CT tests cover the Tab focus trap (forward / backward / focus-leaked-outside), the authenticated user-actions section, `runAndClose` for both nav items and user actions, the signed-out `handleLogin` path, and the toggle / sheet ARIA semantics.

- **Tokenize hardcoded navigation colors and tighten test-wrapper / focus-effect conventions in the Corpuses split** (`frontend/src/views/Corpuses.styles.ts`, `frontend/src/views/CorpusQueryView.tsx`, `frontend/tests/CorpusQueryViewTestWrapper.tsx`, `frontend/src/assets/configurations/osLegalStyles.ts`). Follow-up to PR #1578 review feedback:
  - Added `OS_LEGAL_COLORS.navBlue` (`#4a90e2`) and `navIndigo` (`#6366f1`) tokens plus matching `navBlueAlpha` / `navIndigoAlpha` rgba helpers. These are intentionally distinct from `primaryBlue` (`#3b82f6`) — they capture the slightly-different navigation/sidebar blue that recurs across legacy chat, search, and note components, so multiple files can converge on a single named source. Cross-file harmonization with `primaryBlue` is tracked in #1446.
  - Replaced ten hardcoded `rgba(74, 144, 226, ...)` / `rgba(99, 102, 241, ...)` / `#4a90e2` / `#6366f1` literals across `NavigationItem`, `NavigationToggle`, `NavigationItems` (scrollbar focus ring), and `BackNavButton` with the new tokens / helpers — visual fidelity preserved exactly.
  - Replaced `React.useState(() => { showQueryViewState(initialQueryViewState); return null; })` in `CorpusQueryViewTestWrapper.tsx` with a plain `useEffect` keyed on `[initialQueryViewState]`. The `useState` initializer pattern was an unconventional way to fire a side effect during render; `useEffect` clearly signals "side effect on mount" and is lint-clean. All 7 `CorpusQueryViewCoverage.ct.tsx` tests continue to pass.
  - Annotated the `eslint-disable react-hooks/exhaustive-deps` on the mount-only focus effect in `CorpusQueryView.tsx` with a one-line WHY: `isDesktop` is read once to avoid re-firing on viewport resize (which would pop the mobile keyboard mid-session).

### Added

- **Agent tools for step-by-step CAML article citation review**
  (`opencontractserver/llms/tools/core_tools/caml_article.py`,
  registered in `opencontractserver/llms/tools/tool_registry.py`). Three new
  async tools compose into a "find a candidate → verify the match → ask the
  user → replace one location" loop the agent can run against the corpus's
  `Readme.CAML` article:

  - `read_corpus_caml_article` (read-only, `requires_corpus`) loads the
    article via `Document.objects.visible_to_user(user)` and returns it as
    blank-line-delimited blocks with each block's existing `{{@cite ...}}`
    directives plus a `needs_citation_candidate` heuristic that flags prose
    blocks lacking a citation. Headings, lists, code fences, blockquotes,
    table rows, and `[component:…]` markers are excluded so the candidate
    list stays focused on text the user would actually want to cite.
  - `propose_caml_citation_match` (read-only, `requires_corpus`) runs the
    same `CoreAnnotationVectorStore.async_search` path the renderer's
    `useCiteHandler` uses, scoped to the invoking user's `user_id` so
    `Annotation` visibility is honoured. Returns ranked
    `{annotation_id, raw_text, label_text, label_color, document_id,
document_title, corpus_id, page, similarity_score}` candidates capped at
    25 entries.
  - `apply_caml_article_edit` (`requires_approval=True`,
    `requires_write_permission=True`, `requires_corpus`) replaces a single
    occurrence of `target_text` inside `Readme.CAML` with `replacement_text`.
    Fails closed if the substring matches zero or more than one location, and
    layers a defense-in-depth UPDATE check (`is_superuser` /
    `creator_id == user.id` / `user_has_permission_for_obj(... UPDATE)`) on
    top of the wrapper's READ check on `deps.corpus_id`. The edit rewrites
    `Document.txt_extract_file` in place via `FieldFile.save`, so the
    existing Document row is preserved and frontend deep-links to
    `Readme.CAML` continue to resolve. The `rationale` argument is surfaced
    in the approval modal so the user can see why each replacement was
    proposed.
  - Agent flow is one approval prompt per replacement, matching the issue
    request to "step through changes with the user — one citation at a
    time". The directive registry on the frontend is already extensible
    (`frontend/src/components/corpuses/caml/directiveRegistry.ts`), so the
    inserted `{{@cite scope}}` markers are resolved at render time by the
    existing `useCiteHandler` semantic search; no frontend changes needed.
  - Tests in `opencontractserver/tests/test_caml_review_tools.py` cover
    block parsing, candidate heuristics, IDOR-safe error messages, the
    single-occurrence rule, the creator/superuser/UPDATE check matrix, and
    registry resolution including the `requires_approval` /
    `requires_write_permission` / `requires_corpus` flags.
  - Each `apply_caml_article_edit` call rotates `Document.txt_extract_file`
    to a fresh storage blob and queues an `on_commit` cleanup that deletes
    the previous blob, so long-lived corpora don't accumulate orphan files
    on every citation edit. The UPDATE permission check runs **inside** the
    `select_for_update` transaction so a permission revocation between the
    initial load and the locked write cannot slip through. The prose
    heuristic also rejects `***` / `* * *` thematic breaks (previously only
    `-`/`_`/`=` runs were excluded). `CAML_CITATION_MAX_CANDIDATES` and
    `CAML_EDIT_PREVIEW_RADIUS_CHARS` moved to
    `opencontractserver/constants/document_processing.py`.
  - `propose_caml_citation_match` now wraps **both** `CoreAnnotationVectorStore`
    construction and the `async_search` call in the same try/except, so a
    corpus that has no `preferred_embedder` and no `PipelineSettings` default
    surfaces the same friendly "Semantic search failed" `ValueError` the
    agent already knows how to recover from instead of leaking the raw
    `get_embedder() resolved no embedder_path` error from `__init__`. Pinned
    by a new `test_constructor_failure_surfaces_as_value_error` test.
  - `ProposeCamlCitationMatchTests` now patches `__init__` alongside
    `async_search` on `CoreAnnotationVectorStore`. The real constructor calls
    `get_embedder()` against the migration-seeded `PipelineSettings`
    singleton, but `TransactionTestCase` truncates that row between tests
    (`serialized_rollback=False` default), so any other class running first
    on the same pytest-xdist `--dist loadscope` worker would leave us with no
    default embedder and the constructor would raise before the patched
    `async_search` could execute. Coverage of `caml_article.py` is now 100%
    via additional tests for `_safe_delete_storage_path`, `_read_caml_content`
    on a falsy `txt_extract_file`, the `User.DoesNotExist` branches in
    `_read_corpus_caml_article` / `_apply_caml_article_edit` /
    `_assert_corpus_visible_to_user`, the empty-`target_text` guard, and
    component-marker / numbered-list rejection in `_looks_like_prose`.

- **PII scan & auto-annotate agent tool** (`scan_and_annotate_pii` / `ascan_and_annotate_pii`)
  - Scans documents via the new `privacy_filter` Docker service
    ([Open-Source-Legal/privacy-filter](https://github.com/Open-Source-Legal/privacy-filter))
    for 8 PII categories (email, phone, person name, address, account number, URL,
    date, secret).
  - Creates one labeled annotation per detection — TOKEN_LABEL via PlasmaPDF for PDFs,
    SPAN_LABEL for plain text.
  - Agent knobs: `min_score` (default 0.5), `entity_groups` allowlist, `dry_run` preview
    mode, `start_char` / `end_char` to scope the scan to a slice.
  - Auto-creates 8 colored/iconed labels via `Corpus.ensure_label_and_labelset`.
  - Files: `opencontractserver/llms/tools/core_tools/pii.py`,
    `opencontractserver/llms/tools/core_tools/_privacy_filter_client.py`,
    `opencontractserver/llms/tools/tool_registry.py`.
  - New `privacy_filter` service in `local.yml` (with `required: false` so devs without
    the image pulled can still bring up the stack) and `production.yml` (with mandatory
    `PRIVACY_FILTER_API_KEY` env var via `${...:?}` syntax).
  - Settings: `PRIVACY_FILTER_URL`, `PRIVACY_FILTER_API_KEY`,
    `PRIVACY_FILTER_TIMEOUT_SECONDS` in `config/settings/base.py`.
  - 18 unit tests (12 tool + 6 client); the client is mocked in tool tests so CI does
    not need the privacy-filter container.

### Changed

- **Adaptive `load_document_text` tool sizes its slice to the agent's remaining context budget** (`opencontractserver/llms/tools/pydantic_ai_tools.py`, `opencontractserver/llms/agents/pydantic_ai_agents.py`, `opencontractserver/llms/tools/tool_registry.py`). Whole-document tasks like summarisation used to fan out into hundreds of `load_document_text` calls because the tool description hard-coded a 5K-50K char chunk recommendation regardless of how much room the model actually had. The agent now snapshots its per-turn budget onto `PydanticAIDependencies` and the document-agent factory swaps the registry-built tool for a budget-aware variant.
  - `PydanticAIDependencies` gains four new fields populated at agent construction and refreshed each turn: `model_name`, `context_window_tokens`, `estimated_used_tokens`, and `compaction_threshold_ratio`. Two helpers — `remaining_tokens_until_compaction()` and `recommended_chunk_chars(reserve_ratio=0.25)` — derive the budget the next read should target. The default `reserve_ratio` of 0.25 leaves 25% of the remaining tokens for the assistant's own response and the ~10% over-estimate baked into `estimate_token_count`.
  - `PydanticAICoreAgent._refresh_context_budget()` is invoked after every `_get_message_history()` call (`_chat_raw`, `_stream_core`, the structured run path, and `resume_with_approval`), so the snapshot reflects the post-compaction state entering each turn. Tools read the budget through closure-by-reference on the shared `agent_deps` instance — Pydantic v2 BaseModels mutate cleanly and the agent factory now constructs `agent_deps_instance` _before_ the custom tool closures so they see live values at runtime.
  - In the document-agent factory, `load_document_text` is no longer auto-built from the registry; a custom adaptive variant takes its place. When `end` is omitted the tool defaults to `min(total_chars, start + recommended_chunk_chars())` (with a `MIN_IMPLICIT_DOCUMENT_CHUNK_CHARS` floor — 5,000 chars — for starved budgets). The return value is now a structured dict — `{text, total_chars, returned_range, chars_remaining, suggested_next_start, context_budget_chars, budget_was_applied}` — which both reports the budget back to the LLM for planning and bypasses the wrapper's string-only 50K truncation so large slices reach the model intact when the budget allows. `budget_was_applied` is `True` iff the slice size was driven by the budget heuristic (`end` was omitted) so the LLM can tell whether `context_budget_chars` shaped the response or merely advertises the next implicit-call budget.
  - New `get_remaining_context_budget` tool (registered under `ToolCategory.DOCUMENT`, custom-built in the document-agent factory) lets the agent inspect its budget without performing a load — useful for multi-step planning before whole-document reads.
  - Both `PydanticAIDocumentAgent` and `PydanticAICorpusAgent` factories now thread `model_name`, `context_window_tokens`, and `compaction_threshold_ratio` into their `PydanticAIDependencies` instance so future budget-sensitive tools on either agent inherit the snapshot. The corpus agent does not currently expose `load_document_text` (the tool is per-document), but the wiring is consistent so the ground is prepared.
  - Tool registry description for `load_document_text` updated to advertise the adaptive defaulting, and the tool-registry NOTE listing runtime-context-only tools now includes `get_remaining_context_budget`.
  - Multi-call drift fix: `PydanticAIDependencies.turn_implicit_doc_text_chars` accumulates the implicit-chunk bytes already returned within a single turn and is folded into the budget so successive `end`-less reads back off proportionally; `_refresh_context_budget` resets it at the start of every turn. When `_stream_core` is called with an explicit `message_history` (the `resume_with_approval` path), a fresh `_HistoryResult` is now built from that explicit list via `_history_result_from_messages` and run through the same `_refresh_context_budget` codepath — so `estimated_used_tokens`, `context_window_tokens`, and `compaction_threshold_ratio` all refresh, not just the per-turn tally.
  - `_refresh_context_budget` now delegates to a `@staticmethod _apply_context_budget(deps, config, history_result)` helper, and `_history_result_from_messages` is also a `@staticmethod` taking an explicit `config`. The transformations are unit-testable without bypassing `__init__`, and the indirection lets the explicit-history and DB-history paths share one codepath without leaking class-internal state.
  - Cache hygiene: `core_tools.text_extracts` exposes `get_cached_txt_extract_length(document_id)` and `_DOC_TXT_CACHE` is no longer re-exported from `core_tools/__init__.py` — consumers must use the accessor. The cache backend can later evolve (async-safe structure, Redis) without rippling across consumers.
  - The closure body of the adaptive tool was extracted into a module-level `_make_load_document_text_tool(agent_deps, doc_id)` builder so it can be exercised end-to-end in unit tests instead of requiring a full `DocumentAgentContext`.
  - Regression coverage in `opencontractserver/tests/test_context_guardrails.py` pins the budget arithmetic — `TestContextBudgetSnapshot` covers positive remaining tokens, zero-floor when usage exceeds the threshold, default 25% reserve, and explicit `reserve_ratio` overrides; `TestLoadDocumentTextDriftMath` pins the multi-call drift behaviour (second implicit call backs off, drift floors at `MIN_IMPLICIT_DOCUMENT_CHUNK_CHARS`, `_refresh_context_budget` reset returns to the full budget); `TestRefreshContextBudgetFallback` pins the fallback semantics for zero vs non-zero `_HistoryResult.context_window`; `TestLoadDocumentTextClosureIntegration` drives the actual closure with a stubbed cache and pins the dict-shape contract (including `budget_was_applied`) plus the in-turn tally mutation through the closure reference.
  - Defensive-zero fallback: `_refresh_context_budget` no longer uses `or` to choose between `_HistoryResult.context_window` and `get_context_window_for_model(...)` — `or` would conflate "missing" with a legitimate zero, so the snapshot now uses an explicit `> 0` check instead.
  - Observability: `load_document_text` emits `logger.warning` when `recommended_chunk_chars()` exceeds `LARGE_IMPLICIT_CHUNK_WARN_RATIO` (50%) of the model's full context window in characters. The threshold is _relative_ so 1M-token models (Gemini 1.5, Claude Opus large-context) don't spam the warning on legitimately large recommendations — only real `CHARS_PER_TOKEN_ESTIMATE` drift trips it. This is a cheap production signal when the heuristic drifts (e.g. for multilingual or code-heavy documents whose chars-per-token ratio differs from the `CHARS_PER_TOKEN_ESTIMATE` default).
  - Breaking-shape note for downstream callers: `load_document_text` now returns `dict[str, Any]` rather than `str`. Python code that called the tool programmatically and unwrapped a string return will break at runtime. The change is intentional — the wrapper's 50K-char string truncation is bypassed so large slices reach the model intact, and the dict carries the budget-planning fields the LLM uses to decide whether to chunk further.
  - Review round 5 follow-up: (1) empty-document edge case — `_make_load_document_text_tool` previously detected "never cached" via `cached_len == 0`, which collided with "loaded and empty" and forced a redundant cache-prime fetch on every call for a 0-byte document. `core_tools.text_extracts` now exports a `is_txt_extract_cached(document_id)` membership predicate and the closure uses that instead; the prime now fires exactly once for an empty document, not once per call. (2) `PydanticAIDependencies.recommended_chunk_chars` now emits a `logger.warning` when `reserve_ratio` falls outside `[0.0, 1.0]` — the value is still clamped (preserving back-compat) but the warning makes "unexplained 0-char budget" diagnosable instead of silent. (3) Tool-registry description for `load_document_text` no longer mentions the document-agent's adaptive override (which is implementation-noise the LLM doesn't need to see); the explanation moved to a `# NOTE` code comment. (4) The `get_remaining_context_budget` registry description was tightened to enumerate all six returned fields (it previously omitted `compaction_threshold_ratio`, drift-prone if the in-agent helper grows). (5) Regression coverage in `test_context_guardrails.py`: `TestIsTxtExtractCached` pins the membership-vs-length distinction; `TestLoadDocumentTextClosureIntegration.test_empty_document_does_not_repopulate_cache` asserts the empty-document prime fires exactly once across two implicit calls; `TestContextBudgetSnapshot.test_recommended_chunk_chars_reserve_above_one_clamps_and_warns` / `..._below_zero_...` pin the warning + clamp behaviour.
  - Review round 6 follow-up (issue #1611): (1) `get_document_text_length_tool` (in `pydantic_ai_agents.py`) had the same "empty document re-loads on every call" pitfall round 5 fixed in `_make_load_document_text_tool` but missed at this sibling site — the guard was `cached_len > 0`, which is false for a legitimately-empty document so a second full-document load fired on every call. Switched to the `is_txt_extract_cached` membership predicate so the prime is consulted once and the second load only fires on the genuine cache-miss fallback. (2) `_history_result_from_messages._part_text` used `getattr(part, "content", None) or getattr(part, "args", None) or ""` — the `or` chain conflates an empty string (e.g. a `ToolReturnPart` with empty content) with "missing" and falls through to `args`, double-counting tokens in the budget estimate. Replaced with an explicit `is None` check matching the defensive-zero pattern in `_refresh_context_budget`. (3) `TestLoadDocumentTextClosureIntegration._patches()` now also patches `is_txt_extract_cached` to return `True`; without this the real predicate returned `False` for the fake-loaded `DOC_ID` and the prime path was being silently exercised on every call in three of the four tests, so the warm-cache code path the tests claimed to exercise wasn't actually covered. The fourth test (`test_empty_document_does_not_repopulate_cache`) still leaves the predicate unpatched on purpose to drive the membership-vs-length distinction end-to-end. (4) Documented at the `PydanticAIDependencies.turn_implicit_doc_text_chars` field that the in-turn drift correction is _partial_ — only implicit `load_document_text` calls feed the tally; other heavy tool outputs (vector-search results, annotation lists) consume context but are not tracked. Future maintainers adding a budget-aware tool now have a pointer to the design constraint at the source of truth.

### Fixed

- **`get_embedder` redundant tuple cast** (`opencontractserver/utils/embeddings.py:132`). The follow-up patch in commit 85496ad re-introduced a `cast(tuple[Optional[type[BaseEmbedder]], Optional[str]], ...)` on the return statement that mypy now flags as redundant -- the function's declared return type already matches what mypy infers, so the cast trips `warn_redundant_casts = True`. This was the only mypy error on the branch and was blocking pre-commit (and therefore Backend CI's lint step) for any new commit. Reverted to the bare `return embedder_class, embedder_path` from PR #1545, which both satisfies mypy and matches the function's declared return type.
- **Reddit-style auto-assigned user handles** (issue #1574). Auth0 social-login users without a populated `name`/`given_name`/`first_name` previously rendered as the redacted `user_<id>` fallback in the GraphQL `displayName` resolver chain. New `User.handle` field (`opencontractserver/users/models.py`) is auto-assigned on `save()` from a curated `adjective × noun` namespace (`opencontractserver/users/handle_wordlists.py`, ~56k base combos) via `opencontractserver/users/handle_generator.py`. Handles surface as a higher-priority branch in `UserType.displayName` (`config/graphql/user_types.py`) — chain order: `name` → `given_name + family_name` → `first_name + last_name` → `handle` → `username` (only when not a `provider|sub` Auth0 identifier) → redacted `user_<id>` fallback. Schema + data migrations (`opencontractserver/users/migrations/0027_user_handle.py`, `0028_backfill_user_handles.py`) backfill existing users in one pass; the same generator powers the `regenerate_user_handles` management command for re-running with `--reroll-suffixed` (after the word list grows) or `--reroll-all`. User-facing handle editing is intentionally out of scope; the resolver still degrades to the redacted fallback for any user the migration hasn't reached. Coverage in `opencontractserver/tests/test_user_handle.py` pins generator uniqueness, no-PII-leakage, fixed-seed determinism, suffix promotion under saturation, the resolver chain priority, and the management command's three modes.
  - The django-guardian Anonymous system user is excluded from auto-assignment, the migration backfill, and the `regenerate_user_handles` queryset — it never surfaces to other users and so never needs a handle. Without this exclusion the post-migrate signal that creates Anonymous (after `0028_backfill_user_handles` runs) could leave it with `handle=NULL`, which made `regenerate_user_handles` see two candidates instead of one in a freshly-bootstrapped DB.
  - `User.save()` no longer string-parses the `IntegrityError` message to detect handle-uniqueness collisions; instead it queries for an existing row holding the chosen handle and only re-rolls when that hit confirms the conflict. The previous `"handle" in str(exc)` heuristic was Postgres-specific and would silently swallow non-handle integrity errors if the constraint were ever renamed.
  - Curated word list cleanup: `flora` and `ladybug` moved from `ADJECTIVES` to `NOUNS` so `flying ladybug` no longer surfaces as a noun-noun pairing in generated handles.
  - `regenerate_user_handles --reroll-suffixed`: keep the user's own row in `scope_qs` (rather than excluding by pk) so its current DB handle blocks `generate_handle` from re-selecting the same value. Setting `user.handle = None` in Python alone does not reach the DB, so excluding the pk could let `cleverFox42` round-trip through the generator and back to itself, defeating the flag's purpose. Same change applies to `--reroll-all`. Regression coverage in `test_user_handle.py::RegenerateUserHandlesCommandTests::test_reroll_suffixed_targets_only_numeric_tails` (now reliably re-rolls under the fix).
  - Anonymous-user exclusion is now pinned by tests in `test_user_handle.py::UserModelHandleAutoAssignTests` (one test for `User.save()`, one for `regenerate_user_handles --reroll-all`) so the four exclusion sites (model save, schema migration, data migration, management command) cannot drift apart silently.
  - Architecture doc at `docs/architecture/user_handles.md` (wired into `mkdocs.yml`) documents the resolution chain, generator semantics, migrations, management command modes, and tunables.
  - Post-merge cleanup pass on the handles PR: (1) `regenerate_user_handles` now raises `CommandError` when `--reroll-all` and `--reroll-suffixed` are passed together (previously the narrower flag was silently dropped); (2) the `\d+$` "is suffixed" regex is now sourced from a single `_SUFFIXED_RE` constant used by both the queryset filter and the compiled `_SUFFIXED_PATTERN` (eliminates a drift trap); (3) removed the unused `handle_field` parameter from `generate_handle` per YAGNI — every caller used the default; (4) added explanatory comment in `User.save()` that `self.pk` stays `None` across failed INSERT retries (so `scope_qs.exclude(pk=self.pk) if self.pk else scope_qs` is correct without a committed-row guard); (5) trimmed `"ox"` from `NOUNS` so the wordlist actually honours its documented 3-12 character bound; (6) tightened the `test_wordlists_provide_a_meaningful_namespace` floor from 10k to 40k and added `test_wordlists_respect_length_bounds` plus `test_reroll_all_and_reroll_suffixed_are_mutually_exclusive`.

### Changed

- **`DocumentKnowledgeBase` decomposed into focused modules** (`frontend/src/components/knowledge_base/document/DocumentKnowledgeBase.tsx`, `frontend/src/components/knowledge_base/document/document_kb/{SidebarTabs,DocumentViewer,HeaderBar,helpers,useResizeHandle,useDocumentMarkdown,useDocumentLoader,useStructuralAnnotations,useContainerWidth}.{ts,tsx}`). Extracts sidebar tabs (desktop + mobile), the central document viewer (PDF / TXT / DOCX dispatch), the resize handle hook, the markdown summary fetcher, the structural-annotations loader, the container-width tracker, and the entire data-loading orchestration into dedicated modules. Behavior preserved (no UI/UX change), but the main file shrinks ~60% and each extracted module is independently testable. Tightens prop types — `DocumentViewer.createAnnotationHandler` is now `(annotation: ServerTokenAnnotation) => Promise<void>` instead of `any`. PDF.js types (`PDFDocumentProxy`, `PDFPageProxy`, `PageTokens`) replace three `any`s in `useDocumentLoader.buildPdfPages`. `processAnnotationsData` collapses two `setPdfAnnotations` calls into one to avoid an intermediate render in Apollo's `onCompleted` (outside React's automatic batching). `useContainerWidth` now creates/disposes its `ResizeObserver` inside the ref callback, so the observer correctly reattaches on conditional re-mounts. The internal `setViewState` setter is no longer exported from `useDocumentLoader` — callers should not poke at the hook's internal state. Discussions tab in `SidebarTabs` switches to `MessagesSquare` to disambiguate from the Chat tab's `MessageSquare`. Header date no longer flashes today's date when `metadata.created` is null — renders an em-dash placeholder instead. Follow-up review remediation: `SelectionLayer.finalizeSelection` no longer mutates `finalizingRef.current` from inside a `setMultiSelections` updater (the new line was paired with pre-existing side effects in the same updater — both updater functions are now pure, with the side effects sequenced after); the text-document branch of `DocumentViewer` now uses `id="text-container"` instead of inheriting the misleading `pdf-container` id from the pre-refactor monolith; `MENU_INTERACTION_COOLDOWN` is hoisted into `frontend/src/assets/configurations/constants.ts` as `SELECTION_MENU_COOLDOWN_MS`; `PANEL_WIDTH_FULL_PCT` gains a comment clarifying that the "full" preset deliberately stops at 90 %; `DesktopSidebarTabsProps` drops the unused `showRightPanel` from its prop bag via `Omit<CommonProps, "showRightPanel">`; and `useDocumentLoader.ts` gains a six-test smoke suite covering the auth-gated skip path, the document-only TXT happy path, the null-document ERROR path, the `threadCount` plumbing, the corpus-context skip, and the annotations-only refetch skip.

### Fixed

- **PDF text selection: action menu silently dropped when mouseup landed on overlapping fixed-position UI** (`frontend/src/components/annotator/renderers/pdf/SelectionLayer.tsx`). The selection finalisation handler was bound to React's `onMouseUp` on the SelectionLayer element itself, so a drag that released over an overlapping `position: fixed` sibling — most notably the right-edge `SidebarTab` buttons whose z-index sits above the selection layer — fired mouseup on the sibling instead. The selection state was left mid-flight: `localPageSelection` set, no `setShowActionMenu(true)`, and no `pendingSelections` flush. Surfaced by `tests/DocumentRenderingCornerCases.ct.tsx` "selection menu off-screen on mobile" once the `useContainerWidth` extraction below started reporting the post-modal-animation container width (causing the test's calculated drag end coordinates to land exactly on a sidebar tab); was a latent UX bug for mobile users selecting near the viewport's right edge in production. Fix: extract a `finalizeSelection(clientX, clientY, shiftKey)` helper and attach a document-level `mouseup` (and `mousemove` for off-element drift) listener while `localPageSelection` is non-null. The listeners detach as soon as the selection finalises or the component unmounts. Both the React-bound `onMouseUp` and the document fallback call the same helper, so single-page release behaviour is preserved exactly.

### Breaking Changes

- **`UserType.canImportCorpus` is now nullable** (`Boolean` instead of `Boolean!`). Cross-user / anonymous viewers receive `null`; only self-views still get a concrete `true` / `false`. Hand-written client queries that destructured the field without a default will start receiving `null`. TypeScript codegen picks up the change automatically; check unstructured consumers. Driven by the slug-only privacy policy below.
- **`objectSharedWith` payload shape change**: collaborator entries on shared corpuses / documents / labelsets no longer expose `email` or `username` — only `id`, `slug`, and the permissions list. Because `objectSharedWith` is typed as `GenericScalar` (no GraphQL schema enforcement of inner shape), external clients reading `objectSharedWith[].email` / `.username` will see those keys silently disappear at runtime instead of getting a schema error. Migrate consumers to `objectSharedWith[].slug`. Driven by the slug-only privacy policy below.

### Security

- **Slug-only user privacy policy across the GraphQL surface** (`config/graphql/user_types.py`, `config/graphql/permissioning/permission_annotator/mixins.py`, `config/graphql/og_metadata_queries.py`, `config/graphql/og_metadata_types.py`, `config/graphql/worker_queries.py`, `opencontractserver/users/models.py`, `opencontractserver/users/api/serializers.py`, `opencontractserver/llms/tools/core_tools/links.py`). Hardens user-PII exposure so non-self viewers see only the public `slug`; everyone else (including superusers via the public API) gets `null` for `email`, `username`, `name`, `firstName`, `lastName`, `givenName`, `familyName`, `phone`, `emailVerified`, and `isSocialUser`. PII access is reserved for Django admin (server-side, gated), not the GraphQL layer. Specifically:

  - `UserType` declares each PII field as a custom `graphene.String`/`Boolean` so the resolver gate runs even though `DjangoObjectType` would otherwise auto-resolve from the model. A shared `_is_self_view(user_obj, info)` helper enforces the contract uniformly; `Meta.exclude` adds a defensive layer that hides `password`, `last_ip`, `last_login`, `last_synced`, `synced`, `auth0_Id`, and `first_signed_in` from every viewer (including the user themselves — those have no client use case and were leaking operational metadata).
  - `displayName` resolver: non-self viewers always receive the user's `slug` (or a stable `user_<pk-suffix>` redaction when slug is unset). Self-views still walk the rich profile-name fallback so settings UIs can still greet users by name; the OAuth `provider|sub` redaction for social-login users still applies. The `email` resolver lost its superuser exception — admin tooling continues to work because the Django admin UI is the official PII path.
  - `User.__str__` no longer embeds the email (was `f"{self.username}: {self.email}"`, now `self.slug or self.username`) — closes a small log/admin leakage surface.
  - `AnnotatePermissionsForReadMixin.resolve_object_shared_with` (`config/graphql/permissioning/permission_annotator/mixins.py`) previously emitted `email` and `username` for every collaborator on a shared corpus / document / labelset to every viewer of that object. The payload now carries only `id`, `slug`, and the permissions list — every PII consumer in the existing client trees migrates to `slug`. **Breaking change for external consumers**: this field is typed as `GenericScalar` (no schema enforcement of payload shape), so external GraphQL clients that read `objectSharedWith[].email` / `.username` will see those keys disappear at runtime rather than getting a schema error. External integrations should migrate to `objectSharedWith[].slug`.
  - `og_metadata_queries.py` (corpus / document / document-in-corpus / thread / extract OG cards) and `worker_queries.py` (worker-account creator) switched `creator.username` → `creator.slug` so social-media link previews and worker-account dropdowns never surface the raw OAuth `sub`. OG `creator_name` field descriptions updated from "Username of …" to "Public slug of …".
  - `opencontractserver/llms/tools/core_tools/links.py` (Markdown link generator that the agent toolchain uses to surface annotation/corpus/document URLs to the LLM) was reading `creator.username` to populate the path's `userSlug` segment — wrong for OAuth users since the URL routing layer expects the case-sensitive slug. Now reads `creator.slug` everywhere; the variable was already named `user_slug`.
  - `opencontractserver/users/api/serializers.py` (`UserSerializer`) — currently unwired-but-defined DRF serializer reduced from `["username", "name", "url"]` to `["slug", "url"]`, with `lookup_field` switched to `slug` so any future view that wires it up follows the same contract by default.
  - Frontend leaderboards (`frontend/src/components/community/Leaderboard.tsx:582`, `frontend/src/components/landing/CompactLeaderboard.tsx:364,370`) now render `entry.user.slug` / `contributor.slug` directly instead of `displayName`. The backend already redacts `displayName` to slug for non-self viewers, but reading the slug field explicitly removes any chance of a stale Apollo cache leaking the self-view rendering when a logged-in user looks at their own row, and makes the privacy contract visible in the JSX.
  - Coverage: 12 new tests in `opencontractserver/tests/test_user_privacy.py` (self-view PII visibility, cross-user redaction, superuser redaction, anonymous redaction, redacted-handle fallback, `__str__` privacy, `object_shared_with` slug-only payload). Existing `test_user_display_name.py::EmailResolverTestCase` updated to assert that superusers no longer see other users' emails through the GraphQL API; new `CrossUserDisplayNameTestCase` pins the slug-fallback contract for `displayName`. `test_og_metadata_queries.py` `creatorName` assertions updated to read `self.user.slug` (underscores in usernames normalise to hyphens via the slug sanitizer). Follow-up review remediation: 4 additional `test_user_privacy.py` cases pin the self-only gate on `canImportCorpus` (cross-user → null, anonymous → null, self-view → boolean) and the `is_active=False` gate on `_is_self_view` (a deactivated user with a still-live session must not pass the gate, since `AbstractBaseUser.is_authenticated` is statically `True`); `_is_self_view` now also checks `is_active`. The deferred `from django.conf import settings` inside `resolve_can_import_corpus` was hoisted to module scope. Frontend `getCreatorDisplay` (`frontend/src/utils/userDisplay.ts`) now decodes the Relay global ID and uses the last 6 chars of the numeric pk so its no-slug fallback matches the backend `_redacted_handle` output (`user_<pk-suffix>`) instead of producing a different string for the same user — covered by 7 new `userDisplay.test.ts` cases.
  - **Out of scope for this PR / tracked in #1608**: V2 corpus export (`opencontractserver/utils/export_v2.py`) still writes `author_email` / `creator_email` into the export ZIP — this is a deliberate data-export feature owned by the user doing the export, but does cross the same privacy boundary for collaborator emails on a shared corpus. Switching to slug-based author resolution would be a breaking change to the export format and is left for a follow-up (`# TODO(#1608)` markers added at the four leak sites in `export_v2.py`).
  - **Schema-type break**: `UserType.canImportCorpus` is now `Boolean` (nullable) instead of `Boolean!`. Cross-user / anonymous viewers receive `null`; only self-views still get a concrete `true` / `false`. Hand-written client queries that trusted the non-null contract (e.g. destructured the field without a default) will start receiving `null`. TypeScript codegen picks up the change automatically; check unstructured consumers.

- **Close corpus-less `ModerationAction` auth gap in `resolve_moderation_action`** (`config/graphql/conversation_queries.py`, closes #1594). `resolve_moderation_action` previously short-circuited to `return action` whenever `conversation.chat_with_corpus` was `None`, leaking document-only and orphaned moderation actions to any authenticated user via direct ID lookup. The resolver now routes every case through `Conversation.can_moderate` (the same canonical authorization helper used by the moderation mutations), which accepts: superuser; conversation creator; corpus owner/designated moderator (when `chat_with_corpus` is set); document owner (when `chat_with_document` is set). Actions whose `conversation` FK is `NULL` (a rare historical edge — the column is nullable but every real action has one) fail closed for non-superusers. New regression coverage in `opencontractserver/tests/test_moderation.py` pins all four allowed paths plus the previously-leaking unauthorized-user case for both document-only and corpus-less conversations.
- **Bump `postcss` transitive dependency to `^8.5.10` to resolve GHSA-disclosed XSS** (`frontend/package.json`, `frontend/yarn.lock`, dependabot alert #87). `styled-components@6.1.19` pinned `postcss@8.4.49` exactly, and `tailwindcss`/`tinyglobby` resolved a separate `^8.5.6` line to `8.5.8`; both fall under the moderate-severity advisory ("PostCSS has XSS via Unescaped `</style>` in its CSS Stringify Output", patched in `8.5.10`). Added a `"postcss": "^8.5.10"` entry to the `resolutions` block so both spec ranges collapse to a single patched `8.5.14` lockfile entry. PostCSS runs in build-time tooling (Vite, tailwindcss), so the practical exposure was bounded, but the alert is closed regardless.
- **Stop echoing `str(exception)` from document-import REST views** (`opencontractserver/document_imports/services.py`, `opencontractserver/document_imports/views.py`, CodeQL `py/stack-trace-exposure` alerts #74 and #75). The `DocumentImportView.post` and `DocumentsZipImportView.post` 403 paths returned `{"error": str(e)}` for any `PermissionError` raised by the service layer, which CodeQL flags as exception-data flow into an HTTP response. The two service-layer raises were intentionally human-readable (usage-cap and bulk-upload-denied messages), but the data-flow link itself is the issue. Fix: introduce `DocumentImportPermissionError(PermissionError)` carrying a stable `code` (`USAGE_CAP`, `BULK_UPLOAD_DENIED`); both raises switch to the new class. The REST views catch the subclass and build the response body from a view-local `_public_permission_message(code)` literal-string lookup — exception state never reaches the response. The exception still inherits from `PermissionError` and `str(e)` returns the same human-readable message, so the GraphQL upload mutation (`config/graphql/document_mutations.py:207`), which lets `PermissionError` propagate as a Graphene error, and the assertion in `opencontractserver/tests/test_usage_caps.py` that pins the user-visible cap message keep working unchanged.

### Changed

- **Removed legacy `@testing-library/react-hooks` from frontend test infrastructure** (closes #1576). The package still calls the React 17-era `ReactDOM.render` API, so every hook test that imported from it printed a "ReactDOM.render is no longer supported in React 18" warning. The repo already had a thin React-18-native replacement at `frontend/src/test-utils/renderHook.tsx`, but 16 test files still imported from the legacy package. Migration:
  - Extended `frontend/src/test-utils/renderHook.tsx` with `RenderHookOptions<TProps>` (the legacy `{ wrapper, initialProps }` shape), plus `waitForNextUpdate` and `waitFor` on the hook return so it is a drop-in for every call site we use. The new helpers wrap each polling iteration in `act` so React 18 flushes pending state updates (Apollo cache writes, `fetch().then(setState)` resolutions, reactive-var subscribers) between polls — a single act boundary across the whole loop would defer the flush until the loop exits, leaving `renderCount` and `result.current` perpetually stale.
  - The standalone `waitFor` export matches `@testing-library/react-hooks` v8 semantics: a callback returning `undefined` (e.g. `() => expect(...)`) succeeds when it stops throwing; a callback returning a boolean succeeds on truthy. This lets `AnalysisHooks.sync.test.tsx` keep its `expect(...)`-inside-waitFor pattern after swapping `import { waitFor } from "@testing-library/react"` for `import { waitFor } from "../../../../test-utils/renderHook"`. The RTL version polls without `act`, so under `IS_REACT_ACT_ENVIRONMENT = true` (set in `setupTests.ts`) async hook updates never become visible to the probe and the predicate never sees the post-resolve state.
  - Updated the 16 test files (`useAnnotationImages`, `AnalysisHooks.sync`, `useClearTextBlockOnInteraction`, `useTextSearch`, `AnnotationHooks`, `useVisibleAnnotations`, `useUploadMutations`, `useMessageBadges`, `useAgentChat`, `useCacheManager`, `useFeatureAvailability`, `useBadgeCelebration`, `useAuthReady`, `useNetworkStatus`, `useCorpusMdDescription`, `useRefetchOnAuthChange`) to import from `../../test-utils/renderHook` (or the equivalent relative path).
  - Updated `useWebSocketAuth.test.ts:167` to pass `{ initialProps: { enabled: false } }` instead of the bare `{ enabled: false }` second arg now that the helper accepts an options object instead of a raw `initialProps` value.
  - Removed `@testing-library/react-hooks` from `frontend/package.json` `devDependencies` and pruned its entry from `yarn.lock`. The transitive `react-error-boundary` lockfile entry will be GC'd on the next `yarn install`.

### Fixed

- **Infinite navigation loop on `/e/:user/:extractId` extract route** (`frontend/src/App.tsx`, `frontend/src/components/routes/ExtractLandingRoute.tsx` removed). Visiting any extract by its canonical `/e/<creator>/<extractId>` URL tripped the navigation circuit breaker with `"/e/<...>" appeared 3 times`. Two pieces of routing code were redirecting in opposite directions and fighting each other:

  1. `ExtractLandingRoute.tsx:32-36` (mounted on `/e/:userIdent/:extractIdent`) redirected to `/extracts/<extract.id>` as soon as `openedExtract` was populated.
  2. `CentralRouteManager` Phase 3 then read `buildCanonicalPath(...)` (`frontend/src/utils/navigationUtils.ts:213`), which returns `/e/<creator.slug>/<extract.id>` as the canonical path for an extract, and redirected `/extracts/<id>` back to `/e/<...>`. Loop.
     Fix: route `/e/:userIdent/:extractIdent` directly to `ExtractDetailRoute` (whose JSDoc and the routing-guide table at `docs/frontend/routing_system.md:178-179` already declared this as the intended setup), and delete the dead `ExtractLandingRoute.tsx` + its test. `/extracts/<id>` still works — Phase 3 redirects it once to the canonical `/e/<creator>/<id>` and stops, since the next render is already on the canonical path. Updated `docs/frontend/routing_system.md` to reflect the removed component and the actual extract-routing flow.

- **Document deletion follow-up hardening** (issue #1572). Six surgical fixes to the orphan-blob reclaim path landed in PR #1487 / issue #1492:

  - `Document.blob_field_names()` (`opencontractserver/documents/models.py:365-389`) now returns an immutable `tuple[str, ...]` cached on the class. The previous implementation returned the same shared `list` on every call, so a misbehaving caller doing `names.append(...)` would corrupt the cache for every subsequent caller in the process. Latent footgun, no current caller mutated the result.
  - `_schedule_blob_cleanup_post_delete` (`opencontractserver/documents/signals.py:298-343`) now registers AT MOST ONE `transaction.on_commit` flush callback per atomic context, instead of one per `post_delete` signal. The flush callback was promoted to module scope and tagged with a sentinel attribute (`_FLUSH_CALLBACK_MARKER`) so we can detect prior registrations in `connection.run_on_commit` without comparing closure identity. `QuerySet.delete()` of N rows now costs O(1) on_commit registrations instead of O(N). Rollback safety preserved: Django drops our callback from the queue on savepoint/transaction rollback, so the next `post_delete` in a fresh transaction re-registers cleanly. The orphan re-check inside `cleanup_orphaned_document_blobs_task` continues to backstop the cross-transaction accumulator bleed for rolled-back paths.
  - `cleanup_orphaned_document_blobs_task` (`opencontractserver/tasks/cleanup_tasks.py:25-131`) now calls `default_storage.delete()` unconditionally instead of probing with `default_storage.exists()` first, closing a TOCTOU window where a concurrent process could remove the blob between the check and the delete and turn a benign race into an alarming ERROR log. `FileNotFoundError` is now caught at DEBUG level for backends that raise on missing paths; `FileSystemStorage` and `S3Boto3Storage` are idempotent on missing paths so the deleted-count semantics shift slightly (a successful delete of an already-absent path counts as 1).
  - `cleanup_orphaned_document_blobs_task` gained `autoretry_for=(Exception,), max_retries=3, default_retry_delay=60`. The task is idempotent (the orphan re-check absorbs whatever happened on a previous attempt), so transient failures (worker crash mid-execution, transient DB / storage error during the orphan check) now retry cleanly instead of silently losing the remaining paths.
  - The `__in` lookup in the orphan check (`opencontractserver/tasks/cleanup_tasks.py:90-97`) now passes `sorted(candidate_paths)` (a `list`) instead of a `set`, eliminating non-deterministic query plans on backends sensitive to set input ordering.
  - The undocumented "store an arbitrary attribute on `django.db.connections[alias]`" pattern used by the per-DB-connection accumulator in `opencontractserver/documents/signals.py:175-187` is now flagged in a comment with a documented escape hatch (switch to `WeakValueDictionary` keyed by `id(connection)` or `threading.local`) should the assumption ever break.
  - Regression coverage: new `test_bulk_delete_registers_single_on_commit_callback` in `opencontractserver/tests/test_blob_retention.py` deletes 20 rows inside a single atomic block and asserts that exactly one blob-cleanup callback (filtered by the sentinel marker) is appended to `connection.run_on_commit`. The pre-fix behavior would have been 20.

- **Conflicting migrations on `annotations` app block all `migrate` runs on `main`** (`opencontractserver/annotations/migrations/0071_grounding_annotation_unique_constraints.py`, `opencontractserver/tests/test_extraction_grounding.py`). PR #1418 (`Back grounding get_or_create with DB-level uniqueness`) merged after PR #1503 (`Add default LabelSet seeding`), but its migration branched off the older parent `0068_enforce_embedder_path_not_null` and was named `0069_grounding_annotation_unique_constraints`. That created two leaf nodes — `0069_grounding_annotation_unique_constraints` and `0070_seed_default_labelset` (already applied in production on top of `0069_labelset_is_default_and_seed_default`) — and Django bailed with `CommandError: Conflicting migrations detected; multiple leaf nodes …`, breaking Backend CI and Frontend E2E (the latter shells out to `migrate` during test setup). Resolution: renumber the unmerged grounding migration in place — `0069_grounding_annotation_unique_constraints` → `0071_grounding_annotation_unique_constraints` — and re-parent it on `0070_seed_default_labelset`. This is safe because PR #1418 had only just merged (35 minutes before this fix) and CI had been red the whole time, so the migration had not been recorded in any environment's `django_migrations` table. The renumber yields a single linear leaf with no merge migration carrying forward. Test `TestMigration0069M2MDedup`, which loads the migration file by name to exercise its `_repoint_cross_references` helper directly, was renamed and updated to reference the new filename.

### Changed

- **Shared view layout primitives** (`frontend/src/components/layout/PageLayout.tsx`, `frontend/src/components/layout/DiscoveryViewLayout.ts`). Eight top-level views in `frontend/src/views/` had been hand-rolling near-identical page chrome (`PageContainer`, `ContentContainer`, `HeroTitle`, `HeroSubtitle`, `StatsContainer`, `SectionHeader`, `SectionTitle`, `EmptyStateWrapper`); two more (`GlobalDiscussions`, `DiscoverSearchResults`) duplicated the discussion-style page primitives. Pure refactor with no functional or layout changes — net **-321 lines** (583 deleted from views, 262 added across two new primitive modules). `SectionHeader` exposes `$gap` and `$wrap` transient props so DiscoveryLanding's section rows preserve their pre-refactor single-line layout (no gap, no wrap), while the four list views keep their canonical `gap: 16px; flex-wrap: wrap` defaults.

- **Consolidated inline corpus moderation checks onto `Corpus.user_can_moderate`** (issue #1450, cluster 3). `config/graphql/conversation_queries.py:467-505` (`resolve_moderation_action`) and `:514-550` (`resolve_moderation_metrics`) each repeated the same hand-rolled `if not user.is_superuser: is_owner = corpus.creator == user; is_moderator = corpus.moderators.filter(user=user).exists(); if not is_owner and not is_moderator: return None` block — the exact "drift" pattern the issue calls out. Extracted into a single `Corpus.user_can_moderate(user)` instance method on `opencontractserver/corpuses/models.py:670` that owns the canonical superuser/creator/designated-moderator decision and short-circuits anonymous / `None` users. Behavior is preserved verbatim — the helper intentionally counts ANY `CorpusModerator` row (regardless of the `permissions` list), matching the previous queries; the divergence with the stricter `Conversation.can_moderate()` (which additionally requires `bool(moderator.permissions)`) is documented in the helper's docstring and tracked separately so this consolidation PR doesn't double as a behavior-change PR. Regression coverage in `opencontractserver/tests/test_corpus_user_can_moderate.py`: superuser, creator, designated-moderator-with-permissions, designated-moderator-with-empty-permissions, outsider, anonymous, `None`, and a cross-corpus leak guard.

- **Cookie consent dialog redesign** (`frontend/src/components/cookies/CookieConsent.tsx`). Two-column data grid for the "Data we collect" and "Data you agree to share" sections, semantic HTML (`<section>`, `<aside>`, `<ul>`), responsive collapse to a single column below 720px, and an icon badge in the header. Adds tokens to `frontend/src/assets/configurations/osLegalStyles.ts` (`OS_LEGAL_SPACING.modalMaxWidth`, `modalSideGutter`, `borderRadiusListItem`, `borderAccentWidth`, `iconBadgeDesktop`, `iconBadgeMobile`; `OS_LEGAL_SHADOWS.modalOverlay`) and `COOKIE_CONSENT_GRID_BREAKPOINT` in `frontend/src/assets/configurations/constants.ts` so the modal no longer relies on inline magic numbers. Disclaimer text is held at 0.75rem (12px) to meet the WCAG-recommended legal-text minimum, and `DataListItem` text colour was raised to `textSecondary` for AA contrast on `surfaceLight`.

### Fixed

- **Leaderboard leaked raw OAuth `provider|sub` IDs into the USER column + polled aggressively** (issue #1557). Two related defects on the Leaderboard / Top Contributors surfaces.

  - **A. Raw OAuth IDs in USER column.** `frontend/src/components/community/Leaderboard.tsx:570` rendered `entry.user.username` directly, and `frontend/src/components/landing/CompactLeaderboard.tsx:398` rendered `contributor.username`. For social-login users, `User.username` is the Auth0 `sub` claim (e.g. `google-oauth2|114688257717759010643`) — see `config/graphql_auth0_auth/utils.py:291` (`jwt_get_username_from_payload_handler`) and `:227` (`get_or_create(username=remote_username)`). Fix: added a `displayName` resolver to `UserType` (`config/graphql/user_types.py`) that resolves in priority `name` → `given_name` + `family_name` → `first_name` + `last_name` → `username` → redacted `user_<last 6 chars of sub>` fallback. The redaction branch fires only when `is_social_user=True` (not just when `|` appears in the username), because the project's `UserUnicodeUsernameValidator` (`opencontractserver/users/validators.py`) explicitly allows `|` in locally-chosen usernames — so a local user named e.g. `alice|admin` is legitimate and must not be redacted. The redacted suffix is taken from after the last `|`, so short subs like `auth0|abcde` redact to `user_abcde` with no `|` leak; suffix length lives in `OAUTH_SUB_DISPLAY_SUFFIX_LENGTH` in `opencontractserver/constants/auth.py`. The raw `sub` is never returned. `GET_LEADERBOARD` (`frontend/src/graphql/queries/leaderboard/queries.ts`) and `GET_DISCOVERY_DATA` (`frontend/src/graphql/landing-queries.ts`) now select `displayName` instead of `username`; both leaderboard components and their TS types (`frontend/src/types/leaderboard.ts`, `LeaderboardEntry` in `landing-queries.ts`) read `displayName`. Backend coverage in `opencontractserver/tests/test_user_display_name.py` pins down the priority order, the social-user gate, and the redaction guarantees so a regression cannot quietly re-expose the raw `sub` or falsely redact a local username.
  - **B. Over-aggressive polling.** `Leaderboard.tsx` had `pollInterval: 60000` on `GET_LEADERBOARD` and `pollInterval: 120000` on `GET_COMMUNITY_STATS` — the queries refetched on a fixed clock regardless of tab visibility, re-rendering rows and resetting in-flight UI state (rank chip animations, hover state). Compounded by the fact that `metric` / `scope` / `limit` are query variables, so every filter change _also_ issued a fresh request on top of the cadence. Fix: dropped both `pollInterval`s; queries now use `fetchPolicy: "cache-and-network"` + `nextFetchPolicy: "cache-first"` + `notifyOnNetworkStatusChange: false` so the user sees instant cached data while a single background refresh runs on mount or filter change. A `visibilitychange` listener performs one refetch when the tab returns to the foreground — hidden tabs no longer hit the network at all.
  - **C. PII-gated `email` resolver.** Even with the leaderboard query no longer selecting `email`, `UserType` is a `DjangoObjectType` so the field would auto-resolve to any client that selected it on a public `user` subtree. Added a `resolve_email` method on `UserType` (`config/graphql/user_types.py`) that returns `None` unless the requester is authenticated AND (viewing themselves OR a superuser). The `me` query and admin tooling continue to see the real address. Coverage: `EmailResolverTestCase` in `opencontractserver/tests/test_user_display_name.py` pins the self-only / superuser / cross-user / anonymous / blank-email / context-missing matrix. Frontend follow-through: `email` is dropped from the `LeaderboardUser` GraphQL selection set, the TypeScript `LeaderboardUser` interface, and all leaderboard test mocks (`Leaderboard.ct.tsx`, `LeaderboardTestWrapper.tsx`).
  - **D. Avatar initials regression on leading-whitespace single-token names.** `getLeaderboardInitials` (`frontend/src/utils/leaderboardAvatar.ts`) used `name.substring(0, 2)` on the un-trimmed input in the single-token branch, so `"  alice"` rendered as `"  "` instead of `"AL"`. Fixed by using the trimmed token; whitespace-only names now return `"?"` instead of falling through to silent whitespace. Test coverage in `frontend/src/utils/__tests__/leaderboardAvatar.test.ts`.
  - **E. `useTabVisibilityRefresh` no longer silently swallows errors in production.** Promise rejections and synchronous throws from a refresh callback (e.g. Apollo query torn down between visibility transitions, network outage) were only logged in dev. Switched to unconditional `console.error` so consistently failing refreshes surface in production browser consoles / Sentry. Test updates in `frontend/src/hooks/__tests__/useTabVisibilityRefresh.test.ts`.

- **SPA layout: redundant 100vh on inner wrapper exposed gradient strip before footer** (`frontend/src/App.tsx`, issue #1558). The shell stacked three full-viewport-height constraints — `minHeight: 100vh` on the outer wrapper, a second wrapper with `minHeight: 100vh` + `overflow: hidden`, and a third with `minHeight: 100vh` + `maxHeight: 100vh` + `height: 100vh` + `overflow: hidden` — clamping the routed content area to exactly the viewport. On short pages (e.g. discovery landing with the empty-state CompactLeaderboard) the `CallToAction` panel that follows in the natural document flow was clipped to a thin band rather than shown in full, producing a "blue-to-purple gradient strip" between a flat light-gray gap and the dark footer. The footer wrapper compensated with a `marginTop: -1.5rem` hack to hide the resulting visual seam. Fix: collapse the layout to one `minHeight: 100vh` flex column on the outer wrapper, a single inner flex shell with `flex: 1, minHeight: 0`, and `flex: 1` on the `AppContainer` main slot so it grows to push the footer down naturally. Drop `overflow: hidden` from inner wrappers and the `-1.5rem` margin compensation. Pages that scroll internally (DiscoveryLanding, Leaderboard, Documents, Corpuses, etc.) keep working because they own their own `height: 100%; overflow-y: auto` containers; pages without internal scroll containers (PrivacyPolicy, TermsOfService) now scroll the body instead of being clipped. Also drops `justifyContent: center` from the outer wrapper, which would have negatively offset overflow content if any page exceeded 100vh under the new flex flow. Also drops the carried-over `minWidth: 100vw` from `APP_CONTAINER_STYLE` — `100vw` includes the scrollbar width but the parent flex container does not, so under the new flow (no `overflow: hidden` to mask it) it would have forced a horizontal scrollbar on systems with non-overlay scrollbars whenever vertical scroll was active; `width: 100%` already fills the parent. Regression coverage: layout invariants are pinned in `frontend/src/styles/__tests__/appShellLayout.test.ts` (style constants) and `frontend/src/components/layout/__tests__/AppShell.test.tsx` (rendered DOM); the modal stack that hangs off the shell — `SelectAnalyzerOrFieldsetModal`, `DocumentUploadModal`, the EDIT/VIEW `CRUDModal` pair — was also extracted into `frontend/src/components/layout/AppDocumentModals.tsx` so the conditional/ternary branches that decide which modal renders can be unit-tested without standing up the routing tree (`frontend/src/components/layout/__tests__/AppDocumentModals.test.tsx`).

- **`user_has_permission_for_obj` per-row N+1 on Document GraphQL resolvers** (`opencontractserver/utils/permissioning.py`, `opencontractserver/shared/Managers.py`, `config/graphql/permissioning/permission_annotator/mixins.py`, issue #1555). `DocumentType.canViewHistory` (`config/graphql/document_types.py:879-897`) and `DocumentType.canRetry` (`config/graphql/document_types.py:927-957`) — both selected by the shared `GET_DOCUMENTS` query — were calling `user_has_permission_for_obj` per row, and that helper's underlying `get_users_permissions_for_obj` issued `getattr(instance, "<model>userobjectpermission_set").filter(user_id=user.id)` and `…groupobjectpermission_set.filter(group_id__in=…)` on each call. `.filter(...)` on a related manager bypasses any default `prefetch_related` cache, so the prefetches set up by `_apply_document_prefetches` (`_prefetched_user_perms`, `documentgroupobjectpermission_set__permission/__group`) were silently ignored — every paginated `documents` page was charged 2-3 extra DB round trips per row. The same bypass affected `resolve_my_permissions` for group permissions. Fix:

  - `_apply_document_prefetches` now writes user-id-suffixed attributes — `_prefetched_user_perms_uid_<user.id>` and `_prefetched_user_group_perms_uid_<user.id>` — both pre-joined to `permission` via `select_related`. The group-perm prefetch is now filtered to the user's own group ids at the SQL layer (replacing the unfiltered `documentgroupobjectpermission_set__permission/__group` lookups), so the fetched row count is bounded by the user's group membership rather than the total number of groups with perms on the document.
  - `get_users_permissions_for_obj` now consumes those prefetched lists directly (reading `perm.permission.codename` from the in-memory rows) when the user-id-suffixed attribute is present. The user-id suffix is what makes the cache safe to consume: a lookup for a different user finds no attribute and cleanly falls through to the legacy guardian path.
  - `resolve_my_permissions` updated to read from the same attributes — eliminates the second N+1 it was paying via its own `…groupobjectpermission_set.filter(group_id__in=…)` call.
  - Regression coverage in `opencontractserver/tests/permissioning/test_permissioned_querysets.py::UserHasPermissionForObjPrefetchFastPathTest`: pins zero queries across 5 documents for both direct and group-permission lookups, verifies the fallback path still works for instances loaded without the prefetch, and verifies a different-user lookup correctly falls through (the suffix makes the cache safe under multi-user lookups).

- **Loading-overlay UX regression carried over to seven other infinite-scroll lists** (follow-up to PR #1565 / issue #1559). PR #1565 fixed the `LoadingOverlay`-during-`fetchMore` antipattern in `frontend/src/views/Documents.tsx` only, but seven sibling infinite-scroll views/components copied the same shape and exhibited the same bug: a full-cover `LoadingOverlay` (most of them `inverted`, painting a near-black band) was shown for the entire `loading === true` window, including `fetchMore` requests, dimming or covering already-rendered rows mid-scroll. Fixed in: `frontend/src/views/Extracts.tsx`, `frontend/src/views/LabelSets.tsx`, `frontend/src/components/exports/ExportList.tsx`, `frontend/src/components/extracts/ExtractCards.tsx`, `frontend/src/components/corpuses/CorpusListView.tsx`, `frontend/src/components/analyses/AnalysesCards.tsx`, `frontend/src/components/documents/DocumentCards.tsx`. Each call site now scopes `LoadingOverlay.active` to the initial load only (`loading && filteredItems.length === 0`, or for `DocumentCards` the existing `showEmptyState` gate which already encodes "no items and no folders") and drops the `inverted` flag so the dark variant is reserved for genuinely blocking states. To DRY the spinner that replaces the overlay during `fetchMore`, the `FetchMoreFooter` styled-component originally inlined in `Documents.tsx` was extracted to `frontend/src/components/widgets/infinite_scroll/FetchMoreFooter.tsx` (a small, accessible `role="status" aria-live="polite"` strip with a Lucide `Loader2` icon and an entity-specific message like "Loading more extracts…"), and Documents.tsx was migrated to consume the shared component while preserving its existing `data-testid="documents-fetch-more-spinner"`. Each new call site gets a parallel `data-testid` (`extracts-fetch-more-spinner`, `labelsets-fetch-more-spinner`, `exports-fetch-more-spinner`, `extract-cards-fetch-more-spinner`, `corpuses-fetch-more-spinner`, `analyses-fetch-more-spinner`, `document-cards-fetch-more-spinner`) so future regression tests have a stable hook. The shared `FetchMoreOnVisible` stale-closure / `rootMargin` fixes from PR #1565 already covered every consumer because the bug was in the shared component itself; this follow-up only covers the _overlay_ half of the regression for the consumers PR #1565 didn't touch.

- **Documents view (`/documents`) infinite scroll never fires; dim overlay obscures grid during load** (issue #1559). Two cooperating bugs prevented the second-and-later pages from loading and made the page look broken on the first paint:

  1. `frontend/src/views/Documents.tsx` rendered an `inverted` full-cover `LoadingOverlay` over the grid container for the entire `documents_loading === true` window — including refetches and `fetchMore` — which painted a near-black band over already-rendered rows. Replaced with: (a) an unobtrusive non-inverted overlay scoped to the _initial_ load only (`documents_loading && filteredDocuments.length === 0`), and (b) a small footer-pinned spinner ("Loading more documents…", `data-testid="documents-fetch-more-spinner"`, `role="status"` for a11y) that renders below the grid only while a `fetchMore` request is in flight and there is still a next page. The dark/inverted variant is now reserved for genuinely blocking states.
  2. `frontend/src/components/widgets/infinite_scroll/FetchMoreOnVisible.tsx` had two compounding issues: its `useEffect` deps were `[entry, vertical, inView]` and did **not** include the `fetchNextPage` / `fetchPreviousPage` callbacks, so once Apollo's `loading` flag toggled true→false during the first `fetchMore`, the parent re-created its `useCallback` handler and the observer's effect kept calling the stale closure (which captured `loading=true` or a now-replaced Apollo handle) — silently dropping every subsequent fetch. Additionally, the `useInView` config had no `rootMargin`, so the sentinel only fired when the user reached the absolute bottom. Fix: hold the latest callbacks in `useRef` slots that are refreshed every render and dereferenced inside the effect, so observer events always invoke the current closure regardless of how many times the parent's handler identity has churned mid-flight; and default `rootMargin` to `"200px 0px"` so the sentinel triggers ~one viewport's worth before the absolute bottom. The new `rootMargin` prop is opt-in for callers that need a different value; existing call sites (`AnnotationList`, `ExportList`, `ExtractCards`, `SearchResults`, `UnifiedContentFeed`, `ConversationListView` ×2, `CorpusListView`, `AnnotationsPanel`, `ThreadList`, `AnalysesCards`, `DocumentMetadataGrid`, `DocumentCards`, `Extracts`) inherit the safer default with no API change.
  3. Regression coverage: new Playwright CT in `frontend/tests/Documents.ct.tsx` ("Documents View - Infinite Scroll (issue #1559)") mounts the view with a 20-doc first page (`hasNextPage: true`, `endCursor: "cursor-page-1-end"`) and a single distinctively-named second-page mock keyed on `{ limit: 20, cursor: "cursor-page-1-end" }`, scrolls the `.FetchMoreOnVisible` sentinel into view, and asserts the second-page document title appears. The test uses an inline `InMemoryCache` with the same `relayStylePagination` keyArgs as `frontend/src/graphql/cache.ts` so `fetchMore` merges into the existing connection. On the pre-fix code the second mock is never consumed (stale `handleFetchMore` closure no-ops on the captured `documents_loading=true`), so the assertion times out.

- **`DocumentType.versionCount` per-row N+1** (`config/graphql/document_types.py:632`, `opencontractserver/documents/query_optimizer.py`, closes #1554). `resolve_version_count` was issuing one `COUNT(*)` query per document row (`Document.objects.filter(version_tree_id=self.version_tree_id).count()`), so any GraphQL query that selected `versionCount` on a paginated documents connection paid up to 100 extra COUNT queries per page. The shared `GET_DOCUMENTS` query selects this field, so every active consumer (`CorpusDocumentCards`, `SelectDocumentsModal`, `DocumentRelationshipModal`, corpus tabs, metadata workflow) was hit. Fix mirrors the request-level cache pattern already established for `doc_relationship_count` in the same file (`document_types.py:395-432`):

  - New `DocumentVersionQueryOptimizer.get_version_counts_by_tree` (`opencontractserver/documents/query_optimizer.py`) computes `{version_tree_id: count}` for every visible tree in **one** aggregated SQL query and caches the result on `info.context`. Subsequent per-row resolvers in the same GraphQL request collapse to dict lookups.
  - The aggregation is scoped to `Document.objects.visible_to_user(user)` so the badge cannot leak the existence of versions hidden from the requesting user — the previous unscoped `.count()` would have surfaced private versions in the visible count.
  - Resolver returns `1` (not `0`) when the tree is absent from the cache: the resolver is only reachable on a document the user can already see, so its tree always has at least one visible version.
  - Out of scope (tracked separately): `canViewHistory` / `canRetry` per-row permission checks.
  - Regression coverage: `opencontractserver/tests/test_document_version_count_optimizer.py` covers the normal multi-tree case, partial-visibility (count must collapse to only the visible subset), outsider (no counts surface), superuser, request-level cache memoisation, and an end-to-end integration test that resolves `versionCount` on 15 documents in a shared request and asserts no `version_tree_id` SQL runs after the first batched aggregation.

- **Browse Annotations (`/annotations`): tall dark loading band + image cards rendering "Failed" + misleading Doc Labels stat** (issue #1560).

  - **Loading band (issue A).** `frontend/src/components/annotations/AnnotationsPanel.tsx` previously rendered a `LoadingOverlay` with `inverted` (rgba(0,0,0,0.85)) anchored by `position: absolute; inset: 0` over a flex container that had no positive `min-height` while `items` was empty — producing a tall, near-black band against the otherwise light page. The overlay is replaced with shimmer-animated card-shaped skeletons (`AnnotationCardSkeleton`) that match the eventual `ModernAnnotationCard` two-up grid layout (label row, label color, body, footer). Skeletons participate in normal flow, so the grid has predictable height while data loads. The now-unused `loadingMessage` prop is dropped from `AnnotationsPanelProps` and from both call sites (`Annotations.tsx`, `CorpusAnnotationCards.tsx`).
  - **"Failed" image cards (issue B).** `frontend/src/components/annotator/hooks/useAnnotationImages.tsx` flagged any non-200 REST response as `error`, which the preview rendered as a dashed-border "Failed" placeholder on every image annotation when the JWT wasn't yet in the `authToken` reactive var (the `IsAuthenticated`-gated `/api/annotations/{id}/images/` endpoint then returns 401). The hook now also tracks `hasFetchedEmpty` and treats 401/403/404/429 the same as a successful empty payload — the backend's IDOR-safe contract already returns `{images: []}` for missing/unauthorized/throttled, so we surface the same gap as a neutral placeholder rather than an alarming failure. `AnnotationImagePreview` accepts the new prop and renders a `NoThumbnailState` (image icon + "No thumbnail" + dashed border + "No thumbnail generated for this annotation" tooltip) for the empty-success case while keeping the genuine `error` branch for 5xx/network/parse failures. Both consumers — `ModernAnnotationCard` (Browse Annotations cards) and `HighlightItem` (sidebar) — wire `hasFetchedEmpty` through. Cache-poisoning fix: 401/403/429 are explicitly **not** memoized in the module-level `imageCache`, so a render that lands before the JWT resolves doesn't lock the annotation into "no thumbnail" forever — only 404 (permanently no thumbnail row) and 200-empty are cached. `AnnotationImagePreview` short-circuits to `null` while `images === null` so the placeholder doesn't flash before the loading spinner appears. New regression tests in `frontend/src/components/annotator/hooks/__tests__/useAnnotationImages.test.tsx` pin the matrix: no fetch when not IMAGE, empty 200 → `hasFetchedEmpty`, 401/404 → `hasFetchedEmpty`, 500 → `error`, populated 200 → images, plus the cache-policy guards (401 → re-fetches on next mount, 404 → second mount skips network). The `HighlightItem.scroll` test mock and the `AnnotationsPanel` Loading State test were updated to match the new contract.
  - **Misleading Doc Labels stat (issue C).** `frontend/src/views/Annotations.tsx:455` computed `docLabels`/`textLabels`/`humanAnnotated` from the first 20 loaded annotations while `Total` correctly used the server-side `totalCount`. With ~30k annotations the breakdown tiles read "0" and looked authoritative. The `stats` memo now also exposes `loadedCount` and an `isPartial` flag; when partial, the breakdown labels gain an "(in view)" suffix and the tile gets a tooltip ("Counted from X of Y loaded so far. Scroll to load more."). The Total tile is unchanged — it was already accurate.

- **Documents view (`/documents`) refetch storm + over-fetched query** (`frontend/src/views/Documents.tsx`, `frontend/src/graphql/queries.ts`, `frontend/src/services/cacheManager.ts`, `frontend/src/components/widgets/modals/UploadModal/hooks/useUploadMutations.ts`). The top-level Documents view was issuing ~7 `GET_DOCUMENTS` requests on every visit — one from `useQuery` plus six redundant `useEffect(() => refetchDocuments())` hooks watching `current_user`, `location`, `document_search_term`, `filtered_to_label_id`, `filtered_to_labelset_id`, `filtered_to_corpus`, all of which fired on mount. `nextFetchPolicy: "network-only"` forced every refetch to skip the cache. The query itself selected ~22 fields per row when the UI only renders 7 — including the per-row N+1 `versionCount` (`Document.objects.filter(version_tree_id=…).count()` per row), per-row `canViewHistory` / `canRetry` `user_has_permission_for_obj` checks, four file-URL signing fields the list never displays, and the `doc_label_annotations` connection. With `RELAY_CONNECTION_MAX_LIMIT = 100` and no explicit `first:` argument, the first paint was charged for 100 fully-resolved DocumentType rows. Fix:
  - New focused `GET_DOCUMENTS_FOR_LIST` query selects only what the list view renders (`id slug title fileType backendLock pageCount icon created creator { id slug email }`). The shared `GET_DOCUMENTS` is preserved for its other consumers (`CorpusDocumentCards`, `SelectDocumentsModal`, `DocumentRelationshipModal`, corpus tabs, metadata workflow tests).
  - Six redundant `useEffect(refetchDocuments)` hooks deleted. Apollo's `useQuery` already refetches on `variables` change (deep-compared), so the reactive-var values that _should_ drive a refetch are now memoized into `documentVariables` and the ones that shouldn't (`location` re-renders, `current_user` settling, `filtered_to_labelset_id`) no longer trigger one. `filtered_to_labelset_id` is intentionally not a query variable — it only scopes the label-picker dropdown.
  - `nextFetchPolicy: "network-only"` removed; default cache-first behavior restored.
  - Initial query now passes explicit `first: DOCUMENTS_PAGE_SIZE` (20) so first paint matches the paged-load size.
  - `document_items` memoized on the stable Apollo `edges` reference (was being rebuilt via `.map().filter()` on every render, churning `filteredDocuments` / `stats` / `statusFilterItems` deps — same shape of bug fixed in PR #1517 for `CorpusDocumentCards`).
  - Latent bug: query used to ask for `creator { id slug }` while the UI called `getInitials(doc.creator?.email)`; `email` was never selected and avatar initials always defaulted. Fixed in the new query's selection set.
  - `useUploadMutations` and `cacheManager.invalidateRelatedQueries` updated to include `GET_DOCUMENTS_FOR_LIST` alongside `GET_DOCUMENTS` in their refetch lists, so an upload from the Documents page still refreshes the list. Apollo's `refetchQueries` ignores entries with no active observable, so listing both is a safe union.
  - Deleted unused `frontend/tests/DocumentsTestWrapper.tsx` (no consumers in the codebase).
  - Regression coverage: new Vitest source-level test (`frontend/src/views/Documents.refetch-shape.test.ts`) pins the four structural fixes (no `network-only`, no `useEffect(refetchDocuments)`, slim query in use, explicit page-size constant). Behavioral counter-style tests cannot catch the storm because Apollo's built-in query deduplication merges concurrent in-flight requests with the same key into a single network call before they reach `MockLink`.

### Changed

- **Typing: graduated `opencontractserver.llms.*` (non-tools) modules out of the mypy baseline** (issue #1484, follow-up to #1468). Removed the `[mypy-…]` `ignore_errors` blocks for `llms.agents.agent_factory`, `llms.agents.core_agents`, `llms.agents.pydantic_ai_agents`, `llms.agents.timeline_stream_mixin`, `llms.api`, `llms.client`, `llms.vector_stores.core_conversation_vector_stores`, `llms.vector_stores.core_vector_stores`, and `llms.vector_stores.pydantic_ai_vector_stores`. Fixes (~125 errors) included: replacing `callable`/`any`/`dict[str, any]` with `typing.Callable[..., Any]` / `typing.Any`; adding explicit `Optional[...]` to satisfy `no_implicit_optional`; tightening `_resolve_framework`/`_resolve_tools` helpers in `llms.api`; typing `Annotation`/`ChatMessage` querysets as their custom `AnnotationQuerySet`/`ChatMessageQuerySet` so the `search_by_embedding` extension method resolves; switching `CorpusAgentContext.documents` to a non-Optional `list[Document]` with `default_factory=list`; aligning `CoreAgent.stream` / `CoreAgent.resume_with_approval` Protocol signatures with their PydanticAI implementations; and pruning unused `# type: ignore` comments. Pre-existing errors outside `llms.*` are untouched.

- **`opencontractserver.tasks.*` graduated out of the mypy baseline** (issue #1482, refs #1447). All sixteen task modules (`agent_tasks`, `analyzer_tasks`, `badge_tasks`, `corpus_tasks`, `data_extract_tasks`, `doc_analysis_tasks`, `doc_tasks`, `embeddings_task`, `export_tasks`, `export_tasks_v2`, `extract_orchestrator_tasks`, `fork_tasks`, `import_tasks`, `import_tasks_v2`, `lookup_tasks`, `memory_tasks`) now type-check cleanly. Their `[mypy-…]` blocks are removed from `mypy.ini` and the corresponding 227 lines are pruned from `docs/typing/mypy_baseline.txt`. Additional tightening: `LabelLookupPythonType.{text_labels,doc_labels}` narrowed from `dict[str | int, …]` to `dict[str, …]` (the producer in `utils.etl.build_label_lookups` already emits string keys), and the `corpus_id` parameter on `build_label_lookups` was corrected to `int`. `utils.importing.{load_or_create_labels, import_doc_annotations}` accept `Mapping` instead of `dict` so callers can pass `OpenContractDocExport` / `AnnotationLabelPythonType` TypedDicts without `cast()`.

- **Typing: graduate `opencontractserver.conversations.models` from mypy baseline** (refs Issue #1447, closes #1478): removed the last remaining `[mypy-opencontractserver.conversations.models]` `ignore_errors` block from `mypy.ini` and pruned 29 baseline error lines from `docs/typing/mypy_baseline.txt`. Per-file fixes in `opencontractserver/conversations/models.py`:

  - Replaced the `Optional["AbstractBaseUser"]` parameter on `ConversationQuerySet.visible_to_user` and `ChatMessageQuerySet.visible_to_user` with `"UserModel | AnonymousUser | None"` (forward-referenced via `TYPE_CHECKING` import of `opencontractserver.users.models.User as UserModel`). Cleared the 2× `AnonymousUser → AbstractBaseUser | None` assignment errors at the `if user is None: user = AnonymousUser()` reassignment, the 6× `[union-attr]` errors on `.is_superuser` / `.is_anonymous` / `.id`, and the `Incompatible type for lookup 'creator'` error on `Document.objects.filter(creator=user)` (after narrowing past the `is_anonymous` early-return, mypy resolves user to `User`).
  - Aligned `SoftDeleteManager.visible_to_user`, `ConversationManager.visible_to_user`, and `ChatMessageManager.visible_to_user` signatures with `BaseVisibilityManager.visible_to_user` (added `lightweight: bool = False` and `with_doc_label_annotations: bool = False` kwargs forwarded to `super()` for parity), eliminating the 3× `Signature of "visible_to_user" incompatible with supertype` `[override]` errors without resorting to `# type: ignore[override]`.
  - Stopped reassigning `base_qs` after `.annotate(...)` in `ConversationQuerySet.search_by_embedding` and `ChatMessageQuerySet.search_by_embedding`; the reassignment surfaced a `ConversationQuerySet[Model, Model]` vs `ConversationQuerySet[_Model, _Row]` (and the `ChatMessageQuerySet` equivalent) generic mismatch from django-stubs because `.annotate` widens the row TypeVar. The function now uses a separate `annotated_qs` local for the post-annotate pipeline.
  - Scoped `# type: ignore[misc]` on the `objects = ConversationManager()` and `objects = ChatMessageManager()` class-variable overrides (django-stubs flags re-declaring `BaseOCModel.objects`; the manager substitution is the documented mechanism for plugging in custom QuerySets).
  - Replaced narrowing `assert self.message is not None` in `ModerationAction.__str__` with a defensive `if/else` guard so the fallback path is exercised under `python -O` and covered by tests.
  - Lifted `from django.contrib.auth.models import AnonymousUser` from three local function imports to a module-level import so the type annotation does not need a `TYPE_CHECKING` indirection.

  Verified locally with `mypy --config-file mypy.ini opencontractserver config` reporting "Success: no issues found in 1032 source files".

- **Typing: graduated 11 `opencontractserver.utils.*` modules out of the mypy baseline** (chunk 2 of issue #1481, follow-up to #1331/#1447). Modules graduated and `ignore_errors` removed from `mypy.ini`: `export_v2`, `files`, `import_v2`, `importing`, `logging`, `mention_parser`, `multimodal_embeddings`, `packaging`, `pdf_token_extraction`, `sharing`, `storages`. 54 baseline error entries pruned from `docs/typing/mypy_baseline.txt`. Highlights:

  - V2 export builders now emit narrowly typed `TypedDict` payloads (`StructuralAnnotationSetExport`, `CorpusFolderExport`, `OpenContractsRelationshipPythonType`, `DescriptionRevisionExport`).
  - `packaging.unpack_corpus_from_export` discriminates V1 vs V2 input via the `OpenContractCorpusV2Type` overlay; the V1/V2 detection key is the presence of `post_processors` (legacy V1 exports never carry it).
  - `multimodal_embeddings` propagates `PawlsTokenPythonType` through the image-token pipeline.
  - `import_v2` and `packaging` use `TYPE_CHECKING` `User` aliases so `User = get_user_model()` is no longer used as a runtime annotation.
  - `sharing.make_analysis_public` now guards against analyses with no `analyzed_corpus` and returns a graceful error response instead of crashing on `AttributeError`.
  - `files.split_pdf_into_images` preserves its `Literal["PNG", "JPEG"]` parameter narrowing across the case-normalisation step.
  - **Latent bug fix uncovered by typing**: `packaging.unpack_label_set_from_export` and `unpack_corpus_from_export` had inverted `isinstance(user, str)` checks on parameters typed `int | UserModel`; the `int` branch was unreachable. Pinned by `opencontractserver/tests/test_packaging_typing_bug_fixes.py`.
  - Pre-commit `mypy` continues to run clean on the full project surface (`opencontractserver` + `config`).

- **typing: graduate `opencontractserver.mcp.server` and `opencontractserver.mcp.tools` from the mypy baseline** (issue #1486, sub-issue of #1447). Both modules pass `mypy` cleanly and their `[mypy-...]` blocks are removed from `mypy.ini`; the corresponding 7 entries are pruned from `docs/typing/mypy_baseline.txt`. Real-typing changes (no `# type: ignore` shortcuts where a proper annotation would do):

  - `opencontractserver/mcp/server.py` — `resources` now annotated as `list[Resource]`; resource `uri` strings wrapped in `pydantic.AnyUrl(...)` to match `Resource.uri`'s `Annotated[AnyUrl, ...]` field; `MCPLifespanManager._run_context` and `ScopedMCPLifespanManager._run_context` typed as `AbstractAsyncContextManager[None] | None` instead of bare `None`, fixing the `__aenter__` / assignment errors; ASGI `app(scope, receive, send)` and the inner `_handle_mcp_request` now use a shared `MutableMapping[str, Any]`-based `ASGIScope`/`ASGISend`/`ASGIReceive` alias compatible with Starlette and `StreamableHTTPSessionManager.handle_request`; `handle_sse_connection` annotated as `(request: Request) -> Response`; `__init__` and `main` get explicit `-> None` returns.
  - `opencontractserver/mcp/tools.py` — single `# type: ignore[attr-defined]` on `.search_by_embedding(...)` (mirrors the existing pattern in `opencontractserver/discovery/views.py`); django-stubs cannot model the custom `DocumentQuerySet.search_by_embedding` through the `from_queryset(...)` manager. `_text_search_fallback` parameters typed (`corpus: Corpus`, `user: User | AnonymousUser`).
  - Cross-module ripples to keep ASGI types honest (helper signatures previously used `dict[str, Any]`, which is invariant in `Callable` argument position and would have rejected our `MutableMapping`-based `send`/`receive`): `config/ratelimit/keys.py:139` (`get_client_ip_from_scope`), `config/ratelimit/decorators.py:371` (`check_mcp_rate_limit`), and `opencontractserver/mcp/telemetry.py:320` (`get_user_agent_from_scope`) widened to `MutableMapping[str, Any]`. `opencontractserver/corpuses/folder_service.py` — `check_corpus_read_permission` and `get_corpus_documents` widened from `user: User` to `user: User | AnonymousUser`. Both methods already handled the anonymous case at runtime via `user.is_anonymous`/`user.is_superuser` branches; the type widening just makes the public-corpus MCP entry points type-check without casts.

  Per the issue's recommendation, `opencontractserver/mcp/tests/test_mcp` (the 319-error worst-offender file) is intentionally **not** included here — it's tracked for a dedicated follow-up PR so the runtime-code graduation lands first and clean.

- **Docker image hardening follow-ups to PR #1500** (issue #1505). Six review observations on the slim-production-image PR are addressed without altering the deployed runtime semantics:

  - **Python pinned to `3.11.15-slim-bookworm`** in both `compose/production/django/Dockerfile` and `compose/local/django/Dockerfile` (was the floating `3.11-slim-bookworm` alias). Rebuilds on different dates can no longer silently pick up a new CPython point release that changes runtime behaviour. Bumps are now an intentional edit, not a side effect of the calendar.
  - **spaCy model install switched to the pinned-URL `wget` pattern in production** (`compose/production/django/Dockerfile`). The runtime stage previously called `python -m spacy download en_core_web_sm` / `en_core_web_lg`, which resolves the wheel URL at build time and could silently drift to a different patch. The production build stage now downloads the same explicit GitHub release URLs already used in `compose/local/django/Dockerfile` into `/usr/src/app/wheels`, with the same retry semantics; the runtime stage installs them via the existing `pip install --no-index --find-links=/wheels/` path and runs a `spacy.load(...)` smoke test. `wget` is added to the build stage only — it never reaches the runtime image.
  - **Release workflow split for the Django image** (`.github/workflows/docker-build-release.yml`). The previous step combined `load: true` (so the size-budget gate could `docker image inspect` the result) with `push: true`. That combination only works on single-platform builds, is not well-documented across Buildx versions, and some runner/daemon configurations reject it. The Django image is now built with `load: true, push: false`, gated on the 1.5 GiB size budget, then pushed via `docker push <tag>` per tag — the daemon already has the layers from the load step, so no rebuild occurs. Non-Django images keep the original combined `push: true, load: false` path.
  - **`.dockerignore` documents why `locale/` is safely excluded** — the tree contains no `.po`/`.mo` files (only a placeholder `README.rst`) and no `compose/*/start*` script invokes `manage.py compilemessages`. The runtime image still installs `gettext` so future i18n work has a build hook ready; if `.po` files are added, drop the `locale/` line so the build context picks them up.
  - **`brotli>=1.1.0` floor rationale documented** in `requirements/base.txt`. The constraint was not chosen incidentally — 1.1.0 is the current major-version line under active upstream maintenance. The underlying `_brotli` C-extension API that `httpx` looks up has been stable since 1.0.x, so a future transitive constraint can lower the floor safely if needed.
  - **`ffmpeg` removal verified** by an exhaustive `grep` across `opencontractserver/`, `compose/`, `config/`, `frontend/`, and the workflow tree: no Python `import`, `subprocess`, or shell invocation of `ffmpeg` exists. The PR #1500 removal of the apt package is safe; this entry records the verification trail rather than re-removing anything.
  - **SHA-256 checksums added for spaCy wheel downloads** in both `compose/production/django/Dockerfile` and `compose/local/django/Dockerfile`. Pinning to an explicit release URL is good but without a hash check the build is still vulnerable to a GitHub release being retroactively modified or a CDN MITM. Each `wget` step is now followed by a `sha256sum -c` verification that fails the build immediately if the fetched wheel doesn't match the expected digest.
  - **Size-budget gate confirmed intact** after the `id: build` step-ID removal. The gate at `.github/workflows/docker-build-release.yml:93` uses `${{ steps.meta.outputs.tags }}` (not the removed `steps.build.outputs.imageid`) to identify the locally-loaded image for `docker image inspect`, so the split build path works correctly.

- **Graduated remaining `config.graphql.*` modules from the mypy baseline** (issue #1485, sub-issue of #1447). Removed the per-module `ignore_errors` blocks for the final 14 modules (`annotation_mutations`, `annotation_queries`, `base`, `conversation_queries`, `corpus_mutations`, `document_mutations`, `document_types`, `extract_mutations`, `extract_queries`, `label_mutations`, `permissioning.permission_annotator.{middleware,mixins,utils}`, `social_queries`) and pruned 174 corresponding lines from `docs/typing/mypy_baseline.txt`. The entire `config.graphql.*` surface is now type-checked. Notable code changes made in service of typing:

  - `AnnotatePermissionsForReadMixin` declares `_meta` / `is_public` under `TYPE_CHECKING` so the mixin no longer trips `attr-defined` errors when combined with a `DjangoObjectType` (`config/graphql/permissioning/permission_annotator/mixins.py`).
  - All `from_global_id(kwargs.get("id", None))[1]` Relay node resolver patterns rewritten to `from_global_id(kwargs["id"])[1]` (the `id` arg is required for relay, so the `.get(..., None)` was dead defensiveness that produced an `Any | None` mypy can't accept).
  - Optimizer call-sites (`AnnotationQueryOptimizer.get_document_annotations`, `RelationshipQueryOptimizer.*`, `BadgeQueryOptimizer.check_user_badge_visibility`) now coerce the `from_global_id` string pk to `int` instead of relying on Django's lenient `pk` lookup.
  - PEP 484 implicit-Optional defaults converted to explicit `T | None` in `corpus_mutations.CreateCorpusAction.mutate`, `corpus_mutations.UpdateCorpusAction.mutate`, and `document_mutations.StartCorpusExport.mutate`.
  - Removed dead `language_model_id` parameter and assignment in `UpdateColumnMutation` — the `language_model` field was deleted from `Column` in migration `0012_auto_20240622_2225` and the parameter was unreachable from the mutation's `Arguments`.
  - `timezone.timedelta(...)` (django-stubs no longer re-exports `timedelta`) replaced with a direct `from datetime import timedelta` in `conversation_queries.resolve_corpus_moderation_stats`.

- **Type checking: graduated `opencontractserver.discovery.tests.test_discovery_views`** (`opencontractserver/discovery/tests/test_discovery_views.py`, closes #1479, refs #1447). Removed the module's `[mypy-…]` `ignore_errors` block and pruned the corresponding 44 baseline entries from `docs/typing/mypy_baseline.txt`. The 49 errors mypy reported were all of the same shape — class-level attributes assigned inside `setUpTestData(cls)` that mypy doesn't follow because the assignments live in a `@classmethod` body. Fix uses Option 1 from the issue: declare the attributes as class-level annotations (`owner: User`, `public_corpus: Corpus`, etc.) on each affected test class (`LlmsTxtTest`, `LlmsFullTxtTest`, `SitemapXmlTest`, `WellKnownMcpTest`, `SearchApiTest`) so mypy sees the names without changing runtime behavior. Replaced `User = get_user_model()` with a direct `from opencontractserver.users.models import User` so the annotations resolve to the real model class (the dynamic `get_user_model()` callable returns `type[AbstractBaseUser]`, which mypy cannot use as a type).

- **Consolidated corpus-modify permission checks behind `user_can_modify_corpus`** (`opencontractserver/utils/permissioning.py`, refs #1450). Four GraphQL mutations were repeating the same inline `corpus.creator == user or user_has_permission_for_obj(user, corpus, UPDATE, include_group_permissions=True)` check (`config/graphql/badge_mutations.py:91, 211, 328` and `config/graphql/document_mutations.py:416`). The pattern is the cluster-2 anti-pattern flagged in the issue: as the permission model evolves (sharing, group inheritance, MIN(doc, corpus)) inline copies drift apart. Replaced each with `user_can_modify_corpus(user, corpus)`, which encodes the canonical "is_superuser OR creator OR explicit guardian UPDATE" semantics in one place. Behaviorally identical: superuser was already admitted via `user_has_permission_for_obj` (returns True for superusers on guardian-enabled models); the helper just makes the short-circuit explicit. Anonymous and `None` callers correctly return `False`. New tests in `opencontractserver/tests/test_user_can_modify_corpus.py` pin the matrix (superuser, creator, user-level UPDATE, group-level UPDATE, group-perm-disabled, outsider, anonymous, `None`, id-by-int, id-by-str). Out of scope for this consolidation PR (flagged for separate review per the "consolidation PRs should not also be behavior-change PRs" rule): `corpus_mutations.py` `UpdateCorpusDescription`/`ReEmbedCorpus` (intentionally creator-only per docstring), `corpus_mutations.py` `CreateCorpusAction`/`AddTemplateToCorpus` (use `PermissionTypes.CRUD` not UPDATE — switching widens access), `worker_mutations.py:154` (no group-perm fallback today). Also removed two stale `resolve_oc_model_queryset` references from `opencontractserver/shared/Managers.py:139` and the docstring of `opencontractserver/tests/test_visibility_managers.py` — that helper has no live callers and visibility lives in the manager method now.

- **Graduated `opencontractserver.pipeline.*` from the mypy baseline** (issue #1483, refs #1447, #1331). All fourteen pipeline modules — the `base` package (`base_component`, `chunked_parser`, `embedder`, `parser`, `post_processor`), the embedders (`multimodal_microservice`, `openai_embedder`, `sent_transformer_microservice`), the parsers (`docling_parser_rest`, `docxodus_parser`, `llamaparse_parser`, `oc_markdown_parser`, `oc_text_parser`), `post_processors.pdf_redactor`, plus `pipeline.registry` and `pipeline.utils` — are now type-checked from scratch (103 baseline entries removed from `docs/typing/mypy_baseline.txt`). Notable changes that surfaced from the graduation: shared component metadata (`title`, `description`, `author`, `dependencies`, `input_schema`) is now declared on `PipelineComponentBase` so `type[PipelineComponentBase]` lookups in `pipeline.utils.get_metadata_for_component` resolve attributes through the base class instead of bare `type`; `BasePostProcessor` now declares `supported_file_types: list[FileTypeEnum]` (previously accessed but never declared on the base); `BaseMultimodalMicroserviceEmbedder.url_setting_name` is now a typed class attribute (referenced in error messages, never declared); `pipeline.utils.run_post_processors` validates that the class loaded by `get_component_by_name` is actually a `BasePostProcessor` subclass before calling `process_export`. Image-token construction in `docling_parser_rest` and `llamaparse_parser` only sets optional `PawlsTokenPythonType` fields when present, eliminating spurious `None` writes against `NotRequired`-but-non-`None` slots. `llamaparse_parser._build_image_token` extracted as a shared helper. No runtime behaviour change; all interactions with broader union types are preserved through targeted `cast()` calls at the boundary.
- **Type-checking: graduated `opencontractserver.utils.*` chunk 1 from mypy baseline** (issue #1480, sub-issue of #1447). Removed `[mypy-…]` ignore blocks for `utils.analysis`, `utils.analyzer`, `utils.cleanup`, `utils.corpus_forking`, `utils.embeddings`, and `utils.etl` and fixed all surfaced errors. Notable production fixes uncovered by typing:
  - `opencontractserver/utils/cleanup.py:34` — `delete_analysis_and_annotations` was calling `Analysis.annotations.filter(...)` on the class-level reverse descriptor (would fail at runtime with `AttributeError`); now correctly uses `analysis.annotations.all().delete()`.
  - `opencontractserver/utils/analyzer.py:204` — `run_analysis` now raises `ValueError` early when `analyzer.host_gremlin` is `None` instead of crashing on `gremlin.url` later.
  - `opencontractserver/utils/analyzer.py:272` — Submission-response logging used `f"{submission_response.content}"` which produced `b'…'` rather than the response body; switched to `!r` for safe bytes formatting.
  - `opencontractserver/utils/analyzer.py:install_analyzers` and `opencontractserver/tasks/analyzer_tasks.py:install_analyzer_task` — return type corrected from `list[int]` to `list[str]` (Analyzer PKs are CharField).
  - `opencontractserver/utils/embeddings.py` — `get_embedder` / `aget_embedder` parameter and return types now reflect that any of `corpus_id`, `mimetype_or_enum`, `embedder_class`, and `embedder_path` may be `None`. Removed an outdated `# type: ignore[attr-defined]` on `embed_text`.
  - `opencontractserver/utils/etl.py:404` — Renamed inner PDF-burn-in loop variable from `page` to `pdf_page` so it no longer collides with the `PageAnnotationData` `page` from `iter_page_annotations`. Color tuple is now an explicit `tuple[float, float, float]` instead of variable-length.
  - Pruned 52 corresponding lines from `docs/typing/mypy_baseline.txt`.

### Fixed

- **typing: graduate `opencontractserver.shared.{Managers,decorators}` from mypy baseline** (refs #1447, closes #1477). Drains the last two `[mypy-opencontractserver.shared.*]` ignore-blocks from `mypy.ini` (37 baseline lines pruned: 6813 → 6776 in `docs/typing/mypy_baseline.txt`). Per-file fixes:

  - `opencontractserver/shared/Managers.py`: drop the `if TYPE_CHECKING / else AbstractUser as User` aliasing block (mypy treated `User` as a value, not a type) — replace with explicit `AbstractBaseUser` annotations on `EmbeddingManager.store_embedding(creator=...)` and `UserFeedbackManager.by_creator(creator=...)`, and `Any` for the duck-typed `_apply_document_prefetches(user=...)` argument; cast `self.model` to `Any` once in `BaseVisibilityManager.visible_to_user` so downstream `model_cls.objects.*` and `cast(Any, self.model).DoesNotExist` (in `UserFeedbackManager.get_or_none`) typecheck without `_default_manager` semantic drift; coalesce `self.model._meta.model_name or ""` so the `.upper()` calls don't union-attr against `None`; align `PermissionManager.visible_to_user` and `UserFeedbackManager.visible_to_user` with `BaseVisibilityManager.visible_to_user(user, lightweight, with_doc_label_annotations)` (kwargs accepted for parity, no behaviour change); silence `[misc]` on the `PermissionManager.from_queryset(...)` dynamic base for `AnnotationManager` / `NoteManager`. Removes three unused `for_user(...)` delegation methods on `PermissionManager`, `AnnotationManager`, `NoteManager` that called `self.get_queryset().for_user(...)` against querysets that never implemented it — would have raised `AttributeError` at runtime. No callers in `opencontractserver/`, `config/`, or tests.
  - `opencontractserver/shared/decorators.py`: type `pdf_data_layer` and `pdf_text_extract` so the PDF / TXT branches of both `doc_analyzer_task` and `async_doc_analyzer_task` typecheck. `pdf_text_extract` is annotated `Optional[str]` and asserted non-`None` in the TXT branch (only reachable when `txt_extract_file` is non-empty); `pdf_data_layer` is annotated `Any` to absorb the `[] | PdfDataLayer` union without forcing `plasmapdf` stubs. Three inner helpers (`get_analysis_creator`, `get_analysis_analyzer`, `celery_task_with_async_to_sync.wrapper`) gain `-> Any` return annotations. Final return-annotation coverage on the graduated modules: 100% on `Managers.py` and 100% on `decorators.py`.

- **Document deletes now reclaim orphaned file blobs from storage** (`opencontractserver/documents/signals.py:170-300`, `opencontractserver/tasks/cleanup_tasks.py:25-110`, `opencontractserver/shared/Managers.py:413-470`, issue #1492). Previously, deleting a `Document` row left its `pdf_file`, `txt_extract_file`, `pawls_parse_file`, `icon`, and `md_summary_file` blobs in storage forever, so storage cost grew unbounded. Two new signal handlers on `Document` capture every populated FileField path in `pre_delete` and schedule a single Celery task per transaction (via `transaction.on_commit`) that removes paths whose live reference count hit zero. The task re-checks orphan status at execution time across every FileField, so corpus-isolated copies (issue #1464) and races with concurrent uploads are handled correctly. Bulk `QuerySet.delete()` is covered by the same signals — once listeners are connected Django disables its fast-delete path and dispatches per-instance signals. New manager method `Document.objects.unique_blob_paths_for_many(qs_or_pks)` gives callers a 2-queries-per-FileField primitive for queryset-scale orphan checks (vs N+1 per row in `unique_blob_paths`). Transaction safety: `on_commit` callbacks never fire on rollback, so a failed delete never reaches the storage layer. Regression tests in `opencontractserver/tests/test_blob_retention.py` cover per-row delete, queryset bulk delete (one-of-pair vs both-of-pair), `permanently_delete_document` (Rule Q1 cascade), `DocumentPath` PROTECT semantics, transaction rollback, sparse FileField sets, and direct task contract. Coverage spans every FileField on Document so adding a new file field is automatically protected.

- **STRIDE P0 follow-up review fixes** (issue #1466, follow-up to PR #1463).

  - `_add_doc_type_annotation` test helper in
    `opencontractserver/tests/test_add_annotation_idor.py:157` now passes the
    `user` argument it accepts into `_MutationContext` instead of the
    hard-coded `self.attacker`. The previous version made the helper appear
    parameterised but always executed as the attacker, so any future caller
    that passed `self.victim` would have silently re-run the attacker case
    and could miss a regression.
  - `_resolve_annotation_parents` in
    `config/graphql/annotation_mutations.py:218` now imports `DocumentPath`
    at module level alongside `Document` (no circular dependency exists),
    removing the unexplained lazy import.
  - `resolve_bulk_document_upload_status` in
    `config/graphql/document_queries.py:215` now constructs the opaque
    "not found" `BulkDocumentUploadStatusType` only inside the rejection
    branch instead of allocating it unconditionally on every status query.
    (Code-quality cleanup, not a security fix — the previous code
    allocated the response eagerly but only ever returned it on the
    rejection path, so externally-observable behaviour is unchanged.)
  - `BULK_UPLOAD_OWNER_CACHE_TTL_SECONDS` in
    `opencontractserver/constants/zip_import.py:60` carries a comment
    explaining why the 24-hour default is intentionally generous (a large
    zip can take many hours and the user must remain able to poll progress
    for the full lifetime of the job).
  - `ingest_doc` in `opencontractserver/tasks/doc_tasks.py:350` carries a
    comment explaining why the worker-side check uses `PermissionTypes.READ`
    while the parallel `retry_document_processing` path uses
    `PermissionTypes.UPDATE`.
  - `retry_document_processing` in
    `opencontractserver/tasks/doc_tasks.py:846` now emits a
    `[SECURITY]`-tagged log line on the `User.DoesNotExist` early-exit and
    returns the more accurate `"Invalid user for retry"` message instead
    of the misleading `"Document not found"`. Audit symmetry with the
    `ingest_doc` path is now preserved.
  - `Corpus._propagate_public_status_to_documents` in
    `opencontractserver/corpuses/models.py:522` moves the
    `cross_owner_blocked_ids` query inside the `transaction.atomic()`
    block, closing the narrow race where a document added to a new
    cross-owner private corpus between the membership check and the
    update could be incorrectly publicized. The same method now passes
    `batch_size=500` to `Notification.objects.bulk_create` so a corpus
    with thousands of cross-owner documents does not blow up a single
    SQL statement.

- **Bulk and single document import no longer crash with Apollo "Payload allocation size overflow"** before the network request fires. The previous transport base64-encoded the entire file (PDF or zip) into a single GraphQL `String` variable; Apollo then `JSON.stringify`'d the request body, which V8 cannot allocate as a contiguous string for large files (~33% inflation pushes a moderately sized zip past the V8 string limit / heap fragmentation ceiling). The frontend now streams uploads as `multipart/form-data` via `fetch` to two new REST endpoints, so the binary bytes never become a JS string. Files affected:

  - `frontend/src/components/widgets/modals/UploadModal/hooks/useUploadMutations.ts` — replaces the `UPLOAD_DOCUMENT` and `UPLOAD_DOCUMENTS_ZIP` Apollo mutations with `fetch` calls.
  - `frontend/src/utils/importHttp.ts` (new) — multipart helpers (`importDocumentMultipart`, `importDocumentsZipMultipart`) that attach the JWT bearer token and parse REST error bodies into structured results.
  - `frontend/src/utils/files.ts` — `toBase64` removed (last call site was the upload hook).

- **CorpusChat source citation deep-link: silent no-ops + an annotation-id fallback** (`frontend/src/views/Corpuses.tsx`, `frontend/src/components/corpuses/CorpusChat.tsx`). When asking questions of a corpus, clicking a source citation could silently do nothing. `handleSourceNavigate` had three guarded silent-return paths (no `document_id`, missing corpus slugs, empty `textBlock`); each one now emits a console.warn naming the failing condition so the cause is observable in DevTools. CorpusChat's source `onClick` also logs every click reaching the handler so we can confirm the click chain is intact. The third silent-return (text-block construction failed — typically a PDF source whose multipage `annotation_json` has page keys but empty `tokensJsons` arrays, leaving `encodePdfBlock` to return `""`) now falls back to a `?ann=<global-annotation-id>` deep-link when the source has a real (positive) `annotation_id`. As a last resort, navigates to the document with no highlight rather than no-op. Follows the routing-system contract: navigation goes through `navigate(url)` with Relay/global IDs in the path so CentralRouteManager Phase 1 can canonicalize, and `?ann=`/`?tb=` are picked up by Phase 2 — no direct reactive-var sets.

- **Drop the standalone "AI Assistant is thinking…" pill in favor of an animated ticker icon** (`frontend/src/components/widgets/chat/ChatMessage.tsx`, `frontend/src/components/corpuses/CorpusChat.tsx`, `frontend/src/components/knowledge_base/document/right_tray/ChatTray.tsx`). The old pill (pulse-dots + spinner + "AI Assistant is thinking…") rendered as a separate banner under the messages and duplicated the per-message in-flight signal. Replaced with: (a) a soft-pulse + glow halo on the `StreamingThoughtTicker` icon plus a tiny breathing dot at the right edge — the icon itself is the activity cue, so a tool_call still reads as a wrench, a thought still reads as a lightning bolt; (b) a generic "Thinking" placeholder ticker rendered when the assistant is mid-stream but no timeline entry has arrived yet, replacing the per-message `ProcessingIndicator`; (c) a standalone warm-up ticker rendered between user-send and the first assistant message in both `CorpusChat` and `ChatTray`, so the in-flight cue is universal across corpus-chat and document-chat. The mid-stream rendering rule was also tightened: the ticker only shows when the assistant is streaming AND no content has arrived yet (as soon as text tokens stream in, the bubble takes over and accumulates the response). The bordered `TimelinePreview` panel is suppressed entirely mid-stream and only appears post-completion as a collapsible summary inside the bubble. Removed the dead `ProcessingIndicator` component + its testid `processing-indicator`. Tests updated: rewrote `chat-message-processing-indicator.ct.tsx` to assert the ticker contract; updated the user-message check in `ChatMessageCoverage.ct.tsx`; new regression test `the legacy 'AI Assistant is thinking…' pill is gone after a send` in `CorpusChat.ct.tsx`.

- **Streaming chat timeline replaced with a single-line "now-thinking" ticker** (`frontend/src/components/widgets/chat/ChatMessage.tsx`). The bordered, scrollable `TimelinePreview` panel rendered while the assistant was streaming pushed the chat input off-screen on multi-step responses and shifted with every new step. New `StreamingThoughtTicker` shows only the latest step (icon + truncated title) on a single ~24px line and fades the previous step out as a new one arrives (subtle 250ms y-translate + opacity transition via `AnimatePresence exitBeforeEnter`). The full bordered timeline is no longer rendered mid-stream — it returns automatically once the message finalizes via the existing collapsible Timeline inside the message bubble (unchanged). Cleaned up the now-unused `expandLatestOnly` prop on `TimelinePreview` and the dormant `compact` mode added in the previous iteration; `TimelineContent` `max-height` reduction 400 → 220 px is kept so the post-completion expanded view stays bounded for long step lists. Updated the existing "should show timeline instead of processing indicator when timeline arrives first" test to assert the ticker, not the panel; new regression tests `streaming renders the latest-step ticker, not the bordered timeline panel` and `once the message is complete, the ticker disappears and the collapsible Timeline returns` pin both halves of the streaming → finalized handoff.

- **CorpusChat: duplicate Back button on the conversation-list views** (`frontend/src/components/corpuses/CorpusChat.tsx`, `frontend/src/views/Corpuses.tsx`). Same class of bug as the conversation-view header but on the list-view side: the `ConversationListView` filter-bar always rendered its own `←` Back next to "+ New Chat" whenever an `onBack` callback was passed, while the parent (`renderNavigationHeader` for the "Conversation History" / chatExpanded list views, and `TabNavigationHeader` on the corpus "Chats" tab) was simultaneously rendering its own outer Back. Fix: new `hideListBackButton` prop on `CorpusChat` forwards `onBack` to the list view as `undefined` when the parent already provides Back navigation. Wired up at all three call sites — chatExpanded path and VIEW-mode path use `hideListBackButton={isDesktop}` (mobile keeps the inner Back since the outer header is desktop-only), and the chats tab uses `hideListBackButton` unconditionally because its `TabNavigationHeader` renders on every screen size. New regression test: `hideListBackButton hides the filter-bar Back in the conversation list`.

- **CorpusChat: duplicate stacked Back buttons + sluggish chat input** (`frontend/src/views/Corpuses.tsx`, `frontend/src/components/corpuses/CorpusChat.tsx`, `frontend/src/components/corpuses/corpus_chat/styles.ts`). Two issues in the corpus chat surface:

  - The dashboard "search → chat" flow rendered the parent `Back / Chat` navigation header on top of `CorpusChat`'s own `← / Conversation` header, leaving two visually-stacked back buttons inside a conversation. The parent's header is now suppressed whenever the inner CorpusChat is in a conversation (mirroring the pattern already in use for the corpus "Chats" tab), so a single Back affordance is on screen at any time. Clicking it returns to the chat list, the parent header reappears, and clicking again returns to the search view — a single button that the user can click twice. New `chatExpandedInConversation` mirror state in `CorpusQueryView` is pre-set when entering chat to avoid a one-frame flash of the outer header.
  - The chat textarea started at ~2 visible rows (~73px) and never auto-grew — typing past the visible area scrolled inside the static box. The textarea now starts at one row (40–48px), auto-resizes to its `scrollHeight` up to a 140px content-box cap, then becomes scrollable. Container padding was tightened (1.25rem 1.5rem → 0.625rem 0.875rem) and the send button was resized to 40×40 (38×38 mobile) so the input area is no longer a giant white rectangle when empty. Shift+Enter inserts a newline, bare Enter sends. Mirrors the existing pattern in `ChatTray`.
  - New regression tests in `frontend/tests/CorpusChat.ct.tsx`: "conversation view shows exactly one Back button (not two stacked)", "onViewModeChange notifies parent when entering / leaving conversation", and "chat input starts compact and grows with multi-line content". New auto screenshots `corpus--chat--input-compact` and `corpus--chat--input-expanded`.

- **`GetDocumentKnowledgeAndAnnotations` no longer eagerly loads structural annotations & relationships** (`frontend/src/graphql/queries.ts`, `config/graphql/document_types.py`, `frontend/src/components/knowledge_base/document/DocumentKnowledgeBase.tsx`, `frontend/src/components/annotator/context/AnnotationAtoms.tsx`). The initial document-open query was returning every structural annotation and structural relationship for the document — for documents with thousands of structural items, this meant ~30s to first render even though the UI hides them by default. The frontend already had a separate lazy `GET_DOCUMENT_STRUCTURAL_ANNOTATIONS` query that fires when the user toggles structural visibility, but the main query never passed `isStructural: false`, so the work was being done twice. Fix:

  - The main query now passes `isStructural: false` to both `allAnnotations` and `allRelationships` (and the same for `GET_DOCUMENT_ANNOTATIONS_ONLY` used during analysis switching).
  - `Document.allRelationships` GraphQL field gains an `isStructural: Boolean` argument (mirroring `allAnnotations`); the underlying `RelationshipQueryOptimizer.get_document_relationships` already supported the filter.
  - New `Document.allStructuralRelationships` field — the relationship analogue of `allStructuralAnnotations` — and the lazy `GET_DOCUMENT_STRUCTURAL_ANNOTATIONS` query now fetches both in a single round-trip.
  - New `structuralRelationshipsAtom` (paralleling `structuralAnnotationsAtom`) plus a derived `allRelationsAtom` and `useAllRelations` hook for consumers that need the full set (`useVisibleAnnotations`, `AnnotationList`, `UnifiedContentFeed`). User-editable relation CRUD continues to read/write `pdfAnnotationsAtom.relations` only.
  - Backend coverage: new tests in `test_structural_annotations_graphql_backwards_compat.py` for `allRelationships(isStructural: false|true)` and `allStructuralRelationships` (pre- and post-migration).

- **Bound peak memory in PDF image extraction step** (`opencontractserver/utils/pdf_token_extraction.py`, issue #1498). The `extract_images_from_pdf` step run after document reassembly in the Celery `ingest_doc` pipeline previously rasterised each page once _per embedded image_ via `pdf2image.convert_from_bytes` and never explicitly closed PIL Image / BytesIO buffers, so peak RSS scaled linearly with `page_count * images_per_page`. A 127-page PDF would OOM Celery workers (cgroup SIGKILL, exit 137) that comfortably handled 30-page documents. Fix: pages are now processed one at a time; rasterisation is cached _per page_ via the new `_render_pdf_page` / `_crop_region_from_rendered_page` split, so a page with N images is rendered exactly once; PIL Image and BytesIO buffers are closed in inner `finally` blocks; pdfplumber's per-page parse cache is flushed via `page.flush_cache()`; and `gc.collect()` runs every `IMAGE_EXTRACTION_GC_INTERVAL_PAGES` pages (default 1) to reclaim Poppler subprocess buffers. Peak working set per call is now `len(pdf_bytes) + 1 rendered page + 1 decoded image bytes` — independent of total page count. Two new env vars: `IMAGE_EXTRACTION_DPI` (default 150, page-render RSS scales as ~DPI²) and `IMAGE_EXTRACTION_GC_INTERVAL_PAGES` (default 1, set to 0 to disable explicit collection). Regression tests in `opencontractserver/tests/test_pdf_token_extraction.py::TestExtractImagesMemoryBound` pin: render-once-per-page contract, rendered-page closure, pdfplumber cache flush, gc.collect cadence, eager per-image buffer close, and synthetic 100-page scaling.

- **Doc-blob deletion guard follow-up** (issue #1496). Tightened the safety net introduced in #1464:

  - `Document.safe_delete_field_blob` (`opencontractserver/documents/models.py:415-421`) now catches the specific `django.core.exceptions.FieldDoesNotExist` raised by `Options.get_field()` instead of bare `Exception`, so unrelated runtime errors (e.g. `AttributeError`, ORM internals) are no longer re-raised as misleading `ValueError`s.
  - The docstring now calls out the inherent TOCTOU window between `unique_blob_paths` and the storage delete, and tells callers in high-concurrency paths that a row-level lock on the single mutated document is not sufficient — sibling rows must be locked too.
  - `opencontractserver/shared/Managers.py` gains `from __future__ import annotations` so the PEP 585 `set[str]` return annotation on `DocumentManager.unique_blob_paths` evaluates lazily and remains importable on Python 3.8.
  - `unique_blob_paths(self, doc: "Document")` replaces the looser `doc: Any` so type checkers can flag invalid callers; the `Document` type is imported under `TYPE_CHECKING` to avoid a circular import.
  - Added a comment on the subTest loop in `test_shared_blob_is_preserved_for_sibling` (`opencontractserver/tests/test_blob_retention.py`) explaining why the `refresh_from_db()` calls at the top of each iteration are required for isolation as the loop progressively clears fields on `copy`.

- **`PipelineSettings.get_parser_kwargs` now merges encrypted secrets** (`opencontractserver/documents/models.py:1260`, issue #1515). The runtime path that resolves parser configuration in `opencontractserver/tasks/doc_tasks.py:424` previously returned only `parser_kwargs[parser_class_path]` and silently dropped any secret stored under `encrypted_secrets[parser_class_path]`. Parsers like `LlamaParseParser` whose API key lives only in `encrypted_secrets` (the documented secure location) were invoked with an empty `api_key=""` placeholder and failed with `"LlamaParse API key not configured"`. Fix: `get_parser_kwargs` now overlays decrypted `encrypted_secrets[parser_class_path]` on top of `parser_kwargs[parser_class_path]`, with secrets winning on key collision. Operators may keep `parser_kwargs.api_key = ""` as a schema marker without clobbering the real secret. The merged dict is built fresh on every call so a long-lived `PipelineSettings` reference does not retain decrypted secrets in memory. Regression tests in `opencontractserver/tests/test_pipeline_settings.py` cover merge, secret-wins, secret-only, no-mutation, and log-redaction.

- **`PipelineSettings.get_component_settings` now merges encrypted secrets** (`opencontractserver/documents/models.py:1297`, follow-up to #1515). The model-level component settings getter previously returned only the plaintext `component_settings[component_class_path]` entry, which would silently drop encrypted values for any direct caller that bypassed `get_full_component_settings`. The method now overlays decrypted `encrypted_secrets[component_class_path]` on top of the plaintext settings using the same precedence as `get_parser_kwargs` (secrets win on key collision; empty placeholders allowed; merged dict built per call so secrets are not retained on the model). New regression test `test_get_component_settings_merges_encrypted_secrets`.

### Added

- **Multipart REST import endpoints under `/api/imports/`** (`opencontractserver/document_imports/`, new app):

  - `POST /api/imports/documents/` — single-document import (`DocumentImportView`).
  - `POST /api/imports/documents-zip/` — bulk-zip import (`DocumentsZipImportView`); stages the archive in a `TemporaryFileHandle` and queues `process_documents_zip` (`opencontractserver/tasks/import_tasks.py`) exactly as the GraphQL path did.
  - `opencontractserver/document_imports/services.py` exposes `import_document_for_user` and `import_documents_zip_for_user`. Both `UploadDocument` / `UploadDocumentsZip` GraphQL mutations now delegate to these services so the two transports validate identically (visibility + EDIT permission with unified IDOR-safe error message; usage-cap enforcement; folder-in-corpus check; allowed-MIME enforcement; bulk-job owner cache binding).
  - New setting `MAX_DOCUMENT_IMPORT_SIZE_BYTES` (env-overridable, defaults to `MAX_FILE_UPLOAD_SIZE_BYTES`) caps multipart import size at the view layer with an explicit `413` response. Set to `0` to disable.
  - Auth: bearer JWT only. Both views pin `authentication_classes = [GraphQLJWTAuthentication]` rather than inheriting the DRF default (`JWT + Session + Token`), which narrows the threat model: browsers do not auto-attach `Authorization: Bearer`, so the endpoints are CSRF-safe by construction and any future change to `DEFAULT_AUTHENTICATION_CLASSES` cannot silently widen the surface here. Session callers (e.g. Django admin) must keep using GraphQL.
  - Per-endpoint throttle: both views apply `DocumentImportThrottle` (scope `document_imports`, default `120/hour` via `REST_FRAMEWORK["DEFAULT_THROTTLE_RATES"]`). The global `user` rate (`1000/hour`) is far too permissive for an endpoint where a single request can be hundreds of MB; `120/hour` is a workable default and operators can tune via the standard DRF setting.
  - Tests: `opencontractserver/tests/test_document_imports_rest.py` covers both views (auth, owned/inaccessible/read-only corpus IDOR uniformity, folder-in-corpus check, unsupported MIME, oversize 413, usage-cap, custom-meta JSON pass-through, owner-cache binding) plus direct service tests for the bytes/UploadedFile branches. Three new security-focused classes round out the suite: `DocumentImportRealJWTAuthTests` exercises the actual JWT validation path (valid bearer → 201, missing/tampered/expired → 401/403; the previous tests used `force_authenticate` which bypasses auth classes), `DocumentsZipNonZipRejectionTests` verifies `_peek_zip_magic` end-to-end against a PDF-as-zip and random bytes, and `DocumentImportFolderIDORTests` confirms a folder ID belonging to another user's corpus is rejected without document creation in both attacker shapes (own corpus + foreign folder, foreign corpus + foreign folder). `frontend/src/components/widgets/modals/UploadModal/hooks/__tests__/importHttp.test.ts` (8 cases) verifies the request body is `FormData` — not JSON — with correct URL, bearer auth header, blank-string omission, and structured error parsing. `useUploadMutations.test.tsx` (8 cases) covers sequential upload semantics, status-callback ordering, partial-failure resilience, and refetch+`onComplete` exactly-once.

- **Discover search crash on null annotation/note documents** (`frontend/src/views/DiscoverSearchResults.tsx:467-469, 600-603` and `frontend/src/utils/navigationUtils.ts:326-342, 358-400`). Running a search on the Discover view could throw `TypeError: can't access property "slug", e is null` when the backend returned an annotation or note edge whose `document` field was null (e.g., document filtered out by visibility or recently soft-deleted). `getDocumentUrl` accessed `document.slug` directly, and the section render lists never filtered for the missing relation. Fix: filter edges with `node.document == null` out of the annotation and note lists, and widen `getDocumentUrl` / `getCorpusUrl` to accept nullable inputs and fall through to the existing `"#"` no-op return path instead of throwing.

### Changed

- **Reduced chatty permissioning INFO logs** (`opencontractserver/utils/permissioning.py:365-373`, issue #1436). `user_has_permission_for_obj` previously emitted two `INFO` log lines on every call ("Starting check for ..." and "App name: ..."), which fired on essentially every authenticated request and dominated Django pod logs in production. The two lines are now collapsed into a single `logger.debug(...)` call using lazy `%`-formatting, so the strings are not built unless DEBUG logging is enabled. Permission denial logs (structural-write denial, analysis/extract privacy denials) remain at `INFO`, and all `WARNING`/`ERROR` paths are unchanged.
- **`supportedMimeTypes` GraphQL query is now accessible to anonymous users** (`config/graphql/pipeline_queries.py:205`). Removed the `@login_required` decorator from `resolve_supported_mime_types` so that uploaders, landing pages, and other unauthenticated UI surfaces can advertise the accepted file formats without forcing a login. The data returned is derived purely from the pipeline registry and exposes no user-specific information. Test updated: `opencontractserver/tests/test_pipeline_component_queries.py::test_supported_mime_types_allows_anonymous`.

### Added

- **CAML README references in bulk-import zips** (`opencontractserver/utils/caml_rewrite.py`, `opencontractserver/utils/import_v2.py:285`, `opencontractserver/tasks/import_tasks_v2.py:357`). The corpus README (`md_description`) shipped inside a V2 import zip now supports `oc-import://document/<filename-in-zip>` and `oc-import://annotation/<id-in-data.json>` placeholder URLs that are rewritten to live `/d/<user-slug>/<corpus-slug>/<doc-slug>[?ann=<new-pk>]` URLs after all referenced objects have been created. Lets a zip author hand-write a README that links to bundled documents and annotations without knowing the destination deployment's primary keys or slugs. Document references key off the same filename used as the `annotated_docs` key in `data.json`; annotation references key off the export-time `"id"` already aggregated by the importer as `all_annot_id_maps`. Unresolved references are left intact and warned about (no silent stripping). Revision snapshots are intentionally not rewritten to preserve the existing checksum chain. New tests in `opencontractserver/tests/test_caml_rewrite.py`. Documented in `docs/upload_methods/corpus_export_import.md`.
- **Server-derived `me.canImportCorpus` field** (`config/graphql/user_types.py:43-60`). A new boolean on `UserType` that mirrors the backend permission check enforced by `UploadCorpusImportZip` / `ImportZipToCorpus`: returns `false` for anonymous users and for usage-capped users when `USAGE_CAPPED_USER_CAN_IMPORT_CORPUS` is disabled, otherwise `true`. Exposed through `GET_ME` (`frontend/src/graphql/queries.ts`) so the frontend can gate corpus-import UI without re-implementing the rule. Tests: `opencontractserver/tests/test_user_can_import_corpus.py` covers the three states (capped+disabled, capped+enabled, uncapped).
- **Dedicated `ImportCorpusModal`** (`frontend/src/components/widgets/modals/ImportCorpusModal.tsx`). Replaces the previous hidden `<input type="file">` flow on the Corpuses page with a confirm/upload/progress modal that calls `importOpenContractsZip` to create a brand-new corpus from an OpenContracts export ZIP. The trigger button on `CorpusListView` is now hidden for users whose `me.canImportCorpus` is `false`, eliminating the prior UX where usage-capped users could pick a file and only then receive a backend permission error.

### Security

- **`updatePipelineSettings` mutation rejects plaintext secrets in `parser_kwargs` and `component_settings`** (`config/graphql/pipeline_settings_mutations.py`, related to issue #1515). Schema-declared `SettingType.SECRET` fields (e.g. `api_key`, `secret_token`) can no longer be stored as non-empty values in either of these unencrypted JSON fields; the mutation returns an error directing the operator to `updateComponentSecrets`. Empty-string and `null` placeholders are still permitted as schema markers, paired with the secret-wins merge in `get_parser_kwargs`. Closes the foot-gun where the now-fixed merge bug pressured operators to copy real API keys into `parser_kwargs` (stored unencrypted, returned by GraphQL queries, more likely to leak into logs). The error message echoes the offending field name only — never the supplied value. Tests in `PipelineSettingsGraphQLTestCase` cover both `parser_kwargs` and `component_settings` rejection paths and the empty-placeholder allow-list.
- **WebSocket authentication tokens no longer travel in URL query strings**
  (issue: JWT leakage via nginx logs, browser history, `Referer` headers,
  Sentry breadcrumbs, and CDN/WAF logs). All WS connections now authenticate
  via the standard `Sec-WebSocket-Protocol` handshake header
  (`opencontracts.jwt.v1, <jwt>`). Token rotation is in-band via
  `{"type":"AUTH","token":"..."}` frames — no socket churn on refresh. Hard
  cutover: `?token=` URL parameters are stripped by the new middleware
  (`config/websocket/middleware.py`) and ignored. `Authorization: Bearer`
  headers are also no longer consulted for WS auth (browsers cannot set them
  on the WebSocket constructor anyway). Stale browser tabs from before the
  deploy must reload to recover.

  - **`AuthHandshakeMixin`** (`config/websocket/auth_handshake.py`) — added
    to all three consumers. Refuses user-pk swap mid-connection
    (`USER_MISMATCH` → close 4002), re-runs resource permission checks on
    refresh (`PERMISSION_REVOKED` → close 4003), supports server-nudged
    refresh via `AUTH_REFRESH_REQUIRED` with a grace timer that closes 4001
    on timeout.

  - **Frontend `useWebSocketAuth` hook**
    (`frontend/src/hooks/useWebSocketAuth.ts`) — single shared lifecycle
    owner used by `useAgentChat`, `useNotificationWebSocket`, and
    `CorpusChat`. Token rotation no longer reconnects the socket.

  - **Removed**: deprecated `getDocumentQueryWebSocket` and
    `getCorpusQueryWebSocket` URL helpers (forwarded to the unified
    endpoint and now gone per the no-dead-code rule). Removed the
    `autoReconnect` / `reconnectDelay` options on `useNotificationWebSocket`
    — reconnect is owned by the shared hook.

  - **Hook return-shape change** (`useNotificationWebSocket`): the previous
    `{ connect, disconnect }` actions are replaced by `{ reconnect }`. The
    socket is now opened by the shared `useWebSocketAuth` lifecycle on mount
    and closed on unmount; manual open/close was redundant. External
    consumers that imported the hook directly will fail typecheck.

  - **Tests**: `opencontractserver/tests/test_websocket_auth.py` gains four
    new test classes (`JWTAuthMiddlewareSubprotocolTests`,
    `AuthHandshakeMixinTests`, `UnifiedAgentHandshakeTests`,
    `ThreadUpdatesHandshakeTests`, `NotificationUpdatesHandshakeTests`) —
    ~30 cases covering middleware, mixin, per-consumer integration,
    user-mismatch refusal, and `?token=` regression. Frontend adds
    `useWebSocketAuth.test.ts` and `websocketAuth.test.ts`; the existing
    `useNotificationWebSocket.auth.test.ts` is rewritten as a no-token-in-URL
    regression suite.

  - **Manual test script**:
    `docs/test_scripts/websocket-auth-handshake.md` — verifies subprotocol
    transport, in-band refresh, user-pk-swap refusal, and DevTools sanity
    check that the JWT never appears in the request URL.

  - **Full-stack E2E spec**:
    `frontend/tests/e2e/websocket-auth.spec.ts` drives the real-browser →
    Vite proxy → Daphne ASGI → consumer pipeline for all three websocket
    consumers. Asserts on actual frames captured via Playwright's
    `page.on('websocket')` so the wire protocol (subprotocol echo, AUTH_OK,
    NOTIFICATION_CREATED, in-band refresh) is the source of truth — no
    mocks. Triggers a real BADGE notification via `docker compose exec` to
    prove signal-driven broadcast end-to-end. Companion CI workflow
    `.github/workflows/frontend-e2e-websocket.yml` boots the local stack
    (Daphne, not runserver), runs under VCR replay so the LLM-touching
    test needs no real OpenAI key, and uploads coverage to Codecov under
    flags `frontend-e2e,frontend,websocket` and `backend-e2e,websocket`.
    The agent-chat path of `UnifiedAgentConsumer` (`_stream_agent_response`

    - `_handle_approval_decision`) is now wrapped in `maybe_vcr_cassette()`
      so the cassette covers both branches.

  - **Browser close-code surface is documented**: real browsers see close
    code `1006` (abnormal closure) for any auth rejection that happens
    before the consumer calls `accept()` — Channels translates a pre-accept
    `close()` into an HTTP 403 handshake rejection, which never produces
    a WebSocket close frame with the application code. The
    `WebsocketCommunicator` tests still see `4001`/`4002`/`4003` because
    Channels exposes the intended code to its test runner. The new e2e
    spec asserts the real browser surface (1006) for all pre-accept paths
    so a future change to accept-then-close (which would let the browser
    see the application code) is caught and the test gets updated.

  - **Per-connection AUTH-frame cooldown** in `AuthHandshakeMixin`: a
    1-second floor between accepted AUTH frames per socket prevents a
    malicious client from spamming refresh frames to burn DB queries on
    `_get_user_from_token` + `_validate_resource_permissions`. Auth0 silent
    renewal happens roughly every 50 minutes, so legitimate refreshes are
    unaffected.

  - **Auth-failure close codes are terminal in the frontend**:
    `useWebSocketAuth` treats close codes 4000 (UNAUTHENTICATED), 4001
    (TOKEN_EXPIRED), and 4002 (TOKEN_INVALID) as auth-invalid signals —
    `onAuthInvalid` fires once and the hook stops spawning new sockets.
    Previously 4000/4001 fell through to exponential-backoff reconnect,
    which would just be rejected again in a loop until the user
    re-authenticated. The hook also no longer returns the underlying
    `WebSocket` reference (it was stale on the first render), to remove a
    footgun for downstream consumers.

  - **`CorpusChat` and `ChatTray` migrated to `useWebSocketAuth`**: both
    chat surfaces previously instantiated raw `new WebSocket(url)` without
    subprotocols, so authenticated users would have hit the new middleware
    as anonymous after this deploy. Both now compose the shared hook
    (subprotocol auth on connect, in-band AUTH refresh on token rotation,
    close-code-aware reconnect policy).

  - **Backend `receive()` cleanup**: each consumer now parses the incoming
    text frame once and dispatches AUTH/non-AUTH from the same payload
    (was double-parsing). `thread_updates.py` consolidates redundant
    nested imports of `Conversation` / `Corpus` / `Document` to module
    level.

### Changed

- **Decluttered floating layout around the chat tray input** (`frontend/src/components/knowledge_base/document/FloatingDocumentControls.tsx`, `frontend/src/components/knowledge_base/document/DocumentKnowledgeBase.tsx`). When the right-hand sliding panel is open in chat mode, the document-tool FABs (extracts, analyses, create-analysis) are now hidden so they no longer crowd the panel boundary next to the chat input and sidebar tabs. The panel-width control remains visible since it directly affects the open panel. Closing the chat tray (or switching the sidebar to a non-chat view) restores the FAB stack. Implemented via a new `hideDocumentTools` prop on `FloatingDocumentControls`, applied to both the desktop FAB column and the mobile speed-dial entries.
- **Tightened chat tray input container so it stops drifting past the panel edge** (`frontend/src/components/knowledge_base/document/ChatContainers.tsx`). The `ChatInputContainer` had `padding: 1.25rem 1.5rem`, a `gap: 0.875rem`, and the `ChatInputWrapper` carried a `2px` border with a `0 0 0 4px` focus halo. On narrower panel widths the halo bled past the wrapper bounds and got clipped by `ChatContainer`'s rounded right edge, while the send button ended up hugging the panel wall. Reduced container padding to `0.875rem 1rem` (with an `env(safe-area-inset-bottom)` aware bottom inset for mobile), shrank the gap to `0.625rem`, dropped the wrapper border to `1px` / `12px` radius, and tightened the focus halo to `0 0 0 2px`. Added `min-width: 0` to both the container and the wrapper plus `flex: 1 1 0` on the wrapper so the textarea actually shrinks within flex instead of forcing horizontal overflow.
- **Corpus-import button gating** (`frontend/src/views/Corpuses.tsx`). The "Import" action (both the dropdown action and the list-view button) is now gated on `REACT_APP_ALLOW_IMPORTS && backendUser.canImportCorpus` instead of `REACT_APP_ALLOW_IMPORTS && Boolean(currentUser)`. Deploy-level kill switch is preserved; user-level permission now matches the backend check. The legacy `corpusUploadRef` / `onImportFileChange` / inline `START_IMPORT_CORPUS` mutation hook in `Corpuses.tsx` are removed in favor of the new modal.
- **`StartImportCorpusExport` TypeScript shape corrected** (`frontend/src/graphql/mutations.ts`). The output type now correctly nests `{ ok, message, corpus }` under `importOpenContractsZip` to match the GraphQL mutation response (the previous flat shape was technically wrong but never accessed because the only consumer relied on `onCompleted` callbacks).
- **Slimmed Django/Celery production image from ~6.3 GB to a target ≤1.5 GB** (Issue #1494). The image previously inherited from `pytorch/pytorch:2.7.1-cuda12.6-cudnn9-runtime`, dragging in CUDA 12.6, cuDNN 9, and a full PyTorch installation that nothing in the Django runtime imports — the only consumer was the optional in-process `CrossEncoderReranker`, which has been removed (see _Removed_ below). Cold pod pulls dropped from ~5+ minutes to seconds, dramatically narrowing the recovery window when nodes reschedule.
  - `compose/production/django/Dockerfile` and `compose/local/django/Dockerfile` now base on `python:3.11-slim-bookworm` for both build and runtime stages. The build toolchain (`build-essential`, `cmake`, `libtesseract-dev`, `libpq-dev`, etc.) is confined to the builder stage; the runtime stage installs only the libraries actually needed at execution time (`libpq5`, `gettext`, `poppler-utils`, `tesseract-ocr` + `tesseract-ocr-eng`, `libgl1`, `libsm6`, `libxext6`).
  - Dead CUDA environment variables (`CUDA_MODULE_LOADING`, `TORCH_CUDA_ARCH_LIST`, `CUDA_VISIBLE_DEVICES`, `PYTORCH_CUDA_ALLOC_CONF`) and the now-unused `ffmpeg` apt package were removed from both Dockerfiles.
  - `.dockerignore` was significantly tightened — `frontend/`, `fixtures/`, `docs/`, `cloudflare-og-worker/`, `tools/`, `model_preloaders/`, `locale/`, `.github/`, `.vscode/`, `.cursor/`, `.claude/`, `.envs/.local/`, `*.md` (except `CHANGELOG.md`), and Python caches are now excluded from the build context, both shrinking image size and reducing accidental secret/source leakage into runtime images.
  - **CI image-size budget** added to `.github/workflows/docker-build-release.yml`: the release build now fails if the Django image exceeds 1.5 GiB (`1610612736` bytes), surfacing regressions immediately rather than at the next cold pod pull.
- **Celery configured for at-least-once delivery so worker death no longer silently loses tasks** (Issue #1493, `config/settings/base.py`). Previously Celery acked messages on receive (at-most-once): if a worker died mid-task — OOM, host failure, deploy, eviction, SIGKILL — the broker had already removed the message and the task was silently lost, leaving long-running ingest/parse/embed work with documents stuck at `backend_lock=True` and no parsed content. With the new settings the broker only acks after the task returns successfully, and hard-killed workers cause redelivery rather than silent disappearance. The redundant per-task `acks_late=True` on `opencontractserver/worker_uploads/tasks.py` is left in place as an explicit declaration of intent (it is now equivalent to the global default). Documented in `docs/architecture/asynchronous-processing.md`, including the contract, known non-idempotent tasks to harden in follow-up work, and the per-task opt-out for tasks that genuinely cannot be made idempotent.

  **Technical details**

  - `CELERY_TASK_ACKS_LATE = True` — broker only removes the message after the task returns successfully.
  - `CELERY_TASK_REJECT_ON_WORKER_LOST = True` — SIGKILL'd workers cause redelivery rather than silent task loss.
  - `CELERY_BROKER_TRANSPORT_OPTIONS = {"visibility_timeout": CELERY_REDIS_VISIBILITY_TIMEOUT_SECONDS}` (12 hours) — prevents Redis from double-delivering tasks that run longer than the 1-hour Celery default. The constant lives in `opencontractserver/constants/celery.py` (Issue #1504).
  - **Trade-off**: at-least-once delivery means tasks may run twice. All tasks must be idempotent; per-task opt-out is available via `@shared_task(acks_late=False, reject_on_worker_lost=False)`.
  - Regression test in `opencontractserver/tests/test_celery_worker_death_resilience.py` pins all three settings _and_ verifies the namespaced Django→Celery wiring propagates each of them into `celery_app.conf`, so a future settings refactor cannot silently revert the resilience guarantee (Issue #1504).

### Removed

- **In-process `CrossEncoderReranker`** (Issue #1494). The lazily-loaded `sentence_transformers.CrossEncoder` reranker added in PR #845 was the sole reason the Django/Celery image inherited a CUDA-enabled PyTorch base (~4–5 GB of bloat for a feature that few deployments enabled). Reranking is still available via `MicroserviceReranker` (HTTP side-car) and `CohereReranker` (API), both of which are framework-agnostic and require no in-container model weights.
  - Deleted `opencontractserver/pipeline/rerankers/cross_encoder_reranker.py`. The pipeline registry auto-discovers reranker classes from the package, so removal cleanly drops `CrossEncoderReranker` from `get_registry().rerankers` with no further wiring changes.
  - Deleted `CrossEncoderRerankerTest` (`opencontractserver/tests/test_reranker.py`) and `CrossEncoderRerankerTests` (`opencontractserver/tests/test_data_extract_helpers.py`). Both used in-process stubs of `_load_cross_encoder` to avoid downloading weights in CI; with the backend gone, their coverage target is gone too. The orphaned `from unittest.mock import MagicMock` import in `test_data_extract_helpers.py` was removed.
  - Removed orphaned configuration: `SENTENCE_TRANSFORMER_MODELS_PATH` (`config/settings/base.py:986-988`) and the matching `SENTENCE_TRANSFORMER_MODELS_PATH=/models/sentence-transformers` line in `.github/workflows/production-stack.yml:107` were the only consumers of the deleted backend's model-cache directory.
  - Removed the `tokenizers>=0.23.1,<0.24` pin from `requirements/base.txt`. Its inline comment ("Pin to prevent conflicts with transformers") referred to a transitive constraint that was only relevant while `sentence-transformers` was installed; nothing in the codebase imports `tokenizers` directly.

### Fixed

- **`brotli` decoder added to runtime deps** (`requirements/base.txt`). The previous PyTorch base image shipped `brotli` incidentally via conda, so `httpx` registered a `br` decoder and transparently decompressed `Content-Encoding: br` responses. The new `python:3.11-slim-bookworm` base does not, leaving `httpx._decoders.SUPPORTED_DECODERS` at `{identity, gzip, deflate, zstd}` only. Without a brotli decoder, httpx returns the still-compressed bytes to callers and the OpenAI SDK then raises `json.JSONDecodeError: Expecting value: line 1 column 1 (char 0)` when it tries to parse them. This surfaced as `test_structured_response_uses_document_tools` (the only VCR cassette in the repo recorded with `Content-Encoding: br`) failing on the slim image; live OpenAI traffic was equally affected whenever the API negotiated brotli. Adding `brotli>=1.1.0` re-enables `httpx`'s `br` decoder for both VCR replay and production HTTP.
  - Deleted `.github/workflows/docker-build-cuda.yml`. The CUDA-tagged image build had no remaining purpose once the only torch consumer was removed.
  - Deleted `docs/deployment/docker-gpu-setup.md` and its `mkdocs.yml` nav entry. The guide documented the now-removed CUDA/PyTorch base image and was the only page under the _Deployment_ nav section.

### Added

- **Pre-fill defaults in the new-corpus modal**:

  - **Default LabelSet seeded at install** (`opencontractserver/annotations/label_set_seeds.py`, migration `annotations/0069_labelset_is_default_and_seed_default.py`): a public `LabelSet` titled "Default Labels" is now created on install (owned by the first superuser) with a starter palette aimed at corporate operations review — three general research labels (Important, Question, Reference) plus a light contract palette (Definition, Party, Governing Law, Effective Date, Termination Date, Expiration, Termination, Renewal, Limitation of Liability). New `LabelSet.is_default` BooleanField with a partial unique constraint ensures at most one labelset is flagged as the default. Idempotent management command `python manage.py seed_default_labelset` (`opencontractserver/annotations/management/commands/seed_default_labelset.py`) re-runs the seed safely after the first superuser is created and appends any newly-added labels to an already-seeded labelset.
  - **`defaultLabelset` GraphQL query** (`config/graphql/annotation_queries.py`): returns the install-wide default LabelSet visible to the current user (or null). `LabelSetType` exposes the new `is_default` field via `DjangoObjectType`'s default field set.
  - **`CreateCorpusMutation` auto-fills `label_set`** (`config/graphql/corpus_mutations.py`): when the mutation is called without a label_set the install-wide default is substituted, so corpuses created via the API land with a usable starter palette. Defaulting is intentionally at the mutation layer, not in `Corpus.save()`, so direct ORM creates in tests/scripts remain opt-in.
  - **CorpusModal CREATE mode pre-fills** (`frontend/src/components/corpuses/CorpusModal.tsx`, `frontend/src/graphql/queries.ts`): on open, the modal fetches `pipelineSettings.defaultEmbedder` + `defaultLabelset` via a single `GET_CORPUS_CREATE_DEFAULTS` query and seeds the embedder and labelset selectors. License pre-selects `CC-BY-4.0` via the new `DEFAULT_NEW_CORPUS_LICENSE` constant in `frontend/src/assets/configurations/constants.ts`. All three pre-fills are pure UX nudges — users can clear them before submitting.

- **Cross-content Discover search** at `/discover/search`. The Discover hero
  search box now lands on a unified results page that fans out across
  discussions, annotations, collections (corpuses), and notes in parallel,
  rather than redirecting to discussion-only search.

  - Backend: new `searchNotesForMention` resolver in
    `config/graphql/search_queries.py` (visibility via
    `Note.objects.visible_to_user`, optional `corpusId` / `documentId`
    scoping, eager-loads `document`, `corpus`, and creators).
    `NoteType` is exported through the same `DjangoConnectionField` shape
    used by the other mention searches and goes through `NoteType.get_queryset`
    for a second visibility pass.
  - Frontend view: `frontend/src/views/DiscoverSearchResults.tsx` reuses
    `ThreadListItem` for discussions and a single shared `ResultRow` for
    annotations / collections / notes. Tabbed UI: All (preview of 5 each),
    Discussions, Annotations, Collections, Notes.
  - Routing: `?q=` and `?type=` are URL-driven (replace, no history spam).
    Result rows deep-link via `getCorpusUrl` / `getDocumentUrl`; annotation
    rows preserve `?ann=`, note rows pass `?note=` to the document URL.
  - New `?note=<id>` deep-link param wired through `selectedNoteId`
    (`frontend/src/graphql/cache.ts`), `QueryParams.noteId`
    (`frontend/src/utils/navigationUtils.ts`), and the central manager
    Phase 2/4 sync (`frontend/src/routing/CentralRouteManager.tsx`).
    `DocumentKnowledgeBase` now consumes the var: when the loaded
    document contains a note matching the URL id, the existing note
    detail modal opens once and the param is consumed (one-shot
    deep-link, mirrored back to the URL via Phase 4).
  - Hero wiring: `NewHeroSection.handleSearchSubmit` now navigates to
    `/discover/search?q=...`. The legacy `/discussions?search=...` route is
    preserved for callers that want a discussion-only listing.
  - Tests: `MentionSearchTestCase` in
    `opencontractserver/tests/test_mentions.py` covers visibility
    filtering, content-substring matching, corpus / document scoping,
    anonymous-user filtering, wrong-type global-id rejection, the
    `-modified` ordering contract, and both
    `content_preview` paths (DB-annotated `Left('content', 400)` for
    search results, Python fallback for per-id note fetch).
  - Component test: `frontend/tests/DiscoverSearchResults.ct.tsx`
    covers the empty-prompt state, the four-section header render
    after typing, and a populated-results render exercising every
    section's row primitives. Captures both
    `discover--search-results--empty-prompt.png` and
    `discover--search-results--with-results.png` for docs.

- **Loud guardrail against the `system_prompt=` foot-gun in pydantic-ai** (Issue #1451): `pydantic_ai.Agent` accepts both `system_prompt=` and `instructions=`, but the `system_prompt` value is _only_ materialised into the model request when `message_history` is `None`. OpenContracts' `chat()` flow always persists the user's HUMAN message before calling `Agent.run()`, so `message_history` is never empty in practice and any `system_prompt=` argument is silently dropped — the LLM runs without any system instruction. CLAUDE.md pitfall #14 documented the workaround (use `instructions=`), but a future pydantic-ai bump that renames or re-precedences these parameters could re-introduce the regression silently.

  - **Single construction path** (`opencontractserver/llms/agents/pydantic_ai_factory.py`): new `make_pydantic_ai_agent(...)` factory is now the only place in the codebase that instantiates `pydantic_ai.Agent`. The factory uses a sentinel-based check (not `is not None`) to refuse `system_prompt=` outright — even `system_prompt=None` raises `TypeError` so the lesson cannot be re-learned by accident. The error message references issue #1451 and CLAUDE.md pitfall #14.
  - **All call sites refactored** (in `opencontractserver/llms/agents/pydantic_ai_agents.py`: `_run_structured_extraction`, the document-agent factory, and the corpus-agent factory; in `opencontractserver/tasks/memory_tasks.py`: `summarise_agent` and `curation_agent`). Five direct `PydanticAIAgent(...)` constructions in production code now route through the factory.
  - **Self-contained factory** (`opencontractserver/llms/agents/pydantic_ai_factory.py`): the factory imports `from pydantic_ai.agent import Agent as PydanticAIAgent` directly and calls `PydanticAIAgent(...)` with no sideways indirection through `pydantic_ai_agents`. Tests that need to intercept agent construction now patch `opencontractserver.llms.agents.pydantic_ai_factory.PydanticAIAgent` (44 patch sites updated across 7 test files: `test_pydantic_ai_agents.py`, `test_nested_approval_gates.py`, `test_agent_memory.py`, `test_duplicate_tool_registration.py`, `test_tool_approval_gate.py`, `test_long_conversation_api.py`). The chokepoint is now both production-code clean and test-visible at the same path.
  - **Version-pinning regression test** (`opencontractserver/tests/test_pydantic_ai_factory.py`): four tests cover the loud-failure paths (`system_prompt=<str>`, `system_prompt=None`) and the precedence behaviour itself — `test_instructions_survive_non_empty_message_history` constructs an agent with `instructions=<sentinel>` via the factory, runs it against `pydantic_ai.models.test.TestModel` with a non-empty `message_history`, and asserts the sentinel is delivered to the model (either as a `SystemPromptPart` or via `ModelRequest.instructions`). If a future pydantic-ai release changes precedence so `instructions=` is also dropped under non-empty history, this test fails loudly so the regression is caught before silently shipping.
  - **Dependency pin commentary** (`requirements/base.txt:51`): the `pydantic-ai>=1.89.1,<2` line now carries a multi-line comment documenting that this codebase relies on the current `instructions=` precedence rule and pointing future maintainers at the regression test before widening the upper bound.

- **E2E spec for threads/discussions** (`frontend/tests/e2e/threads-discussions.spec.ts`):
  - Anonymous pass renders `/discussions` (filter pills, search box, "Corpus Discussions" section header) and `/threads` (search route) without authentication.
  - Authenticated pass logs in, creates a per-run corpus (suffixed with `Date.now()` to avoid collisions across retries), opens its inline discussions view via `?view=discussions`, fills the `CreateThreadForm` modal (title, optional description, ProseMirror initial message), posts a top-level reply through the `ReplyForm` composer, navigates back to the thread list, and confirms the new thread surfaces in the global `/discussions` "Corpus Discussions" section before clicking through to the detail.
  - Picked up automatically by the existing `frontend-e2e.yml` workflow — no CI changes required.
- **Three reusable E2E helpers** in `frontend/tests/e2e/helpers.ts`:
  - `openCorpusDiscussionsViaUI` — SPA-navigates to a corpus and opens its inline discussions view; waits on the "All" filter pill as the readiness signal.
  - `createThreadViaUI` — clicks the `aria-label="Create new discussion"` CTA, fills the modal (scoped via the `#thread-title` anchor), submits the composer, and waits for the thread-detail header.
  - `postThreadReplyViaUI` — types into the bottom `ReplyForm` ProseMirror editor and clicks Send, returning once the new message text is visible.

### Removed

- **Stray planning artifacts at repo root** (Issue #1453): deleted `plan.md` (CorpusHome query-performance optimization plan) and `plans/routing-audit-report.md` (frontend routing audit). Both described work that has already shipped — `CountableConnection.resolve_total_count()` already calls `.count()` directly (`config/graphql/base.py:88`) and `browseRoutes` already includes `"discussions"` (`frontend/src/utils/navigationUtils.ts:177`). The `plans/` directory is now gone; future planning docs belong under `docs/plans/` or `docs/architecture/` per existing conventions.

### Fixed

- **Discover search section header miscounted threads when soft-deleted ones were present** (`frontend/src/views/DiscoverSearchResults.tsx`): the discussions section displayed `data.conversations.totalCount` as the count, but rendered rows are filtered client-side via `!n?.deletedAt`. With even one tombstoned thread in the results, the header read e.g. "5 threads" while only 4 rows appeared. Switched the count to `threads.length` (post-filter) so the badge always matches what the user sees.

- **`?note=<id>` deep-link could pin in the URL forever for documents with no notes** (`frontend/src/components/knowledge_base/document/DocumentKnowledgeBase.tsx`): the auto-open effect early-returned on `notes.length === 0`, which is _also_ true while the document query is loading. The intent comment said "Clear regardless of match" but the early return skipped the clear call entirely. If the underlying document genuinely had no notes (or the loaded notes simply did not contain the deep-linked id) the reactive var stayed set, so `CentralRouteManager` Phase 4 kept writing `?note=<id>` back into the URL on every render — the deep-link became sticky with no user-visible escape hatch. Effect now gates only on `combinedData?.document` being loaded; once the query has resolved, the effect runs `find` (possibly empty) and always clears the var.

- **`NewHeroSection` had a dead `isAuthenticated` prop** (`frontend/src/components/landing/NewHeroSection.tsx`, `frontend/src/views/DiscoveryLanding.tsx`): the prop was declared on the interface and threaded through `DiscoveryLanding` but never read inside the component. Removed from the interface and the call site.

- **Discover search rows silently no-op'd on `"#"` URLs from missing slugs** (`frontend/src/views/DiscoverSearchResults.tsx`): `getDocumentUrl` / `getCorpusUrl` return `"#"` when slugs are missing on the underlying entities. Each section's `onClick` did `if (url !== "#") navigate(url)`, so the row remained visually clickable, hover styles fired, and the click was swallowed with no feedback. `ResultRow` now accepts `disabled` / `disabledReason`; sections compute `unrouteable = url === "#"` once and forward both. Disabled rows render with `opacity 0.55`, `cursor: not-allowed`, native `disabled` (and `aria-disabled`), and a tooltip explaining the missing data — replacing the silent failure with a discoverable one. `disabled:not(:disabled)` scoping on the hover style stops the disabled rows from animating on hover.

- **Annotation deep-links from the corpus-home Table of Contents silently no-op'd** (`frontend/src/components/corpuses/DocumentAnnotationIndex.tsx`, `frontend/src/components/knowledge_base/document/document_kb/RightPanelContent.tsx`): clicking a structural section in the corpus-home document index (e.g. "Subchapter I. Formation, p. 2") was supposed to open the document with the annotation pre-selected and scrolled into view. Instead it appeared to do nothing. Root cause: `DocumentAnnotationIndex` overloaded a single `embedded` prop with two semantics — visual layout ("render without an outer container") _and_ click routing ("we are already on the document page, just rewrite `?ann=`"). The corpus-home call site (`DocumentTableOfContents.tsx:919`) needed the visual flavor but absolutely was _not_ on a document page, so `handleSectionClick` took the wrong branch and wrote `?ann=<id>` onto the corpus URL — no navigation, no scroll. Fix splits the prop: `embedded` is now purely visual, and a new explicit `onDocumentPage` prop controls click routing. The single call site that's actually on a document page (`RightPanelContent.tsx`) opts in. Regression test in `frontend/src/components/corpuses/__tests__/DocumentAnnotationIndex.test.tsx` pins the new contract: a click from a corpus URL must produce a string-form `navigate("/d/.../doc?ann=<id>")` (full deep link), while a click from a document URL with `onDocumentPage` produces `navigate({ search: "...ann=<id>..." }, { replace: true })`.

- **Zip importer reported `success: True` even when sidecars failed (silent annotation loss)** (`opencontractserver/tasks/import_tasks.py:421`, `:1411`): `_read_sidecar` raises `ValueError` when a sidecar exceeds `ZIP_MAX_SIDECAR_SIZE_BYTES`; malformed JSON, schema failures, and missing labels for sidecar-declared annotations all bump `annotation_sidecars_errored` and append to `errors`. The success determinations only checked `files_errored` (`import_zip_with_folder_structure`) or the user-cap message (`process_documents_zip`), so callers observed `success: True, completed: True` while annotations were silently dropped — exactly the silent data-loss path called out in PR #1489 review feedback. `import_zip_with_folder_structure` now requires `annotation_sidecars_errored == 0` in addition to the existing `files_errored == 0` and user-cap check; `process_documents_zip` now requires `error_files == 0` in addition to the user-cap check. `relationship_errors` is intentionally not folded in — those are surfaced separately via `relationships_skipped` + `relationship_errors` and the documents themselves are imported correctly. Two tests in `test_sidecar_import.py` (`test_skip_pipeline_without_labels_json`) and a new regression test (`test_sidecar_error_drops_overall_success_flag`) lock down the new contract.

- **`_create_analysis_notification` was reading a non-existent field on `Analyzer`** (Issue #1471, `opencontractserver/analyzer/views.py:50`). The notification builder for completed/failed analyses was reading `analysis.analyzer.analyzer_id`, but `Analyzer` has no such field — its primary key is `id` (a `CharField(max_length=1024, primary_key=True)`). On a real ORM instance every successful or failed remote-analyzer notification raised `AttributeError` inside the builder before the `Notification` row was written. Production tests in `opencontractserver/tests/test_job_notifications.py` masked the bug by mocking the entire `Analysis` instance with `MagicMock()` and setting `analysis.analyzer.analyzer_id = "test-analyzer"` (MagicMock auto-creates any attribute on access). Fix replaces the bad attribute with `analysis.analyzer.id`. A follow-up integration test using a real `Analyzer` instance is tracked in the issue.
- **Preferred Embedder dropdown showed disabled embedders for superusers** (`frontend/src/components/widgets/CRUD/EmbedderSelector.tsx`, `frontend/src/graphql/queries.ts`): the create/edit corpus form's `EmbedderSelector` listed every embedder returned by `pipelineComponents.embedders`. The backend resolver in `config/graphql/pipeline_queries.py` only filters by configured/preferred class names for non-superusers, so a superuser saw embedders that `PipelineSettings.enabled_components` had explicitly disallowed and could pick one the rest of the system would refuse to use. The `enabled` flag was already computed and exposed on `PipelineComponentType` but the `GET_EMBEDDERS` query did not request it. Added `enabled` to the query and filter `embedder.enabled !== false` on the client (`undefined` from older backends still passes through). Test mocks in `frontend/tests/EmbedderSelector.ct.tsx` and `frontend/tests/corpus-modal.ct.tsx` updated, and a new test case verifies a `enabled: false` embedder is omitted from the dropdown.

### Changed

- **Typing: graduate `opencontractserver.documents.*` from mypy baseline** (refs Issue #1447): removed eight `[mypy-…]` `ignore_errors` blocks (`opencontractserver.documents.{admin,checks,models,query_optimizer,versioning}`, `documents.tests.test_pdf_hash`, `documents.management.commands.{compute_document_hashes,migrate_pipeline_settings}`); added a scoped `disable_error_code = django-manager-missing` block for `documents.models` (DocumentPath self-relation `path_records` / `children` and `IngestionSource.document_paths` are django-tree-queries reverse-relation surfaces django-stubs cannot resolve). Pruned 40 baseline error lines from `docs/typing/mypy_baseline.txt` (7124 → 7084). Per-file fixes:

  - `documents/versioning.py` — replaced the runtime `User = get_user_model()` with a TYPE_CHECKING import of `opencontractserver.users.models.User` and quoted six `user: "User"` annotations; widened `move_document(new_folder=...)` from `Optional[CorpusFolder] = "UNSET"` to `Optional[CorpusFolder] | str = "UNSET"` and added an `assert not isinstance(new_folder, str)` after the sentinel check so `folder_to_use: Optional[CorpusFolder]` narrows correctly.
  - `documents/models.py` — scoped `# type: ignore[misc]` on the `objects = DocumentManager()` manager override; added `author_obj: AbstractBaseUser` annotation in `Document.update_summary` plus `# type: ignore[misc]` on the corresponding `author=author_obj` kwarg passed to `DocumentSummaryRevision.objects.create(...)`; typed two `all_components: list[Any] = []` accumulators in pipeline-component check helpers.
  - `documents/query_optimizer.py` — typed `result: dict[str, list[Any]] = {...}` in the related-object collector; added `Any` to the `typing` import.
  - `documents/checks.py` — typed three `errors: list[Warning] = []` accumulators (one widened to `list[Error | Warning]` because the body appends both severity classes).
  - `documents/admin.py` — scoped `# type: ignore[attr-defined]` on the three Django-admin `method.short_description = "..."` / `admin_order_field = "..."` annotations.
  - `documents/tests/test_pdf_hash.py` — added narrowing `assert computed_hash is not None` before the SHA-256 length assertion; cast `_meta.get_field("pdf_file_hash")` to `Any` in the field-introspection helper since the abstract `Field` return type doesn't expose `db_index` / `max_length`.
  - `documents/management/commands/compute_document_hashes.py` — added `assert document.pdf_file.name is not None` before `storage.exists(...)`; coalesced `old_hash[:8]` to `(old_hash or "")[:8]` for the no-change branch.
  - `documents/management/commands/migrate_pipeline_settings.py` — typed two `all_components: list[Any] = []` accumulators; widened the heterogeneous `stats` dict to `dict[str, Any]`; renamed the inner iteration variable to `group_components` and rebound `components: list[Any] = list(group_components)` so the list-comp re-assignment doesn't conflict with the source tuple type.

- **Annotation sidecar size limit raised and made env-configurable** (`opencontractserver/constants/zip_import.py:49`, `config/settings/base.py:730`): the per-sidecar JSON cap during zip import was 10 MB, which silently dropped annotations for larger corpora — `_read_sidecar` (`opencontractserver/tasks/import_tasks.py:572`) raised `ValueError` and the importer recorded it under `errors` while still returning `success: True, completed: True`, so callers had no signal that annotations were lost. Default raised to 50 MB and the setting is now read from the `ZIP_MAX_SIDECAR_SIZE_BYTES` env var via `django-environ` in `config/settings/base.py`, mirroring the pattern used for `MAX_WORKER_UPLOAD_SIZE_BYTES` and friends. The constants module's `getattr(settings, ...)` fallback was bumped to 50 MB so non-Django importers see the same default. Existing tests in `opencontractserver/tests/test_sidecar_import.py` patch the constant directly and are unaffected.

- **Remaining zip-import limits are now env-configurable** (`config/settings/base.py:730`, `opencontractserver/constants/zip_import.py`): `ZIP_MAX_FILE_COUNT`, `ZIP_MAX_TOTAL_SIZE_BYTES`, `ZIP_MAX_SINGLE_FILE_SIZE_BYTES`, `ZIP_MAX_COMPRESSION_RATIO`, `ZIP_MAX_FOLDER_DEPTH`, `ZIP_MAX_FOLDER_COUNT`, `ZIP_MAX_PATH_COMPONENT_LENGTH`, `ZIP_MAX_PATH_LENGTH`, and `ZIP_DOCUMENT_BATCH_SIZE` previously honored Django settings overrides via `getattr(settings, ...)` in the constants module but were not surfaced in `base.py`, so operators could only override them in code, not via environment variables. Each now has a matching `int(env(...))` entry in `base.py` with the same default the constants module already used, and the constants-module docstring documents the env-var path. No behavior change at the defaults.

- **Typing: graduate `opencontractserver.{agents,analyzer,discovery.views}` from mypy baseline** (refs Issue #1447): removed seven `[mypy-…]` `ignore_errors` blocks (`opencontractserver.agents.{admin,memory,models}`, `opencontractserver.analyzer.{utils,views}`, `opencontractserver.discovery.views`) and pruned 32 baseline error lines from `docs/typing/mypy_baseline.txt`. The `discovery.tests.test_discovery_views` test module stays on the baseline — its 49 errors come from `setUpTestData` class-attribute assignments that mypy can't follow without per-test refactoring. Per-file fixes:

  - `analyzer/views.py` — fixed real bug: replaced `analysis.analyzer.analyzer_id` with `analysis.analyzer.id` (see Fixed above).
  - `analyzer/utils.py` — added `# type: ignore[assignment]` on the `get_doc_analyzer_task_by_name = None` ImportError fallback (mypy sees the original as `Callable`); switched the gating check from `not get_doc_analyzer_task_by_name` (truthy-function warning) to `is None`; cast `AnalyzerManifest` to `dict(...)` before `is_dict_instance_of_typed_dict()` since the helper expects a plain dict at runtime.
  - `agents/models.py` — scoped `# type: ignore[override]` on two `visible_to_user(user)` overrides in `AgentConfigurationManager` / `AgentActionResultManager` (django-stubs flags signature drift from `BaseVisibilityManager`); scoped `# type: ignore[misc]` on the two `objects = ...Manager()` declarations (manager class-vs-instance reassignment); added narrowing assertion on `self.corpus.title` in `AgentConfiguration.__str__` (the model `CheckConstraint` already enforces `corpus__isnull=False` when `scope == "CORPUS"`).
  - `agents/memory.py` — added narrowing `assert corpus.memory_document is not None` in the `IntegrityError` re-read branch (the truthy `corpus.memory_document_id` already proves the FK is set); added `list[str]` annotation on `result_parts` in the no-query memory-injection branch.
  - `agents/admin.py` — scoped `# type: ignore[attr-defined]` on the eight `admin_method.short_description = "..."` and one `admin_order_field = "..."` Django-admin annotation patterns mypy can't model; collapsed `obj.document.title[:30]` truncation into a single `doc_title or ""` binding so the slice no longer hits `str | None`.
  - `discovery/views.py` — scoped `# type: ignore[attr-defined]` on `Document.objects.filter(...).search_by_embedding(...)` (django-stubs returns generic `QuerySet[Document, Document]`, not the project's `DocumentQuerySet` which mixes in `VectorSearchViaEmbeddingMixin`).

- **Typing: graduate `opencontractserver.corpuses.*` from mypy baseline** (refs Issue #1447): removed five `[mypy-…]` `ignore_errors` blocks (`opencontractserver.corpuses.{admin,checks,folder_service,managers,models}`) and pruned 166 baseline error lines from `docs/typing/mypy_baseline.txt`. Per-file fixes:

  - `corpuses/checks.py` — typed `errors: list[Warning] = []` accumulator.
  - `corpuses/folder_service.py` — replaced the `if TYPE_CHECKING: User = get_user_model()` runtime-call-as-type pattern with a direct `from opencontractserver.users.models import User` (cleared 50+ "User is not valid as a type" errors with one edit).
  - `corpuses/managers.py` — added `lightweight: bool = False` parameter to `CorpusActionExecutionManager.visible_to_user` and forwarded it to `super()`, aligning the signature with `BaseVisibilityManager` (proper signature fix instead of suppression).
  - `corpuses/admin.py` — scoped `# type: ignore[attr-defined]` on the eight `admin_method.short_description = "..."` and one `admin_order_field = "..."` Django-admin annotation patterns mypy can't model.
  - `corpuses/models.py` — added `author_obj: AbstractBaseUser` annotation in `update_description` so the `int → User` and `User → User` branches share a unified type; scoped `# type: ignore[misc]` on five `creator=user` kwargs and one `author=author_obj` kwarg passed to `Document` / `DocumentPath` / `CorpusDescriptionRevision.objects.create(...)` (django-stubs widens the FK target type to `User | Combinable` while the call sites pass `AbstractBaseUser`); scoped `# type: ignore[arg-type]` on `set_permissions_for_obj_to_user(user, ...)`; scoped `# type: ignore[misc]` on the lambda in `transaction.on_commit(...)` (mypy cannot infer the captured-default-arg lambda); added narrowing `assert document is not None` in the `remove_document` else-branch (the early `if not document and not path: raise` guard already establishes the invariant); scoped `# type: ignore[assignment]` on the `CorpusActionTemplate.creator = ForeignKey(..., null=True)` field redeclaration (intentional override of `BaseOCModel.creator` to use `SET_NULL`); scoped `# type: ignore[misc]` on the `objects = CorpusActionExecutionManager()` manager override; scoped `# type: ignore[return-value,arg-type]` on the `cls.objects.bulk_create(executions)` return in `bulk_queue_executions` (the `cls(...)` instances are typed as `Self` but the function explicitly returns `list[CorpusActionExecution]`); dropped two now-dead `# type: ignore[arg-type]` comments on `md_description.open(...)` calls and one on `descendants(include_self=True)`.

- **Typing: graduate `opencontractserver.llms.tools.*` from mypy baseline** (refs Issue #1447): removed seven `[mypy-…]` `ignore_errors` blocks (`core_tools` + `core_tools.*` wildcard, `image_tools`, `moderation_tools`, `pydantic_ai_tools`, `tool_factory`, `tool_registry`) and pruned 25 baseline error lines from `docs/typing/mypy_baseline.txt`. Per-file fixes:

  - `core_tools/_helpers.py` — dropped three dead `# type: ignore` comments on `channels.db` / `asgiref.sync` imports (the partial-call concern doesn't surface under current type stubs).
  - `core_tools/text_extracts.py` / `core_tools/descriptions.py` — dropped four dead `# type: ignore[arg-type]` comments on `FieldFile.read()` / `.open()` calls.
  - `core_tools/links.py` — added a trailing `raise ValueError(f"Unhandled entity_type: {entity_type!r}")` to both `create_markdown_link` and `acreate_markdown_link` so mypy sees the function as exhaustive (without converting an unreachable to an opaque `AssertionError`).
  - `core_tools/notes.py` — widened `search_document_notes` / `asearch_document_notes` return type from `list[dict[str, str | int]]` to `list[dict[str, str | int | None]]`. The previous annotation was a real lie: `note.created.isoformat() if note.created else None` could already produce `None` values.
  - `core_tools/document_summaries.py` — replaced `author or author_id` with an explicit `summary_author = author if author is not None else author_id` + narrowing assertion; the early `if author is None and author_id is None: raise ValueError(...)` guard establishes the invariant the assertion locks down.
  - `image_tools.py` — removed redundant `images: list[ImageData] = []` re-annotation that conflicted with an earlier `images = _load_images_from_annotation_file(...)` binding; scoped a `# type: ignore[attr-defined]` on `Document.objects.filter(...).visible_to_user(user)` (django-stubs returns generic `QuerySet[Document, Document]`, not the project's custom `PermissionQuerySet`).
  - `moderation_tools.py` — scoped `# type: ignore[attr-defined]` on `message._skip_signals = True` (runtime convention, not a model field).
  - `pydantic_ai_tools.py` — typed `PydanticAIDependencies.vector_store` as `Optional[CoreAnnotationVectorStore]` (was `CoreAnnotationVectorStore` with `default=None` — a latent Pydantic validation bug); scoped `# type: ignore[attr-defined]` on the four `async_wrapper.<attr> = ...` metadata assignments used for pydantic-ai introspection and approval-gate wiring.
  - `tool_factory.py` — hoisted `param_descriptions = self.metadata.parameter_descriptions or {}` out of the `sig.parameters` loop so `.get()` no longer hits `Optional[dict[str, str]]`.
  - `tool_registry.py` — tightened `tool_class: type | None` to `tool_class: type[BaseTool] | None`; added `TYPE_CHECKING` imports for `BaseTool` / `CoreTool`; dropped the now-unnecessary `# noqa: F821` on `to_core_tool`.

- **Typing: graduate `opencontractserver.shared.{QuerySets,fields,mixins}` from mypy baseline** (refs Issue #1447): removed three `[mypy-…]` `ignore_errors` blocks (`opencontractserver.shared.QuerySets`, `opencontractserver.shared.fields`, `opencontractserver.shared.mixins`) and pruned 10 baseline error lines from `docs/typing/mypy_baseline.txt`. The remaining `Managers` and `decorators` modules still have substantial typing surfaces (django-stubs limitations around dynamic `from_queryset`, `self.model.objects` generic narrowing, etc.) and stay on the baseline for follow-up work. Per-file fixes:

  - `opencontractserver/shared/QuerySets.py` — replaced two `timezone.timedelta(...)` calls with `datetime.timedelta(...)`. `django.utils.timezone` re-exports `timedelta` only as a runtime alias; the django-stubs package does not expose it on the public `timezone` module surface.
  - `opencontractserver/shared/fields.py` — scoped `# type: ignore[override]` on `NullableJSONField.formfield(**kwargs)` since the variadic kwargs signature intentionally differs from `Field.formfield`'s positional defaults.
  - `opencontractserver/shared/mixins.py` — scoped `# type: ignore[attr-defined]` on `self.filter(...)` in `VectorSearchViaEmbeddingMixin.search_by_embedding` (provided by the QuerySet base the mixin is combined with at the concrete subclass level), and on two `self.creator` references in `HasEmbeddingMixin.add_embedding` / `add_embeddings` (provided by the model base the mixin is combined with).

- **Typing: graduate small single-file packages from mypy baseline** (refs Issue #1447): removed eight `[mypy-…]` `ignore_errors` blocks (`config.urls`, `config.jwt_utils`, `config.ratelimit.decorators`, `config.graphql_auth0_auth.utils`, `opencontractserver.conftest`, `opencontractserver.feedback.models`, `opencontractserver.worker_uploads.views`, `opencontractserver.examples.structured_response_example`) and pruned 14 baseline error lines from `docs/typing/mypy_baseline.txt`. Per-file fixes:

  - `config/urls.py` — replaced `settings.USE_SILK` with `getattr(settings, "USE_SILK", False)`. `USE_SILK` is only defined in `config/settings/local.py`, so the bare `settings.USE_SILK` access only worked because `mypy.ini` was muting the file; the `getattr` form is what other optional settings already use.
  - `config/jwt_utils.py` — replaced the runtime `User = get_user_model()` (which mypy treats as a value, not a type) with a `TYPE_CHECKING` import of `opencontractserver.users.models.User` and quoted return-type strings.
  - `config/ratelimit/decorators.py` — added `count: int | str` annotation so the `count, period_name, retry_after = "?", "period", 60` fallback (taken when `parse_rate(rate)` raises `ValueError`/`IndexError`) doesn't conflict with the `int` inferred from the success branch.
  - `config/graphql_auth0_auth/utils.py` — JWKS endpoints publish public keys only, but `cryptography` stubs widen `RSAAlgorithm.from_jwk` to `RSAPrivateKey | RSAPublicKey`. Added a localised `assert isinstance(public_key, RSAPublicKey)` before passing the key to `jwt.decode` so the union narrows correctly.
  - `opencontractserver/conftest.py` — wrapped `UserFactory()` in `cast(User, ...)` since the factory class returns a generic `UserFactory` instance under mypy.
  - `opencontractserver/feedback/models.py` — scoped `# type: ignore[misc]` on the `objects = UserFeedbackManager()` manager override (django-stubs flags the re-declaration as overriding a class variable).
  - `opencontractserver/worker_uploads/views.py` — replaced three `token: CorpusAccessToken = request.auth` assignments with `cast(CorpusAccessToken, request.auth)` so the DRF `Token | Any` return type narrows to the project's `CorpusAccessToken` subclass.
  - `opencontractserver/examples/structured_response_example.py` — added `set[str]` annotation on the `all_parties` aggregator.

- **Typing: graduate `opencontractserver.annotations.*` from mypy baseline** (refs Issue #1447): removed seven `[mypy-…]` `ignore_errors` blocks (`opencontractserver.annotations.{admin,compact_json,models,query_optimizer,utils}` plus the two `management.commands.{migrate_structural_annotations,populate_content_modalities}` blocks); added a scoped `disable_error_code = django-manager-missing` block for `annotations.models` (mirrors the existing `users.models` exception — django-stubs / django-tree-queries cannot resolve the `Corpus.label_set → LabelSet.used_by_corpuses` reverse relation). Pruned 27 baseline error lines from `docs/typing/mypy_baseline.txt` (7124 → 7097). Per-file fixes:

  - `annotations/compact_json.py` — collapsed an `isinstance(bounds, dict)` re-binding into a single `bounds_raw → bounds` assignment so the variable isn't typed `Any | None` first; renamed `token_indices: list[int]` to drop the redundant re-annotation that conflicted with the v2 branch's untyped binding.
  - `annotations/query_optimizer.py` — narrowed `path_query.first()` into a separate `first_path: DocumentPath | None` binding before the `is None` guard so mypy doesn't see the `DocumentPath | None` value flowing into the previously `DocumentPath`-typed `document_path` variable.
  - `annotations/models.py` — added `author_obj: AbstractBaseUser` annotation in `Note.update_content_with_revision`; scoped `# type: ignore[misc]` on three manager-override `objects = ...Manager()` declarations (`Embedding`, `Annotation`, `Note`); scoped `# type: ignore[assignment]` on the `StructuralAnnotationSet.creator` `ForeignKey(..., null=True)` redeclaration (intentional override of `BaseOCModel.creator`); scoped `# type: ignore[misc]` on the lambda inside `transaction.on_commit(...)` and on `author=author_obj` passed to `NoteRevision.objects.create(...)`; dropped the `: str` re-annotation on `embedder_path = CharField(...)` (CharField field declarations don't accept Python type annotations on the LHS).
  - `annotations/utils.py` — typed `all_tokens: list[Any] = []` accumulator.
  - `annotations/management/commands/populate_content_modalities.py` — typed `all_token_refs: list[Any] = []`.
  - `annotations/management/commands/migrate_structural_annotations.py` — widened the heterogeneous `stats` dict to `dict[str, Any]` so the mixed counter / error-list values type-check.

- **Badge tile overflow fix on community leaderboard** (`frontend/src/components/badges/Badge.tsx`, `frontend/src/components/community/Leaderboard.tsx`): long badge names previously broke out of their `BadgeCard` containers in the leaderboard grid because the card was a horizontal flex (`flex-direction: row`) with no flex-shrink hint, and `StyledBadge`/`BadgeName` did not allow word-break. Switched `BadgeCard` to a vertical layout (`flex-direction: column`, `align-items: flex-start`, `gap: 12px`, `min-width: 0`) so the icon stacks above the metadata and the badge can fully consume the card width; added `overflow-wrap: anywhere` plus `max-width: 100%` to `StyledBadge` and `overflow-wrap: anywhere` to `BadgeName` so very long names break at any character. `BadgeMeta` was switched from `flex: 1` to `width: 100%` to size correctly under the new column direction. `Badge.tsx`'s wrapper `<motion.span>` also gets an inline `maxWidth: "100%"` so badges constrained by ancestor flex layouts cannot escape their parent. Affects every screen that renders a `Badge` (the column-direction `BadgeCard` change is leaderboard-scoped).

- **Consolidate inline document READ checks onto `visible_to_user`** (Issue #1450, `config/graphql/document_types.py`): four resolvers (`resolve_page_annotations`, `resolve_page_relationships`, `resolve_relationship_summary`, `resolve_extract_annotation_summary`) each repeated the same hand-rolled `if not self.is_public: ... user != self.creator and not user.is_superuser ... user_has_permission_for_obj(user, self, READ)` block. The fallback to `user_has_permission_for_obj` for a Document READ check is the exact anti-pattern the helper's docstring warns against — it ignores corpus context and other visibility rules. Replaced all four blocks with a single private helper, `DocumentType._assert_user_can_read`, which delegates to `Document.objects.visible_to_user(user).filter(id=self.id).exists()` and raises a `GraphQLError` (`Authentication required` for anonymous, `do not have access` for authenticated). Behavior is preserved for the public/anonymous/owner/superuser/sharee/no-access matrix and is now driven by the canonical visibility manager. New unit suite `opencontractserver/tests/test_document_type_read_permission.py` locks the helper's contract.

- **Leaderboard page redesigned to match the OS Legal design system** (`frontend/src/components/community/Leaderboard.tsx`):

  - Replaced custom gradient/inline-style sections with a hero header, `StatGrid`-based community stats, and named styled components for every compound element (`UsernameCell`, `ScoreCell`, `DetailsCell`, `RisingStarTag`, `BadgeCard`, `BadgeMeta`, `BadgeName`, `BadgeStats`).
  - `RankBadge` palette tokenised into a file-local `RANK_COLORS` map and `RisingStarTag` colours into `RISING_STAR_COLORS`; rank-2 surface/border/text now reuse `OS_LEGAL_COLORS` tokens directly.
  - Bumped `RisingStarTag` font-size from 11px to 12px (WCAG 2.1 small-text floor).
  - Removed unused `GradientSegment` import and `statsLoading` variable.

- **Consistent MCP discoverability across corpus tiles and detail pages** (`frontend/src/components/common/MCPShareButton.tsx`, `frontend/src/components/corpuses/CorpusListView.tsx`, `frontend/src/components/corpuses/CorpusHome/CorpusLandingView.tsx`, `frontend/src/components/corpuses/CorpusHome/CorpusDetailsView.tsx`):

  - The MCP share button now renders for every corpus that has a slug, not just public ones, and the tile overlay is always visible (no longer fades in on hover).
  - `MCPShareButton` accepts a new `isPublic` prop. For private corpora the popover shows a `Lock` icon and explains that the corpus must be made public before an MCP endpoint is exposed (the backend MCP server only serves public corpora — `opencontractserver/mcp/server.py`).
  - The redundant "MCP Endpoint" entry in the corpus tile context menu was removed; the always-visible overlay button is now the single canonical entry point.

- **Extract pipeline cleanup pass** (Issue #1410): hardens the extraction-related plumbing introduced by #1381 / #1380 / #1399 against three medium-impact failure modes and four documentation/observability gaps surfaced during code review.
  - **`opencontractserver/tasks/embeddings_task.py:_batch_embed_text_annotations`** — Transient HTTP errors (`requests.Timeout`, `requests.ConnectionError`, `EmbeddingServerError`) now drop queued sub-batches and shut down the `ThreadPoolExecutor` with `wait=False, cancel_futures=True` before re-raising. Previously the exception propagated out of the `with` block, which calls `shutdown(wait=True, cancel_futures=False)` by default and blocked Celery autoretry by up to ~`max_workers` × the per-sub-batch round-trip latency. In-flight HTTP calls cannot be torn down from Python, but at least we no longer wait on them.
  - **`opencontractserver/benchmarks/loader.py:force_celery_eager`** — Refuses to mutate the global Celery config unless `settings.MODE == "TEST"` _or_ the new `OC_BENCHMARK_CLI` env var is set; outside test mode also refuses when `task_always_eager` is already `True` (concurrent benchmark runs would race on the global save/restore). In test mode an already-`True` flag is the ambient state imposed by `CELERY_TASK_ALWAYS_EAGER=True`, so the helper is a no-op there. The `run_benchmark` management command now sets `OC_BENCHMARK_CLI=1` automatically. Prevents the benchmark helper from silently routing every task in a live web/worker process through the in-process executor.
  - **`opencontractserver/llms/agents/pydantic_ai_agents.py`** — `PydanticAICorpusAgent._build_structured_system_prompt` now says "most legal corpora need multiple targeted queries" instead of "most legal documents …" — corpora-scoped wording for the corpus agent. The document-agent prompt is unchanged.
  - **`config/settings/test.py`** — The `DEFAULT_EMBEDDER` env-var override is now gated behind an explicit `BENCHMARK_MODE=1` env var. Without that opt-in, a stray `DEFAULT_EMBEDDER` value in the CI environment would silently push the regular test suite onto a real embedder and start making live network calls. The default `TestEmbedder` keeps regular `pytest` runs hermetic.
  - **`opencontractserver/tasks/data_extract_tasks.py`** — When `model_override` is accepted in the unrestricted (operator-only) path because `BENCHMARK_ALLOWED_MODEL_OVERRIDES` is unset, a `WARNING`-level log line now records the override + cell ID. WARNING (not INFO) because the open mode is a misconfiguration / abuse signal — anything firing it from a web/GraphQL caller is a real anomaly that should reach log aggregation. Lets operators `grep` production logs to confirm the open mode is fired only by the benchmark tooling and never by an unexpected web/GraphQL caller.
  - **`opencontractserver/pipeline/utils.py:get_default_reranker_instance`** — Docstring now warns that `bulk_update` / `QuerySet.update` / data-migration writes to `PipelineSettings.default_reranker` bypass the `auto_now` `modified` field used as part of the cache key, so callers who mutate the singleton via those paths must also call `invalidate_reranker_cache()` (or touch `modified` explicitly).

### Documentation

- **Auth0 admin-claim namespace mismatch** (`docs/configuration/authentication.md`): documented a silent failure mode where the Post-Login Action's `namespace` constant doesn't match `AUTH0_ADMIN_CLAIM_NAMESPACE` (e.g. `https://opencontracts.opensource.legal/` vs the default `https://contracts.opensource.legal/`). When the namespaces don't match byte-for-byte, the backend's fail-closed sync (`config/graphql_auth0_auth/utils.py:sync_admin_claims_from_payload`) treats the claims as missing and overwrites `is_staff` / `is_superuser` to `False` on every authenticated request, so the frontend admin link in the user dropdown never appears even though `app_metadata.is_superuser = true` is set in Auth0. Added a `!!! danger` callout to the Post-Login Action setup section and expanded the "Admin claim missing, defaulting to False" troubleshooting entry with the namespace-mismatch case and a `!!! info` note explaining why missing claims revoke admin instead of being ignored.

### Security

- **Audit + IDOR-pattern hardening of the four largest GraphQL mutation modules** (Issue #1449): completed the preventative permission audit called for in #1449 across `config/graphql/document_mutations.py` (1,673 lines), `config/graphql/corpus_mutations.py` (1,480 lines), `config/graphql/extract_mutations.py` (1,312 lines), and `config/graphql/annotation_mutations.py` (883 lines). Findings + fixes:
  - **Per-file verdict**: `extract_mutations.py` is fully covered (every user-supplied ID is loaded through `Model.objects.visible_to_user(user)` or guarded by `user_has_permission_for_obj` with creator/public Q-filters, and error messages are uniform). `corpus_mutations.py`, `document_mutations.py`, and `annotation_mutations.py` were mostly safe but had a handful of resolvers that loaded objects via raw `Model.objects.get(pk=...)` and then either skipped the visibility check, deferred it to a service, or returned a _different_ error message for "not found" vs "permission denied" — the exact IDOR-enumeration smell `CLAUDE.md` warns about.
  - **`config/graphql/annotation_mutations.py`** — `RemoveRelationships` (line ~489), `UpdateRelationship` (line ~550), `UpdateRelations` (line ~690), and `UpdateNote` (line ~741) now load the target object through `Relationship.objects.visible_to_user(user)` / `Note.objects.visible_to_user(user)` before any permission gating. All branches ("relationship not found", "permission denied", "note not found", "not creator") collapse to one shared error string per mutation so an attacker cannot distinguish "ID does not exist" from "ID exists but is not yours". `UpdateRelationship` additionally filters add-source / add-target annotation IDs through `Annotation.objects.visible_to_user(user)` instead of `Annotation.objects.filter(id__in=...)`, and short-circuits with the same unified message when any requested ID is invisible. Removal paths now restrict deletion to annotations actually attached to the relationship (via `relationship.source_annotations.filter(id__in=...)`) so removal cannot leak the existence of unrelated annotation IDs even though the relationship-level UPDATE check still gates the operation.
  - **`config/graphql/document_mutations.py`** — `EmptyTrash` (line ~1470), `RestoreDocumentToVersion` (lines ~1529–1530), `UploadDocument` (line ~200, corpus load), `UploadDocumentsZip` (line ~575), and `ImportZipToCorpus` (line ~922) now route corpus/document loads through `Corpus.objects.visible_to_user(user)` / `Document.objects.visible_to_user(user)`. The "corpus not found" and "you don't have permission to add documents to this corpus" branches in the upload mutations were merged into a single unified message, and `RestoreDocumentToVersion`'s separate "document version not found" / "corpus not found" / "you don't have permission to restore this document" / "you don't have permission to modify this corpus" branches were collapsed into one IDOR-safe string.
  - **`config/graphql/corpus_mutations.py`** — `UpdateCorpusDescription` (line ~266) now goes through `Corpus.objects.visible_to_user(user)` and returns the same message whether the corpus does not exist, the caller cannot see it, or the caller can see it but is not the creator. `AddDocumentsToCorpus` (line ~365) and `RemoveDocumentsFromCorpus` (line ~424) likewise route the corpus load through `visible_to_user(user)` instead of relying solely on the `DocumentFolderService` write-permission check that runs immediately afterwards. `RunCorpusAction` (line ~1284) now carries an explicit `assert user.is_superuser` and a comment documenting that its raw `CorpusAction.objects.get(...)` / `Document.objects.get(...)` calls are intentional because the resolver is gated by `@user_passes_test(lambda user: user.is_superuser)`.
  - **Test updates** (`opencontractserver/tests/test_permission_fixes.py`): the existing `RemoveRelationships` and `UpdateRelations` IDOR tests were updated to assert the new unified error string instead of the old per-branch messages. Test _intent_ is unchanged — they still verify that an unauthorized user and a user supplying a non-existent ID receive byte-identical responses — only the expected literal changed because the mutations now use the harmonised wording. No production behaviour outside the audited mutations was modified.
  - **What was not changed**: `extract_mutations.py` was left as-is (already compliant); `RetryDocumentProcessing` in `document_mutations.py` was left as-is because its raw load is followed immediately by a `user_has_permission_for_obj` check that already returns the _same_ "Document not found" string for both branches, so the IDOR pattern is preserved despite the load shape; `AddTemplateToCorpus`'s raw `CorpusActionTemplate.objects.get(pk=...)` is intentional per the in-line docstring (templates are global) and was left in place pending a separate decision on whether template publishing should ever become user-scoped.

### Fixed

- **Extraction grounding follow-up #2** (Issue #1407, follow-up to PR #1397):

  - **DB-level idempotency for grounding `get_or_create`** (`opencontractserver/annotations/migrations/0069_grounding_annotation_unique_constraints.py`, `opencontractserver/annotations/models.py`, `opencontractserver/utils/extraction_grounding.py`): added two partial `UniqueConstraint`s on `Annotation` (`annotation_unique_token_label_grounding_key`, `annotation_unique_span_label_grounding_key`) that exactly match the grounding pipeline's `get_or_create` lookup keys, scoped to a new `Annotation.is_grounding_source` boolean so non-grounding flows producing the same key tuple are unaffected. Promotes the application-level `get_or_create` from best-effort to a correctness invariant against concurrent Celery retries. `creator_id` and `structural=False` were moved into the lookup so the constraints back the application-level call. Migration backfills `is_grounding_source=True` on pre-existing `OC_EXTRACT_SOURCE` rows, dedupes survivors (keeping lowest-pk and re-pointing M2M / FK references on `Datacell.sources`, `ChatMessage.{source,created}_annotations`, `Assignment.resulting_annotations`, `Relationship.{source,target}_annotations`, `UserFeedback.commented_annotation`), then adds the constraints.
  - **Documented PostgreSQL-specific `json` lookup semantics** (`_create_span_annotation` docstring) and the dedup-by-pk loop's rationale (`_create_grounding_annotations`).

- **Blob retention across corpus-isolated Document copies** (Issue #1464): `Corpus.add_document()` and the document-versioning helpers create corpus-isolated _copies_ of a `Document` whose file fields (`pdf_file`, `txt_extract_file`, `pawls_parse_file`, `icon`, `md_summary_file`) intentionally point to the SAME S3 blobs as the source. Until now, exactly one production code path (`update_memory_content` in `opencontractserver/agents/memory.py:237`) called `txt_extract_file.delete(save=False)` unconditionally — meaning any memory document copy-shared between two corpora via `Corpus.add_document()` would see its blob silently destroyed in the sibling corpus on the next memory update. Detection was masked by `FileSystemStorage`'s path-reuse semantics: `delete()` followed by `save(SAME_NAME, …)` writes to the same path, so the sibling's `txt_extract_file.name` still resolves but reads back the _new_ (foreign-corpus) content — a quiet data-corruption failure, not a 404.
- **Defensive coverage spans every Document `FileField`, not just the one current call site.** The fix introduces a generic primitive that protects all five file fields uniformly so that the upcoming orphan-cleanup work (Issue #1492) inherits the sharing-check for free.
- **New manager method** (`opencontractserver/shared/Managers.py`): `Document.objects.unique_blob_paths(doc)` returns the set of file-field blob paths on `doc` that are NOT referenced by any other live `Document` row. The method derives its FileField list from `Document._meta.get_fields()`, so adding a new `FileField` to the model extends coverage automatically.
- **New model method** (`opencontractserver/documents/models.py`): `Document.safe_delete_field_blob(field_name, *, save=False)` is the single, public chokepoint for freeing a blob in storage. Empty field → no-op; unique blob → `FieldFile.delete()`; shared blob → field cleared on this row only, blob retained in storage. Validates that `field_name` resolves to a `FileField` on the model and raises `ValueError` for typos so silent no-ops can't hide. Any code in this codebase that needs to delete a blob from storage MUST go through this primitive instead of calling `FieldFile.delete()` directly.
- **Guard at the one current production call site** (`opencontractserver/agents/memory.py:241`): the inline shared-blob check is replaced with a single call to `locked_doc.safe_delete_field_blob("txt_extract_file")`. The behavioural contract is identical; the logic now lives on the model where every future caller can find it.
- **Regression coverage** (`opencontractserver/tests/test_blob_retention.py`, 4 `TransactionTestCase` classes):
  - `UniqueBlobPathsTestCase` (4 tests) — pins the manager method's contract: solitary docs own all blobs, shared blobs are excluded for both source and copy, per-field uniqueness after partial overwrite, empty fields produce no `""` entries.
  - `SafeDeleteFieldBlobTestCase` (5 tests) — exercises the primitive across **every** `FileField` on Document (parameterized via `DOCUMENT_FILE_FIELDS`): unique blobs are removed from storage, shared blobs are retained for the sibling, empty fields are no-ops, invalid field names raise, non-FileField names raise.
  - `MemoryDocumentBlobRetentionTestCase` (2 tests) — content-equality assertion that catches the actual failure mode: after `update_memory_content` on corpus A, sibling in corpus B still reads its original content. Failed pre-fix, passes post-fix.
  - `DocumentDeleteBlobRetentionTestCase` (2 tests, parameterized across all FileFields) — issue-spec invariant: `Document.delete()` does not destroy a blob still referenced by a sibling. Passes today (default `Model.delete()` doesn't touch FileField blobs) and is the contract test for the row-delete path; it will become live coverage once the orphan-cleanup work in Issue #1492 lands the deletion mechanic.
- **Forward pointer to Issue #1492**: this PR is the _defensive_ half of the blob-retention contract ("don't delete shared blobs"). The complementary half ("reclaim truly orphaned blobs at row-delete time") is tracked separately and will reuse `Document.safe_delete_field_blob` so the sharing-check does not need to be re-implemented.
- **Audit findings** (preserved in PR description): `django-cleanup` is not installed; no `pre_delete`/`post_delete` handlers on `Document` delete blobs (the existing handlers only GC `StructuralAnnotationSet`); `worker_uploads/tasks.py` deletes belong to `WorkerDocumentUpload.file`, a separate model; default `Model.delete()` does not delete FileField blobs.

- **MCP share popover clipped/obscured on corpus list tiles** (`frontend/src/components/common/MCPShareButton.tsx`): the popover was rendered inline inside `Container` with `position: absolute; z-index: 1000`, which left it subject to two ancestor problems on `CorpusListView`. (1) `PageContainer` sets `overflow-x: hidden`, so the popover's right edge was clipped on narrow viewports — particularly in responsive mode where the trigger sits near the right edge of the card. (2) Sibling card stacking contexts (each card's `MCPButtonOverlay` carries its own `z-index: 10` over `position: relative` cards) caused the popover from one card to render _behind_ the next card down. Fix: portal the popover into `document.body`, switch it to `position: fixed`, compute its viewport coordinates from the trigger's `getBoundingClientRect()` (clamped within `POPOVER_VIEWPORT_MARGIN` so it can't overflow the viewport on either edge), and recompute on `scroll` (capture phase, to catch nested scrollers) and `resize`. Click-outside detection now checks both `containerRef` and the new `popoverRef` since the portaled popover is no longer a DOM descendant of the trigger.

- **Slow document loading on corpora with hundreds of documents** — opening the Documents tab on a large corpus blocked behind a 5+ second "Documents Loading…" spinner because the list query did O(N) work per document for relationship metadata:

  - **Frontend** (`frontend/src/graphql/queries.ts`, `frontend/src/components/documents/ModernDocumentItem.tsx`): `GET_DOCUMENTS` no longer eagerly fetches `allDocRelationships` for every document on the page (each list page was pulling source/target/label rows for every linked document — 5,000+ extra fields on a corpus with 100 docs averaging 50 links). Added a new lazy `GET_DOC_RELATIONSHIPS_FOR_DOC` query (resolves through the existing `bulkDocRelationships` field) that fires the first time the user hovers/focuses a relationship badge on a document card. The hover popup's existing "Loading relationships…" placeholder covers the brief network round-trip. The `docRelationshipCount` badge still renders synchronously from the list query.
  - **Backend** (`opencontractserver/documents/query_optimizer.py`, `config/graphql/document_types.py`): replaced the per-document `.count()` call in `resolve_doc_relationship_count` with `DocumentRelationshipQueryOptimizer.get_relationship_counts_by_document` — a single pair of aggregated `GROUP BY` queries (one for source, one for target) that returns `{document_id: count}` for every document the user can see (optionally scoped by corpus). Result is cached on `info.context` keyed by user+corpus so all N badges in a single GraphQL response share the work. Eliminates the `Document.objects.get` + permission re-check + `select_related` query that previously ran once per document. For a 20-item page this collapses ~80 DB round-trips into 2.
  - **Backend** (`config/graphql/filters.py:in_folder`): the folder filter was materialising `DocumentPath` IDs into a Python `set()` (forcing eager evaluation) and running two extra `COUNT` queries purely for `logger.info` debug lines. Switched to a lazy `values()` queryset so Django emits a SQL subquery for the `__in` lookup and dropped the diagnostic counts — keeps the queryset fully lazy for downstream pagination.

- **Backend hardening for GraphQL CSRF / `Authorization` semantics** (Issue #1432, follow-up to #1431): the original fix already normalised empty / whitespace-only `Authorization` headers, but any _non-empty_ value still bypassed CSRF. That left a defense-in-depth gap (a malformed scheme such as `Authorization: Basic …` was treated as evidence of token auth) and the production logs were still drowning in benign `Forbidden (CSRF token missing.)` warnings. Two-part hardening:

  - **Strict scheme validation** (`config/graphql/security.py:45-104`): added `_is_recognised_token_credential` which only accepts the `Authorization` header as a token credential when it splits into exactly `<scheme> <credential>`, the credential is non-empty, and the scheme matches a recognised prefix (currently `Bearer` plus the configured `API_TOKEN_PREFIX` when API-key auth is enabled). Scheme matching is case-insensitive per RFC 7235. Empty, whitespace-only, scheme-only (`Bearer`, `Bearer  `), and unrecognised-scheme (`Basic …`) values now fall through to the cookie-based path so a malformed header cannot smuggle a session-cookie request into the token-auth bypass. `conditional_csrf_exempt` was rewritten on top of this helper; the no-cookie anonymous bypass is preserved unchanged so Bearer-only SPA boot races still get a 200 on cold start.
  - **Log volume control** (`config/graphql/security.py:157-198`, `config/settings/base.py`, `config/settings/production.py`): introduced `CsrfRejectLogFilter`, a `logging.Filter` wired into the `django.security.csrf` logger that demotes the predictable `Forbidden (CSRF token missing. | CSRF cookie not set.)` WARNING to INFO. Genuine anomalies — origin mismatch, bad referer, mismatched token — keep their WARNING level so log shipping continues to surface them. The filter inspects `record.args[0]` against a small allow-list of benign reason strings; anything else is passed through unchanged.
  - **Test matrix expansion** (`opencontractserver/tests/test_security_hardening.py`): added the missing legs called out in #1432. New `TestConditionalCsrfExempt` cases — `test_unrecognized_scheme_with_session_enforces_csrf`, `test_unrecognized_scheme_without_session_bypasses_csrf`, `test_bearer_without_credential_with_session_enforces_csrf` (covers `"Bearer"`, `"Bearer "`, `"Bearer    "`), `test_bearer_with_credential_bypasses_csrf`, `test_bearer_scheme_is_case_insensitive`, `test_api_key_scheme_bypasses_csrf` (with `API_TOKEN_PREFIX` overridden so the test is independent of `ALLOW_API_KEYS`), and `test_session_with_csrf_token_passes` — close the positive session+CSRF path that the suite previously lacked. New `TestCsrfRejectLogVolume` covers the filter contract directly: `test_filter_demotes_csrf_token_missing_warning_to_info`, `test_filter_does_not_demote_other_csrf_reasons` (origin mismatch, bad referer, `CSRF token incorrect.`), and `test_filter_passes_unrelated_records_unchanged`.
  - **Out of scope** (per the issue): no SPA auth-model changes, no CSRF-token plumbing on the frontend.

- **System Settings page failed to load with `Enum 'FileTypeEnum' cannot represent value: 'md'`** (`config/graphql/pipeline_types.py:12`): the GraphQL `FileTypeEnum` was hand-maintained and only declared `PDF`/`TXT`/`DOCX`, but the backend `FileTypeEnum` has `MD = "md"` and `oc_markdown_parser.py` declares `supported_file_types = [FileTypeEnum.MD]`. When the pipeline-components query serialized the markdown parser's supported file types, the GraphQL enum could not represent `"md"` and the entire query failed. Replaced the hand-maintained class with `graphene.Enum.from_enum(BackendFileTypeEnum)` so the GraphQL enum is generated from the backend source of truth and the two can never drift again — adding a member to the backend enum now exposes it through the schema automatically.

- **Document annotator right sidebar: crowded tabs and unresponsive Chat panel**:

  - **Tabs visually merge when panel is open** (`frontend/src/components/knowledge_base/document/styled/SidebarTabs.tsx:78-93`): `SidebarTabsContainer` set `gap: 0` whenever the panel was open, so the four vertical tabs (Index, Chat, Feed, Discussions) butted up against each other along the panel's left edge with no separation. Replaced the conditional gap with a constant `gap: 6px`, restoring per-tab visual identity in both open and closed states.
  - **`NewChatFloatingButton` ignored panel bounds** (`frontend/src/components/knowledge_base/document/ChatContainers.tsx:587-606`): the FAB used `position: fixed; bottom: 2rem; right: 2rem`, anchoring it to the viewport instead of the chat panel — at 50% panel width the "+" button rendered over the document, not the chat tray, and didn't follow panel resizes. Switched to `position: absolute` so it tracks the `SlidingPanel`, with a softer offset (`1.5rem`) and a more visible shadow.
  - **Empty conversation list collapsed to a thin strip** (`frontend/src/components/knowledge_base/document/right_tray/ConversationListView.tsx:87-104`, `frontend/src/components/knowledge_base/document/ChatContainers.tsx:389-399`): the list view's outer wrapper had no `flex: 1`/height, and `ConversationGrid` likewise didn't grow, so when no conversations existed the panel showed only the filter row over a sea of empty white. Made both flex-grow within their parent and added `grid-auto-rows: max-content` plus bottom padding so cards stack from the top and don't collide with the FAB.
  - **No empty-state messaging for Chat** (`frontend/src/components/knowledge_base/document/right_tray/ConversationListView.tsx:185-231`): added a centered empty state with a chat icon, "No conversations yet" heading, and a hint that nudges users toward the new-chat FAB. Gated on `conversations.length === 0` so it disappears as soon as data lands.

- **GraphQL POST 403 storm when Auth0 token is empty** (Issue #1431): production logs were filling with `Forbidden (CSRF token missing.): /graphql/` because the React frontend always sent `Authorization: ""` whenever the Auth0 access token was momentarily missing (startup race, silent-refresh failure, post-expiry re-mount), and `conditional_csrf_exempt` only bypassed CSRF when the header was _truthy_. An empty header fell through to Django's session/CSRF path even though the frontend never carries a CSRF token. Two-sided fix:

  - **Frontend** (`frontend/src/index.tsx:36-50`): conditional-spread `Authorization` so the header is omitted entirely when the token is empty rather than sent as the empty string.
  - **Backend** (`config/graphql/security.py:43-86`): refactored `conditional_csrf_exempt` to skip CSRF when **either** an `Authorization` header is present (Bearer / API-key — browsers don't auto-attach it) **or** no session cookie is present (the request is fully anonymous; without a cookie there is nothing for a cross-origin attacker to ride). Whitespace-only `Authorization` headers are now treated as missing. Session-cookie requests without a Bearer token still go through the existing CSRF check, so authenticated session traffic remains protected.
  - **Backend** (`config/graphql_auth0_auth/settings.py:73-79`): `Auth0JWTSettings.__getattr__` now raises `AttributeError` quietly for any name beginning with `_`, eliminating the `ERROR settings  Auth0JWTSettings.__getattr__() - Invalid setting requested: _user_settings` log noise emitted whenever Python internals (deepcopy / pickle / `hasattr`) probed dunder/private attributes. Unknown public settings still log an error.
  - **Tests** (`opencontractserver/tests/test_security_hardening.py`, `opencontractserver/tests/test_auth0_jwks_cache.py`): updated `test_session_auth_without_csrf_rejected` to inject a session cookie (so the CSRF path is actually exercised); added `test_anonymous_no_session_bypasses_csrf`, `test_empty_authorization_header_treated_as_missing`, and `test_empty_authorization_with_session_still_enforces_csrf` to lock in the new semantics; added `TestAuth0JWTSettingsDunderProbe` to assert that dunder/private probes don't emit error logs while unknown public settings still do.

- **Shared-protocol contract drift surfaced by PR #1400 follow-up review** (Issue #1408):
  - **`PermissionedQueryManagerProtocol.visible_to_user` signature mismatch** (`opencontractserver/types/protocols.py:147`): the protocol declared `visible_to_user(self, user: Any = None)` but `PermissionManager.visible_to_user` and `UserFeedbackManager.visible_to_user` have no default. A caller holding a protocol-typed reference could call `.visible_to_user()` with no args and trigger a runtime `TypeError`, and mypy would also reject the concrete classes as structural matches. Dropped the `= None` default so the protocol pins the strictest contract; the docstring now explains that callers must pass an `AnonymousUser` when no authenticated principal is available. Lenient managers (e.g. `BaseVisibilityManager` with `user=None`) still satisfy the protocol — verified with both `isinstance` and `issubclass` checks.
  - **`ToolProtocol` `@property` descriptors against a `@dataclass`** (`opencontractserver/types/protocols.py:97`): `CoreTool` is a `@dataclass` exposing `name` / `description` / `parameters` / `requires_approval` as plain instance attributes. Mypy accepted dataclass fields against the property-shaped protocol, but the asymmetry was a footgun for future implementors who would reach for `@property` based on the protocol surface. Converted to plain attribute declarations matching the concrete class.
  - **`StreamObserverProtocol` drift hazard** (`opencontractserver/types/protocols.py:152`): the protocol duplicated `opencontractserver.llms.types.StreamObserver` with no automated enforcement. A naive re-export creates a real circular-import risk because `protocols.py` is imported by `opencontractserver.shared.Managers` during Django app loading and `opencontractserver.llms.__init__` eagerly pulls in the heavy LLM stack (`api` → conversation models, agent factories). Kept duplicated definitions but added explicit "Must be kept in sync with…" notices on both sides (`opencontractserver/llms/types.py:18` and `opencontractserver/types/protocols.py:152`) explaining the rationale, and aligned the `__call__` return type on `llms.types.StreamObserver` (`-> None`, dropping the redundant `Awaitable[None]` wrapper that was already implicit in `async def`).
  - **`VectorStoreProtocol` test missed renames** (`opencontractserver/tests/test_protocols.py:24`): the canonical-implementation test used `hasattr` against hard-coded method names instead of the `@runtime_checkable` machinery used by the other three tests in the file. Renaming `search` / `async_search` would have left the test passing as long as the attribute names happened to exist via some other code path. Switched to `issubclass(CoreAnnotationVectorStore, VectorStoreProtocol)`, which executes the protocol's structural check without instantiating the class. The negative test for plain `object` was tightened from a manual `hasattr` to `issubclass(object, VectorStoreProtocol)` for symmetry.

### Added

- **Tests for issue #1410 fixes**:

  - `opencontractserver/tests/test_data_extract_failure_classification.py::PydanticAiSchemaCanaryTests` — Pin the pydantic-ai message-schema discriminators (`ModelResponse.kind == "response"`, `ToolCallPart.part_kind == "tool-call"`, `TextPart.part_kind == "text"`) so a future minor version that renames them surfaces immediately rather than silently flipping every `None` extraction into the `empty_history` mis-classification mode.
  - `opencontractserver/tests/test_batch_embedding.py::TestBatchEmbedTextAnnotations::test_transient_error_does_not_block_on_in_flight_peers` — Asserts the executor fast-fail path completes well inside the 10 s peer-block window. A regression that waits on in-flight peers (the previous default-shutdown behaviour) would take ~10 s and fail this test.
  - `opencontractserver/tests/test_benchmarks.py::ForceCeleryEagerSafetyGuardsTestCase` — Pins the three new `force_celery_eager` safety refusals: non-test mode without the CLI env var, non-test mode _with_ the CLI env var (allowed), and stacked invocations (`task_always_eager` already `True`).

- **Extract iterations & cell-level diff** — surfaces the three eval workflows callers asked for (model drift, document-version drift, fieldset-config sweeps) without forking the data model. Two new fields on `Extract` chain a parent/child relationship and capture run-time model config; everything else reuses the existing extract → fieldset → datacell pipeline so storage, permissions, and the runner are unchanged.

  - **Migration `extracts/0029_extract_iterations.py`** — adds `Extract.parent_extract` (self-FK, indexed) and `Extract.model_config` (`NullableJSONField`). Both nullable so legacy extracts continue to load with no backfill.
  - **`opencontractserver/extracts/diff.py`** — pure helper (no Graphene, no DB writes) that takes two pre-permission-filtered `Datacell` iterables and produces aligned `(row_key, column_key)` rows classified `UNCHANGED | CHANGED | ONLY_IN_A | ONLY_IN_B`. Rows are aligned by `Document.version_tree_id` so a single logical document still appears once even when the two iterations point at different content versions. `corrected_data` wins over `data` when computing equality. `_column_config_signature` gates the `column_config_changed` flag on prompt/instructions/output_type/etc., not name — so a rename alone doesn't trip it.
  - **`config/graphql/extract_mutations.py:CreateExtractIteration`** — single mutation that forks an extract along `MODEL`, `DOCUMENT_VERSIONS`, or `FIELDSET`. MODEL and DOCUMENT_VERSIONS share the parent's fieldset (column definitions stay byte-identical for apples-to-apples comparison); FIELDSET deep-clones `Fieldset` + `Column` rows via `_clone_fieldset_for_iteration` with optional per-column overrides. DOCUMENT_VERSIONS re-resolves each parent doc to the row with `is_current=True` in its `version_tree_id`. `auto_start=True` queues the existing `run_extract` task on commit — no new orchestration code path. `parent_extract` is set on the new extract; default name is `<source> (iteration N)` with N counting existing siblings + 1. Permission gate: caller needs `READ` on the source extract.
  - **`opencontractserver/tasks/extract_orchestrator_tasks.py`** — `run_extract` reads `extract.model_config.get("model")` and forwards it as the existing `model_override` kwarg of `doc_extract_query_task` (originally added for the benchmark runner). New `_task_accepts_kwarg(task_func, name)` (LRU-cached) inspects the column task's signature so the kwarg is only forwarded to tasks that accept it — custom column tasks predating `model_override` keep working unchanged. No model_config → behaviour byte-identical to before.
  - **`config/graphql/extract_types.py`** — exposes `modelConfig`, `parentExtract`, `fullIterationList`, plus a derived `iterationAxis` field (`MODEL` if `model_config` differs from parent, `FIELDSET` if fieldset differs, `DOCUMENT_VERSIONS` if doc set differs, else null) so the frontend can badge iterations without persisting the axis.
  - **`config/graphql/extract_queries.py:compareExtracts(extractAId, extractBId)`** — returns `ExtractDiffType { extractA, extractB, cells, summary }` where `cells` is a list of `ExtractCellDiffType { rowKey, columnKey, document, documentA, documentB, cellA, cellB, status, columnConfigChanged }`. Permission check reuses `ExtractQueryOptimizer.check_extract_permission` so visibility rules stay consistent across the existing `extract` resolver and the diff resolver. Cells pulled via `ExtractQueryOptimizer.get_extract_datacells` — same optimizer (and permission filtering) on both sides of the diff.
  - **Frontend — `frontend/src/components/extracts/iterations/`**:
    - `ExtractIterationsTab.tsx` lives as a 4th tab (`Iterations`) on `ExtractDetailContent`. Lists every extract in the series (parent + current + iterations) with a status bullet, axis chip, and captured-model chip. Clicking two rows toggles a comparison view inline.
    - `NewIterationDialog.tsx` is the focused fork modal — three radio-style cards (model, document versions, schema), an optional name, an optional model identifier, and a "Run immediately" toggle that maps to `autoStart`.
    - `ExtractCompareView.tsx` runs `compareExtracts` and renders the heatmap: rows are documents, columns are the union of column names, each cell is colored by `DiffStatus` and clickable to open a fixed-position side panel showing the A/B JSON. Reuses `formatCellValue` from `frontend/src/utils/formatters.ts` so cell values look identical to the main extract grid. Summary chips (changed / only-in-A / only-in-B / unchanged) above the grid use the same palette as the cell borders so the legend and grid never drift.
  - **Frontend GraphQL** — `REQUEST_GET_EXTRACT` extended with `modelConfig`, `iterationAxis`, `parentExtract`, `fullIterationList`. New `COMPARE_EXTRACTS` query and `REQUEST_CREATE_EXTRACT_ITERATION` mutation live alongside the existing extract operations in `frontend/src/graphql/queries.ts` / `mutations.ts`. `ExtractType` in `frontend/src/types/graphql-api.ts` gained the same four fields.
  - **Tests — `opencontractserver/tests/test_extract_iterations.py`** — `DiffHelperTestCase` exercises the diff algorithm directly (alignment by `version_tree_id`, `corrected_data` precedence, only-in-A/B, summary counts); `CreateExtractIterationMutationTestCase` covers all three axes, the unknown-axis error path, and the auto-incrementing default name; `CompareExtractsResolverTestCase` smoke-tests the GraphQL resolver end-to-end. Respects the `Document.version_tree_id` unique constraint by demoting `is_current` before creating successor versions.
  - **E2E coverage — `frontend/tests/e2e/extract-pdf-workflow.spec.ts`** — extends the existing pipeline spec with `forkExtractIterationViaUI` + `selectIterationsForCompare` helpers (in `frontend/tests/e2e/helpers.ts`) to drive the `Iterations` tab end-to-end after the parent extract finishes: opens `NewIterationDialog`, picks the MODEL axis, names the iteration, submits with `Run immediately` OFF (the cassette only covers parent-extract LLM calls — see helper docstring), and asserts the new row renders with its axis chip. Then selects parent + new iteration to load `ExtractCompareView`, asserting the summary chips show `Only in A: 2 / Changed: 0 / 2 cells compared` and the heatmap header includes the column name. Proves `createExtractIteration → fullIterationList → compareExtracts` end-to-end without burning a second LLM round-trip.

- **VCR.py wrapper for LLM calls in `doc_extract_query_task`** — `opencontractserver/utils/vcr_replay.py` exposes a `maybe_vcr_cassette()` context manager that, when `OC_LLM_VCR_MODE` and `OC_LLM_VCR_CASSETTE` are set on the celery worker, records or replays every HTTP call to LLM provider hosts (currently `api.openai.com` / `api.anthropic.com`). A custom request-body matcher strips volatile values (millisecond timestamps, Django document PKs, OpenAI tool-call IDs, UUIDs) so a cassette recorded against one DB replays cleanly against another. With the env vars unset the wrapper is a no-op — production behavior is unchanged. Pre-recorded cassette for the E2E extract spec lives at `opencontractserver/tests/fixtures/cassettes/e2e_extract_pdf_workflow/extract.yaml`. Replay was verified end-to-end against a deliberately-fake `OPENAI_API_KEY` to confirm no real network call is made. See `docs/development/e2e_vcr.md` for record / replay / debug instructions.
- **`.github/workflows/frontend-e2e-extract.yml`** — CI workflow that runs the E2E extract spec against the full `local.yml` stack with `OC_LLM_VCR_MODE=replay` and a fake `OPENAI_API_KEY`. Triggered on `pull_request` (path-filtered to changes that can affect extract behaviour) in addition to `workflow_dispatch`. PDF parsing routes to the in-stack Docling microservice (`PDF_PARSER=docling`); the workflow forces `PipelineSettings.preferred_parsers["application/pdf"]` to Docling explicitly after `migrate` so a pre-existing DB singleton can't silently route through LlamaParse. No external service credentials required.
- **`frontend/tests/e2e/extract-pdf-workflow.spec.ts`** — full-stack Playwright E2E spec for the extract pipeline: login → create corpus → upload two PDFs (`frontend/tests/fixtures/{usc-title-1,eton-agreement}.pdf`) → wait for parse + embedding → create extract with one column → run with a real OpenAI call → CSV export → assert non-empty cells. Adds new helpers to `frontend/tests/e2e/helpers.ts` (`uploadPdfViaUI`, `waitForDocumentReady`, `createExtractViaUI`, `openExtractByName`, `addColumnViaUI`, `addDocumentsToExtractViaUI`, `runExtractAndWaitForFinish`). Gated on `E2E_RUN_LLM_TESTS=true`; skipped in CI until LLM responses can be mocked over the wire. Runs on the live `local.yml` stack; required tweaks to disable Auth0 (`.envs/.local/.django USE_AUTH0=false`) and to widen the celeryworker `watchfiles --ignore-paths` (in `compose/local/django/celery/worker/start` and the `local.yml` command pointer) so editor / Playwright artifact writes don't hot-reload the worker mid-task. Also adds `data-testid="document-card"` (+ `data-processing` on the `/documents`-view variants) to `frontend/src/views/Documents.tsx` and `data-testid="document-card"` to `frontend/src/components/documents/ModernDocumentItem.tsx`, so tests can poll for the `backendLock` UI signal without depending on hover-only action menus. Cards are matched by `[data-testid="document-card"]` filtered with the visible title text — the standard Playwright pattern.
- **Mypy: type analyzer, shared, agents, badges, worker_uploads; introduce shared protocols** (Issue #1335): Brought the five smaller, interface-rich target packages over the ≥70% return-annotation bar called for by the issue and seeded `opencontractserver/types/protocols.py` with the four protocols requested in the scope:
  - `VectorStoreProtocol` — minimum surface (`search` / `async_search`) implemented by `CoreAnnotationVectorStore` (`opencontractserver/llms/vector_stores/core_vector_stores.py`); imported and re-exported from that module so consumers can annotate against the protocol rather than the concrete dataclass.
  - `PipelineComponentProtocol` — `title` / `description` / `author` / `dependencies` surface that the pipeline registry duck-types against; imported from `opencontractserver/pipeline/base/base_component.py` so any future parser/embedder/thumbnailer registered outside the inheritance hierarchy still type-checks against the same contract.
  - `ToolProtocol` — `name` / `description` / `parameters` / `requires_approval` mirror of `CoreTool` (`opencontractserver/llms/tools/tool_factory.py`); framework adapters can accept any object satisfying it, no inheritance required.
  - `PermissionedQueryManagerProtocol` — `visible_to_user(user) -> QuerySet` contract that `BaseVisibilityManager`, `PermissionManager`, `DocumentManager`, `AnnotationManager`, and `NoteManager` all satisfy (`opencontractserver/shared/Managers.py`); imported there so callers receiving "a permissioned manager" can type against the protocol instead of a concrete class.
  - `StreamObserverProtocol` — duplicated `__call__(event)` shape from `opencontractserver/llms/types.StreamObserver` for callers in non-LLM modules (notifications, websockets) that need the contract without importing `llms.types`.
  - **Coverage delta** (return-annotation coverage measured by AST walk, excluding `__init__.py`):
    - `analyzer/`: 12.5% → **87.5%** (target ≥70%)
    - `shared/`: 38.1% → **96.3%** (target ≥70%)
    - `agents/`: 34.4% → **71.9%** (target ≥70%)
    - `badges/`: 47.1% → **100%** (target ≥70%)
    - `worker_uploads/`: 46.4% → **100%** (target ≥70%)
  - **Files touched** (annotations only, zero behavior changes): `analyzer/{apps,checks,signals,startup,utils,admin,admin_views}.py`, `analyzer/management/commands/sync_doc_analyzers.py`, `agents/{apps,admin,memory}.py`, `badges/{apps,signals,models}.py`, `worker_uploads/{apps,auth,serializers,views,models,tasks}.py`, `shared/{utils,defaults,fields,mixins,Managers,QuerySets,decorators}.py`, plus the four protocol-consumer wirings above.
  - **Bare-generic promotion**: every `dict` / `list` / `set` in newly-touched public signatures was widened to a parametrised form (`dict[str, Any]`, `list[Any]`, `dict[str, int]`, etc.). The `HasEmbeddingMixin` docstring example was tightened to match (`-> dict[str, Any]`).
  - **`# type: ignore` audit**: every bare `# type: ignore` comment in the codebase now carries a specific error code. `opencontractserver/llms/tools/core_tools.py:413,416,418,420` (channels/asgiref import + `partial` kwarg-aware wrapper) tightened to `[import-not-found]` / `[call-arg]` with a one-line comment explaining why; `opencontractserver/utils/embeddings.py:171` to `[attr-defined]`; `opencontractserver/tests/test_pipeline_utils.py:395,423,431,438,446` cleaned up the duplicated `# type: ignore; type: ignore` markers and scoped them to `[import-not-found]`; `opencontractserver/tests/test_core_tool_factory.py:34,35,47` widened from bare to `[attr-defined]`. The total count went from 62 → 61, and the ratio of bare-to-scoped dropped to zero.
- **Return-type annotations across `config/graphql/` resolvers and mutations** (Issue #1332, follow-up to #1331): The largest, least-typed subtree in the backend (459 function definitions, ~4.4% return-annotation coverage at baseline) is now at 100% return-annotation coverage. Touched files include every `*_mutations.py`, every `*_queries.py`, every `*_types.py`, plus `filters.py`, `base.py`, `base_types.py`, `security.py`, `optimized_file_resolvers.py`, `permissioning/permission_annotator/{middleware,mixins,utils}.py`, and the small utility modules. No behavioral changes — annotations only.
  - **`mutate(...)` on `graphene.Mutation` subclasses**: typed as forward references to the enclosing class (`-> "ClassName"`). Discovered and fixed the latent bug in `config/graphql/analysis_mutations.py:179` (`DeleteAnalysisMutation.mutate`) where the success path had no `return` statement; annotation is `-> "DeleteAnalysisMutation | None"` and an explicit `return None` was added to preserve the original implicit-None behavior on success.
  - **`resolve_*` methods**: typed as `-> Any` by default, refined where the GraphQL field type makes the runtime return obvious (e.g. `resolve_in_use -> bool`, `resolve_datacell_count -> int`).
  - **`AnnotatePermissionsForReadMixin`** (`config/graphql/permissioning/permission_annotator/mixins.py`): per the issue's specific guidance, `resolve_my_permissions -> list[str]`, `resolve_is_published -> bool`, `resolve_object_shared_with -> list[dict[str, Any]]`. The pre-existing wrong annotation `list[PermissionTypes]` (an Enum, while the implementation returns plain strings) was corrected to `list[str]`. The now-unused `PermissionTypes` import was removed.
  - **Filter / queryset helpers** (`filter_by_*`, `text_search_method`, `get_node`, `get_queryset`, `_get_*`, etc.) typed as `-> Any` to keep the change conservative; tightening to `QuerySet[Model]` is a follow-up.
  - **`config/graphql/permissioning/permission_annotator/utils.py`** had a broken import (`config.graphql.permission_annotator.middleware` instead of `config.graphql.permissioning.permission_annotator.middleware`) — fixed in passing.
  - **`config/graphql/conversation_types.py`**: replaced `base64.binascii.Error` with a direct `binascii.Error` import (pre-existing — `base64` re-exports `binascii` at runtime but `mypy` doesn't see the re-export).
  - **Var-annotated additions**: `id_to_children: dict[Any, list[Any]]` in `base_types.py`, `read_only_fields: list[str]` in `serializers.py`, `this_model_permission_id_map: dict[int, str]` etc. in middleware.
  - **Five modules graduated from the mypy baseline** (`mypy.ini` → no longer `ignore_errors = True`): `config.graphql.base_types`, `config.graphql.conversation_types`, `config.graphql.permissioning.permission_annotator.middleware`, `config.graphql.permissioning.permission_annotator.utils`, `config.graphql.serializers`. Their entries in `docs/typing/mypy_baseline.txt` (11 lines) were also pruned. Future PRs can graduate the remaining baselined files as the structural issues they expose (custom `visible_to_user` manager method not seen by `django-stubs`, `set_permissions_for_obj_to_user` signature mismatch, mixin `_meta` access) are addressed.
  - **Tooling**: zero new `# type: ignore` markers; black & isort applied; `flake8 config/graphql/` clean. `mypy --config-file mypy.ini opencontractserver config` passes with the updated baseline.
- **Mypy: graduated `opencontractserver/users/tasks.py` out of the baseline** (Issue #1333 follow-up): `tasks.py` was the last `opencontractserver.users` module still suppressed in `mypy.ini`. PR #1370 left it untyped because the file is only loaded when `settings.USE_AUTH0=True`, so it never failed at runtime under the test settings; the typing gap kept the package short of the issue's "all four packages at ≥80% return-annotation coverage" Done-When criterion. Added return + parameter annotations to all five Auth0 sync tasks (`get_new_auth0_token`, `apply_data_to_user`, `sync_remote_user`, `ensure_valid_auth0_token`, `get_user_details_async`), introduced a module-level docstring documenting the `USE_AUTH0` gating, and removed the `[mypy-opencontractserver.users.tasks] ignore_errors = True` section. Local `data` rebound from request body (`dict[str, str]`) to response payload (`dict[str, Any]`) was split into two distinctly-named variables (`request_data` / `payload`) so the types are unambiguous; behavior is unchanged. No callers needed updating — `config/graphql_auth0_auth/utils.py` still consumes `sync_remote_user.delay(...)` exactly as before.

### Fixed

- **Structured extraction agent loops on tool calls instead of committing** (Issue #1414, follow-up to #1381 / #1413): The E2E spec from PR #1413 surfaced a real `failure_mode=no_final_response` (the classifier added by PR #1399). With `gpt-4o-mini`, a strict prompt ("verbatim title from page 1"), and a 9-page PDF, the agent received the answer in its very first `load_document_text(start=0, end=500)` call and then made **97 more sequential 500-byte byte-range reads** through the entire body without ever calling `final_result*`. The structured runner exhausted `output_retries=3` and returned `None`. Two underlying biases were addressed in the structured-extraction system prompts:
  - **Tool-loop without commit** — the agent had the answer but treated reading as the entire task. The `2-3 distinct search queries` rule (added in #1381) was worded as an unconditional precondition to committing, which encouraged exhaustive reading whenever the prompt was constrained. All three `_build_structured_system_prompt` overrides in `opencontractserver/llms/agents/pydantic_ai_agents.py` (`PydanticAICoreAgent`, `PydanticAIDocumentAgent`, `PydanticAICorpusAgent`) now lead with a **COMMIT-EARLY rule**: as soon as a tool result contains a confident answer, the agent MUST stop calling tools and commit by calling the result tool. The 2-3-search rule is now explicitly **negative-case-only** — it bounds giving up, not committing — and rule precedence is stated inline so the model can't conflate the two.
  - **Tool selection bias** — for prompts that map cleanly onto pre-chunked, embedded structural annotations (titles, parties, defined terms, specific clauses), the agent skipped `similarity_search` entirely and went straight to byte-range reads. The same prompt overrides now state that `similarity_search` is the **preferred first step** for fact-finding queries because the search index is already chunked and ranked over structural annotations, and reserve `load_document_text` (and other byte-range readers) for whole-document tasks (summaries, exhaustive review) or as a fallback when search clearly misses. The corpus override carries the matching guidance for document-coordination tools.
  - **Tests** — `opencontractserver/tests/test_pydantic_ai_agents.py` extended: `test_document_agent_structured_prompt_commits_to_result`, `test_corpus_agent_structured_prompt_commits_to_result`, and `test_core_agent_base_structured_prompt_commits_to_result` now assert on the new `COMMIT-EARLY` and `similarity_search` / `load_document_text` guidance, and the base-agent test asserts that the old unconditional "Before concluding the requested information is absent, you MUST issue at least 2-3 distinct search queries…" wording is **gone** — that exact phrasing is what produced the loop in #1414.
  - **Out of scope** — no model-side changes (no temperature override, no `output_retries` change), no rework of the structural-annotation similarity index. The classifier from #1381 / #1399 still surfaces any residual `no_final_response` if the model continues to loop on a different prompt shape.
  - **End-to-end verification (PR follow-up)** — re-recorded `opencontractserver/tests/fixtures/cassettes/e2e_extract_pdf_workflow/extract.yaml` against the new prompt with the in-stack Docling parser, then replayed the spec from PR #1413 cassette-only. The originally-failing USC cell now produces `"TITLE 1-GENERAL PROVISIONS"` (body text from page 1) and the Eton cell commits its description fallback — both non-empty. The cassette has 13 OpenAI interactions and contains zero occurrences of the old `Before concluding the requested information is absent…` wording, confirming the prompt the agent was asked under is the new one.
  - **VCR matcher hardening (`opencontractserver/utils/vcr_replay.py`)** — required for the re-record to replay against fresh DBs. (1) Strip Django auto-increment PKs (`annotation_id` / `corpus_id` / `document_id` / `label_id` / generic `id`) from request bodies; PKs are echoed back into subsequent assistant calls inside JSON-encoded tool messages, so the patterns match both the raw-JSON form and the escaped-string form (`\"annotation_id\\":N`). (2) Strip `similarity_score` / `score` floats — the embedder microservice and pgvector hybrid-search fusion produce slightly different numeric scores even for identical inputs across runs. (3) Replay drops body matching entirely — `record_mode="none"` now matches on `(method, scheme, host, port, path)` only and plays interactions back in cassette order. The vector-embedder microservice is real (not cassette-wrapped because it is not an LLM provider) and returns slightly different similarity orderings against a fresh corpus, which leaks chunk content into subsequent request bodies that no static stripper can normalize. Sequential playback is sufficient because the celery worker processes datacells sequentially, the model is effectively deterministic at the temperatures used, and the cassette captures the full successful conversation per document. Record mode keeps strict body matching so partial re-records still produce a deterministic cassette. New unit tests in `opencontractserver/tests/test_vcr_replay.py` cover the PK stripper (raw + escaped forms) and the similarity-score stripper.
  - **CI activation (`.github/workflows/frontend-e2e-extract.yml`)** — flipped from `workflow_dispatch`-only to `pull_request` (path-filtered to changes that can affect the extract agent or the cassette / matcher / spec). Forces `PDF_PARSER=docling` and explicitly resets `PipelineSettings.preferred_parsers` to Docling after `migrate` so a stale DB singleton can't silently route through LlamaParse. No external service credentials needed.
  - **Test-helper fixes for `frontend/tests/e2e/helpers.ts`** uncovered while replaying the spec end-to-end:
    - `addDocumentsToExtractViaUI` no longer hard-fails when the "Add documents" modal renders zero cards — the corpus auto-populates extract rows on creation, so the second `addDocuments` call legitimately has nothing to add. Probes for cards with a short timeout and bails to "Cancel" on the empty-modal path.
    - `runExtractAndWaitForFinish` now waits for the AG-Grid `Loading…` placeholder to detach after the "Extraction in progress" overlay clears. Previously the helper returned as soon as the backend marked the Extract row complete, but the per-cell `data` payloads were still mid-flight via Apollo, which made cell-text assertions intermittently see whitespace. Source of the "row has no non-empty extracted cell" false-negative when the spec was first re-run.
    - `waitForDocumentReady` no longer gates on the outer "Your documents" page heading. The route can render the document-card grid before its surrounding chrome (the heading lives in an outer layout component that occasionally hydrates last); the helper now waits directly on the document card it cares about with an inner 30s ceiling, since the card is a sufficient signal that the documents view loaded. Source of an intermittent 8-minute hang on `/documents` re-navigation.
- **Anthropic models silently fail in `doc_extract_query_task`** (Issue #1381): When `doc_extract_query_task` was run with an Anthropic / Claude model, ~85% of cells failed with the canonical "extraction returned None — the requested information may not be present" message even though the document contained the answer. Inspecting `Datacell.llm_call_log` for failed cells showed Claude's last assistant message was always `text` + `tool_use` parts, never a final structured response — pydantic-ai's structured-response runner treated this as no result and returned `None`. The error message conflated three distinct outcomes ("agent committed to None", "agent never produced a final structured response", "agent looped on the same tool call") under one ambiguous string. Three coordinated changes:
  - **`opencontractserver/llms/agents/pydantic_ai_agents.py`** — All three `_build_structured_system_prompt` overrides (`PydanticAICoreAgent`, `PydanticAIDocumentAgent`, `PydanticAICorpusAgent`) now explicitly tell the agent that **after gathering enough information from the tools, it MUST commit to the final structured response by calling the result tool**. Wording is universal — harmless for OpenAI and necessary for Claude. `_structured_response_raw` now passes `output_retries=STRUCTURED_OUTPUT_RETRIES` (=3) to `PydanticAIAgent` so pydantic-ai retries the final-result tool call when the model fails to commit on the first pass. When an Anthropic model is used and the caller did not pin a temperature, structured runs force `temperature=0` (Claude is reluctant to commit; non-zero temperature pushes it toward more exploratory text). **Cost note**: bumping `output_retries` from pydantic-ai's default of 1 to 3 means a cell that fails the first commit attempt can incur up to 3 result-tool round trips. In the previous behaviour those cells silently returned `None` instead of retrying, so the worst-case per-cell LLM cost on Anthropic models can roughly triple compared to the broken baseline. This is the correct tradeoff — the previous baseline produced no answer — but operators should anticipate the cost shift in billing for Anthropic-driven extractions.
  - **`opencontractserver/constants/llm.py`** — Hosts `STRUCTURED_OUTPUT_RETRIES`, `TOOL_LOOP_THRESHOLD`, `EXTRACT_DEFAULT_TEMPERATURE`, and the `NONE_RESULT_*` vocabulary used by the classifier (see below).
  - **`opencontractserver/utils/llm.py`** — New `is_anthropic_model()` helper so call sites outside the agents layer (notably `data_extract_tasks.doc_extract_query_task`) can decide whether to pass `temperature=None` and let the Anthropic guard activate.
  - **`opencontractserver/tasks/data_extract_tasks.py`** — New `_classify_none_result(messages)` helper inspects the captured pydantic-ai message log and returns one of `agent_committed_none` (a `final_result*` tool call appears — legitimate "data not present"), `no_final_response` (no `final_result*` anywhere — pipeline integration failure), `tool_loop_no_output` (same tool call repeated ≥ `TOOL_LOOP_THRESHOLD`× without final — pipeline bug), or `unknown`. The `Datacell.stacktrace` now records `failure_mode=<classification>` plus a human-readable message (the integration-failure variants reference issue #1381) so operators can `grep failure_mode=` to separate legitimate "not present" outcomes from pipeline bugs. New `_resolve_extract_temperature(model_name)` helper picks the temperature passed to the structured runner: returns `None` for Anthropic models so `_structured_response_raw`'s `temperature=0` override fires automatically, and `EXTRACT_DEFAULT_TEMPERATURE` (0.3) otherwise. This closes the latent footgun where flipping `DEFAULT_EXTRACT_MODEL` to a Claude model would have silently bypassed the reliability fix because `temperature=EXTRACT_DEFAULT_TEMPERATURE` was passed unconditionally.
  - **`opencontractserver/tests/test_data_extract_failure_classification.py`** — `SimpleTestCase` suite covering the classifier (empty input, `final_result` detection, `final_result_<TypeName>` suffix variants, no-tool-calls path, tool-call-without-final path, repeated-call loop detection, threshold-minus-one boundary, loop-then-commit precedence, mixed text + tool path, non-`ModelResponse` skip, JSON-string `args` normalisation, malformed JSON `args` defensive path, unhashable `args` `repr` fallback), `is_anthropic_model` (prefix, bare-name, OpenAI / `gpt-4` / `o1` rejection, empty/None), and `_resolve_extract_temperature` (Anthropic→None, OpenAI→default, unknown→default, current-default sanity check).
  - **`opencontractserver/tests/test_pydantic_ai_agents.py`** — New `_structured_response_raw` tests pinning the Anthropic temperature override (forces 0 when caller passes `temperature=None`, respects function-level pin, respects `config.temperature` pin, leaves OpenAI runs untouched), and three smoke tests for the strengthened `_build_structured_system_prompt` overrides covering the document, corpus, and core base agents.
- **Extraction grounding follow-up** (Issue #1246, follow-up to original #1245 grounding pipeline):
  - **Bug — silent `page=1` fallback corrupted multi-page PDF grounding** (`opencontractserver/utils/extraction_grounding.py`, `_create_pdf_annotation`): when PlasmaPDF could not determine a page for a span, the previous code logged a warning and saved the annotation on page 1 anyway. For multi-page PDFs this produced a structurally incorrect annotation pinned to the wrong page (and therefore the wrong bounding box context), so users clicking through to the source landed on a different page than the one containing the extracted text. Fixed: `_create_pdf_annotation` now raises `ValueError` inside its `transaction.atomic()` savepoint, the savepoint rolls back, and the outer per-result `try/except` in `_create_grounding_annotations` logs it as a failed grounding attempt. Best-effort grounding is preserved (other annotations in the batch are unaffected) but no annotation is ever saved with a wrong page.
  - **Bug — label-set lookup outside the per-annotation guard caused all-or-nothing failure** (`opencontractserver/utils/extraction_grounding.py`, `_create_grounding_annotations`): `corpus.ensure_label_and_labelset(...)` was invoked once before the per-annotation `try/with transaction.atomic()` loop. A failure to materialise the label-set (e.g. a transient DB error or a pre-existing constraint conflict) propagated out, was caught by the outer `try/except` in `data_extract_tasks.py`, and silently dropped _all_ groundings for the datacell. Moved the call inside the savepoint so a label-lookup failure only skips the affected annotation.
  - **Bug — duplicate `OC_EXTRACT_SOURCE` annotations on Celery retry** (`opencontractserver/utils/extraction_grounding.py`, `_create_pdf_annotation` & `_create_span_annotation`): nothing prevented the grounding pipeline from creating fresh annotations and re-linking them via `datacell.sources.add(*annotations)` if `ground_extraction_to_annotations` ran twice on the same datacell (Celery retry after partial failure was the realistic trigger). Replaced the construct-then-`save()` flow with `Annotation.objects.get_or_create()` keyed on `(document, annotation_label, annotation_type, raw_text, …)` so retries reuse existing rows. `datacell.sources` is a `ManyToManyField`, so re-linking the same row is already a no-op once the row is shared.
  - **Constant — extracted `DOCX_MIME_TYPE`** (`opencontractserver/constants/document_processing.py`): the long `application/vnd.openxmlformats-officedocument.wordprocessingml.document` literal previously lived inline in `_load_document_text_and_layer`. Per the project's no-magic-strings rule it now sits next to `MARKDOWN_MIME_TYPE` and is imported from one place.
  - **Type annotations** (`opencontractserver/utils/extraction_grounding.py`): `Document`, `Corpus`, `Datacell`, `Annotation`, and `AnnotationLabel` parameters and return types added via a `TYPE_CHECKING` block on every public and helper function. No runtime change.
  - **Documentation** — the `page=1` placeholder for SPAN_LABEL annotations (text/DOCX) is now documented in the function docstring, explaining that the `txt_extract_file` pipeline does not preserve a page-break map and the actual location lives in the character offsets in `json`.
  - **Tests** — `opencontractserver/tests/test_extraction_grounding.py`:
    - `TestGroundingPipelinePDFIntegration` (new class): builds a synthetic two-page PAWLS payload (no real PDF binary needed), runs grounding through `build_translation_layer`, and verifies (a) annotations land on the correct page, (b) re-running grounding is idempotent, and (c) when PlasmaPDF returns `page=None` the annotation is **skipped** instead of being saved on page 1.
    - `test_ground_text_document_is_idempotent`: regression for the duplicate-annotation bug on the SPAN_LABEL path.
- **`CreateCorpusActionModal` opened with the wrong default agent instructions for document triggers** (Issue #1385, `frontend/src/components/corpuses/CreateCorpusActionModal.tsx:136-144,168-171`): the `inlineAgentInstructions` state was initialised with `DEFAULT_MODERATOR_INSTRUCTIONS` even though the default trigger is `add_document` (a document trigger). The trigger-change handler at line 611 swaps to `DEFAULT_DOCUMENT_AGENT_INSTRUCTIONS`, but a user who created an inline agent on the default-selected trigger without first re-selecting the trigger would submit the moderator copy as the new agent's system instructions. Initialised both the `useState` default and `resetForm()` to `DEFAULT_DOCUMENT_AGENT_INSTRUCTIONS` so the pre-interaction value matches the default trigger. Updated `frontend/tests/CreateCorpusActionModal.ct.tsx` "inline-agent create: full happy path" mutation mock to expect `DEFAULT_DOCUMENT_AGENT_INSTRUCTIONS` — the previous mock variable masked this bug because `MockedProvider` was matching the stale moderator default rather than the trigger-appropriate one.

### Changed

- **Test/type cleanup follow-ups from the PR #1383 review** (Issue #1385):

  - Pinned the `isProcessing` contract for SYNC_CONTENT in `frontend/tests/CorpusChat.ct.tsx` "SYNC_CONTENT renders a complete message immediately": added an `expect(input).toBeEnabled()` assertion after the reply renders, locking the documented invariant that `setIsProcessing(true)` is owned solely by `ASYNC_START` and that a SYNC_CONTENT-only reply must never disable the input.
  - Consolidated the duplicated `::: oc-component` fence dispatcher: extracted `OcComponentBlock` interface and a new `buildOcComponentCustomBlocks(renderMarkdown)` helper into `frontend/src/utils/camlComponents.ts`. Both `frontend/src/hooks/useCamlComponentRenderer.tsx` and `frontend/src/components/corpuses/caml/CamlDirectiveRenderer.tsx` now share the same helper instead of each casting `block` independently.
  - Replaced `route: any` and `page: any` escape hatches with the proper `Route` and `Page` types from `@playwright/test` in `frontend/tests/CorpusDescriptionEditor.ct.tsx` (`setupMdRoute` and the abort-route test).
  - Migrated `.version-number` CSS-class locators in `frontend/tests/CorpusDescriptionEditor.ct.tsx` to a semantic `data-testid="version-number"` matcher (`page.getByTestId("version-number")`); added the test id to the rendered version-row in `frontend/src/components/corpuses/CorpusDescriptionEditor.tsx`.

- **`test_superuser_sees_all_queryset` miscounts personal corpuses by 1** (Issue #1394, `opencontractserver/tests/test_visibility_managers.py`, `opencontractserver/tests/test_resolvers.py`): Two `VisibleToUserTests.test_superuser_sees_all_queryset` cases asserted that `Corpus.objects.visible_to_user(superuser).count() == 4` (public + private + 2 personal), but the actual count is 5 because the test DB starts with a pre-existing personal corpus owned by django-guardian's `AnonymousUser` (created during fixture setup before/around the username-based skip in `opencontractserver/users/signals.py::user_created_signal`). The assertion is now scoped to corpuses created by the test's two users (`creator__in=[self.user, self.superuser]`), making it resilient to any fixture-level corpuses that exist at test DB init time. Production code is unchanged.
- **Merged `frontend` Codecov flag drops to ~33% on every commit where Frontend CI's CT job fails** (`frontend/package.json` `test:coverage:ct`): the script chained `playwright test ... && mkdir -p ... && nyc report ...`, so a failing CT run short-circuited before `nyc report` could turn the per-test JSON files in `.nyc_output` into an `lcov.info`. The downstream `Upload CT Coverage to Codecov` step (`if: success() || failure()`) then errored with "No coverage reports found" and `frontend-component` did not upload for that SHA. Codecov's server-side aggregation of the `frontend` flag was left with only `frontend-unit` (~23%) and `frontend-e2e` (~24%), pulling the merged number down to ~33% even though the previous commit was at ~67% — observed on six consecutive main commits 2026-04-26T01:02..02:58Z (`2d7033f8`..`be5bcfc8`) before recovering on `30298391`. Mirrored the existing `test:e2e:coverage` pattern (`; CT_EXIT=$?; nyc report ... || echo "No coverage data to report"; exit $CT_EXIT`) so `nyc report` runs regardless of test outcome and the lcov ships even on red CT runs. `frontend-component` will still report a slightly lower number when tests fail (failed tests register fewer hits), but it will report — keeping the merged `frontend` flag's denominator stable.
- **`User.__init__` shared-state mutation re-introduced by branch merge** (`opencontractserver/users/models.py:172-180` removed): PR #1374 (commit `50ed6740`) deleted the `User.__init__` override that mutated `Field.validators[0]` on every instantiation, but a subsequent merge (`b68c1cb4 → 6d2cddbf`) resurrected the override along with its mypy-narrowing changes. The current main on commit `6d2cddbf` therefore reproduced the original `#1358` bug: `User(...)` rebound `username_field.validators[0]` and clobbered any third-party validator prepended to the list. Removed the `__init__` override entirely; the class-body declaration `validators=[UserUnicodeUsernameValidator()]` on the `username` field (still present from PR #1374) is the canonical and only declaration. Also dropped the now-unused `Field` import. Regression coverage from PR #1374 (`opencontractserver/tests/test_user_username_validator.py`) was already on main and is what surfaced the regression in CI.
- **Decomposed `opencontractserver/llms/tools/core_tools.py` into a package** (Issue #1445): The 2,981-line monolith has been split into the `opencontractserver/llms/tools/core_tools/` package with one submodule per tool category — `md_summaries`, `notes`, `text_extracts`, `descriptions`, `document_summaries`, `annotations`, `document_indexing`, `search`, `page_images`, `links`, `documents`, `memory`, plus a private `_helpers` for shared utilities (`_token_count`, `_db_sync_to_async`, `_apply_ndiff_patch`). The package's `__init__.py` re-exports every previously top-level name (functions, classes, `_DOC_TXT_CACHE`, and the `Document`/`Note`/`Corpus`/`CorpusDescriptionRevision`/`NoteRevision` model references that tests patch directly), so no consumer import paths change. Behavior-preserving — each function body is identical to the original; this is purely a structural refactor to make individual tool families reviewable in isolation. Documentation references in `docs/architecture/pawls-format.md`, `docs/architecture/document_annotation_index.md`, `docs/architecture/llms/README.md`, `docs/walkthrough/step-7-query-corpus.md`, and the `opencontractserver/corpuses/template_seeds.py` comment were updated to point at the new package paths.

### Security

- **`RemoveAnnotationLabelsFromLabelsetMutation` allowed any user to strip labels from public labelsets** (`config/graphql/label_mutations.py`): The resolver used `Q(creator=user) | Q(is_public=True)` with no further check, so `is_public` (a read flag) effectively granted write access to non-owners. Replaced with `user_has_permission_for_obj(user, labelset, PermissionTypes.UPDATE, include_group_permissions=True)`. Denial raises `LabelSet.DoesNotExist` so the response is byte-identical to the not-found case (no IDOR information leak). Coverage: `test_remove_labels_rejects_non_owner_even_when_labelset_is_public`.
- **`CreateLabelForLabelsetMutation` ignored guardian UPDATE grants** (`config/graphql/label_mutations.py`): The resolver was creator-only, so collaborators with explicit `update_labelset` guardian permission (or a group grant) could not add labels even though the consolidated permissioning guide says edit rights on a `LabelSet` should permit add/remove/edit of its labels. Now uses the same `PermissionTypes.UPDATE` gate as `Remove`, with the matching IDOR-safe deny path. Coverage: `test_non_owner_with_explicit_update_permission_can_create`. Both mutations now have a dedicated `LabelSet.DoesNotExist` handler that logs at WARNING (not ERROR) so auth probes don't pollute logs with stack traces.
- **Cross-corpus structural-annotation leak in `CoreAnnotationVectorStore`** (`opencontractserver/llms/vector_stores/core_vector_stores.py:296-326,371-413`): The corpus-wide retrieval path (`corpus_id` set, `document_id=None`) returned every structural annotation in the database regardless of corpus. Two collaborating defects caused the leak:
  1. `Q(structural=True)` in the corpus-only branch had **no corpus constraint** — parser-produced structural annotations have `Annotation.document_id = corpus_id = NULL`, so corpus membership is only knowable through `structural_set → Document.structural_annotation_set (reverse FK) → DocumentPath.corpus_id`, a join the previous code did not perform.
  2. The `check_corpus_deletion` block (default `True`) added `Q(document_id__in=active_doc_ids)`, and `__in` lookups never match `NULL`, so structural annotations were silently dropped on the production-default path. Bypassing this filter with `check_corpus_deletion=False` exposed defect #1 directly.
  - **Impact**: Multi-tenant deployments leaked structural annotations across tenant corpora — a real security boundary violation since the upfront IDOR check only validated the _requested_ `corpus_id`, not the _returned_ rows. Single-tenant deployments saw it as a corpus-scoping / search-quality bug (e.g. corpus-wide benchmark runs returned chunks from abandoned corpora). The standard `Annotation.objects.visible_to_user()` permission filter was bypassed entirely because the vector store builds its own filter chain rather than going through that manager method.
  - **Fix** (corpus boundary + per-document visibility for the structural class): the corpus-only branch now requires `structural_set_id__in=<sets reachable from a document in this corpus that is visible to the user>`, joining through `Document.objects.visible_to_user(user).filter(path_records__corpus_id=...)`. The deletion-aware filter accepts both `document_id__in=active` AND structural rows whose set links to one of those active documents, so parser-produced structural annotations remain reachable on the default path. `CoreAnnotationVectorStore.global_search()` was already correct (it explicitly joins via `structural_set__documents__in=accessible_doc_ids`) and is unchanged.
  - Regression coverage: `opencontractserver/tests/test_corpus_isolation_vector_store.py` — six tests covering cross-corpus leak, deletion-aware drop, orphan-set leak, document-scoped retrieval still returns structural rows, viewer-without-doc-permission excluded, creator still sees own row.
- **Test-only**: `opencontractserver/tests/test_pydantic_ai_agents.py`, `opencontractserver/tests/test_structural_annotation_portability.py` — `Document.objects.create(...)` calls in `TransactionTestCase` setUp now pass `processing_started=timezone.now()` to short-circuit `process_doc_on_create_atomic`, which would otherwise eagerly chain a Celery PDF-ingest task that fails on the (file-less) test document and aborts the whole test class. Pre-existing failure, exposed cleanly when the regression suite was added.

### Fixed

- **`Embedding.embedder_path` could be NULL but was typed `str`** (Issue #1357, `opencontractserver/annotations/models.py:461-465`, `opencontractserver/annotations/models.py:584-585`, `opencontractserver/annotations/migrations/0068_enforce_embedder_path_not_null.py`): The Django field was declared `null=True, blank=True` while the Python annotation claimed `str`, causing a long-standing mypy `assignment` error and — more importantly — silently gutting the partial unique constraints added in migration 0059. Each `unique_embedding_per_{document,annotation,note,conversation,message}_embedder` constraint is conditioned on `<parent>__isnull=False` and keys on `(embedder_path, <parent>)`, so any row with `embedder_path IS NULL` bypassed duplicate prevention for its parent. Every production code path that creates an `Embedding` (`Embedding.objects.store_embedding()`, `HasEmbeddingMixin.add_embedding()`, `worker_uploads._store_embeddings()`) already supplies a concrete `embedder_path` or skips creation when empty, so enforcing non-null at the DB level matches actual behaviour rather than constraining it. New migration 0068 backfills any legacy NULL rows with `settings.DEFAULT_EMBEDDER` (deleting rows that would collide with an existing `(default_embedder_path, parent)` row under the partial unique constraint — they were previously unreachable via any query path since all call sites filter on a concrete embedder path), then `AlterField`s the column to `NOT NULL`. Removed the now-unreachable `or 'Unknown Model'` fallback in `Embedding.__str__`. Migration runs with `atomic = False` so the RunPython backfill commits before `AlterField` takes the `ACCESS EXCLUSIVE` lock to set `NOT NULL`, matching the pattern established by migration 0059.

### Added

- **Frontend `any`-baseline gate wired into CI** (Issue #1448): The frontend has no ESLint pipeline, so explicit `any` accretion was silent — ~449 type-position uses across 123 files at the time of this change. New script `frontend/scripts/check-any-baseline.js` walks `frontend/src/**/*.{ts,tsx}`, counts `: any | as any | <any> | any[] | Array<any> | Promise<any> | ReadonlyArray<any>` (skipping comment-only lines), and compares against the committed `frontend/.any-baseline.json` snapshot (total + per-area breakdown across `knowledge_base`, `annotator`, `widgets_chat`, `widgets_other`, `components_other`, `graphql`, `hooks`, `atoms`, `routing`, `utils`, `types`, `other`). New scripts: `yarn any:check` (regression-only), `yarn any:check:strict` (also fails when the count drops without the baseline being lowered, keeping the snapshot honest), `yarn any:write` (regenerate after a reduction). The Lint job in `.github/workflows/frontend.yml` runs `any:check:strict` on every PR. Workflow + rationale documented in `docs/frontend/any-baseline.md`. Per the issue, follow-up PRs are expected to drain the prioritised areas (`knowledge_base` 33 → … → 0, `annotator` 90 → …, `widgets_chat` 2 → 0) by replacing `any` with types from `frontend/src/types/graphql-api.ts` / GraphQL codegen output, calling out the per-area delta in each PR.
- **Pluggable text chunking strategies for `TxtParser`** (Issue #1348, alongside PR #1239): Introduced `opencontractserver/pipeline/parsers/text_chunkers.py` — a small registry-backed abstraction (`BaseTextChunker` + `TextChunk` + `get_chunker`) with three built-in strategies: `SentenceChunker` (spaCy `doc.sents`, preserves pre-#1348 behaviour and emits the existing `SENTENCE` label), `ParagraphChunker` (blank-line split with optional `min_chars` filter and `max_chars` oversize-paragraph fallback, emits `PARAGRAPH`), and `SlidingWindowChunker` (fixed-character window with configurable `overlap` and optional `respect_word_boundaries` snap, emits `WINDOW`). `TxtParser` now declares a `Settings` dataclass with a `chunkers: list[ChunkerSpec]` field (default `[{"name": "sentence"}]`) that can be overridden via `PipelineSettings` _or_ per-call via a `chunkers=[...]` kwarg on `parse_document`; the parser iterates the configured strategies and emits one structural SPAN_LABEL annotation per chunk under each strategy's label, so stacked configurations (e.g. sentence + paragraph) index multiple retrieval granularities simultaneously. Motivates the benchmark work in #1239: the LegalBench-RAG `probe_recall_at_10` gap on `privacy_qa` (0.22 observed vs 0.5–0.8 paper floor) is the thesis for needing paragraph-granularity retrieval units, but this PR is strategy-neutral — which chunker wins for which subset is a follow-up optimisation to be driven by the benchmark harness itself. Regression coverage in `opencontractserver/tests/test_text_chunkers.py` (pure-Python, no Django DB) exercises offset/whitespace invariants, overlap arithmetic, word-boundary snapping, argument validation and registry lookup; `test_txt_ingestor_pipeline.py` gains two integration tests that parse the live fixture with a paragraph-only and a stacked paragraph+sliding_window recipe. Existing sentence-only ingestion path is unchanged.
- **Global post-retrieval reranker for vector search** (Issue #1349): Adds an optional cross-encoder reranking stage that runs after first-stage vector / hybrid retrieval, so OpenContracts can close the gap between vanilla HNSW recall and the accuracy achievable with a cross-encoder scoring pass.
  - New abstract base class `opencontractserver.pipeline.base.reranker.BaseReranker` wired into the existing `PipelineComponentBase` settings machinery: concrete subclasses declare a `Settings` dataclass (loaded from `PipelineSettings` at runtime) and implement `_rerank_impl(query, passages, **kwargs)`. A default `_arerank_impl` wraps the sync implementation via `sync_to_async` so every backend has a working async path without duplicating logic.
  - Fault-tolerant helpers `safe_rerank` / `safe_arerank` swallow reranker failures and return `None` so retrieval degrades gracefully to the first-stage ordering — critical because a misconfigured reranker must never take down semantic search.
  - Four shipped backends in `opencontractserver/pipeline/rerankers/`:
    - `NoopReranker` — identity pass-through for tests and benchmark control conditions.
    - `CrossEncoderReranker` — in-process `sentence_transformers.CrossEncoder` (default `BAAI/bge-reranker-v2-m3` per the issue); lazy model load cached by `(model_name, device)` so workers pay the ~300 MB cost once and reuse it on every query. `sentence-transformers` / `torch` are treated as optional dependencies; a missing install surfaces a clear `ImportError` only when this backend is actually selected.
    - `MicroserviceReranker` — HTTP client that mirrors the shape of `MicroserviceEmbedder` (URL, optional API key, Cloud-Run IAM auth, retry-friendly timeouts). Operators can run any reranker model behind a `/rerank` endpoint and point OpenContracts at it via `RERANKER_MICROSERVICE_URL` (+ secret `RERANKER_MICROSERVICE_API_KEY`).
    - `CohereReranker` — hosted Rerank API (`rerank-v3.5` by default) via the REST endpoint directly (no hard dep on the `cohere` SDK). API key stored in the encrypted `PipelineSettings.encrypted_secrets` bag under `cohere_api_key` (env var `COHERE_API_KEY` at migration time).
  - New `ComponentType.RERANKER` enum value and `rerankers/` auto-discovery in `opencontractserver.pipeline.registry`; `PipelineComponentRegistry` now exposes `.rerankers` / `get_all_rerankers_cached()` alongside parsers, embedders, thumbnailers, and post-processors.
  - `PipelineSettings.default_reranker` (CharField, max_length=512, `documents/models.py:852-980`) — empty string disables reranking; any value is a full class path resolved at runtime. Seeded from `DEFAULT_RERANKER` Django setting at migration time (`documents/migrations/0037_add_default_reranker_to_pipeline_settings.py`). Helpers `get_default_reranker_path()` / `get_default_reranker_class()` / `get_default_reranker_instance()` in `opencontractserver.pipeline.utils`, with a process-local instance cache (cross-encoder model weights are expensive) invalidated via `invalidate_reranker_cache()` on every settings update.
  - `CoreAnnotationVectorStore` (`opencontractserver/llms/vector_stores/core_vector_stores.py:120-1041`) now accepts an optional `reranker` override + `rerank_oversample_factor` kwarg. Every search path — `search`, `async_search`, `hybrid_search`, `async_hybrid_search`, `global_search`, `async_global_search` — oversamples candidates by `RERANK_OVERSAMPLE_FACTOR` (default 3× the requested `top_k`, hard-capped by `RERANK_MAX_CANDIDATES = 128`) when a reranker is active and re-orders results through `_apply_rerank` / `_aapply_rerank` before returning the final `top_k`. All new plumbing is a no-op when `default_reranker` is empty, so zero behavior change for existing deployments.
  - GraphQL surface: `PipelineComponentsType.rerankers`, `PipelineSettingsType.default_reranker`, and `UpdatePipelineSettingsMutation.default_reranker` (validated against the registry, invalidates the reranker instance cache on change).
  - New constants in `opencontractserver.constants.search`: `RERANK_OVERSAMPLE_FACTOR`, `RERANK_MAX_CANDIDATES`, `RERANK_DEFAULT_TOP_K`. New `RERANKER_REQUEST_TIMEOUT_SECONDS` in `opencontractserver.constants.document_processing`.
  - Tests in `opencontractserver/tests/test_reranker.py` cover the base-class contract (sorting, top_k trim, out-of-range indices, max-candidates, async fallback), `safe_rerank` / `safe_arerank` fault-tolerance, all three HTTP backends with mocked `requests.post`, pipeline utility resolution + instance caching, registry auto-discovery, and vector-store integration (oversample factor, reranker failure fallback, re-ordering effects).
- **Mypy graduation: typed GraphQL resolvers, mutations, and filters** (Issue #1332): Raised return-annotation coverage in `config/graphql/` from ~4.8% at the start of #1331 to **91.5%** (421/460 function defs) and removed 22 modules from the `mypy.ini` baseline allow-list.
  - **Root-cause annotation fixes in `opencontractserver/utils/permissioning.py`**: `set_permissions_for_obj_to_user`, `user_has_permission_for_obj`, `get_users_permissions_for_obj`, and `get_permission_id_to_name_map_for_model` were previously annotated with `instance: type[django.db.models.Model]` (a class) despite every call site passing an instance — and with `user: type[User]` instead of the `User` runtime instance. These were annotation bugs (the code was correct, the annotations were inverted), which compounded: every mutation calling `set_permissions_for_obj_to_user(user, obj, ...)` was a single `[arg-type]` error each. Corrected to `instance: django.db.models.Model` / `user: UserModel` (forward-referenced via `TYPE_CHECKING` import of `opencontractserver.users.models.User`). Also added the missing `dict[int, str]` annotation on `this_model_permission_id_map` and removed the `user_instance=User` (class) default on `get_users_group_ids`, which would have exploded at runtime if any caller ever omitted the argument. Module graduated out of the baseline.
  - **Graduated from `mypy.ini` baseline** (22 modules): `config.graphql.{action_queries, agent_mutations, badge_mutations, base_types, conversation_mutations, conversation_types, corpus_types, document_queries, filters, ingestion_source_mutations, moderation_mutations, og_metadata_queries, pipeline_queries, security, serializers, slug_queries, smart_label_mutations, social_types, user_queries, user_types, voting_mutations}` and `opencontractserver.utils.permissioning`. Each had the underlying mypy errors fixed first (root-cause in `permissioning.py` cleared the `set_permissions_for_obj_to_user` cluster across every mutation file above).
  - **Per-file type fixes**:
    - `config/graphql/slug_queries.py` & `config/graphql/user_types.py` & `config/graphql/social_types.py` & `config/graphql/corpus_types.py`: Reversed `.filter(...).visible_to_user(user)` → `.visible_to_user(user).filter(...)` so the custom manager method (typed on the manager) resolves before `.filter()` flattens to the base `QuerySet[Model]` that django-stubs doesn't know carries `visible_to_user`. Semantics are preserved — both orderings AND the conditions. The CLAUDE.md permissioning docs already recommend the manager-first pattern.
    - `config/graphql/og_metadata_queries.py`: Guarded against `Extract.corpus` being `None` (the FK uses `on_delete=SET_NULL`, so `corpus` is nullable in the DB but the OG metadata resolver was treating it as non-null).
    - `config/graphql/pipeline_queries.py`: Narrowed the `mimetype` optional before passing to `get_components_by_mimetype_cached` (which required `str`), and typed `components_data` as `dict[str, Sequence[PipelineComponentDefinition]]` so both branches (which return `list[...]` vs `tuple[...]`) type-check against the unified annotation.
    - `config/graphql/ingestion_source_mutations.py`: Replaced `if error:` with `if pk is None:` in two call sites — the error-then-continue pattern left mypy unable to narrow `pk: str | None` through the conditional. Functionally equivalent (`error is None ⟺ pk is not None` by construction of `_parse_ingestion_source_global_id`).
    - `config/graphql/conversation_types.py`: Fixed `base64.binascii.Error` → `binascii.Error` with an explicit `import binascii` — `base64` doesn't re-export `binascii` as an attribute, so the reference was broken under `warn_unused_ignores`.
    - `config/graphql/filters.py`: Coerced `from_global_id(value)[1]` (returns `str`) to `int` before passing to `folder_id` lookup.
    - `config/graphql/security.py`: Replaced `CsrfViewMiddleware(lambda req: None)` with a typed `_csrf_noop_get_response(request) -> HttpResponse` so the middleware's `get_response` contract is satisfied in-types; switched `wrapped_view.csrf_exempt = True` and `request._dont_enforce_csrf_checks = True` to `setattr(...)` to avoid typing-only attribute errors against Django stubs that don't carry these internal flags. Behaviour identical.
    - `config/graphql/moderation_mutations.py`: Added an explicit `Union[ChatMessage, Conversation, None]` annotation on `target` where the surrounding `if/else` mixes the two types (mypy can't unify across branches without the hint).
    - `config/graphql/action_queries.py`: Coerced `from_global_id(...)[1]` to `int` at the three call sites where it feeds `for_corpus` / `for_document` custom queryset methods (which expect `int` PKs).
  - **Docs & baseline maintenance**: `mypy.ini` baseline section for `config.graphql` trimmed from 35 modules to 14; 63 matching error lines pruned from `docs/typing/mypy_baseline.txt` so the reference file matches the live baseline.
  - **Known remaining bugs surfaced by mypy** (filed as separate issues per the scope rules of #1332):
    - #1359 — `RemoveLabelsFromLabelsetMutation` calls non-existent `labelset.documents`. Silent runtime failure (swallowed by a broad `except Exception`). Blocks `config.graphql.label_mutations` graduation; one-line fix + test needed.
    - #1360 — `DRFMutation.IOSettings` declares `model: django.db.models.Model = None` and `serializer = None`. Non-trivial refactor of the base mutation class; blocks `config.graphql.base` graduation.
- **Coverage: raise Corpus Chat & Agent Management component tests** (Issue #1276): added 36 new Playwright CT tests across the four lowest-ROI corpus components to drive coverage toward the ≥60% target. Breakdown:
  - `frontend/tests/CorpusChat.ct.tsx` (+13 tests): `initialQuery` auto-send, tool-call timeline entries (ASYNC_THOUGHT), ASYNC_SOURCES merge, SYNC_CONTENT rendering, ASYNC_RESUME, ask_document sub-tool approval remapping, unknown-type default branch, back-to-list navigation, server-message-with-sources rendering, title-filter debounce, and additional navigation-header coverage. Extended the shared `StubSocket` in `beforeEach` with new query-triggered frame sequences.
  - `frontend/tests/CreateCorpusActionModal.ct.tsx` (+8 tests): analyzer-path validation, inline-agent validation (empty name / empty instructions), existing-agent-selection validation, successful inline-agent mutation, backend error toast, analyzer edit-mode pre-population, and legacy trigger-casing normalization fallback.
  - `frontend/tests/CorpusAgentManagement.ct.tsx` (+8 tests): query loading state, query error state, multi-tool badge overflow, inactive-status badge, update-mutation happy path, create-mutation backend-error toast, tool deselection, and edit-modal cancel.
  - `frontend/tests/CorpusDescriptionEditor.ct.tsx` (+7 tests): save failure (`ok: false`), save network-error path, reapply of snapshot-less version, twice-click collapse, Cancel Version Edit reset, fetch-md URL failure, and version-count pluralization.
  - **Follow-up review polish**: moved `DEFAULT_DOCUMENT_AGENT_INSTRUCTIONS` from `CreateCorpusActionModal.tsx` into `frontend/src/assets/configurations/constants.ts` so both default-instruction strings (moderator + document agent) live in the single constants module per the project's no-magic-strings rule.
- **Return-type annotations across core models and import/export pipeline** (Issue #1334, follow-up to #1331): The mypy gate wired in by #1331 recorded a 7208-error baseline frozen across 357 files. This PR pays down the annotation deficit on the core domain models and the bulk import/export tasks without touching runtime behavior or adding validators. Coverage jumped from the pre-issue numbers to:
  - `opencontractserver/corpuses/` 61.5% → 88.4% (target ≥80%)
  - `opencontractserver/annotations/` 48.1% → 93.8% (target ≥80%)
  - `opencontractserver/documents/` 51.9% → 84.9% (target ≥80%)
  - `opencontractserver/extracts/` 47.1% → 94.1% (target ≥75%)
  - `tasks/import_tasks.py`, `tasks/import_tasks_v2.py`, `tasks/export_tasks.py`, `tasks/export_tasks_v2.py` all at 100% function-signature coverage.
  - **Files touched** (annotations only — zero behavior changes, zero new comments, zero renames): `annotations/{models,signals,admin}.py`, `corpuses/{models,signals,managers}.py`, `documents/{models,signals}.py`, `extracts/{models,signals}.py`, `tasks/{import_tasks,import_tasks_v2,export_tasks,export_tasks_v2}.py`.
  - Each file adopts `from __future__ import annotations` so forward references work without string quoting, and uses a `TYPE_CHECKING` block for the handful of cross-app imports (`AbstractBaseUser`, `Corpus`, `Document`, `Annotation`, etc.) that would otherwise be circular at runtime.
  - **TypedDict adoption** — the import/export task signatures now consume existing TypedDicts from `opencontractserver/types/dicts.py` directly instead of bare `dict`: `OpenContractsExportDataJsonPythonType` / `OpenContractsExportDataJsonV2Type` for the data.json payloads, `OpenContractDocExport` for per-document payloads, `OpenContractsAnnotationPythonType` / `OpenContractsRelationshipPythonType` for annotation/relationship lists, `IngestionSourceExport` / `DocumentPathExport` / `StructuralAnnotationSetExport` / `CorpusFolderExport` / `AgentConfigExport` / `DescriptionRevisionExport` / `ConversationExport` / `ChatMessageExport` / `MessageVoteExport` for V2-specific shapes, and `OpenContractsAnnotatedDocumentImportType` for single-document imports. No new TypedDicts were introduced — all were already defined and previously unused in callers.
  - Signal handlers (`annotations/signals.py`, `corpuses/signals.py`, `documents/signals.py`, `extracts/signals.py`) now have typed `sender` / `instance` / `created` / `**kwargs` parameters with `TYPE_CHECKING` imports of the sender model classes.
  - **Manager methods**: `CorpusActionExecutionManager.visible_to_user` now types the optional `user` param as `Optional[AbstractBaseUser]` (previously bare default), and `summary_by_status` / `summary_by_action` tightened from bare `dict` / `QuerySet` to `dict[str, int]` / `QuerySet[Any]`.
  - **Model method signatures**: `save()`/`clean()`/`delete()` overrides are now `-> None` (Django's `Model.save` returns None) or `-> tuple[int, dict[str, int]]` (Django's `Model.delete` signature); `__str__` returns `str`; `@property` accessors have explicit scalar returns; classmethod factories return `"ClassName"`; tree/versioning helpers on `CorpusFolder` type their descendant queries as `QuerySet[CorpusFolder]` / `list[CorpusFolder]` where applicable (falling back to `Any` where the CTE queryset class isn't easily importable).
  - **Graduation from the mypy baseline is deferred.** The `mypy.ini` `[mypy-…] ignore_errors = True` sections for these modules still contain pre-existing Django-plugin-specific errors (field descriptor `assignment` mismatches, `misc` lambdas in model field declarations, base-class variable override warnings on `embedder_path`/`creator`) that are not caused by missing annotations and therefore cannot be fixed under this issue's "annotations only, no bugfixes" constraint. The annotations landed here make those modules ready for a follow-up PR that either `# type: ignore`s or refactors the remaining Django-model type issues and then removes the baseline entries.
- **Mypy type-checking wired into pre-commit and CI** (Issue #1331): The existing `[mypy]` block in `setup.cfg` and the `mypy==1.20.1` / `django-stubs==6.0.2` / `djangorestframework-stubs==3.16.9` pins in `requirements/local.txt` were never actually enforced, so the investment was drifting (48 pre-existing `# type: ignore` markers, many modules at 0% annotation coverage). This PR turns on the gate without requiring the 7208 existing errors across 357 files to be fixed first — `mypy.ini` lists each of those files under its own `[mypy-<module>] ignore_errors = True` section, so new modules added outside the baseline **are** type-checked and CI / the hook fails on their errors.
  - `mypy.ini` (split out of `setup.cfg` because the per-module baseline is ~1000 lines): `python_version` bumped `3.9` → `3.11` to match the runtime, plugins kept, `django_settings_module` pointed at the new `config/settings/mypy.py` (dummy `DATABASE_URL` default so contributors don't need env vars).
  - `.pre-commit-config.yaml`: new `mirrors-mypy@v1.20.1` hook with `pass_filenames: false` and fully pinned stubs + Django runtime in `additional_dependencies` (pre-commit autoupdate only bumps `rev`, so unpinned stubs would drift).
  - `.github/workflows/backend.yml`: `Run mypy` step added to the `linter` job; the preceding `pip install -r requirements/local.txt` step satisfies the django-stubs plugin.
  - `docs/typing/README.md`: how to run locally / via pre-commit / in the test container, plus the per-file graduation workflow.
  - `docs/typing/mypy_baseline.txt`: frozen error list (sorted for stable diffs) so follow-up issues can measure progress.
- **Mypy: typed the auth, users, and notifications packages** (Issue #1333, follow-up to #1331): Brought `config/admin_auth`, `config/graphql_api_token_auth`, `opencontractserver/users`, and `opencontractserver/notifications` to full return-annotation coverage and graduated them out of the mypy baseline. These packages sit on hot paths for authentication, token verification, session handling, and notification dispatch; typing them documents invariants that used to live only in `CLAUDE.md`.
  - Signal handlers (`opencontractserver/users/signals.py`, `opencontractserver/notifications/signals.py`) now declare `sender: type[Model]`, `instance: Model`, `created: bool`, `**kwargs: Any`, and `-> None` so drive-by edits can't silently change the signal contract.
  - `opencontractserver/notifications/__init__.py` gained a module-level docstring calling out that the app deliberately diverges from `AnnotatePermissionsForReadMixin` — a single `recipient` FK is the authoritative visibility gate. The design note in `CLAUDE.md` is referenced so changes to one file flag the other.
  - `mypy.ini`: dropped `[mypy-config.admin_auth.*]`, `[mypy-config.graphql_api_token_auth.backends]`, and `[mypy-opencontractserver.users.models]` `ignore_errors` sections. Added a narrow `disable_error_code = django-manager-missing` on `opencontractserver.users.models` only, documented inline: `django-tree-queries`' `as_manager(with_tree_fields=True)` pattern used by `Corpus`, `CorpusFolder`, and `DocumentPath` creates manager classes at runtime that the `mypy_django_plugin` can't introspect, so reverse `_set` accessors on `User` can't resolve — a known plugin limitation, not a code error. Graduating those three models with explicit manager typing will let us delete the disable.
  - `docs/typing/mypy_baseline.txt`: pruned all 20 entries for the graduated modules.
  - Bug fixes uncovered by typing:
    - `config/graphql_api_token_auth/backends.py`: `ApiKeyBackend.authenticate_header` referenced `self.keyword` but the attribute was never declared. DRF calls this on 401 responses to build the `WWW-Authenticate` header, so an unauthenticated request would have hit `AttributeError` at response time. Added `keyword: str = "Token"` as a class attribute.
    - `opencontractserver/notifications/signals.py`: `create_moderation_notification` dereferenced `action.moderator.username` unconditionally, but the `ModerationAction.moderator` FK permits `NULL`. A moderation action created without a moderator would crash the signal handler (and therefore the originating save). Added an explicit early return with a debug log when `action.moderator is None`.

### Changed

- **Decomposed `ModernDocumentItem.tsx`** (Issue #1446, `frontend/src/components/documents/ModernDocumentItem.tsx`, `frontend/src/components/documents/ModernDocumentItem.styles.ts` (new), `frontend/src/components/documents/DocumentRelationshipList.tsx` (new)): The component had grown to 1,699 lines with ~755 lines of `styled-components` definitions and a relationship-popup body duplicated verbatim across the card and list view branches. Extracted all styled-components into a sibling `ModernDocumentItem.styles.ts` (re-exported by name) and lifted the duplicated relationship popup body into a shared `DocumentRelationshipList` component used by both views. Net: main component file dropped from 1,699 → 870 lines (49% reduction) while remaining behaviorally identical — the public `ModernDocumentItem` export, prop signature, and DOM structure are unchanged. All 42 component tests in `frontend/tests/ModernDocumentItem.ct.tsx` and `frontend/tests/document-failure-overlay.ct.tsx` pass without modification, confirming the refactor is semantics-preserving. First slice of the multi-PR decomposition tracked by issue #1446.
- **Centralized extract and user-profile route resolution in `CentralRouteManager`** (`frontend/src/routing/CentralRouteManager.tsx`, `frontend/src/components/routes/ExtractDetailRoute.tsx`, `frontend/src/components/routes/UserProfileRoute.tsx`, new `frontend/src/components/routes/ProfileRedirect.tsx`, `frontend/src/utils/navigationUtils.ts`, `frontend/src/graphql/cache.ts`): `ExtractDetailRoute` and `UserProfileRoute` were each calling `useParams()`, running their own GraphQL resolution queries, and writing entity reactive vars — duplicating the four-phase flow documented in `docs/frontend/routing_system.md` and racing the manager during back-navigation. Both are now thin consumers that read `openedExtract` / `openedUser` / `routeLoading` / `routeError`. `parseRoute` learned `/users/:slug` and `/extracts/:extractId`; the manager added a `GET_USER` lazy query and a Phase 1 user branch alongside the existing extract handling. New `openedUser` reactive var (typed `OpenedUserProfile`) joins the routing-owned set. `/profile` is now served by a small `ProfileRedirect` that uses `backendUserObj` to redirect to `/users/<slug>` — auth-driven, not URL-driven, so legitimately outside the manager. Smaller fixes that ride along: `views/Corpuses.tsx:1807` swaps `window.history.replaceState` for `navigate({ pathname, search }, { replace: true })` so query-param mutations stay inside React Router; redundant `openedLabelset(null)` calls in `LabelSetDetailPage` and `LabelSetLandingRoute` handlers were removed (Phase 1 already clears the var on browse navigation).
  - Test coverage: 8 new `parseRoute` tests, 3 new manager tests (user resolve, user not-found, extract by id), updated `beforeEach` to reset `openedExtract` / `openedUser`. New `frontend/src/routing/__tests__/centralRouteDiscipline.test.ts` is a static regression test that grep-walks `frontend/src/` and fails if any production file outside the manager and `cache.ts` SETs one of the 15 routing-owned reactive vars — caught three pre-existing `openedLabelset` writes during development that were also fixed in this PR. New `frontend/tests/e2e/user-and-extract-routes.spec.ts` deep-links `/users/<slug>`, verifies the `/profile` redirect, and exercises the dumb-consumer error path on `/extracts/<unknown-id>`. `tests/e2e/helpers.ts` `VIEWS` catalog now includes `/users/admin` so the existing login-and-navigation walk also covers the user route.
  - Doc updates: `docs/frontend/routing_system.md` route-pattern table now lists `/users/:slug`, `/extracts/:extractId`, and `/label_sets/:labelsetId`; the "ONLY CentralRouteManager may SET" list and the four critical RULE blocks were extended to include `openedThread`, `openedLabelset`, and `openedUser`; the new discipline test is referenced as the CI enforcement mechanism.
- **Pre-existing `tsc` failure on `frontend/src/components/corpuses/caml/CamlDirectiveRenderer.tsx`** resolved by refreshing `@os-legal/caml-react` to `0.1.0` (latest on the registry). The lockfile had been pinned to `0.0.1`, whose published `dist/index.d.ts` lacks the `resolveImageSrc` prop that the consumer passes through; `0.1.0` ships the prop on `CamlArticleProps`, `CamlChapterRendererProps`, and the block-renderer prop interfaces. `yarn upgrade @os-legal/caml-react@^0.1.0 @os-legal/caml@^0.1.0` updates the lockfile; no code changes needed.
- **Simplified `RelationGroup.updateForAnnotationDeletion` pruning logic** (Issue #1317, follow-up to #1314, `frontend/src/components/annotator/types/annotations.ts:40-60`): The method previously branched on four near-duplicate conditions (`sourceEmpty && nowTargetEmpty`, `targetEmpty && nowSourceEmpty`, `!sourceEmpty && nowSourceEmpty`, `!targetEmpty && nowTargetEmpty`) each returning `undefined`. All four are equivalent to a single `nowSourceEmpty || nowTargetEmpty` check (`filter` is monotonic, so an originally-empty side stays empty after filtering). Collapsed the branches and removed the now-unused `sourceEmpty` / `targetEmpty` locals per the project's DRY guideline. Behavior is unchanged; existing regression tests in `frontend/src/components/annotator/types/__tests__/annotations.test.ts` still pass unmodified, confirming the simplification is semantics-preserving.
- **Frontend Vitest unit coverage provider switched from V8 to Istanbul** (`frontend/vite.config.ts:210-230`, `frontend/package.json:154`, `frontend/yarn.lock`): The merged `frontend` Codecov flag was landing at ~44% even though `frontend-component` alone was at ~61% on the same code — impossible for a union-aggregated metric unless the three per-suite uploads were measuring on different yardsticks. Root cause: Vitest's `@vitest/coverage-v8` provider emits ~183 `DA:` records per source file (~86k across ~480 files) because V8's native coverage API reports hits for nearly every executable line — imports, declarations, block-closing `}`, etc. — as needed for engine profiling and DevTools. `vite-plugin-istanbul` (used by Playwright CT and E2E suites) emits ~61 `DA:` per file (~27k total) because it only instruments statements. Same code, ~3× denominator mismatch. When Codecov unions multiple uploads under one flag it keeps V8's larger line-number set; Istanbul hits from CT/E2E land on a subset of those line numbers and can never close the gap. Swapped the unit-coverage provider to Istanbul so all three suites report on the same universe. Swapped the `@vitest/coverage-v8` devDep for `@vitest/coverage-istanbul` at the same `^3.1.2` version to match the `vitest` major.minor, changed `coverage.provider` to `"istanbul"` in the Vitest config, regenerated `yarn.lock`, and updated the `all: true` rationale comment to stop singling out V8. The per-suite `flags:` tagging in `frontend.yml` / `frontend-e2e.yml` is unchanged — server-side aggregation still handles the merge. Verified locally: the new unit lcov reports ~451 `SF:` and ~24k `DA:` (previously ~86k), matching the CT/E2E scale. Trade-off: unit-test runtime under coverage grows somewhat (Istanbul transforms source pre-execution), likely 10–30s on this suite. `tsc --noEmit` clean.

### Removed

- **Dead GraphQL operations and result types removed** (Issue #1244, `frontend/src/graphql/landing-queries.ts`, `frontend/src/graphql/queries/folders.ts`, `frontend/src/graphql/metadataOperations.ts`, `frontend/src/graphql/queries.ts`, `frontend/src/graphql/mutations.ts`): Removed GraphQL operations and associated TypeScript Input/Output types that were not referenced anywhere in `frontend/src/` or `frontend/tests/`.
  - `landing-queries.ts`: `GET_TRENDING_CORPUSES`, `GET_RECENT_DOCUMENTS` (+ `GetRecentDocumentsOutput`), `GET_COMMUNITY_STATS` (query only — `GetCommunityStatsOutput` interface is still referenced internally by `GetDiscoveryDataOutput`), `GET_GLOBAL_LEADERBOARD` (+ `GetGlobalLeaderboardOutput`).
  - `queries/folders.ts`: `GET_CORPUS_FOLDER` (singular — kept the live `GET_CORPUS_FOLDERS` plural query), `MOVE_DOCUMENTS_TO_FOLDER` (plural — kept the live `MOVE_DOCUMENT_TO_FOLDER` singular), plus the `GetCorpusFolderInputs/Outputs`, `MoveDocumentsToFolderInputs/Outputs`, and `GetDeletedDocumentsInputs/Outputs` type pairs. Kept `GET_DELETED_DOCUMENTS_IN_CORPUS` and `DeletedDocumentPathType` (used by `TrashFolderView.tsx`).
  - `metadataOperations.ts`: `GET_METADATA_COMPLETION_STATUS`, `GetDocumentMetadataDatacellsInput/Output`, `GetMetadataCompletionStatusInput/Output`, `DocumentMetadataResult` (inlined into `GetDocumentsMetadataBatchOutput`).
  - `queries.ts`: `USER_BY_SLUG`, `CORPUS_BY_SLUGS`, `DOCUMENT_BY_SLUGS` (kept live `DOCUMENT_IN_CORPUS_BY_SLUGS` and `RESOLVE_*_FULL` variants), `GET_LABELSET_BY_ID_FOR_REDIRECT`, `REQUEST_PAGE_ANNOTATION_DATA`, `GET_EXPORT` (kept live `GET_EXPORTS`), `GET_FIELDSET` (kept live `GET_FIELDSETS`), `GET_DOCUMENT_ANNOTATIONS_AND_RELATIONSHIPS`, `getAnnotationsByDocumentId`, `listAnnotations`, `GET_DOCUMENT_DETAILS` (kept live `GET_DOCUMENT_KNOWLEDGE_AND_ANNOTATIONS`), `GET_UNREAD_NOTIFICATION_COUNT`, `GET_DOCUMENT_RELATIONSHIP_COUNT`, plus all associated Input/Output interfaces.
  - `mutations.ts`: `UPDATE_LABELSET`, `CREATE_ANNOTATION_LABEL` (kept live `CREATE_ANNOTATION_LABEL_FOR_LABELSET`), `SMART_LABEL_LIST` (kept live `SMART_LABEL_SEARCH_OR_CREATE`), `REMOVE_ANNOTATION_LABELS_FROM_LABELSET`, `DELETE_ANNOTATION_LABEL` (kept live `DELETE_MULTIPLE_ANNOTATION_LABELS`), `DELETE_DOCUMENT` (kept live `DELETE_MULTIPLE_DOCUMENTS`), `REQUEST_DELETE_DOC_TYPE_ANNOTATION`, `UPDATE_CORPUS_SETTINGS`, `UPDATE_BADGE`, `AWARD_BADGE`, `REVOKE_BADGE` (kept live `DELETE_BADGE`), `DELETE_CONVERSATION`, `MARK_NOTIFICATION_READ`, `MARK_NOTIFICATION_UNREAD`, `MARK_ALL_NOTIFICATIONS_READ`, `DELETE_NOTIFICATION` (entire notification-mutations block had no call sites), `PERMANENTLY_DELETE_DOCUMENT`, `UPDATE_DOCUMENT_RELATIONSHIP`, `DELETE_DOCUMENT_RELATIONSHIPS` (kept live `DELETE_DOCUMENT_RELATIONSHIP` singular), plus all associated Input/Output interfaces.
  - Verification: every deletion was preceded by a word-boundary grep (`\bSYMBOL\b`) across both `frontend/src/` and `frontend/tests/`; interfaces that were only referenced by the deleted operations were removed, while interfaces still referenced internally (e.g. `GetCommunityStatsOutput` used by `GetDiscoveryDataOutput`) were preserved. `tsc --noEmit` and `yarn build` pass cleanly.
  - `types/graphql-api.ts` and `types/graphql-slug-queries.ts` were intentionally left untouched per the issue scope — test wrappers and mock fixtures import many type names from these files, and auto-generated structural types are too risky to prune by hand.

### Fixed

- **DRFMutation/DRFDeletion `IOSettings` misconfiguration surfaces as a generic internal error** (Issue #1360, `config/graphql/base.py`): `DRFMutation.IOSettings` declared `model: django.db.models.Model = None`, `graphene_model: DjangoObjectType = None` and `serializer = None` — the type annotations were wrong (instance types, not class types, holding `None`), and there was no runtime guard. A subclass that forgot to override one of those would fail deep inside `mutate()` with an `AttributeError` (`'NoneType' object has no attribute 'objects'`) or `TypeError` (`'NoneType' object is not callable`), which the broad `except Exception` then masks as a generic "internal error" message back to the API caller. `DRFDeletion.IOSettings` had the inverse problem — it didn't declare `model` at all, even though `DRFDeletion.mutate` dereferences `cls.IOSettings.model.objects`. Discovered while working on #1332 (typing): mypy flagged four errors on `base.py:128/204/238/252` plus `base.py:93 "type[IOSettings]" has no attribute "model"`.
  - Retyped both `IOSettings` declarations with `ClassVar[Optional[type[...]]] = None` so the annotation matches reality (holding a class, not an instance, and nullable by default). `pk_fields` narrowed from `list[str | int]` to `list[str]` — the only call sites use it as a dict key.
  - Added `_require_io_setting(mutation_cls, name)` helper that fails fast with `NotImplementedError("<Subclass>.IOSettings.<name> must be set by the subclass.")` when a required attribute is `None` or missing. Called for `model`/`serializer`/`graphene_model` in `DRFMutation.mutate` and for `model` in `DRFDeletion.mutate`. In `DRFMutation` the error is caught by the broad `except Exception` and surfaced as "internal error" to the API, with the full descriptive message in the server log via `logger.error(traceback.format_exc())`. `DRFDeletion.mutate` has no equivalent `try/except` (matching its pre-PR behavior), so the `NotImplementedError` propagates to graphene as a GraphQL execution error — also a fail-fast outcome, just on a different surface.
  - Tightened `DRFDeletion.mutate` to raise `ValueError` when `kwargs[lookup_field]` is missing (the `id` GraphQL argument is `required=False`), instead of passing `None` into `from_global_id` and getting an opaque `AttributeError`. Same propagation surface as the `_require_io_setting` guard above.
  - Fixed pre-existing `to_global_id(graphene_model.__class__.__name__, obj.id)` in both `DRFMutation` create/update return paths: `graphene_model` is the `DjangoObjectType` class, so `.__class__` referenced graphene's metaclass (`SubclassWithMeta_Meta`) and the resulting global id used the wrong type name. Switched to `graphene_model.__name__` to match the GraphQL type name (e.g. `"CorpusType"`) used everywhere else in `config/graphql/`. Surfaced by the new explicit `ClassVar[Optional[type[...]]]` annotations — both call sites stayed dormant because the returned `obj_id` is rarely consumed by the frontend (which re-queries by global id from its own cache).
  - Graduated `config.graphql.base` out of the mypy baseline: removed the `[mypy-config.graphql.base]` section from `mypy.ini` and pruned the ten corresponding lines from `docs/typing/mypy_baseline.txt`. `mypy --config-file mypy.ini` is now clean on this module.
  - Regression tests in `opencontractserver/tests/test_security_hardening.py::TestIOSettingsRequiredFieldsGuard`: the helper raises `NotImplementedError` for each of `model`/`serializer`/`graphene_model` when missing or `None`, returns the configured value when present, and the base classes expose the `None` defaults the guard relies on.
- **`CorpusChat` dropped `SYNC_CONTENT` messages from the visible chat** (Issue #1276, `frontend/src/components/corpuses/CorpusChat.tsx:468-505`): The `SYNC_CONTENT` WebSocket frame is a standalone, non-streaming assistant reply used for synchronous server responses. `ChatTray` (document chat) appends these directly to its `chat` state; the corpus-level chat only forwarded the content to `handleCompleteMessage`, which stores sources in `ChatSourceAtom` but never pushes a message to the visible list. As a result, any `SYNC_CONTENT` the backend sent over the corpus socket rendered nothing. Fixed by mirroring the `ChatTray` pattern — push a new complete assistant message into `chat` before persisting sources/timeline. The fallback `crypto.randomUUID()` is also now captured in a single local variable so the visible chat entry and the `ChatSourceAtom` record share the same id when the server omits `message_id`. New regression test in `frontend/tests/CorpusChat.ct.tsx` ("SYNC_CONTENT renders a complete message immediately") pins the behavior.
- **CAML article preview crashed when inserting an extract grid embed** (`frontend/src/utils/camlComponents.ts`, `frontend/src/hooks/useCamlComponentRenderer.tsx`, `frontend/src/components/corpuses/CamlArticleEditor.tsx`, `frontend/src/components/corpuses/caml/CamlDirectiveRenderer.tsx`): the editor wrapped each newly-inserted `[component:TYPE ...]` marker in a `::: prose` fence, but `@os-legal/caml`'s parser has no `case "prose"` in `parseBlock`, so the resulting block carried `body` instead of `content`. `ProseBlock` then crashed inside `splitPullquotes(undefined)`, which unmounted the entire editor modal and made the "ArrowDown then Enter inserts the extract-grid component marker" CT test fail. Switched the fence to a project-specific `::: oc-component` block and routed it through `CamlArticle`'s `customBlocks` slot, where the marker text is handed back to the existing `[component:...]` resolver. The keyboard handler in `CamlArticleEditor` was also tightened to read the active picker index from a `useRef` mirror so back-to-back ArrowDown/Enter keystrokes don't observe a stale closure value of `-1` and bail out before insertion.
- **PR #1177 follow-up: CAML extract embed polish** (Issue #1227):
  - **`fullDatacellList` payload now bounded server-side**: `ExtractType.full_datacell_list` accepts optional `limit` / `offset` arguments and the resolver clamps `limit` to `MAX_FULL_DATACELL_LIST_LIMIT` (`opencontractserver/constants/extracts.py`, currently `500`) after permission filtering (`config/graphql/extract_types.py`). `GET_EXTRACT_GRID_EMBED` passes `limit: EXTRACT_GRID_EMBED_CELL_LIMIT` (mirrored at `500` in `frontend/src/assets/configurations/constants.ts`) so pathological extracts no longer transmit thousands of cells just to trigger the too-many-rows guard (`frontend/src/graphql/queries.ts`, `frontend/src/components/extracts/ExtractGridEmbed.tsx`). Full server-side pagination is still tracked in #1204.
  - **`resolveComponentMarker` now receives a stable React key from both call sites**: `useCamlComponentRenderer` and `CamlDirectiveRenderer` pass the marker string as the `key` argument so multiple `[component:...]` blocks in a single article reconcile correctly without React's "missing key prop" warnings (`frontend/src/hooks/useCamlComponentRenderer.tsx`, `frontend/src/components/corpuses/caml/CamlDirectiveRenderer.tsx`). Added regression tests in `frontend/src/utils/__tests__/camlComponents.test.ts`.
  - **Code-point-safe cell value truncation**: `formatCellValue` in `ExtractGridEmbed` now slices on `Array.from(json)` instead of `String.substring`, so cell values containing emoji or other non-BMP characters are no longer truncated mid-surrogate-pair (which previously emitted `U+FFFD` replacement glyphs).
  - **Keyboard navigation in CAML extract picker**: Added a listbox keyboard handler to the "Insert Extract Grid" dropdown in `CamlArticleEditor` — Arrow keys / Home / End move the focused option, Enter selects, Escape closes and returns focus to the trigger button. Active option is reflected via `aria-activedescendant` and an `$active` highlight on `ExtractPickerItem`.
  - **`buildSourceLink` page-indexing convention documented**: Added an explanatory comment clarifying that `Annotation.page` is 1-based (model default=1) and is only used for the chip label (`p.{page}`) — the document viewer navigates by `annotationId` alone, so there is no URL-layer indexing convention to worry about (`frontend/src/components/extracts/ExtractGridEmbed.tsx`).
  - **Verified `GET_EXTRACTS` already selects `fullDocumentList`** (`frontend/src/graphql/queries.ts:1901`) and that `CamlDirectiveRenderer` still wires `resolveImageSrc` through to `MarkdownMessageRenderer` — both no-ops requested by the issue's verification items.
- **`User.__init__` mutated the shared `Field.validators` list on every instantiation** (Issue #1358, `opencontractserver/users/models.py:150-153`): `User.__init__` overwrote `self._meta.get_field("username").validators[0]` with a fresh `UserUnicodeUsernameValidator()` on every `User(...)` construction — including every ORM hydration, form save, and QuerySet materialisation. Two problems: (1) per-instance allocation + assignment on a hot path, and (2) a hard-coded `validators[0]` index that would silently flip the slot back to Django's default `UnicodeUsernameValidator` if any third-party code or future migration prepended a validator, breaking usernames that rely on `\` or `|`. Fix: declare the validator on the `username` field at class-body time (`opencontractserver/users/models.py:78-90`) and drop the `__init__` override. Added migration `0026_alter_user_username_validator.py` to update the stored field definition. Regression tests in `opencontractserver/tests/test_user_username_validator.py` verify the validator survives repeated instantiation and continues to accept `\`, `|`, `*`.
- **`RemoveLabelsFromLabelsetMutation` silently did nothing** (Issue #1359, `config/graphql/label_mutations.py:296`): The resolver referenced `labelset.documents.filter(pk__in=label_pks)`, but `LabelSet` has no `documents` relation — the M2M to labels is `annotation_labels` (see `opencontractserver/annotations/models.py:1284`). Every invocation therefore raised `AttributeError`, which was swallowed by the surrounding `except Exception as e:` block and returned as a generic `"Error removing label(s) from labelset: ..."` with `ok=False`. Because the frontend `REMOVE_ANNOTATION_LABELS_FROM_LABELSET` mutation was itself unused (#1244 swept it out), this bug went unnoticed in production for an unknown length of time; discovered while grading mypy errors for #1332 (`config/graphql/label_mutations.py:296: error: "LabelSet" has no attribute "documents"  [attr-defined]`). Swapped `documents` → `annotation_labels`, removed the now-resolved error from `docs/typing/mypy_baseline.txt`, and added `opencontractserver/tests/test_label_mutations.py` with four regression cases covering: labels are actually removed from the M2M, IDs not in the labelset are silently ignored, a non-owner / non-public caller cannot mutate the labelset, and a public labelset remains editable (pinning the current `Q(creator=user) | Q(is_public=True)` resolver behaviour so any future permission hardening is explicit).
- **`package_annotated_docs` silently corrupted exports when a document failed to burn** (Issue #1356, `opencontractserver/tasks/export_tasks.py:150-212`, `opencontractserver/utils/etl.py:198-463`, `opencontractserver/tasks/doc_tasks.py:463-504`): `build_document_export()` returns `("", "", None, {}, {})` when a per-document export fails (e.g. the underlying file cannot be loaded). The V1 consumer `package_annotated_docs` had no guard for that placeholder — it ran `doc[1].encode("utf-8")` on the empty string (harmless, no crash), wrote an empty-named entry into the zip, and inserted `annotated_docs[""] = None` into the final `data.json`, so a single failed document silently poisoned the export. The V2 pipeline in `export_tasks_v2.py:126-128` already has this guard; V1 did not. Added `if not doc_name or doc_export is None: continue` (mirroring V2's check) and logged a warning identifying the skipped doc. Also tightened the return-type annotations on `build_document_export` and `burn_doc_annotations`: slots 0 and 1 are always `str` (never `None`; they are empty strings on the failure path), so the signature is now `tuple[str, str, OpenContractDocExport | None, dict[...], dict[...]]`. Corrected the `burned_docs` parameter annotation on `package_annotated_docs` from a single-element tuple-of-tuples to a variadic `tuple[tuple[...], ...]` — the runtime iteration is variadic, and the previous annotation was a red herring uncovered while typing for #1334. Regression test in `opencontractserver/tests/test_package_annotated_docs.py` covers both the mixed-success and all-failed scenarios, asserting the zip contains no empty-named entries and `annotated_docs` holds no `None` values.
- **Frontend coverage badge stuck on "unknown"** (`README.md:12`, `.github/workflows/codecov-notify.yml`, `.github/workflows/frontend.yml`, `.github/workflows/frontend-e2e.yml`, `frontend/package.json`, `frontend/yarn.lock`): PR #1322 pointed the README badge at a new merged `frontend` flag fed by a cross-workflow `lcov-result-merger@5` step inside `codecov-notify.yml`. Every `frontend-merged-coverage` upload since has landed at Codecov with `state: error` / `totals: null`, so the badge rendered "unknown" even though `frontend-unit` (31%), `frontend-component` (61%), and `frontend-e2e` (24%) were all processing correctly. Two defects in the merged lcov confirmed by local repro of the CI merge step: (1) `lcov-result-merger@5` emits a stripped lcov containing only `SF:`, `DA:`, `BRDA:`, `end_of_record` — it drops `TN:`, `FN`, `FNDA`, `FNF`, `FNH`, `LF`, `LH`, `BRF`, `BRH`, so line-summary fields required by Codecov's parser are absent; (2) Vitest v8 emits `src/...` (relative to `frontend/`) while `vite-plugin-istanbul` + `nyc report` emit `/home/runner/work/OpenContracts/OpenContracts/frontend/src/...` (absolute), and the merger keys on the literal path string so the same file appears as two records with conflicting hit counts. `codecov-notify.yml` also ran without `actions/checkout`, which Codecov's action docs explicitly recommend against. Fix: stop merging client-side and let Codecov aggregate server-side, since that is what flags are for. Each per-suite upload now declares two flags — `frontend-unit,frontend`, `frontend-component,frontend`, `frontend-e2e,frontend` — so the `frontend` flag total is the union of the three uploads computed by Codecov. `codecov-notify.yml` is reduced to its original gate-and-notify role (no artifact downloads, no `lcov-result-merger`, no merged upload). Deleted the `frontend-{unit,ct,e2e}-lcov` artifact publishes in `frontend.yml` / `frontend-e2e.yml`, removed the `lcov-result-merger@^5.0.1` devDep and the `coverage:merge` script from `frontend/package.json`, and pruned the orphaned entries from `frontend/yarn.lock`. README badge URL unchanged (`flag=frontend`).

### Changed

- **PR #1297 follow-up — tighten component tests and unit-test coverage** (Issue #1321):
  - `frontend/src/components/documents/__tests__/DocumentRelationshipModal.test.ts`: added a test that pins the `labelType: undefined` edge case for `filterRelationshipLabels` (previously only `null` non-relationship labels were covered), so the strict-equality guard against non-`RelationshipLabel` types is explicit.
  - `frontend/tests/ModernDocumentItem.ct.tsx`:
    - Extended the `__reactProps$` upgrade-risk comments on `clickViaReact` / `openContextMenu` to call out the build-hash suffix explicitly — the suffix rotates on every React build, so `Object.keys(el).find(k => k.startsWith("__reactProps$"))` is the only correct lookup pattern.
    - `openContextMenu` now falls back to a full-document scan for an element with a React `onContextMenu` prop when the `.checkbox` anchor is absent (e.g. future conditional rendering of read-only items). The primary checkbox path is unchanged.
    - Added `NOTE` comments on the relationship-popup `toBeAttached()` assertions explaining that they only verify DOM presence (the popup uses `visibility: hidden` + hover reveal) and when to switch to `toBeVisible()` instead.
  - `frontend/tests/DocumentRelationshipModal.ct.tsx`:
    - Replaced the DOM-order `removeButtons.nth(1)` assertion in the "removes document from target column" test with an XPath-based walk from the pill's visible title, so re-ordering the source/target layout can no longer silently test the wrong button.
    - Strengthened the mutation-failure negative assertion: after submitting a failing mutation, wait for the submit button to re-enable (a positive signal that the `isSubmitting` finally-block has executed) before polling `onSuccessCalled === false`. Prevents the poll from passing immediately on the initial `false` value.
  - `frontend/tests/utils/ReactiveVarObserver.tsx`: expanded the doc comment with a three-step recipe for extending the observer to additional reactive vars, matching the existing `viewingDocument` / `editingDocument` convention.

### Fixed

- **Backend CI build aborted on transient SSL failure mid-wheel-download** (`compose/local/django/Dockerfile`, `compose/production/django/Dockerfile`): Backend CI run `24646911374` on commit `233a9b67` (push to `main`) failed in the `Build the Stack` step with `ssl.SSLError: [SSL] record layer failure (_ssl.c:2590)` at 22 MB of 60.4 MB while `pip wheel` was downloading `opencv-python-headless`, aborting the entire build and cascading skips through `Run DB Migrations`, `Verify Docker Containers`, and `pytest`. Pip's default `--retries 5` only covers connection setup and does not resume broken mid-stream downloads — that behaviour is gated on `--resume-retries`, added in pip 24.1 (env var `PIP_RESUME_RETRIES`). Added `ENV PIP_RETRIES=10 PIP_TIMEOUT=60 PIP_RESUME_RETRIES=5` to both the `python-build-stage` and `python-run-stage` of both Dockerfiles so every `pip install` / `pip wheel` invocation (wheel build, `--upgrade pip`, spacy model downloads) picks up the hardened settings. Verified the base image (`pytorch/pytorch:2.7.1-cuda12.6-cudnn9-runtime`) ships pip 25.1.1 and recognises all three env vars (`pip config list` reports `:env:.resume-retries='5'`, `pip install --help` shows `--timeout (default 60.0 seconds)`).
- **`fullDatacellList` no-args path was unbounded** (Issue #1256, follow-up to PR #1235, `config/graphql/extract_types.py:131-148`): `ExtractType.resolve_full_datacell_list` capped the `limit` and `offset-only` branches at `MAX_FULL_DATACELL_LIST_LIMIT` but returned the entire queryset when called with no arguments. Authenticated callers hitting `fullDatacellList` directly (no `limit`, no `offset`) could bypass the payload bound. Collapsed all three code paths so every call — no-args, offset-only, or `limit`+`offset` — returns `qs[start : start + min(limit_or_max, MAX_FULL_DATACELL_LIST_LIMIT)]`. The embed UI is unaffected (it already passes `limit: 500`); direct API callers now receive at most 500 cells and must paginate via `offset` to walk the rest. Added regression test `test_full_datacell_list_no_args_capped_at_server_max` in `opencontractserver/tests/test_extract_queries.py` which creates 501 cells and asserts the no-args response returns exactly `MAX_FULL_DATACELL_LIST_LIMIT` while `datacellCount` reports the true total.

### Changed

- **Dual-constant sync warnings for `MAX_FULL_DATACELL_LIST_LIMIT` / `EXTRACT_GRID_EMBED_CELL_LIMIT`** (Issue #1256, `opencontractserver/constants/extracts.py`, `frontend/src/assets/configurations/constants.ts:351-364`): Both constants now carry explicit `IMPORTANT: ... CI sync-check tracked in issue #1256` warnings so future edits are obvious. Updated the GraphQL schema description on `fullDatacellList(limit:)` to document the server cap rather than claiming the omitted-limit case returns everything. Corrected the stale "N+1: issues a COUNT(\*) per ExtractType instance" comment on `resolve_datacell_count` to reflect that the embed path issues a `COUNT(*)` in addition to the main list query. Added a `TODO` reference to #1256 next to the `ExtractQueryOptimizer` circular-import inline import in `_get_datacell_qs`.
- **Backend CI `changes` job failed on every push to main** (`.github/workflows/backend.yml`): `dorny/paths-filter@v3` was declared without a prior `actions/checkout`, on the stated premise that the action "fetches diffs via the GitHub API". That is only true for `pull_request` events; on `push` events the action shells out to `git branch --show-current`, which fails with `fatal: not a git repository` when the workspace is empty. Every push to a protected branch therefore produced a red X on the `changes` job, silently masked by `continue-on-error: true` plus a `|| github.event_name == 'push'` fail-open gate on downstream jobs. Scoped the `changes` job to `pull_request` events (`if: github.event_name == 'pull_request'`) — where paths-filter actually works — and rewrote the downstream gates on `linter` / `pytest` as `if: always() && (github.event_name == 'push' || needs.changes.outputs.backend != 'false')` so the skip cascade from a `needs`-target that is now intentionally skipped on push does not block the real test jobs. Behaviour preserved: push always runs `linter` + `pytest`; PRs with no backend changes still skip both; PRs where the filter errors transiently (outputs.backend == '') still fail open.

- **Frontend CI tippy-debug step removed** (`.github/workflows/frontend.yml`): Dropped the `Debug - Check for tippy references` step in the `lint` job — a leftover investigation artifact that ran on every PR/push and produced noise with no diagnostic value today. Flagged in Issue #1319.

- **Codecov-notify `matching[0]` relied on undocumented API ordering** (`.github/workflows/codecov-notify.yml`): The cross-workflow coordinator picked the "most recent" workflow run per expected name by indexing `matching[0]` on the filter result, leaning on the GitHub Actions REST API returning results newest-first. That ordering is not guaranteed. Added an explicit `created_at` descending sort before taking `matching[0]`, so re-run detection is correct regardless of API-side quirks. Flagged in Issue #1319.

- **Frontend coverage badge reported ~31% despite months of added tests** (`README.md:12`, `.github/workflows/codecov-notify.yml`, `.codecov.yml`, `frontend/vite.config.ts:210-223`): The README's "Frontend coverage" badge was pointing at `flag=frontend-unit` — the Vitest slice only. The three frontend suites (Vitest unit, Playwright component via `vite-plugin-istanbul`, Playwright E2E via `vite-plugin-istanbul`) upload to separate Codecov flags (`frontend-unit`, `frontend-component`, `frontend-e2e`) and were never merged into a single lcov. Recent PRs almost exclusively added Playwright component and E2E tests, so their coverage landed in `frontend-component`/`frontend-e2e` while the badge stayed stuck reading the Vitest slice.
  - Added an `Upload {unit,CT,E2E} lcov artifact` step (using `actions/upload-artifact@v7`) to each producing job in `.github/workflows/frontend.yml` (`component-test` and `unit-test`) and `.github/workflows/frontend-e2e.yml` (`e2e`). The existing per-flag Codecov uploads are untouched, so per-suite drill-in still works.
  - Extended the existing cross-workflow coordinator at `.github/workflows/codecov-notify.yml` to download the three artifacts by run id (via `actions/download-artifact@v7` with `continue-on-error: true` to tolerate path-filtered skips and upload failures), merge them with `npx lcov-result-merger@5`, and upload the combined lcov to Codecov under a new `frontend` flag before calling `send-notifications`. The existing `listWorkflowRunsForRepo` check already discovers the producing runs by SHA — it now also emits `frontend_ci_run_id` and `frontend_e2e_run_id` outputs for the downloads to consume.
  - Added the `frontend` flag to `.codecov.yml` (paths: `frontend/src/`, `carryforward: true`). Intentionally does not match the `frontend-.*` regex used by the `frontend` component's `flag_regexes`, so the component keeps aggregating only the three per-suite flags and does not double-count the merged upload.
  - Pointed the README badge at `flag=frontend` so the displayed number reflects the union of all three suites.
  - Fixed a secondary v8 issue at `frontend/vite.config.ts:210-223`: added `all: true` to the Vitest coverage block. Without it, v8 silently drops files not imported by any unit test, inflating the `frontend-unit` ratio and misaligning the v8 lcov's file universe with the Istanbul-based component/E2E lcovs (which do enumerate all source files). Aligning the two is required for the merged `frontend` lcov to be meaningful.
  - Added `lcov-result-merger@^5.0.1` as a `frontend/package.json` devDependency with a local `coverage:merge` script so the merge can be reproduced locally (`yarn test:coverage:unit && yarn test:coverage:ct && yarn test:e2e:coverage && yarn coverage:merge`).

### Changed

- **`fullDatacellList` caps caller-supplied `limit` at `MAX_FULL_DATACELL_LIST_LIMIT` (500)** (Issue #1227): The resolver for `ExtractType.fullDatacellList` now clamps the `limit` argument to `MAX_FULL_DATACELL_LIST_LIMIT = 500` (`opencontractserver/constants/extracts.py`) regardless of what the caller requests. Callers that pass no `limit` still get every datacell (full pagination is tracked in #1204); the cap only applies once `limit` is supplied, so a caller asking for `limit: 10_000` is transparently answered with at most 500 rows per request.

- **Simplify `RelationGroup.updateForAnnotationDeletion` pruning branches** (Issue #1316, `frontend/src/components/annotator/types/annotations.ts:40-59`): The four conditional branches that each returned `undefined` collectively covered exactly `newSourceIds.length === 0 || newTargetIds.length === 0`, and the `sourceEmpty` / `targetEmpty` pre-filter variables were only referenced inside those branches. Collapsed the four conditions into a single `if` and removed the now-unused pre-filter variables. No behavior change — existing `RelationGroup > .updateForAnnotationDeletion()` regression tests still pass unchanged (all 28 tests in `annotations.test.ts` pass; `tsc --noEmit` clean).
- **Harden backend CI path-filter job** (Issue #1290, `.github/workflows/backend.yml`): Tightened the `changes` path-filter job so transient `dorny/paths-filter` failures no longer silently skip `linter` / `pytest`.
  - Dropped the redundant `actions/checkout` step in the `changes` job — `dorny/paths-filter@v3` fetches diffs via the GitHub API and does not need a local clone.
  - Added `Dockerfile*` to the backend path filter so root-level Dockerfile changes (which directly affect the backend build environment) correctly trigger backend CI.
  - Marked the `changes` job `continue-on-error: true` so a transient GitHub API failure does not fail the workflow.
  - Changed the downstream gate from `needs.changes.outputs.backend == 'true'` to `needs.changes.outputs.backend != 'false'` on both `linter` and `pytest`. With `continue-on-error: true`, a failed `changes` job produces an empty-string output; the old `== 'true'` gate evaluated to `false` for PR events and silently skipped downstream jobs — the exact failure mode the "fail open" comment claimed to prevent. The new `!= 'false'` gate runs downstream jobs for both the happy path (`'true'`) and the transient-failure path (`''`), only skipping when the filter explicitly reports no backend changes (`'false'`).
  - Reworded the linter path-filter comment to clarify that gating only affects PRs with no backend changes; push events to protected branches still run unconditionally.

### Fixed

- **`ChatMessage` source indicator no longer renders "1 sources"** (`frontend/src/components/widgets/chat/ChatMessage.tsx:1993`): the source indicator always used the plural noun regardless of count, while every other pluralized label in the component ("1 Source", "1 step", "1 tool") switched correctly. Updated to `"1 source" / "N sources"` to match.
- **`DocumentRelationshipModal` "Create label" button never appeared** (Issue #1280, `frontend/src/components/documents/DocumentRelationshipModal.tsx`): `@os-legal/ui`'s Dropdown only fires `onSearchChange` in `searchable="async"` mode — with `searchable="local"` the parent's `labelSearchTerm` state never updated, so the `labelSearchTerm`-gated "Create label: ..." empty-state button was permanently hidden. Switched the Dropdown to `async`, moved option filtering into a `useMemo(filteredRelationshipLabels, [relationLabels, hasCorpus, labelSearchTerm])`, and added the missing dep. Now typing a novel label name surfaces the Create button as designed.
- **`RelationGroup.updateForAnnotationDeletion` pre-filter length check** (Issue #1288): `frontend/src/components/annotator/types/annotations.ts:49-50` computed `nowSourceEmpty` / `nowTargetEmpty` from the pre-filter `this.sourceIds` / `this.targetIds`, so the "now empty" conditions were identical to the "before" conditions and the pruning branches that return `undefined` were dead code. Deleting the sole source or sole target of a relation left the relation orphaned, pointing at a deleted annotation id. Fixed by reading from the post-filter `newSourceIds` / `newTargetIds`. Called from `PdfAnnotations.undoAnnotation()`, so undo now properly drops any relation whose last source or target was the popped annotation. New regression tests under `RelationGroup > .updateForAnnotationDeletion()` cover all four pruning branches plus the survive-with-updated-ids and unchanged cases, and the existing `undoAnnotation` test that pinned the wrong behaviour was corrected.

  - Additionally fixed a surviving-relation identity drop in the same method: the post-prune return previously constructed `new RelationGroup(newSourceIds, newTargetIds, this.label)` without forwarding `this.id` / `this.structural`, so relations that merely lost a member were silently reassigned a fresh uuid and had their `structural` flag cleared. Both fields are now preserved, and the survival tests assert `updated!.id === rel.id` (plus `structural === true` where applicable) to pin the behaviour.

- **`useAgentChat` WebSocket reconnects on every approval-gate transition** (Issue #1296): `frontend/src/hooks/useAgentChat.ts` had `pendingApproval` in the main WebSocket `useEffect` dependency array, so every `ASYNC_APPROVAL_NEEDED` / `ASYNC_APPROVAL_RESULT` / `ASYNC_CONTENT` / `ASYNC_FINISH` message that set or cleared approval state tore the socket down and recreated it mid-conversation. Impacts: approval decisions could race the reconnect and silently fail (`isConnected` flipped to `false` before the fresh socket opened), in-flight streaming tokens were dropped whenever an approval gate opened/closed, and the server saw a brand-new consumer attaching mid-run. Fix: mirror `pendingApproval` in a `pendingApprovalRef` updated by a separate effect (`useAgentChat.ts:291-299`), read `pendingApprovalRef.current` from the `ASYNC_CONTENT` / `ASYNC_APPROVAL_RESULT` / `ASYNC_FINISH` branches of `onmessage` (`useAgentChat.ts:651-752`), and drop `pendingApproval` from the socket effect's dependency array (`useAgentChat.ts:802-818`). Tests in `frontend/src/hooks/__tests__/useAgentChat.test.tsx`: removed the previous `_open()` workarounds around approval events, replaced the old "reconnects on approval state change" regression test with a new `does not reconnect the socket when approval state changes (issue #1296)` guard that asserts `wsInstances.length` is unchanged, `originalSocket.close` was not called, and the approval decision is dispatched through the original socket. All 23 tests still pass.
- **Forced-by-selected-relation visibility now respects explicit user selection regardless of `showStructural`** (Issues #1289, #1294): In `frontend/src/components/annotator/hooks/useVisibleAnnotations.ts:54-62`, the IDs forced visible by a selected relation were previously only merged into `forcedIds` inside the `if (showStructural)` branch. With structural annotations toggled off, clicking a relation in the sidebar failed to highlight its member annotations — making the relation sidebar effectively unusable in that mode. Moved the `forcedBySelectedRelationIds` merge outside the structural branch so explicit user selection always wins (consistent with the existing `forcedBySelection` treatment). Structural-relationships auto-forcing (`showStructuralRelationships`) remains gated on `showStructural`, since that path is implicit rather than user-initiated.

  - Updated `frontend/src/components/annotator/hooks/__tests__/useVisibleAnnotations.test.tsx`: replaced the pinning test that documented the bug ("does NOT apply forced-by-selected-relation when showStructural is false") with two assertions verifying the corrected behavior — forced-by-selected-relation now applies to both non-structural annotations (overriding a label filter) and structural annotations when `showStructural` is false. All 16 tests in the file pass; `tsc --noEmit` clean.

- **`useUpdateAnnotation` lost sibling annotations on update** (`frontend/src/components/annotator/hooks/AnnotationHooks.tsx:386`): The hook called `replaceAnnotations([updatedAnnotation])`, which collapses the whole `annotations` array down to the single passed-in element. On any document with more than one annotation, updating one annotation silently dropped the others. Fixed by using `setPdfAnnotations` with a `.map` that swaps only the matching id; regression test added in `AnnotationHooks.test.tsx` (see `useUpdateAnnotation > updates one annotation in place without dropping siblings`).

### Added

- **Frontend coverage for widgets, modals, and icon-picker** (Issue #1279): Added one vitest suite and three Playwright CT suites targeting ~2,150 uncovered lines in `frontend/src/components/widgets/`:
  - `frontend/src/components/widgets/icon-picker/__tests__/icons.test.ts` — 450 tests (≥90% target). Validates every entry in the Lucide catalog has a valid kebab-case `name`, non-empty trimmed `label`, and a `category` that belongs to the declared `IconCategory` union; asserts name uniqueness, that every declared category contains at least one icon, that the derived `LUCIDE_ICON_NAMES` set and `findIconEntry` map stay in sync with `LUCIDE_ICONS`, and a floor of 300 curated icons as a snapshot guard.
  - `frontend/tests/ChatMessageCoverage.ct.tsx` — 16 tests pushing `ChatMessage.tsx` past the 60% floor (from 50%). Covers user vs assistant header rendering, markdown (bold/code/lists/GFM tables), `onSelect` wiring, source-indicator visibility, all three approval-pill states, sources-preview expand/collapse + per-chip expand + chip `onClick`, timeline collapsed-by-default for long completed messages + header expansion + short-timeline default-expanded path + streaming `showTimelineOnly` branch.
  - `frontend/tests/FieldsetModal.ct.tsx` + `frontend/tests/FieldsetModalTestWrapper.tsx` + `frontend/tests/FieldsetModalMocks.ts` — 11 tests. Covers create-mode header/empty state/disabled-save, footer validation transitions (name-required → column-required), column add via the embedded `CreateColumnModal`, column delete with counter update, edit-mode prefill from `REQUEST_GET_FIELDSET`, and all three close paths (Cancel / X / overlay).
  - `frontend/tests/SelectAnalyzerOrFieldsetModal.ct.tsx` + `frontend/tests/SelectAnalyzerOrFieldsetModalTestWrapper.tsx` + `frontend/tests/SelectAnalyzerOrFieldsetModalMocks.ts` — 16 tests. Covers analyzer-grid rendering, document vs corpus subtitle, result-count pill, debounced search filter, "no match" and "none available" empty states, selection flipping Run↔Configure, schema preview toggle, tab switching (analyzer ↔ fieldset), pagination across 12 analyzers, and all three close paths.
  - Verification: 450/450 vitest + 43/43 Playwright CT (stable across 3 consecutive runs); `tsc --noEmit` clean; prettier clean.
- **Knowledge-base document viewer CT coverage remediation** (Issue #1277): Added ~1,400 lines of new Playwright component tests across the four highest-uncovered files in `frontend/src/components/knowledge_base/document/` — `DocumentKnowledgeBase.tsx` (1,961 LOC / 25.5%), `right_tray/ChatTray.tsx` (1,417 / 25.3%), `unified_feed/RelationshipActionModal.tsx` (549 / 16.2%), and `unified_feed/UnifiedContentFeed.tsx` (577 / 29.8%). All new tests use the existing `*TestWrapper` pattern and drive the UI through `--reporter=list` as required by CLAUDE.md.

  - `frontend/tests/RelationshipActionModal.ct.tsx` — Expanded from 3 to 16 specs. New coverage: corpus-loaded rendering, role-picker + add-to-existing flow with callback assertion, structural-relationship filtering, create-mode search + label-list filtering, "no labelset" warning, create-label form round-trip (open, cancel, submit via `SMART_LABEL_SEARCH_OR_CREATE` mock, change selected label), create-mode submit enablement, full submission (source/target pill assignment → `onCreate(labelId, sourceIds, targetIds)` verification), cancel → `onClose`, `getAnnotationPreview` ellipsis truncation at 30 chars, and singular/plural `Selected: N annotation(s)` rendering.
  - `frontend/tests/RelationshipActionModalTestWrapper.tsx` — Extended to accept `withCorpus` / `hasLabelset` / `relationLabels` / `onAddToExisting` / `onCreate` / `onClose` / `corpusId` props. Seeds `corpusStateAtom` via an internal `CorpusSetupInner` children-wrapping effect component (children-wrapping rather than null-returning to satisfy Playwright CT's babel transform, which silently fails to mount otherwise).
  - `frontend/tests/RelationshipActionModalFixtures.ts` — New plain `.ts` fixtures file hosting `buildRelationLabel`. Kept separate from the `.tsx` wrapper to avoid the Playwright CT split-import rule (pitfall #16 in `CLAUDE.md`).
  - `frontend/tests/UnifiedContentFeed.ct.tsx` — Expanded with 8 new specs: multi-select selection-toolbar appearance & count, Select All, Clear, "Add to Relationship" opens `RelationshipActionModal`, read-only hides the checkbox, `noCorpus` hides the toolbar entirely, sort-by-type, note search-query filter, annotation rawText search-query filter, structural filter, `STRUCTURAL_LABEL_PREFIX` always-hidden invariant, and content-type routing (relationship-only).
  - `frontend/tests/UnifiedContentFeedTestWrapper.tsx` — Added `noCorpus`, `showStructural`, `searchText`, and `textSearchMatches` props so the new tests can exercise the no-corpus and structural branches without mounting the component directly.
  - `frontend/tests/ChatTray.ct.tsx` — Added 10 new specs covering empty conversation list, Back-to-Conversations (exitConversation + refetch), ASYNC_ERROR → Reconnect banner, new-chat FAB path, no-op Enter on empty input, sub-90% no character counter, Shift+Enter newline, context-meter rendering via `ASYNC_FINISH` `context_status`, compaction banner via `ASYNC_THOUGHT` with `compaction` metadata, SYNC_CONTENT standalone assistant message, and readOnly-style start path.
  - `frontend/tests/DocumentKnowledgeBase.ct.tsx` — Added 2 new specs covering the `!documentId` invalid-document error modal and its Close-button `handleClose` callback.
  - Verification: full CT suite (`yarn test:ct --reporter=list`) — **1,523 passed, 3 skipped, 0 failed**.

- **Component test coverage for `ModernDocumentItem` and `DocumentRelationshipModal`** (Issue #1280): Added ~1,600 lines of Playwright CT coverage for two high-traffic document components that were lacking happy-path tests.
  - `frontend/tests/ModernDocumentItem.ct.tsx` — New 750-line suite covering card/list view rendering (thumbnails, badges, metadata), selection state, permission-gated action buttons (CAN_UPDATE / read-only), action behaviors (view, edit, remove, open, download), backend-locked processing state, context menu actions, and version-history / relationship badges.
  - `frontend/tests/DocumentRelationshipModal.ct.tsx` — Extended suite now covers state transitions (moves / removes across columns), Add Source/Target search flow, label picker with pre-populated labels, label change, RELATIONSHIP and NOTES submit mutations, mutation failure handling, inline create-label flow, `SMART_LABEL_SEARCH_OR_CREATE`, the missing-corpus error state, and the useMemo filter bodies (`availableDocuments`, `filteredRelationshipLabels`).
  - `frontend/tests/DocumentRelationshipModalTestWrapper.tsx` — Added `makeMockRelationLabel` helper, `extraMocks` prop for per-test Apollo mocks, `relationLabels` / `withoutLabelset` / `corpusIdOverride` props, and a DRY `buildDocumentsMock()` factory (replaces duplicated mock blocks).
  - `frontend/tests/utils/ReactiveVarObserver.tsx` — New helper that exposes Apollo reactive-var state via DOM data attributes so Playwright assertions can observe cross-process (browser-side) state without reaching into the Node fixture.
  - Added workarounds for DndContext pointer-event interception (`clickViaReact` / `openContextMenu` using React 18 `__reactProps$` fiber access) with explicit upgrade-risk comments.
- **Route component and ExtractDetail view test coverage** (Issue #1285): Closed coverage gaps for zero-coverage route wrappers and the 7%-covered `ExtractDetail.tsx` orchestrator.

  - `frontend/src/components/routes/__tests__/ExtractDetailRoute.test.tsx` — 6 vitest specs covering missing-ID, reactive-var reuse, loading, error, not-found, and success paths for the slug-resolving extract route.
  - `frontend/src/components/routes/__tests__/ExtractLandingRoute.test.tsx` — 4 specs for the legacy `/e/:user/:extract` route, including the redirect to `/extracts/:id` when the reactive var is populated.
  - `frontend/src/components/routes/__tests__/LabelSetLandingRoute.test.tsx` — 5 specs for loading/error/success state and `onClose` cleanup.
  - `frontend/src/components/routes/__tests__/GlobalDiscussionsRoute.test.tsx` — smoke test for the thin route wrapper.
  - `frontend/src/components/routes/__tests__/NotFound.test.tsx` — renders 404 UI and verifies the "Go to Corpuses" button navigates back.
  - `frontend/src/components/routes/__tests__/UserProfileRoute.test.tsx` — 7 specs covering `/profile` redirects (anonymous → `/login`, logged-in → `/users/<slug>`), loading, error, not-found, and own-vs-other profile flags.
  - `frontend/tests/ExtractDetail.ct.tsx` + `ExtractDetailTestWrapper.tsx` + `ExtractDetailFixtures.ts` — 17 Playwright CT specs exercising the extract-detail view's header, status chips, StatBlocks, running/failed/completed state swaps, Data/Documents/Schema tabs, CreateColumnModal/ConfirmModal integration, and the Export-CSV enable/disable gating.
  - `frontend/tests/e2e/routing-round-trip.spec.ts` — Playwright E2E spec covering the URL → entity → back round-trip for corpuses, documents, deep-link replay, and the unknown-route `NotFound` fallback. Runs after the corpus-workflow spec so the necessary seed data exists.
  - Verification: 36 route vitest specs pass (`yarn test:unit --run src/components/routes/__tests__/`), 17 ExtractDetail CT specs pass (`yarn test:ct --reporter=list ExtractDetail.ct.tsx`), `tsc --noEmit` clean.
  - Surfaced Issue #1295 (UserProfileRoute Rules-of-Hooks violation) as a follow-up.

- **Frontend coverage: admin panel CT tests** (Issue #1281): Added 32 new Playwright component tests to close the coverage gap on the three admin-panel files called out in `docs/coverage/frontend-roi-ranking.md` (ranks 10, 24, 27).
  - `frontend/tests/pipeline-icons.ct.tsx` (+ `PipelineIconsTestWrapper.tsx`) — 5 tests covering `components/admin/PipelineIcons.tsx` (5.1% → target ≥60%). Mounts the full icon catalog (Docling, LlamaParse, TextParser, PdfThumbnail, TextThumbnail, ModernBert, SentenceTransformer, Multimodal, Generic), asserts brand icons render as `<img>` with alt text while inline SVG icons render as `<svg>`, verifies `size`/`className` prop forwarding, exhaustively exercises every branch of the `getComponentIcon` className dispatcher (including the compound `pdf+thumb`, `text+thumb`, and `modern_bert` patterns that must precede the single-token fallbacks), and covers `getComponentDisplayName` including the `title` override and `KNOWN_ACRONYMS` replacement path.
  - `frontend/tests/global-agent-management.ct.tsx` — 15 tests covering `components/admin/GlobalAgentManagement.tsx` (8.8% → target ≥60%). Adds loading/error states, list-rendering branches (missing slug, >3 tools with `+N` overflow badge, long-description truncation, inactive status), and full create/update/delete flows: submit-button disabling until required fields are filled, invalid-JSON `badgeConfig` validation toasts, successful create/update mutations with refetch, server-side `ok=false` error toasts, the `ConfirmModal` cancel path, and confirmed-delete mutation + refetch. Introduces a new `GlobalAgentManagementWithToastsWrapper` in `tests/AdminComponentsTestWrapper.tsx` that provides `ToastContainer` + `MemoryRouter` so tests can assert on react-toastify output without affecting pre-existing wrapper consumers.
  - `frontend/tests/system-settings-flows.ct.tsx` — 12 tests covering additional `components/admin/SystemSettings.tsx` flows not exercised by `admin-components.ct.tsx` (17.9% → target ≥60%): assigning/clearing parsers via the filetype-default dropdown, opening the Default Embedder modal and saving both list-selected and hand-typed class paths, non-secret config save (`num_workers` int + `verbose` bool select) through `AdvancedSettingsPanel`, mobile-tab keyboard navigation (`ArrowLeft`/`ArrowRight`/`Home`/`End` including wrap-around), mutation error branches (network error toast + server-side `ok=false` message), and the `handleToggleEnabled` transition from empty `enabledComponents` ("all enabled") to an explicit list.
  - Verification: 77/77 tests pass (`yarn test:ct --reporter=list admin-components.ct.tsx pipeline-icons.ct.tsx global-agent-management.ct.tsx system-settings-flows.ct.tsx`) with no changes to pre-existing admin-components tests. `tsc --noEmit` and `prettier --check` clean.
- **Annotator hook / renderer / label-selector coverage** (Issue #1284): Added vitest coverage for the annotator hook layer plus Playwright CT coverage for the text and docx renderers and the enhanced label selector.

  - `frontend/src/components/annotator/hooks/__tests__/AnnotationHooks.test.tsx` — 25 tests covering state-wrapper helpers, guard clauses, success paths, and relation side effects. Includes a regression that keeps a sibling annotation in state across a `useUpdateAnnotation` call.
  - `frontend/tests/TxtAnnotator.ct.tsx`, `frontend/tests/DocxAnnotator.ct.tsx`, and `frontend/tests/EnhancedLabelSelector.ct.tsx` — CT suites driving prop-driven scenarios (visibility filtering, search highlights, chat sources, structural filtering, read-only mode).

- **Frontend component-test coverage for Corpus Chat & Agent Management** (Issue #1276): Added Playwright CT coverage for the four corpus components called out as the highest-uncovered surface area in `docs/coverage/frontend-roi-ranking.md` (CorpusChat, CreateCorpusActionModal, CorpusAgentManagement, CorpusDescriptionEditor — combined ~3,300 lines, ~2,800 previously uncovered).
  - `frontend/tests/CorpusChat.ct.tsx`: extended from 3 to 13 tests. Adds a Playwright `WebSocket` stub (mirroring the `ChatTray.ct.tsx` pattern) so we can drive ASYNC_START/CONTENT/FINISH, ASYNC_THOUGHT (with compaction notice), ASYNC_APPROVAL_NEEDED, ASYNC_APPROVAL_RESULT, ASYNC_ERROR (deferred via `setTimeout` to avoid being clobbered by the synchronous `setWsError(null)` after `socket.send`), and CONTEXT_EXHAUSTED. Covers approve/reject flows, context-meter rendering, send-button gating, conversation loading, navigation callbacks, and GraphQL error surface.
  - `frontend/tests/CorpusChatTestWrapper.tsx`: now sets `authToken`/`userObj` synchronously (matches `ChatTrayTestWrapper`) to avoid a re-mount race that closes the stub socket between renders. Forwards `initialQuery`, `onNavigateHome`, and `onMessageSelect` to the component.
  - `frontend/tests/CreateCorpusActionModal.ct.tsx` + `CreateCorpusActionModalTestWrapper.tsx`: 13 tests covering create/edit modes, modal open/close lifecycle, all three action types (fieldset/analyzer/agent), trigger-driven query switching (document tools vs. moderation tools), inline-vs-existing-agent toggle, validation toasts, Clear-All / Select-All tool toggles, and the create + update mutation happy paths.
  - `frontend/tests/CorpusAgentManagement.ct.tsx` + `CorpusAgentManagementTestWrapper.tsx`: 9 tests covering the `canUpdate=false` permission notice, empty state, agent row rendering with status badge and slug, create modal (with disabled-submit gating), edit modal pre-population, delete confirmation flow (yes / no), and the Permission Required Tools section unlocking when an available tool is selected.
  - `frontend/tests/CorpusDescriptionEditor.ct.tsx` + `CorpusDescriptionEditorTestWrapper.tsx`: 10 tests covering closed-modal behavior, fetching markdown via `page.route` interception, empty-corpus state, "Unsaved changes" indicator, Show/Hide history panel, version-row expansion + reapply mutation, edit-from-version state with badge, and unsaved-aware close.
  - All test wrappers mount a `<ToastContainer />` so `react-toastify` calls produce visible DOM in CT (Playwright `index.tsx` does not mount one globally).
  - Verification: 45/45 new + updated tests pass via `yarn test:ct CorpusChat.ct.tsx CreateCorpusActionModal.ct.tsx CorpusAgentManagement.ct.tsx CorpusDescriptionEditor.ct.tsx --reporter=list`; `tsc --noEmit` clean on the new files.

### Added

- **Extracts DataGrid & Detail component test coverage** (Issue #1282): Expanded Playwright component tests for the two biggest uncovered files in `frontend/src/components/extracts/`. Previously at 32.5% and 23.0% line coverage respectively (~990 uncovered lines combined); the new tests exercise the high-signal branches listed in the issue.

  - `frontend/tests/DataGrid.ct.tsx`: grew from 4 to 19 tests. New coverage: loading overlay (idle vs running copy), Document/column sort toggles, row-selection bulk-delete bar + callback, add-column modal, per-column edit modal, per-column delete confirmation (including the `fieldset.inUse` warning copy), add-documents FAB, a 4-type cell-rendering matrix (text/number/boolean/JSON object), corrected-data precedence, and `exportToCsv` via the imperative handle.
  - `frontend/tests/ExtractDetailContent.ct.tsx` + `frontend/tests/ExtractDetailContentTestWrapper.tsx` (both new): 16 tests covering the loading overlay, not-found state, stats panel, Data/Documents/Schema tabs, running-state spinner, failed-state Retry button + `startExtract` mutation, schema tab empty-vs-populated + Add Column + Delete Column confirmation, Documents tab empty state, and both imperative-handle methods (`exportToCsv`, `startExtract`) plus the `onExtractLoaded` callback.
  - `frontend/tests/DataGridTestWrapper.tsx`: added optional callback-spy props (`onAddDocIds`, `onRemoveDocIds`, `onRemoveColumnId`, `onAddColumn`) and a `withExportButton` flag that renders a test-only button bound to the grid's imperative `exportToCsv` handle, so tests can verify the handle without reaching into internal refs.

- **Frontend coverage tests for useAgentChat / LabelSetDetailPage / ModerationDashboard** (Issue #1286): Added 47 new tests lifting the three high-ROI files from ~11–28% line coverage toward the ≥60% target.

  - `frontend/src/hooks/__tests__/useAgentChat.test.tsx` — 23 new Vitest unit tests wiring a mock `WebSocket` through `renderHook`. Cover connection lifecycle (open, error, close), every streaming message type (`ASYNC_START`, `ASYNC_CONTENT`, `ASYNC_FINISH`, `ASYNC_THOUGHT`, `ASYNC_SOURCES`, `ASYNC_APPROVAL_NEEDED`, `ASYNC_APPROVAL_RESULT`, `ASYNC_RESUME`, `ASYNC_ERROR`, `SYNC_CONTENT`), malformed JSON handling, approval flow including `sendApprovalDecision`, `sendMessage` guards (empty, disconnected, while-processing, send-throws), `clearError`, and `setSelectedMessageId`. Local coverage now 82.9% lines for `useAgentChat.ts`.
  - `frontend/tests/LabelSetDetailPage.coverage.ct.tsx` — 15 new Playwright CT tests covering previously-uncovered mutation success paths: `UPDATE_ANNOTATION_LABEL` inline edit save, `CREATE_ANNOTATION_LABEL_FOR_LABELSET` submission, `DELETE_MULTIPLE_ANNOTATION_LABELS` delete flow, `DELETE_LABELSET` confirm modal → Yes → mutation, `handleExportJSON` blob download handler, Overview tab Delete button (visible/hidden per permission), Edit Details footer button visibility, Relationships / Doc Labels / Sharing tabs, error state when the query fails, and the empty-state "Add First Label" branch.
  - `frontend/tests/ModerationDashboard.coverage.ct.tsx` — 9 new Playwright CT tests: `ROLLBACK_MODERATION_ACTION` mutation with reason, action-type filter refetch, automated-only toggle refetch, time-range dropdown change refetch, `Load More` cursor pagination via `fetchMore`, actions-query error state, metrics-query error state, rollback-modal cancel, and the System / "No reason provided" rendering branches.
  - **Issue #1296 discovered and filed**: `useAgentChat.ts:810` includes `pendingApproval` in the main WebSocket `useEffect` dependency array, so the socket tears down + reconnects every time approval state changes. This drops in-flight tokens and can race `sendApprovalDecision` against `isConnected=false` during reconnect. A regression test in `useAgentChat.test.tsx` documents the current behaviour; the workaround comments reference #1296 so they can be removed after the fix lands.

- **Codecov components for backend and frontend** (`.codecov.yml`): Added `component_management` with two components — Backend (`opencontractserver/**`, `config/**`) and Frontend (`frontend/src/**`) — grouping all five flags so PR comments and the Codecov UI show a single frontend number (union of unit + component + e2e) instead of five overlapping flag numbers. Component-level patch/project statuses now gate PRs per-area.
- **Frontend coverage ROI audit** (`docs/coverage/frontend-roi-ranking.md`): Ranked every frontend source file by uncovered line count against the Codecov project-level (flag-union) report. Top 50 files, files at 0% coverage with >=100 lines, and per-area totals. Source for prioritizing targeted coverage work.
- **Frontend error-path test coverage** (Issue #1270): Added targeted unit tests covering error boundaries, the Apollo error link, and `catch` blocks in utility modules. Previously these branches were effectively untested, leaving regressions in auth error handling, file download failures, and error boundary fallback UI undetectable in CI.
  - `frontend/src/components/widgets/__tests__/ErrorBoundary.test.tsx` — 7 new tests for `ErrorBoundary.tsx`: default fallback UI on child-thrown errors, `onError` callback invocation, console logging in `componentDidCatch`, custom `fallback` render prop, recovery via the reset button, and persistent error state when the child keeps throwing.
  - `frontend/src/graphql/errorLink.test.ts` — Rewrote the existing smoke test (which only toggled reactive vars) into a full suite that executes the real `errorLink` through an `ApolloLink.from([errorLink, terminating])` chain. Now asserts all auth-error branches (401/403/`UNAUTHENTICATED`, message-based detection, JWT-expired reload via `setTimeout` + `window.location.reload`), the non-auth logging fall-through, and both network-error branches (401/403 vs generic).
  - `frontend/src/utils/__tests__/files.test.ts` — Added tests for `downloadFile` success path (axios → Blob → anchor click) and its `catch (e)` branch (rejects with the original error and logs before re-throwing), plus `toBase64` success and `FileReader.onerror` rejection path.
  - `frontend/src/utils/graphqlGuards.test.ts` — Added explicit coverage for `createSafeQueryExecutor`'s `catch (error)` branch (line 129, logs via `console.error` and surfaces the error) and the validation-blocked `console.warn` branch.
  - Verification: 49/49 new + updated tests pass via `yarn vitest run` on all four files; `tsc --noEmit` and `prettier --check` both clean.

### Fixed

- **`PdfAnnotations.undoAnnotation()` mutated the caller's annotations array** (Issue #1291): `frontend/src/components/annotator/types/annotations.ts:311` called `this.annotations.pop()` before constructing the new `PdfAnnotations`, silently shortening the original instance's `annotations` array even though the field is declared `readonly`. This violated the immutability contract the other methods (`saved()`, `update()`, `fromObject()`) uphold and is a footgun for the upcoming PdfAnnotator package extraction (#1283) — any consumer holding a reference to the pre-undo instance would see a stale length without a state change. The method now derives `popped` via index access and `remaining` via `slice(0, -1)`, so the caller's array is untouched. The pinning test in `frontend/src/components/annotator/types/__tests__/annotations.test.ts` was updated to assert non-mutation (`pdf.annotations` stays `[a, b]` after `pdf.undoAnnotation()`), and the "known wrinkle" comment was removed.

### Removed

- **Frontend dead Jotai atoms and Apollo reactive vars** (Issue #1243): Removed unused state management exports after triple-verifying each against both `frontend/src/` and `frontend/tests/` — atoms consumed only by test wrappers (e.g. `hideLabelsAtom`, `rawPermissionsAtom`, the deprecated `showAnnotation*Atom` set) were left in place per the issue's scope-correction note.

  - **`frontend/src/atoms/folderAtoms.ts`**: Removed 8 unused exports — `currentFolderAtom` (only read by the two dead permission atoms below), `canUpdateCurrentFolderAtom`, `canDeleteCurrentFolderAtom`, `draggingDocumentIdAtom`, `enableDragDropAtom`, `collapseAllFoldersAtom`, `expandAllFoldersAtom`, `openMoveFolderModalAtom`.
  - **`frontend/src/atoms/threadAtoms.ts`**: Removed 5 unused atoms — `selectedCorpusIdAtom`, `currentThreadIdAtom`, `expandedMessageIdsAtom`, `showCreateThreadModalAtom`, `editingMessageIdAtom` — plus the unused `ConversationType` / `ChatMessageType` imports. Unexported the `ThreadFilterOptions` type since it is only used internally by `threadFiltersAtom`.
  - **`frontend/src/components/annotator/context/DocumentAtom.tsx`**: Removed 7 unused atoms and 7 unused hooks: `fileTypeAtom` + `useFileType`, `isLoadingAtom` + `useIsLoading`, `canUpdateDocumentAtom` + `useCanUpdateDocument`, `canDeleteDocumentAtom` + `useCanDeleteDocument`, `hasDocumentPermissionAtom` + `useHasDocumentPermission`, `pageSelectionAtom`, `pageSelectionQueueAtom` + `usePageSelectionQueue`, plus the dead `useViewState` and `usePermissions` hooks (the underlying `viewStateAtom` and `permissionsAtom` are kept because they remain live via `useSetViewStateError` / `useDocumentPermissions` respectively). Dropped the now-unused `BoundingBox` import.
  - **`frontend/src/components/annotator/context/UISettingsAtom.tsx`**: Removed `hasScrolledToAnnotationAtom` (zero references anywhere in the tree).
  - **`frontend/src/graphql/cache.ts`**: Removed 7 unused reactive vars — `addingColumnToExtract`, `editingColumnForExtract`, `editingExtract`, `analyzerSearchTerm`, `editMode`, `allowUserInput`, `filterToAnnotationLabelId` — plus the now-unused `ColumnType` import. Unexported the `LinkDocumentsModalState` interface since it is only used internally as the type argument for `linkDocumentsModalState`.
  - Verification: `tsc --noEmit` passes, `vitest run` passes (935/935 unit tests).

- **Frontend dead styled components in `knowledge_base/document/styled/`** (Issue #1241): Removed unused styled-components from the `frontend/src/components/knowledge_base/document/styled/` folder. None of the deleted exports are referenced by any production code, test wrapper, or `.ct.tsx` test:

  - **`Relationships.tsx` deleted entirely** — all 3 exports (`RelationshipPanel`, `RelationshipCard`, `RelationshipType`) were unused. Note: a GraphQL type with the same name `RelationshipType` lives in `types/graphql-api.ts` and is unaffected.
  - **`LoadingStates.tsx`**: removed `DocumentLoadingContainer` (truly dead — no internal or external use). The other exports (`PlaceholderBase`, `PlaceholderItem`, `SummaryPlaceholder`, `NotePlaceholder`, `RelationshipPlaceholder`) are kept because they are used internally by `LoadingPlaceholders` in the same file.
  - **`RightPanel.tsx`**: removed 5 unused exports (`ControlButtonGroupLeft`, `ControlButtonWrapper`, `ControlButton`, `ChatIndicator`, `ControlButtonGroup`). Kept the only consumed exports: `ConnectionStatus` and `SlidingPanel`.
  - **`styled/index.ts`** barrel pruned to drop the dead re-exports listed above.
  - Verification: `yarn build` succeeds, `yarn test:unit` passes (927/927), `yarn test:ct -g "DocumentKnowledgeBase" --reporter=list` passes (62/62).
  - **Deferred to a follow-up**: dead exports in `ResizeControls.tsx` (`ResizeHandleControl`, `ResizeHandleButton`, `WidthControlBar`, `WidthControlMenu`, `WidthControlToggle`, `WidthButton`, `AutoMinimizeToggle`) — although these are also unreferenced anywhere in the tree, deleting them deterministically exposes a pre-existing race in `tests/DocumentKnowledgeBase.ct.tsx:348 "fullscreen modal covers the entire viewport"` when the suite runs in parallel. The test passes in isolation but fails in the full suite, suggesting the deletion shifts bundle ordering enough to alter timing for the modal mount/measure window. Will be addressed in a separate PR that either patches the test to wait for the modal animation to settle, or finds the underlying race.

- **Frontend dead code cleanup, pass 1** (Issue #1240): Deleted 13 unreachable source files identified by static reachability analysis from `App.tsx`/`index.tsx` and the test entry points. No production importers and no test importers exist for any of these files:
  - Hooks orphaned by refactors: `frontend/src/hooks/useThreadWebSocket.ts` (thread real-time streaming via `ws/thread-updates/` was never wired into any thread component; the backend consumer exists but the frontend hook had no importers — see Issues #623, #697 for future work), `frontend/src/hooks/useUrlAnnotationSync.ts` (the latter was already documented as removed in a comment at `DocumentKnowledgeBase.tsx:1858` but the file itself was never deleted; the stale comment is also removed)
  - Duplicate styled helpers: `frontend/src/components/common.tsx` (duplicated `VerticallyCenteredDiv`/`SidebarContainer` from `components/layout/Wrappers.tsx`) and `frontend/src/components/annotator/common.tsx` (only consumed by the dead file above; also contained a `console.log` that would have appeared in production builds)
  - Dead types: `frontend/src/components/annotator/types/ui.ts` (unused `TabVisibility`) and `frontend/src/components/annotator/renderers/txt/types.ts` (duplicate stub of `MultipageAnnotationJson`; the real one lives in `components/annotator/types/annotations.ts`)
  - Dead barrels (every consumer imports inner files directly, never through these): `frontend/src/components/widgets/icon-picker/index.ts`, `frontend/src/components/corpuses/CorpusAbout/index.ts`, `frontend/src/components/corpuses/CorpusHome/index.ts` (file `CorpusHome.tsx` shadows the directory in Node module resolution), `frontend/src/components/corpuses/caml/index.ts`, `frontend/src/components/corpuses/folders/index.ts`
  - Dead utility: `frontend/src/utils/history.ts` (`createBrowserHistory()` singleton, never imported)
  - Dead mock factory file: `frontend/src/tests/utils/factories.ts` (mock GraphQL response factories with no consumers; its parent directory `src/tests/` is also removed since it's now empty)
  - Updated documentation referencing deleted `useThreadWebSocket`: removed hook section and references from `docs/architecture/websocket/frontend.md`, annotated implementation plan entries in `docs/features/agent_mentions_implementation_plan.md`, removed row from `docs/commenting_system/IMPLEMENTATION_GUIDE.md`
  - Verification: `yarn build` succeeds, all 927 unit tests pass (`yarn test:unit`), and the 16 icon picker component tests still pass (`IconDropdown`, `IconPickerModal`, and `icons.ts` are kept because they have full Playwright `.ct.tsx` test suites in `frontend/tests/`)

### Added

- **Benchmark harness for external RAG datasets** (new app `opencontractserver/benchmarks/`): Generate an OpenContracts corpus from a third-party benchmark (LegalBench-RAG today, pluggable for CUAD/MAUD/etc. via a small adapter interface), run the production extract-grid pipeline against the benchmark's queries with a configurable LLM, probe retrieval independently via `CoreAnnotationVectorStore`, and compute standard metrics (SQuAD-style exact match / token F1 for answers; character-span recall@k / precision@k / IoU for retrieval). Results are written as `report.json` / `report.csv` / `config.json` / `gold.json` under a run directory.
  - Adapter interface and `LegalBenchRAGAdapter` at `opencontractserver/benchmarks/adapters/` (reads the authoritative ZeroEntropy schema — `{"tests": [{"query", "snippets": [{"file_path", "span": [start, end]}], "tags"}]}`)
  - Loader, runner, evaluator, and report modules under `opencontractserver/benchmarks/`
  - Django management command: `python manage.py run_benchmark --benchmark legalbench-rag --path /data/legalbench-rag --user admin --model openai:gpt-4o-mini --top-k 10`
  - Micro fixture under `fixtures/benchmarks/legalbench_rag_micro/` for end-to-end tests without downloading the full dataset
  - Test coverage: `opencontractserver/tests/test_benchmarks.py` (metric unit tests, adapter unit tests, loader materialization test, runner end-to-end test with mocked structured-response agent)
- **`model_override` kwarg on `doc_extract_query_task`** (`opencontractserver/tasks/data_extract_tasks.py`): Optional, backward-compatible kwarg that lets callers override the hardcoded `openai:gpt-4o-mini` default for a single invocation. Consumed by the benchmark runner to sweep models without affecting production defaults; still defaults to `openai:gpt-4o-mini` when not supplied.
- **Frontend unit tests for utils and hooks** (Issue #1267): Added 14 new `*.test.ts(x)` files covering previously-untested utilities and hooks to raise `frontend-unit` coverage on high-ROI pure functions:

  - **Utils**: `formatters.test.ts`, `arrayUtils.test.ts`, `colorUtils.test.ts`, `parseOutputType.test.ts`, `annotationGuards.test.ts`, `env.test.ts`, `extractUtils.test.ts`, `layout.test.ts`, `persistentVar.test.ts`, `routingLogger.test.ts`, `navigationCircuitBreaker.test.ts`, `performance.test.ts`, `jobNotificationCacheUpdates.test.ts`, `compactAnnotationJson.test.ts`.
  - **Hooks**: `useAuthReady.test.tsx`, `useFeatureAvailability.test.ts`, `useMessageBadges.test.tsx`, `useBadgeCelebration.test.tsx` (render-hook based with vi.useFakeTimers/MockedProvider).
  - Adds ~210 new assertions across file-size/date/initial formatting, hex→RGB(A) conversions, Pydantic output-type parsing, per-annotation runtime guards, runtime env coercion, extract status, viewport clamping, session-storage-backed reactive vars, debug logger toggling, navigation circuit breaker tripping/reset/window pruning, performance monitor metric lifecycle, Apollo cache field-level mutation dispatch for job notifications, and v1↔v2 compact annotation JSON round-tripping.
  - Verification: full unit suite passes (1118/1118) and `tsc --noEmit` is clean.

- **Unit tests for Jotai atoms and Apollo reactive vars** (Issue #1268): Added vitest coverage for the global state layer per the ROI audit in PR #1266.

  - **`frontend/src/atoms/__tests__/folderAtoms.test.ts`**: 46 tests covering every primitive atom (initial value + write), every derived atom (`folderTreeAtom`, `folderBreadcrumbAtom`, `folderMapAtom`, `canCreateFoldersAtom`) across multiple dependency states, every write-only action atom (toggle/expand/open/close helpers), and `atomWithStorage` persistence paths (Set ↔ JSON round-trip, malformed-JSON fallback, SSR-safe `sidebarCollapsedAtom` default for mobile vs desktop viewports).
  - **`frontend/src/atoms/__tests__/threadAtoms.test.ts`**: 9 tests covering all five atoms plus a localStorage round-trip and re-hydration path for `threadContextSidebarExpandedAtom`.
  - **`frontend/src/graphql/__tests__/cache.test.ts`**: 24 tests covering initial values of every reactive var exported from `cache.ts` (routing, modals, entity refs, search terms, collections), round-trip writes for representative vars, `mergeArrayByIdFieldPolicy` id-based merge + default-empty branches, `InMemoryCache` presence, and `showKnowledgeBaseModal` (`persistentVar`) first-write persistence, re-hydration from sessionStorage, and malformed-JSON fallback.
  - Coverage (v8 provider): `src/atoms` → 99.23% statements / 93.54% branches; `frontend/src/graphql/cache.ts` → 95.6% statements / 100% branches. Well above the 80% target in the issue.
  - Verification: all 79 new tests green; full unit suite (1014/1014) still passes; `tsc --noEmit` clean.

- **Frontend permission-gating tests** (Issue #1269): Added branch-exhaustive coverage for the permission predicates that gate annotation write UI.

  - New utility `frontend/src/utils/annotationPermissions.ts` centralizes three pure predicates (`canEditAnnotationsInCorpus`, `canDeleteAnnotation`, `canUpdateAnnotation`) that were previously inlined in `DocumentKnowledgeBase.tsx:471` and `HighlightItem.tsx:242`. Both consumers now delegate to the helper, eliminating duplicated logic.
  - New unit tests `frontend/src/utils/__tests__/annotationPermissions.test.ts` cover every branch of the predicates, including the full 2×2 truth table for the corpus/document effective-edit check and the full 2³ truth table for the structural × read-only × `CAN_REMOVE` delete gate (30 assertions).
  - New Playwright CT tests `frontend/tests/HighlightItemPermissions.ct.tsx` (+ harness `HighlightItemPermissionsTestWrapper.tsx`) verify that the sidebar delete affordance renders only on the intersection of the required conditions, and that structural annotations are delete-locked even with `CAN_REMOVE` (6 scenarios).
  - Production behavior is unchanged — the helper is a faithful extraction of the existing inline logic.

- **Frontend E2E integration tests with coverage**: Added a Playwright integration spec (`frontend/tests/e2e/login-and-navigation.spec.ts`) that exercises the full Vite + Django + Postgres stack. The spec logs in via the password form against a real backend and walks every routed view in `src/views/` (Discovery, Corpuses, Documents, LabelSets, Annotations, Extracts, GlobalDiscussions, ThreadSearchRoute, PrivacyPolicy, TermsOfService, UserProfile, Login). New supporting files:
  - `frontend/tests/e2e/fixtures.ts` — Playwright fixture that dumps `window.__coverage__` to `frontend/coverage/e2e/.nyc_output/` after every test (mirrors the CT pattern in `tests/utils/coverage.ts`).
  - `frontend/tests/e2e/helpers.ts` — `VIEWS` catalog, `loginViaUI`, `spaNavigate`, and `expectViewVisible` helpers. Uses `history.pushState` + `popstate` dispatch for SPA navigation so the in-memory `authToken` reactive var survives between routes.
  - `frontend/playwright.config.ts` — Rewritten to manage a `webServer` block that boots vite with `COVERAGE=true` and `REACT_APP_USE_AUTH0=false`, runs the chromium project against `http://127.0.0.1:5173`, and matches `tests/e2e/**/*.spec.ts` (existing `*.spec.tsx` files under `tests/` keep running inside the CT runner).
  - `frontend/package.json` — New `test:e2e:coverage` script that runs the spec with Istanbul instrumentation and merges the per-test JSON dumps into `coverage/e2e/lcov.info` via `nyc report`.
  - `.github/workflows/frontend-e2e.yml` — New CI job that builds the django image from `test.yml`, brings up postgres + redis + django with `--no-deps` (skipping the multi-GB parser/embedder images), waits on `/api/health/`, verifies migration `0003_create_initial_superuser` produced the `admin` user, then runs `yarn test:e2e:coverage` and uploads the lcov to Codecov under the `frontend-e2e` flag.
  - `.codecov.yml` — Declared the `frontend-e2e` flag with `paths: [frontend/src/]` and `carryforward: true` so the E2E upload participates in project coverage aggregation alongside `frontend-unit` and `frontend-component`. Without this declaration the flag was accepted by Codecov but not aggregated into the main-branch project total, which is why coverage did not visibly rise after PR #1251 merged.
- **Bounded `fullDatacellList` payload for extract grid embeds** (Issue #1204): `ExtractType.fullDatacellList` now accepts optional `limit` and `offset` arguments, and a new `ExtractType.datacellCount` field returns the total visible datacell count ignoring pagination. Resolver `resolve_full_datacell_list` applies a deterministic `order_by("document_id", "column_id", "id")` before slicing so limits/offsets produce stable windows across requests (`config/graphql/extract_types.py`, `schema.graphql`). Frontend `GET_EXTRACT_GRID_EMBED` (`frontend/src/graphql/queries.ts`) now passes `limit: EXTRACT_GRID_EMBED_CELL_LIMIT` (500) and fetches `datacellCount`; `ExtractGridEmbed` (`frontend/src/components/extracts/ExtractGridEmbed.tsx`) renders a "Showing N of M cells" / "Showing N of M documents" partial-data footer instead of the previous hard "exceeds limit" error screen. Adds the `EXTRACT_GRID_EMBED_CELL_LIMIT` constant (`frontend/src/assets/configurations/constants.ts`), a new `partial` state to `ExtractGridEmbedTestWrapper`, and backend coverage in `test_extract_queries.py::test_full_datacell_list_limit_offset_and_count`.

- **Agent memory system**: Per-corpus memory that lets agents accumulate reusable insights from conversations. Memory is stored as a first-class markdown Document in the corpus (visible and editable by users). Features include:
  - Corpus model fields: `memory_enabled` toggle and `memory_document` FK (`opencontractserver/corpuses/models.py`)
  - Memory CRUD utilities with hybrid retrieval (full injection for small memory, keyword-scored section filtering for large) (`opencontractserver/agents/memory.py`)
  - End-of-conversation curation via two-stage Celery task: privacy-preserving summarisation followed by LLM-based pattern extraction (`opencontractserver/tasks/memory_tasks.py`)
  - Automatic memory injection into agent system prompts via `UnifiedAgentFactory` (`opencontractserver/llms/agents/agent_factory.py`)
  - New agent tools: `get_corpus_memory` (read) and `suggest_memory_update` (write) (`opencontractserver/llms/tools/core_tools.py`)
  - GraphQL `ToggleCorpusMemory` mutation and `memoryActiveWarning` field with privacy notice (`config/graphql/corpus_mutations.py`, `config/graphql/corpus_types.py`)
  - Periodic Celery beat task to detect idle conversations eligible for curation (`config/settings/base.py`)
  - Constants and configuration in `opencontractserver/constants/agent_memory.py`
- **Frontend code coverage reporting** (PR #1205): Added coverage collection and Codecov integration for both unit tests (Vitest + v8) and component tests (Playwright CT + Istanbul). Coverage is opt-in via `COVERAGE=true` environment variable to avoid performance overhead during normal test runs. Custom Playwright fixture (`frontend/tests/utils/coverage.ts`) extracts Istanbul `__coverage__` data from the browser after each test. CI uploads separate coverage flags (`frontend-unit`, `frontend-component`) with `carryforward: true` for granular tracking. Added `.codecov.yml` configuration, `vite-plugin-istanbul` for build-time instrumentation, and `nyc` for LCOV report generation from collected coverage data.

### Fixed

- **Rules-of-Hooks violation in `UserProfileRoute`** (Issue #1295): `frontend/src/components/routes/UserProfileRoute.tsx` called `useQuery` _after_ a conditional early return for the `!slug` redirect case, so when the same component fiber was reused across the `/profile` → `/users/:slug` redirect (e.g. both routes rendering `UserProfileRoute` in a single `Routes` tree) React threw `Rendered more hooks than during the previous render`. Production flows that unmount/remount across the redirect masked the bug, but any future refactor that kept the fiber alive would start crashing, and test suites that mounted both routes together could not exercise the full redirect → render path. Moved `useQuery` above the early-return block and pass `{ slug: slug ?? "" }` as variables while keeping the existing `skip: !slug` gate on the network call.
- **`PdfAnnotations.undoAnnotation()` mutated the source array** (Issue #1291): `frontend/src/components/annotator/types/annotations.ts` called `this.annotations.pop()` before constructing the returned instance, silently mutating the caller's `annotations` array even though the field is declared `readonly`. Every other method on this reducer-style class returns without touching the input, so the leak was a footgun for downstream consumers of the upcoming PdfAnnotator package extraction (Issue #1283) — any reference to the pre-undo instance would observe a stale shorter array without re-rendering. Replaced `.pop()` with a non-mutating `slice(0, -1)` + index read. Updated `frontend/src/components/annotator/types/__tests__/annotations.test.ts` to remove the pinning comment and add an explicit assertion that the original instance's `annotations` array is unchanged after `undoAnnotation()`.
- **`RelationGroup.updateForAnnotationDeletion` never pruned orphaned relations** (Issue #1292): `frontend/src/components/annotator/types/annotations.ts:49-50` computed `nowSourceEmpty` / `nowTargetEmpty` against `this.sourceIds` / `this.targetIds` (the **pre-filter** arrays) instead of the `newSourceIds` / `newTargetIds` arrays produced a few lines above. Because the "now empty" flags were identical to the "was empty" flags, all four `return undefined` branches intended to prune orphaned relations were unreachable, and the method always returned a fresh `RelationGroup` — even when `PdfAnnotations.undoAnnotation()` had just removed the relation's only source or target annotation. That left zombie relations in `PdfAnnotations.relations` with empty `sourceIds` or `targetIds` and required defensive handling downstream. Fix swaps the two comparisons to the post-filter arrays so the documented deletion conditions fire correctly. `frontend/src/components/annotator/types/__tests__/annotations.test.ts` drops the bug-pinning header note and replaces the ambiguous "prunes any relations" assertion with two explicit cases: the survivor is kept when it still has a source and a target, and a relation whose only target references the popped annotation is now removed (21/21 unit tests pass).
- **O(N) query regression in bulk folder operations** (Issue #1199): `DocumentFolderService.move_documents_to_folder()` and `DocumentFolderService.delete_folder()` in `opencontractserver/corpuses/folder_service.py` previously issued ~3 DB round-trips per document (an EXISTS-style disambiguation fetch, an `UPDATE` via `save(update_fields=…)`, and an `INSERT` via `DocumentPath.objects.create()`). For a 100-document batch that was ~300 queries instead of the single-query `.update()` the old non-lineage code used. Both methods now:
  1. Pre-fetch all occupied paths in the target directory in a **single** query via the new `_fetch_occupied_paths_in_directory` helper and pass the shared mutable set to `_disambiguate_path` via a new `occupied_override` parameter (replaces the single-purpose `extra_occupied` kwarg).
  2. Batch-deactivate every superseded `DocumentPath` row with one `.filter(pk__in=…).update(is_current=False)` call.
  3. Batch-insert every successor row with one `.bulk_create()` call, then manually dispatch `post_save` (`created=True`) via the new `_dispatch_document_path_created_signals` helper so the document-text embedding side effect wired up in `documents/signals.py::process_doc_on_document_path_create` still fires (bulk_create normally bypasses per-row signals).
  4. Use `select_related("document")` + `select_for_update(of=("self",))` on the affected-path query so `current.document` accesses inside the build loop no longer N+1, and the row lock stays scoped to the `DocumentPath` table. Net result: a 100-document bulk move now executes roughly 4 DB round-trips instead of ~300, and the old batched `.update()` performance is restored without sacrificing the Path Tree history nodes introduced in PR #1195.
- **TOCTOU race on `DocumentPath` uniqueness** (Issue #1200): `DocumentFolderService.move_document_to_folder()`, `move_documents_to_folder()`, and `delete_folder()` previously caught `IntegrityError` from the `unique_active_path_per_corpus` partial unique constraint and either bubbled it up to the caller as a "Path conflict, please retry" error (single move) or rolled back the entire batch (bulk move / folder delete). Under concurrent moves of different documents to the same target folder, two transactions could both observe a candidate path as free in `_disambiguate_path()` and race to insert it; the loser hit the partial unique index and the operation failed. New helper `DocumentFolderService._create_successor_path_with_retry()` (`opencontractserver/corpuses/folder_service.py`) wraps the deactivate-then-create pair in a savepoint and retries with a freshly disambiguated path on `IntegrityError`, treating each losing path as occupied. Up to `MAX_PATH_CREATE_RETRIES + 1` attempts (`opencontractserver/constants/document_processing.py`) run before propagating the conflict. The partial unique index added in migration `0023_documentpath_documentpathgroupobjectpermission_and_more` remains the authoritative correctness guarantee. Test class `TestMoveDocumentIntegrityRecovery` covers single-document transient-failure recovery, disambiguated retry path selection, and exhausted-retry rollback. Bulk operations (`move_documents_to_folder`, `delete_folder`) now use the batch approach from Issue #1199 instead of per-row retry, so their former test classes (`TestBulkMoveIntegrityRecovery`, `TestDeleteFolderIntegrityRecovery`) have been replaced by `TestCoverageGapBulkMoveIntegrityErrorRollback` and `TestCoverageGapDeleteFolderIntegrityErrorRollback` which verify full-batch rollback on `IntegrityError` (`opencontractserver/tests/test_document_folder_service.py`).
- **Bulk move loop recomputed target folder path per document** (Issue #1202): `DocumentFolderService.move_documents_to_folder()` (`opencontractserver/corpuses/folder_service.py`) called `_compute_moved_path()` once per document, and each call invoked `target_folder.get_path()` — which walks ancestors via a recursive CTE query. For an N-document bulk move to the same folder, this issued N redundant CTE queries for an O(1) value. The target folder path is now resolved once before the loop and threaded through `_compute_moved_path()` via a new optional `target_folder_path` parameter.
- **Backend coverage disappearing from Codecov dashboard** (`.codecov.yml`): Added a `backend` flag with `carryforward: true` (paths: `opencontractserver/`, `config/`), mirroring the pattern already in place for `frontend-unit` / `frontend-component`. Without carryforward, any commit whose backend `pytest` job fails or times out (e.g. the merge commit `f166a59`, where the push-to-main `pytest` check failed after PR #1213 merged) causes Codecov to record backend files at 0% for that commit, which then replaces the healthy ~90% backend coverage on the dashboard — making it look as if only frontend coverage (~39%) is being collected. With carryforward, a flaky/failing backend run inherits the parent commit's backend coverage so the dashboard continues to reflect both suites.
- **IngestionSource follow-up hardening** (Issue #1228):

  - Added `@graphql_ratelimit` to `CreateIngestionSourceMutation`, `UpdateIngestionSourceMutation`, and `DeleteIngestionSourceMutation` (`config/graphql/ingestion_source_mutations.py`). Previously these mutations only carried `@login_required`, allowing an authenticated user to create thousands of rows (plus Guardian permission entries) in a tight loop. Creates/updates use `WRITE_MEDIUM`, deletes use `WRITE_LIGHT`, matching other CRUD mutations in the codebase.
  - Guarded the fallback `.get()` inside the `except IntegrityError` handler in `_import_ingestion_sources` (`opencontractserver/tasks/import_tasks_v2.py` ~line 463). In the rare scenario where a concurrent request created-then-deleted the row between the `IntegrityError` and the fallback, the unguarded `.get()` would raise `IngestionSource.DoesNotExist` and abort the entire corpus import. Now logs a warning and continues.
  - Added an explicit `fields` allowlist to `IngestionSourceType.Meta` (`config/graphql/document_types.py`) to prevent `user_lock`, `backend_lock`, and `is_public` from leaking through the GraphQL API. `user_lock` in particular would leak the username of whoever currently holds the lock — an information-disclosure issue.
  - Added a clarifying comment documenting the intentional export/import asymmetry for `ingestion_metadata` in `_reconstruct_document_paths` (`opencontractserver/tasks/import_tasks_v2.py`).

- **Document lineage fields dropped on move/delete/restore** (PR #1197): `move_document`, `delete_document`, and `restore_document` in `opencontractserver/documents/versioning.py` now forward `ingestion_source`, `external_id`, and `ingestion_metadata` from the parent path record to newly-created path nodes. Previously these fields were silently dropped, losing lineage provenance on any path operation.

- **`ingestion_metadata` truthiness check** (PR #1197): Changed `if ingestion_metadata:` to `if ingestion_metadata is not None:` in `config/graphql/document_mutations.py` so that an empty dict `{}` is correctly stored rather than silently discarded.

- **Lazy logging in export_v2.py** (PR #1197): Replaced remaining f-string logger calls with `%s`-style lazy formatting in `opencontractserver/utils/export_v2.py` for corpus folders, relationships, conversations, structural annotations, and markdown description error handlers.

- **TOCTOU race in `_import_ingestion_sources`** (PR #1197): Wrapped `get_or_create` in `transaction.atomic()` savepoint so that an `IntegrityError` from a concurrent insert doesn't abort the outer PostgreSQL transaction, which would cause the fallback `.get()` to raise `TransactionManagementError`.

- **`DRFMutation` validation error formatting extracted** (PR #1197): Extracted inline validation-error formatting logic from `DRFMutation.mutate()` into a reusable `format_validation_error()` static method (`config/graphql/base.py`). Updated `TestDRFMutationValidationError` tests to call the real method instead of duplicating the logic inline.

- **Renamed `EXPECTED_GLOBAL_ID_TYPE` to `INGESTION_SOURCE_GLOBAL_ID_TYPE`** (PR #1197): Renamed the generic constant to be self-documenting across `config/graphql/document_types.py`, `document_queries.py`, `document_mutations.py`, and `ingestion_source_mutations.py`.

- **Conversation pagination cache collisions** (PR #1206): Added `keyArgs` configuration for the `conversations` relay pagination cache entry in `frontend/src/graphql/cache.ts`. Previously `conversations` used `relayStylePagination()` with no key arguments, causing all conversation queries (across different corpora, documents, and conversation types) to share a single cache entry. This led to paginated results from one context bleeding into another. Now uses `["documentId", "corpusId", "conversationType", "hasCorpus", "hasDocument"]` to isolate cache entries by filter dimensions.

- **GraphQL security hardening cleanup** (Issue #1198):

  - Corrected misleading `DisableIntrospection` docstring that claimed the class checks DEBUG, when it unconditionally blocks introspection (`config/graphql/security.py`)
  - Rewrote `test_introspection_allowed_in_debug` to use graphql-core's `validate()` directly, as graphene's test Client does not apply validation rules (`opencontractserver/tests/test_security_hardening.py`)
  - Added backslash-prefix check to `_get_safe_redirect_url()` to prevent open-redirect bypass via browser backslash-to-slash normalization (`config/admin_auth/views.py`)

- **Batch embedding cleanup** (Issue #1226): Follow-up on PR #1148:
  - Added backward-compatible re-exports of `EmbeddingClientError` / `EmbeddingServerError` from `opencontractserver/pipeline/embedders/multimodal_microservice.py` so external embedders that imported the symbols from their original module keep working. Canonical location remains `opencontractserver.pipeline.base.exceptions`.
  - `EmbeddingClientError` is now actually raised: batch embedding methods (`MicroserviceEmbedder.embed_texts_batch` and `BaseMultimodalMicroserviceEmbedder.embed_texts_batch` / `embed_images_batch`) now raise `EmbeddingClientError` on 4xx responses instead of silently returning `None`. Callers can now distinguish a client-side failure ("we sent bad data") from a parsing error that still returns `None`. `_batch_embed_text_annotations` in `opencontractserver/tasks/embeddings_task.py` catches `EmbeddingClientError` explicitly and records a permanent per-annotation failure without triggering Celery retries — preventing retries from burning on invalid input that will never succeed. The single-text paths (`_embed_text_impl`, `_embed_image_impl`) are intentionally unchanged because their error semantics must not abort remaining annotations in a task. Tests updated in `test_batch_embedding.py` and `test_multimodal_embedder_unit.py`.
  - Moved `import requests` from inside `BaseEmbedder.embed_texts_batch()` to the top of `opencontractserver/pipeline/base/embedder.py`. Local imports are reserved for breaking circular-import cycles; `requests` is already a listed dependency so module-level import is cheaper on repeated calls.
  - Refactored `_create_embedding_for_annotation` in `opencontractserver/tasks/embeddings_task.py` to remove a misleadingly-indented `else` block. The text-only fallthrough now clearly separates the image-drop debug log from the `return _create_text_embedding(...)` statement.
  - Documented in the `calculate_embeddings_for_annotation_batch` docstring that the returned `result` dict counts reflect only the last attempt on Celery retry (not cumulative work across retries). `add_embedding()` is idempotent so there is no correctness risk, but any external monitoring consuming `result` should read Celery task state for cumulative counts.

### Added

- **Document lineage tracking** (IngestionSource + DocumentPath enrichment): New `IngestionSource` model (`opencontractserver/documents/models.py`) registers named integrations/crawlers/pipelines that produce documents. Three new fields on `DocumentPath` — `ingestion_source` (FK), `external_id` (indexed CharField), and `ingestion_metadata` (JSON) — record which source produced each version of a document and with what context. Lineage kwargs flow through `import_document()` (`opencontractserver/documents/versioning.py`), `Corpus.add_document()` (`opencontractserver/corpuses/models.py`), and the `UploadDocument` GraphQL mutation. New GraphQL types (`IngestionSourceType`, `IngestionSourceTypeEnum`), queries (`ingestionSources`, `ingestionSource`), and CRUD mutations (`createIngestionSource`, `updateIngestionSource`, `deleteIngestionSource`) in `config/graphql/`. Migration: `0036_add_ingestion_source_and_lineage_fields`.

- **Document path history tracking** (PR #1195): Document moves via `DocumentFolderService.move_document_to_folder()`, `move_documents_to_folder()`, and `delete_folder()` now create immutable `DocumentPath` history nodes instead of silent in-place updates. Each move produces a new `DocumentPath` record linked to the previous one via `parent`, enabling full audit trail traversal. Path conflicts are resolved by appending numeric suffixes (e.g. `report_1.pdf`). `versioning.py` `determine_action()` now detects folder-only moves (same path string, different `folder_id`). **Atomic guarantees**: Both `delete_folder()` and `move_documents_to_folder()` use `transaction.atomic()` so that if any document fails to relocate, ALL changes are rolled back (no partial-success state). Within-batch path conflict detection prevents two documents with the same filename from colliding during bulk moves. Failed operations are safe to retry. Comprehensive test coverage in `test_document_folder_service.py` for move tracking, path conflicts, bulk moves, delete-folder tracking, atomic rollback, retry-after-failure, and full lifecycle integration.
- **Move-document agent tool** (PR #1196): New `move_document` / `amove_document` tool that moves a document between folders within the same corpus. Delegates to `DocumentFolderService.move_document_to_folder` for permission checks and path updates. Categorized as `ToolCategory.CORPUS` (corpus-level agents only), requires write permission and approval. Pass `target_folder_id` for a specific folder or omit for corpus root. Uses `visible_to_user()` for document and corpus lookups to prevent IDOR enumeration (`opencontractserver/llms/tools/core_tools.py`, `opencontractserver/llms/tools/tool_registry.py`).

### Changed

- **`move_documents_to_folder()` return value semantics** (PR #1195): The returned `moved_count` now reflects only documents that were actually relocated; documents already in the target folder are skipped and not counted. Previously returned `len(document_ids)` unconditionally.
- **Web search agent tool** (PR #1174): New `aweb_search` tool for agents with pluggable provider backends (Brave Search and Tavily). Introduces `BaseTool` base class (`opencontractserver/llms/tools/base_tool.py`) for tools with database-backed settings and encrypted secrets via `PipelineSettings`. Tools declare a nested `Settings` dataclass with `PipelineSetting` metadata; `is_configured()` gates tool availability at resolution time. `ToolFunctionRegistry` supports `tool_class` entries for database-level enablement gating. New GraphQL mutations `UpdateToolSecretsMutation` and `DeleteToolSecretsMutation` (`config/graphql/pipeline_settings_mutations.py`) allow superusers to manage tool API keys. Includes per-process rate limiting, provider-specific response parsing, and comprehensive test coverage.

### Fixed

- **Quickstart blockers** (PR #1179): Fixed several issues preventing first-time setup from working:
  - Missing `EMBEDDINGS_MICROSERVICE_URL` and `DOCLING_PARSER_SERVICE_URL` env var defaults crash Django on startup (`config/settings/base.py`)
  - Celerybeat race condition on first boot — added Django healthcheck and `service_healthy` dependency for celery services (`local.yml`)
  - Password login broken — JWT middleware was gated behind `USE_AUTH0=True` so `tokenAuth` tokens were never validated (`config/settings/base.py`)
  - Admin user login 500 — `User.save()` re-validated slug on `last_login` updates, and `"admin"` is a reserved slug (`opencontractserver/users/models.py`)
  - Django admin panel 500 on static files — overrode `StaticFilesStorage` in local settings (`config/settings/local.py`)
  - Nginx duplicate `.js` MIME type warning (`frontend/conf/conf.d/default.conf`)
  - Updated README quickstart with missing steps and corrected `docker compose` v2 syntax

### Added

- **CAML Interactive Article System** (PR #1156): Frontend support for corpus articles using CAML (Corpus Article Markup Language). Includes a two-pass parser (tokenizer + block parsers), typed intermediate representation, and composable renderer supporting hero sections, cards, pills, tabs, timelines, CTAs, signup blocks, corpus stats, and pullquotes. Frontend adds `CamlArticleEditor` modal with live preview, `CorpusArticleView` for rendered article display, and `CorpusLandingView` integration for article discovery. Playwright component tests with `docScreenshot` captures for all block types.
- **Extract grid embed in CAML articles** (PR #1177): New `ExtractGridEmbed` component renders extract data tables inline in CAML articles via `[component:extract-grid extractId=...]` marker syntax. Includes `useCamlComponentRenderer` hook and `camlComponents` utility for a generic component marker system, `CAML_COMPONENTS` shared registry (`frontend/src/utils/camlComponentRegistry.ts`), and `ExtractPickerDropdown` toolbar in the editor for inserting extract grids. Query (`GET_EXTRACT_GRID_EMBED`) fetches extract fieldset columns, datacells, and source annotations with deep links to the document viewer.

### Fixed

- **CAML YAML parser nested key bug**: Fixed `parseYamlFrontmatter` in `frontend/src/caml/parser/tokenizer.ts` where `content` used `line.trimEnd()` instead of `line.trimEnd().trimStart()`, causing the key-value regex (`^[a-zA-Z_]...`) to fail for indented nested keys (e.g., `hero.kicker`). Nested frontmatter properties were silently dropped, producing empty objects.

### Changed

- **Replaced hardcoded hex colors with OS_LEGAL_COLORS tokens** in `CamlArticleEditor.tsx` and `CorpusArticleView.tsx`: All hardcoded hex values (`#e2e8f0`, `#fafbfc`, `#f8fafc`, `#64748b`, `#94a3b8`, `#ffffff`, `#fef3c7`, `#92400e`, `#475569`, `#f1f5f9`, `#cbd5e1`) replaced with semantic design tokens from `osLegalStyles.ts`.
- **Extracted `isExternalHref` helper** in `frontend/src/caml/renderer/safeHref.ts`: Deduplicated `href.startsWith("http")` checks across `CamlBlocks.tsx` and `CamlFooter.tsx` into a shared utility function.
- **Removed redundant `articleStats` useMemo** in `CorpusArticleView.tsx`: The memo mirrored its input without transformation; `stats` is now passed directly to `CamlArticle`.

### Fixed

- **Semantic UI removal cleanup** (Closes #1123): Fixed `folderIconAlpha` color mismatch in `ExtractCellFormatter.tsx` StatusDot box-shadows — replaced hardcoded `rgba(245, 158, 11, ...)` with `folderIconAlpha()` to match `getCellBackground()`. Changed Revoke button in `WorkerTokensSection.tsx` from `variant="secondary"` with inline danger color to `variant="danger"` for proper destructive-action styling. Removed conflicting `box-shadow` and `border-radius` from `StyledModalInner` in `EditMessageModal.tsx` to prevent doubled styling with `OsModal` wrapper. Added `role="menu"`, `aria-label`, `aria-haspopup`, keyboard support, and arrow key navigation (roving tabindex) to custom popup in `ExtractCellFormatter.tsx`. Removed redundant Escape key handler from `StatusDot` `onKeyDown` (already handled by global listener). Extracted inline `maxHeight: "70vh"` in `SelectDocumentsModal.tsx` to `MODAL_BODY_MAX_HEIGHT` constant in `constants.ts`.
- **Annotation rendering cleanup** (Closes #1144): Replaced hardcoded `"4px 4px 0 0"` border-radius with `ANNOTATION_BOUNDARY_RADIUS` constant in `SearchResult.tsx` and `ChatSourceResult.tsx`. Removed dead `border` prop from `SelectionInfo` interface and call sites. Used `APPROVED_RGB` constant for approved-state box-shadow in `SelectionBoundary.tsx` (matching `REJECTED_RGB` pattern). Added `TOKEN_EXPANSION_PX` constant and replaced magic `-1`/`+2` token expansion values across `SelectionTokenGroup.tsx`, `Tokens.tsx`, and `ChatSourceTokens.tsx`. Consolidated `[Previous Unreleased]` CHANGELOG section into single `[Unreleased]` block. Removed debug `console.log` statements from `DocumentKnowledgeBase.ct.tsx`, `SearchResult.tsx`, and `SelectionTokenGroup.tsx`. Moved annotation display reactive var initialization into `useEffect` in `DocumentKnowledgeBaseTestWrapper.tsx`.

### Added

- **Index tab in document sidebar**: Added a new "Index" tab (BookOpen icon) to the document viewer sidebar, positioned first before Chat, Feed, and Discussions. The Index tab reuses the `DocumentAnnotationIndex` component to display the document's OC_SECTION-based table of contents directly in the sidebar (`frontend/src/components/knowledge_base/document/document_kb/RightPanelContent.tsx`, `frontend/src/components/knowledge_base/document/DocumentKnowledgeBase.tsx`).
- **OpenAI Embedder pipeline component** (`opencontractserver/pipeline/embedders/openai_embedder.py`): New `OpenAIEmbedder` class supporting `text-embedding-3-small` (default), `text-embedding-3-large`, and `text-embedding-ada-002` models. Features configurable output dimensions, custom API base URL for Azure/proxy support, and graceful error handling. Dynamic `vector_size` property reflects runtime model/dimension configuration to prevent pgvector column size mismatches.
- **OpenAI embedding constants** (`opencontractserver/constants/embeddings.py`): Centralized model dimension mappings and defaults for the OpenAI embedder.

### Fixed

- **BulkImportModal AlertBox hardcoded colors** (Closes #1145): Replaced `OS_LEGAL_COLORS.warning*` / `OS_LEGAL_COLORS.info*` constants with CSS custom properties (`var(--oc-warning-*)`, `var(--oc-info-*)`). Added the six new tokens to `index.css :root` so they participate in theme switching (`frontend/src/index.css`, `frontend/src/components/widgets/modals/UploadModalStyles.ts`).
- **CloudUpload icon inline magic numbers** (Closes #1145): Replaced `style={{ width: 16, height: 16, marginRight: 8 }}` on the Start Import button icon with a new `ButtonIcon` styled component that uses `var(--oc-font-size-md)` and `var(--oc-spacing-xs)` tokens (`frontend/src/components/widgets/modals/BulkImportModal.tsx`, `UploadModalStyles.ts`).
- **Remaining pixel values in styled components** (Closes #1145): Replaced `HeaderIcon` 28px/15px with `calc()` expressions using spacing/font tokens, `DropZone` 200px/160px min-heights with `calc(var(--oc-spacing-xl) * N)`, and `AlertBox` SVG 2px margin-top with `calc(var(--oc-spacing-xs) / 4)` with a documenting comment (`UploadModalStyles.ts`).
- **Missing InMemoryCache in BulkImportTestWrapper** (Closes #1145): Added `InMemoryCache` with relevant type policies and a `mocks` prop to match the `DocumentKnowledgeBaseTestWrapper` pattern (`frontend/tests/wrappers/BulkImportTestWrapper.tsx`).

### Added

- **Sidecar JSON schema validation before annotation import** (Closes #1127): Added `_validate_sidecar_schema()` in `opencontractserver/tasks/import_tasks.py` that validates container types (`labelled_text`, `doc_labels`, `relationships` must be lists) and required keys per entry (`annotationLabel`/`rawText`/`annotation_json` for annotations; `relationshipLabel`/`source_annotation_ids`/`target_annotation_ids` for relationships) before any database work. Invalid sidecars now increment `annotation_sidecars_errored` and log clear error messages instead of crashing or silently skipping data. Comprehensive unit and integration tests added in `opencontractserver/tests/test_sidecar_import.py`.
- **Progress step test coverage for BulkImportModal** (Closes #1145): New test exercises the progress UI (spinner, progress bar, "Importing Documents..." heading) by mocking `IMPORT_ZIP_TO_CORPUS` with a long delay, verifying the loading state is rendered and footer buttons are hidden during import (`frontend/tests/bulk-import-modal.ct.tsx`).

### Added

- **Schema validation for `labels.json` before processing annotation labels** (Closes #1128): Added `validate_labels_data()` in `opencontractserver/utils/importing.py` that validates the structure of `labels.json` before `prepare_import_labels()` is called. Checks that `text_labels`/`doc_labels` are dicts (not lists), each label entry is a dict with a required non-empty `text` field, and optional fields (`label_type`, `color`, `icon`, `description`) have correct types. Validation errors are logged and appended to `results["errors"]` with `labels_loaded` set to `False`. Added unit tests (`TestValidateLabelsData`) and integration tests (`TestMalformedLabelsImport`) in `opencontractserver/tests/test_sidecar_import.py`.

### Changed

- **Batch embedding API calls for annotation embeddings** (PR #1148): Refactored `calculate_embeddings_for_annotation_batch` Celery task to use `embed_texts_batch()` for text-only annotations instead of embedding one-at-a-time, reducing HTTP requests from one-per-annotation to one-per-sub-batch. Key changes:
  - New `embed_texts_batch()` extension point on `BaseEmbedder` (`opencontractserver/pipeline/base/embedder.py`): default sequential fallback that re-raises transient HTTP exceptions; subclasses override for true batch API endpoints.
  - `MicroserviceEmbedder.embed_texts_batch()` calls `/embeddings/batch` with per-item NaN detection, vector-count mismatch validation, and 5xx → `EmbeddingServerError` for Celery retry (`opencontractserver/pipeline/embedders/sent_transformer_microservice.py`).
  - Moved `EmbeddingClientError` / `EmbeddingServerError` to `opencontractserver/pipeline/base/exceptions.py` (breaking import path change for external embedders that imported from `multimodal_microservice`).
  - Django system check `documents.E001` validates `EMBEDDING_API_BATCH_SIZE <= MICROSERVICE_EMBEDDER_MAX_BATCH_SIZE` at startup (`opencontractserver/documents/checks.py`).
  - New constants in `opencontractserver/constants/document_processing.py`: `EMBEDDING_API_BATCH_SIZE` (50), `MICROSERVICE_EMBEDDER_MAX_BATCH_SIZE` (100), `EMBEDDER_SINGLE_REQUEST_TIMEOUT_SECONDS` (30), `EMBEDDER_BATCH_REQUEST_TIMEOUT_SECONDS` (60).
  - Extracted `_get_service_config()` helper in `MicroserviceEmbedder` to eliminate config duplication.
  - Comprehensive test suite (938 lines) covering sequential fallback, partial failures, vector count mismatch, per-item NaN handling, sub-batching, transient error propagation through the full Celery task, and dual-strategy fallback (`opencontractserver/tests/test_batch_embedding.py`).
- **Deferred structural annotation loading for PDF performance**: Structural annotations (headers, sections, paragraphs — often 4,000-6,000 for large documents) are no longer fetched in the initial GraphQL queries. They are loaded lazily via `GET_DOCUMENT_STRUCTURAL_ANNOTATIONS` only when the user toggles structural visibility on. Also removed redundant structural annotation re-fetching during analysis/extract switching (structural annotations are analysis-independent). For a 200-page document, this eliminates ~5-20MB of JSON from the initial payload (`frontend/src/graphql/queries.ts`, `frontend/src/components/knowledge_base/document/DocumentKnowledgeBase.tsx`, `frontend/src/components/annotator/context/AnnotationAtoms.tsx`).
- **OC\_\* annotations filtered from Feed**: Platform-generated annotations with labels prefixed `OC_` (e.g., `OC_SECTION`) are now always hidden from the UnifiedContentFeed, preventing structural index entries from cluttering the annotation feed (`frontend/src/components/knowledge_base/document/unified_feed/UnifiedContentFeed.tsx`).
- **Improved sidebar tab spacing**: Reduced tab dimensions (76px/88px height, 36px/44px width), gap (4px), padding, icon size, and font size to prevent overlap when additional tabs are present (`frontend/src/components/knowledge_base/document/styled/SidebarTabs.tsx`).
- **Softened PDF annotation bounding boxes to diffuse highlighter-pen aesthetic**: Replaced hard-edged borders on annotation boundaries, tokens, and label pills with multi-layer box-shadow glows. Boundaries use a three-layer shadow (outer, mid, inset) that feathers into the page. Tokens use a single-layer soft blur. Approved/rejected states pulse with matching diffuse glows instead of solid borders. Extracted shared `computeAnnotationBoxShadow` utility (`frontend/src/utils/colorUtils.ts`) to eliminate duplicated shadow logic between `SelectionBoundary` and `ResultBoundary`. Added named constants for all shadow radii, opacity levels, border-radius tiers, and status colors (`frontend/src/assets/configurations/constants.ts`). Removed dead code: `getBorderWidthFromBounds`, unused `$border` prop, unused `$isSelected` prop. Fixed `pulseMaroon` animation color mismatch (was `rgba(180, 40, 40)`, now matches static rejected state).
- **Reorganized upload documentation into dedicated `docs/upload_methods/` section**: Consolidated scattered upload-related docs into 8 user-facing reference pages covering single upload, bulk ZIP import, corpus export/import, annotated document import, worker uploads, supported formats, and annotation side effects. Simplified `docs/walkthrough/step-1-add-documents.md` and `docs/walkthrough/advanced/export-import-corpuses.md` to reference the new guides. Trimmed `docs/architecture/bulk-import.md` to focus on internal implementation. Removed obsolete `docs/features/zip_import_with_folders_design.md` design doc (feature is fully implemented).

### Fixed

- **`skip_pipeline=True` silently ignored `meta.csv` metadata** (Closes #1131): Documents created via the `skip_pipeline` path in `import_zip_with_folder_structure` now have their title, description, custom_meta, and is_public fields applied from `meta.csv` and task-level parameters, matching the behavior of the normal pipeline path (`opencontractserver/tasks/import_tasks.py`).
- **Incomplete assertion in `test_malformed_labels_json_records_error`** (Closes #1131): The test now also asserts that `annotation_sidecars_errored` is incremented when labels.json is malformed and a sidecar contains annotations (`opencontractserver/tests/test_sidecar_import.py`).

### Added

- **Test for `skip_pipeline=True` with no `labels.json`** (Closes #1131): New test `test_skip_pipeline_without_labels_json` verifies that pipeline-skipped documents are created successfully even when no labels file is present, and that annotation import errors are recorded properly (`opencontractserver/tests/test_sidecar_import.py`).
- **Test for `skip_pipeline=True` with `meta.csv` metadata** (Closes #1131): New test `test_skip_pipeline_applies_metadata_from_csv` verifies that pipeline-skipped documents correctly receive title and description overrides from `meta.csv`.
- **Per-sidecar JSON size limit** (Closes #1131): Added `ZIP_MAX_SIDECAR_SIZE_BYTES` constant (10 MB default) in `opencontractserver/constants/zip_import.py`. Sidecars exceeding this limit are skipped with an error, preventing oversized JSON from consuming excessive memory during import (`opencontractserver/tasks/import_tasks.py`).

## [Unreleased] - 2026-03-18

### Added

- **`COMMUNITY_STATS_CACHE_TTL` env var for community stats caching** (`config/graphql/social_queries.py`, `opencontractserver/constants/community_stats.py`): The `communityStats` GraphQL resolver now caches results using Django's cache framework with a configurable TTL (default 1 hour). This avoids re-running 7+ COUNT queries on every landing page load. Cache is keyed by user type (anonymous vs authenticated) and optional corpus scope. Also includes an N+1 fix using `in_bulk()` for badge distribution and a subquery optimization in `resolve_corpus_categories`.

### Changed

- **Refactored FullScreenModal to use native `size="fullscreen"` variant** (Closes #1073): Replaced the `createGlobalStyle` workaround in `frontend/src/components/knowledge_base/document/LayoutComponents.tsx` with the native `size="fullscreen"` prop from `@os-legal/ui`, which handles viewport positioning, sizing, border-radius removal, and overlay padding natively. Only minimal body styling overrides (background color, padding, and overflow) remain.
- **Added `overlayClassName` to CorpusModal** (`frontend/src/components/corpuses/CorpusModal.tsx`): Uses the new `overlayClassName` prop to properly apply the `.corpus-modal-overlay` styles to the portal-rendered overlay element, fixing mobile overlay styles that were previously unreachable.
- **Complete removal of Semantic UI React dependency**: Migrated all 19 remaining files that imported from `semantic-ui-react` to use `@os-legal/ui` equivalents and native HTML elements. Component mappings: `Label` → `Chip`, `Modal/ModalHeader/ModalContent/ModalActions` → `Modal/ModalHeader/ModalBody/ModalFooter`, `Button` → `Button`, `Confirm` → custom confirm dialog via `Modal`, `Loader/Dimmer` → `Spinner`, `Dropdown` → `Dropdown`, `Icon` → lucide-react icons, `Segment/Card/Form` → styled HTML elements. Removed `semantic-ui-css` and `semantic-ui-react` from `package.json`, deleted the semantic-ui CSS import from `App.tsx` and `playwright/index.tsx`, and removed the orphaned `semantic.css` asset file. Also replaced the `SemanticICONS` type with `string` in `graphql/mutations.ts`.

### Fixed

- **Undefined ordering in async note retrieval** (Closes #1107): Added missing `.order_by("created")` to `aget_notes_for_document_corpus` in `opencontractserver/llms/tools/core_tools.py`, matching the sync counterpart `get_notes_for_document_corpus` which already had deterministic ordering.
- **Tool name breaking change from async migration** (Closes #1107): `create_document_tools()` in `opencontractserver/llms/tools/tool_factory.py` derived tool names from `func.__name__`, causing names to silently change from e.g. `load_document_md_summary` to `aload_document_md_summary` when the registry switched to async functions. Added explicit `name=` parameters to all `CoreTool.from_function()` calls to preserve the original tool names.
- **Test username typo**: Fixed "toklenuser" → "tokenuser" in `opencontractserver/tests/test_agent_search_tools.py`.
- **Misleading CleanViewContainer comment** (`frontend/src/views/Corpuses.tsx`): Corrected the styled-component comment that claimed height constraints were "intentionally removed" when `height: 100%`, `min-height: 0`, and `overflow: hidden` were still present. The comment now accurately documents the height model and explains that only `max-height: 100dvh` was removed. Closes #1044 (items 1 & 2).

### Added

- **Annotation sidecar import for bulk zip upload** (`opencontractserver/tasks/import_tasks.py`, `opencontractserver/utils/zip_security.py`): The `ImportZipToCorpus` mutation now supports importing pre-calculated annotations alongside source documents. When a document file (e.g. `contracts/master.pdf`) has a co-located `.json` sidecar (e.g. `contracts/master.json`) conforming to `OpenContractDocExport`, annotations and intra-document relationships are imported from the sidecar instead of running the parser pipeline. A root-level `labels.json` file provides label definitions. Documents without sidecars continue through the normal pipeline. This enables mixed-mode imports where some documents have pre-computed annotations and others are parsed fresh.
- **Test coverage for Focus/Power mode toggle** (`frontend/tests/CorpusHome.ct.tsx`, `frontend/tests/CorpusHomeTestWrapper.tsx`): Added four Playwright component tests exercising the `onModeToggle` and `isPowerUserMode` props: toggle hidden when callback absent, toggle visible in focus mode, toggle reflects power-user state, and click fires callback. Updated `CorpusHomeTestWrapper` to forward mode-toggle props. Closes #1044 (item 5).
- **Tighten MCP telemetry input validation and test coverage** (Closes #1106):
  - Bound User-Agent storage to 512 characters (`MAX_USER_AGENT_LENGTH`) in both `set_request_context()` and `get_user_agent_from_scope()` to prevent attacker-controlled multi-megabyte values from reaching telemetry backends (`opencontractserver/mcp/telemetry.py`)
  - Empty-string slugs no longer reach telemetry — changed `if slug is not None` to falsy checks (`if slug`) consistently across all sync and async recording functions
  - Documented the duplicate-header-dropping assumption when converting ASGI header lists to dicts in `get_user_agent_from_scope()` and `get_claimed_client_ip_from_scope()`
  - Added async telemetry tests for `arecord_mcp_tool_call`, `arecord_mcp_resource_read`, and `arecord_mcp_request` (`opencontractserver/tests/test_mcp_extended.py`)
  - Added tests for User-Agent truncation and empty-string slug filtering

## [Unreleased] - 2026-03-15

### Breaking Changes

- **Removed `tokensJsons` and `boundingBox` from GraphQL API**: The `tokensJsons` and `boundingBox` fields have been removed from GraphQL annotation queries and mutations (PR #1100). External API consumers and integrations must update to use the `json` field instead. The `json` field contains either v1 (legacy page-keyed format) or v2 (compact format) annotation data. Use `iter_page_annotations()` (Python) or `iterPageAnnotations()` (TypeScript) for format-agnostic access.

### Added

- **Dynamic MIME type support from pipeline components** (Closes #1059): Supported file types are now derived dynamically from registered pipeline components instead of being hardcoded. Changes include:
  - New `get_supported_mime_types()` and `get_allowed_mime_types()` functions in `opencontractserver/pipeline/registry.py` that compute supported file types by intersecting component coverage across parser, embedder, and thumbnailer stages
  - New `supportedMimeTypes` GraphQL query (`config/graphql/pipeline_queries.py`) returning per-file-type support level with stage coverage details
  - New `SupportedMimeTypeType` and `StageCoverageType` GraphQL types (`config/graphql/pipeline_types.py`)
  - Centralized MIME ↔ file type mappings (`MIME_TO_FILE_TYPE`, `FILE_TYPE_TO_MIME`, `FILE_TYPE_LABELS`, `LEGACY_MIME_ALIASES`) in `opencontractserver/pipeline/base/file_types.py`, replacing scattered inline dicts
  - Upload validation in `document_mutations.py`, `folder_service.py`, and `import_tasks.py` now uses the dynamic registry instead of `settings.ALLOWED_DOCUMENT_MIMETYPES`
  - Frontend `FiletypeDefaults` component fetches supported MIME types via GraphQL query instead of using hardcoded `SUPPORTED_MIME_TYPES` constant
  - Warning icon displayed for partially-supported file types (e.g., DOCX which lacks a thumbnailer)
  - `FileTypeEnum` gains `.mimetype` and `.label` properties and supports legacy MIME aliases in `from_mimetype()`
- **Compact PAWLs v2 format for ~67% storage reduction** (PR #1112): New v2 compact format for PAWLs files (per-page token bounding boxes) that reduces storage from ~500+ KB to ~180 KB for a typical 9-page PDF. Changes include:

  - Core encode/decode in `opencontractserver/utils/compact_pawls.py` (Python) and `frontend/src/utils/compactPawls.ts` (TypeScript)
  - Array-based tokens `[x, y, w, h, "text"]` instead of verbose dicts, shortened page keys, implicit page index, coordinate precision normalization
  - Write paths (parser, import, worker upload) auto-compact on save; read paths transparently expand v2 → v1
  - Safety limit: `COMPACT_PAWLS_MAX_TOKENS_PER_PAGE` (100,000) in `opencontractserver/constants/pawls.py` — graceful fallback to v1 when exceeded
  - Comprehensive Python (5 test classes) and TypeScript (8 tests) unit tests including roundtrip and image token coverage

- **Compact annotation JSON v2 format for ~75% storage reduction** (PR #1100): New compact v2 format for annotation JSON payloads that range-encodes consecutive token indices, compacts bounds to arrays, and drops redundant `pageIndex` and `rawText` from per-page data. Changes include:
  - Core encode/decode in `opencontractserver/annotations/compact_json.py` (Python) and `frontend/src/utils/compactAnnotationJson.ts` (TypeScript)
  - Safety limits: `COMPACT_JSON_MAX_RANGE_SPAN` (10,000) and `COMPACT_JSON_MAX_TOTAL_TOKENS` (50,000) in `opencontractserver/constants/annotations.py`
  - `Annotation.save()` auto-compacts v1 → v2 on write (lazy migration) with exception guard for malformed legacy data
  - Migration `0066` removes redundant `tokens_jsons` and `bounding_box` columns (includes `RunPython` backfill step to preserve any legacy data before column removal — **irreversible migration**)
  - 60+ Python and 48+ TypeScript unit tests for the codec
  - All GraphQL queries/mutations updated to use `json` field instead of `tokensJsons` and `boundingBox`
  - Format-agnostic accessor layer: `iter_page_annotations()`, `offset_annotation_json()`, `has_any_tokens()` (Python) and `iterPageAnnotations()`, `hasAnyTokens()` (TypeScript) — all production code migrated off `expand_annotation_json()`
  - `ServerTokenAnnotation` constructor now accepts both v1 and v2 formats, normalizing internally

### Added

- **Document index feature (within-document TOC)**: Hierarchical, navigable section index for long documents with markdown descriptions. Changes include:
  - New `long_description` (TextField, nullable) on the Annotation model for markdown section summaries (`opencontractserver/annotations/models.py`)
  - Database migration `0066_annotation_long_description`
  - `OC_SECTION` label constant (`opencontractserver/constants/annotations.py`) — `OC_` namespace for platform-generated labels
  - New `create_document_index` agent tool for building hierarchical indexes from exact string matches (`opencontractserver/llms/tools/core_tools.py`)
  - Tool registered in tool registry with approval gate (`opencontractserver/llms/tools/tool_registry.py`)
  - `long_description` exposed in GraphQL AnnotationType (auto-resolved), UpdateAnnotation mutation, and AnnotationSerializer
  - Export/import extended: `long_description` in `OpenContractsAnnotationPythonType`, import reads it, export includes it
  - New `DocumentAnnotationIndex` frontend component with tree rendering, expandable markdown descriptions, page badges, and filter support (`frontend/src/components/corpuses/DocumentAnnotationIndex.tsx`)
  - `DocumentTableOfContents` now renders annotation indexes under each document node; single-doc corpora skip the document header and show sections directly
  - New `GET_DOCUMENT_ANNOTATION_INDEX` GraphQL query and TypeScript types (`frontend/src/graphql/queries.ts`)
  - Frontend constants: `DOCUMENT_ANNOTATION_INDEX_LIMIT`, `OC_SECTION_LABEL` (`frontend/src/assets/configurations/constants.ts`)
- **First-class DOCX document support via Docxodus pipeline**: Added a complete parallel ingestion pipeline and rendering tree for DOCX files, bringing Word document support alongside existing PDF and TXT pipelines. Changes include:
  - **Backend: Docxodus microservice** (`docxodus-service/`): .NET 8 minimal API wrapping `OpenContractExporter.Export()` to produce OpenContractDocExport-compatible JSON with structural annotations and character offsets from DOCX files. Multi-stage Docker build exposed on port 8080.
  - **Backend: DocxodusServiceParser** (`opencontractserver/pipeline/parsers/docxodus_parser.py`): REST parser that sends base64-encoded DOCX to the microservice, normalizes camelCase→snake_case response fields, and handles transient/permanent error classification.
  - **Backend: DocxThumbnailGenerator** (`opencontractserver/pipeline/thumbnailers/docx_thumbnailer.py`): Two-tier thumbnail approach — extracts embedded thumbnails from DOCX ZIP archives (`docProps/thumbnail.jpeg`), falling back to text-based thumbnails via XML parsing of `word/document.xml`.
  - **Frontend: DocxAnnotator** (`frontend/src/components/annotator/renderers/docx/DocxAnnotator.tsx`): WASM-powered DOCX renderer using `docxodus` npm package's `convertDocxToHtmlWithExternalAnnotations()` for annotation projection onto native DOCX HTML output. Supports text selection for new annotation creation via `findTextOccurrences()`.
  - **Frontend: DocxAnnotatorWrapper** (`frontend/src/components/annotator/components/wrappers/DocxAnnotatorWrapper.tsx`): State management wrapper mirroring TxtAnnotatorWrapper pattern — manages annotation CRUD, chat sources, text search, and ref registration.
  - **Frontend: DocumentKnowledgeBase integration**: DOCX loading flow (fetches raw bytes + extracted text) and renderer dispatch added to both query handlers.
  - **Frontend utilities**: `isDocxFileType()` in `frontend/src/utils/files.ts`, `DOCX_MIME_TYPE` constant, `docxBytesAtom` / `useDocxBytes()` hook in DocumentAtom, `getDocxBytes()` in cachedRest.
  - **Docker Compose**: `docxodus-parser` service added to `local.yml`, `production.yml`, and `test.yml` with dependency wiring.
  - **Dependencies**: `docxodus@5.5.0` and `dompurify@3.3.3` added to frontend.
  - **Backend tests**: `test_doc_parser_docxodus.py` with parser unit tests (success, timeout, connection error, normalization) and thumbnailer tests (text preview, embedded thumbnail, invalid DOCX handling).
  - **Frontend tests**: `DocxAnnotator.ct.tsx` component test with `docScreenshot` captures.
- **Richer social media link previews for corpus and document links**: Improved the Cloudflare OG worker to generate better social tags. Changes include:
  - Corpus descriptions are now included in OG/Twitter description tags, combined with document count (e.g. "Corpus description — 15 documents")
  - Document-in-corpus links now surface the parent corpus description when the document lacks its own description
  - Added Twitter structured data tags (`twitter:label1`/`twitter:data1`, `twitter:label2`/`twitter:data2`) showing author, document count, corpus name, etc.
  - New `corpus_description` field on `OGDocumentMetadataType` GraphQL type (`config/graphql/og_metadata_types.py`)
  - Backend resolver returns corpus description for document-in-corpus queries (`config/graphql/og_metadata_queries.py`)
  - Fixed pre-existing TypeScript type errors in worker test suite and metadata extraction
- **Creative Commons license support for corpuses**: Corpuses can now have a license applied, choosing from standard Creative Commons licenses (CC BY, CC BY-SA, CC BY-NC, CC BY-NC-SA, CC BY-ND, CC BY-NC-ND, CC0) or a custom license with a URL. Changes include:
  - New `license` (CharField with SPDX identifiers) and `license_link` (URLField) fields on the Corpus model (`opencontractserver/corpuses/models.py`)
  - License constants in `opencontractserver/constants/licenses.py` and `frontend/src/assets/configurations/constants.ts`
  - Database migration `0047_corpus_license_fields` (single migration with URL validation)
  - GraphQL create/update mutations accept `license` and `licenseLink` arguments
  - New `LicenseSelector` frontend component (`frontend/src/components/widgets/CRUD/LicenseSelector.tsx`)
  - CorpusModal updated with License section for create/edit flows
  - CorpusInfoSection displays the selected license in corpus settings

### Fixed

- **Token bounding box drift in PDF viewer**: Fixed progressive misalignment between token bounding boxes (blue outlines) and rendered PDF text. PAWLs token coordinates are extracted by the parser in its own coordinate system, which can differ from PDF.js's viewport coordinate system. The frontend now normalizes token coordinates by comparing PAWLs page dimensions (`page.width`/`page.height`) against the PDF.js viewport dimensions at scale 1, rescaling tokens when they differ. Added `normalizeTokensToPdfViewport()` utility in `frontend/src/utils/transform.tsx` and applied it in both PDF loading paths in `frontend/src/components/knowledge_base/document/DocumentKnowledgeBase.tsx`.
- **Auth0 SDK blocked by CSP on admin login page** (Closes #1077): The admin login template loads the Auth0 SPA SDK from `cdn.jsdelivr.net`, but the Content-Security-Policy `script-src` directive did not include this origin, causing browsers to block the script. Added `https://cdn.jsdelivr.net` to `script-src` when Auth0 is enabled. (`config/settings/base.py`)
- **PostgreSQL HNSW config warning on startup** (Closes #1074): Fixed `invalid configuration parameter name 'hnsw.iterative_scan'` warning caused by database-level `ALTER DATABASE SET` GUC defaults persisting in `pg_db_role_setting`. The docker-entrypoint-initdb.d phase runs in a temporary postgres without the user's command-line args, so `hnsw.*` GUCs aren't registered when the database-level defaults are later applied on connections. Consolidated shared postgres settings (`shared_preload_libraries`, HNSW, I/O tuning) into `compose/postgres/shared.conf` with a custom entrypoint wrapper that injects them as `-c` flags. Per-environment memory settings remain in compose files. Removed `ALTER DATABASE SET` from `init.sql`. Added migration 0066 to RESET stale database-level defaults. **Upgrade note:** Existing users must rebuild the postgres image (`docker compose build postgres`) to pick up the new entrypoint wrapper and shared config file. (`compose/postgres/shared.conf`, `compose/postgres/docker-entrypoint-wrapper.sh`, `compose/production/postgres/Dockerfile`, `compose/production/postgres/init.sql`, `opencontractserver/annotations/migrations/0066_reset_database_level_hnsw_settings.py`)
- **Single-select Dropdown missing onBlur in MetadataCellEditor** (PR #1076): Two single-select `Dropdown` instances (STRING with choices, CHOICE type) silently dropped the `onBlur` callback during the SUI→@os-legal/ui migration, causing metadata grid cells to never exit edit mode after selection. Initially worked around with `onClose={onBlur}` in v0.1.13; upgraded to proper `onBlur={onBlur}` prop in v0.1.14. (`frontend/src/components/metadata/editors/MetadataCellEditor.tsx`)
- **Dropdown test selectors outdated after @os-legal/ui upgrade**: Updated CSS selectors in `DocumentMetadataGrid.ct.tsx` from `.oc-select-trigger`/`.oc-select-search` to `.oc-dropdown__trigger`/`.oc-dropdown__search-input` to match the v0.1.13 Dropdown component's class names. Changed Enter-key selection to direct `.oc-dropdown__option` click. (`frontend/tests/DocumentMetadataGrid.ct.tsx`)

### Changed

- **Enforced async-only tool registry for LLM agent tools**: Audited and converted the entire tool registry to reject sync functions. Previously `ToolRegistryEntry` carried both `sync_func` and `async_func` fields, `FUNCTION_MAP` registered both versions, and `PydanticAIToolWrapper` had a sync wrapper path that called sync functions without a thread pool (risking `SynchronousOnlyOperation`). Changes include:

  - Removed `sync_func` field from `ToolRegistryEntry` — only `async_func` remains (`opencontractserver/llms/tools/tool_registry.py`)
  - Simplified `FUNCTION_MAP` from 3-tuples `(sync, async, aliases)` to 2-tuples `(async, aliases)`, removing all sync imports from `_populate()`
  - Replaced sync wrapper path in `PydanticAIToolWrapper` with a `TypeError` guard that rejects sync functions at construction time (`opencontractserver/llms/tools/pydantic_ai_tools.py`)
  - Created async versions of `get_note_content_token_length` and `get_partial_note_content` (`aget_note_content_token_length`, `aget_partial_note_content`) which previously only had sync versions that used `Note.objects.get()` (`opencontractserver/llms/tools/core_tools.py`)
  - Updated `create_document_tools()` to use only async functions (`opencontractserver/llms/tools/tool_factory.py`)
  - Updated `__init__.py` exports to async-only (`opencontractserver/llms/tools/__init__.py`)
  - Updated affected tests to use async functions and added new tests for sync rejection

- **SUI Tab migration to @os-legal/ui FilterTabs** (Closes #1022 — tab portion): Removed unused `Tab` and `Menu` imports from `semantic-ui-react` in `frontend/src/views/Corpuses.tsx`. Replaced custom styled `Tab`/`TabContainer` components and custom `SearchInput` in `frontend/src/views/GlobalDiscussions.tsx` with `FilterTabs` and `SearchBox` from `@os-legal/ui`, consistent with the rest of the codebase.

- **Migrated frontend from Semantic UI React to @os-legal/ui design system**: Replaced Modal, Button, Input, and other UI components across the entire frontend with `@os-legal/ui` equivalents. Updated icon system from Semantic UI icons to lucide-react. Introduced `OS_LEGAL_COLORS`, `OS_LEGAL_TYPOGRAPHY`, and `OS_LEGAL_SPACING` design tokens for consistent styling. Upgraded `@os-legal/ui` from 0.1.8 to 0.1.14.
- **Replaced hardcoded hex colors with design system tokens**: Consolidated ~50 color tokens and replaced hardcoded hex color literals across 150+ files with `OS_LEGAL_COLORS` constants from `osLegalStyles.ts`.
- **Extracted shared corpus object collection logic** into `opencontractserver/utils/corpus_collector.py`: New `collect_corpus_objects()` utility and `CorpusObjectCollection` dataclass consolidate duplicated corpus forking/export collection logic (Issue #816)

### Added

- **Dynamic discovery endpoints for crawlers and AI agents**: Replaced static `robots.txt`, `llms.txt`, and `llms-full.txt` files with Django views that dynamically generate content with live data from the database. New endpoints:
  - `robots.txt`: Includes explicit `Allow` directives for AI crawler user-agents (GPTBot, ClaudeBot, anthropic-ai, Google-Extended, PerplexityBot, Bytespider, cohere-ai) and a proper `Sitemap:` reference to `sitemap.xml`
  - `llms.txt` / `llms-full.txt`: Now auto-populate an "Available Collections" section listing all public corpuses with titles, slugs, document counts, and descriptions. Hostnames are resolved from the request instead of using placeholder text. Links use proper inline Markdown format per the llmstxt.org spec
  - `sitemap.xml`: New XML sitemap listing homepage, public corpuses, their documents (via DocumentPath), and discovery endpoints
  - `.well-known/mcp.json`: New MCP server discovery endpoint listing the global MCP server and per-corpus scoped servers
    (`opencontractserver/discovery/views.py`, `opencontractserver/discovery/urls.py`, `config/urls.py`)
- **Traefik routing for discovery endpoints**: Updated production and CI Traefik configs to route `/robots.txt`, `/llms.txt`, `/llms-full.txt`, `/sitemap.xml`, and `/.well-known/*` to Django instead of the frontend nginx container (`compose/production/traefik/traefik.yml`, `compose/production/traefik/traefik-ci.yml`)
- **MCP discovery link in HTML head**: Added `<link rel="alternate" type="application/json" href="/.well-known/mcp.json">` to `frontend/index.html` for agent discovery
- **Comprehensive test suite** for all five discovery endpoints covering content types, spec conformance, public/private corpus filtering, hostname resolution, and edge cases (`opencontractserver/discovery/tests/test_discovery_views.py`)

### Removed

- **Unused `django-crispy-forms` and `crispy-bootstrap5` dependencies**: These were cookiecutter-django boilerplate never used by the project (React frontend uses its own form components). Removed packages from `requirements/base.txt`, `INSTALLED_APPS`, and `CRISPY_*` settings from `config/settings/base.py`.

### Fixed

- **Duplicate enabled-field logic**: `ComponentLibrary` now reads the backend-computed `component.enabled` field instead of recalculating enablement from the `enabledComponents` list, eliminating divergent sources of truth (issue #1036 item 1). (`frontend/src/components/admin/system_settings/ComponentLibrary.tsx`)
- **Silent toggle failure**: Toggle clicks while components are loading now show a toast warning instead of silently failing (issue #1036 item 2). (`frontend/src/components/admin/SystemSettings.tsx`)
- **MIME-type fallback flaw**: `FiletypeDefaults` now falls back to the full MIME string when the short-label lookup misses, preventing all availability checks from failing for unmapped types (issue #1036 item 3). (`frontend/src/components/admin/system_settings/FiletypeDefaults.tsx`)
- **Unused postProcessors fetch**: Removed `postProcessors` from the `GET_PIPELINE_COMPONENTS` GraphQL query since no frontend component consumes it (issue #1036 item 4). (`frontend/src/components/admin/system_settings/graphql.ts`)
- **Potential duplicate class names**: Toggle handler now deduplicates component paths when transitioning from all-enabled to an explicit list (issue #1036 item 5). (`frontend/src/components/admin/SystemSettings.tsx`)
- **Replicated empty-list-as-all-enabled semantics**: Extracted `isComponentEnabled` and `isComponentAvailable` into a shared `utils.ts` module, consolidating the repeated logic into one place (issue #1036 item 6). (`frontend/src/components/admin/system_settings/utils.ts`)
- **Implicit test dependency**: Added explicit class-level constants for test component paths and expanded `test_pipeline_components_query_non_superuser_filters_configured` to verify all component stages, not just parsers (issue #1036 item 8). (`opencontractserver/tests/test_pipeline_component_queries.py`)
- **Corpus preferred_embedder not set when default is empty** (pre-existing): `Corpus.save()` used `if not self.preferred_embedder and default_embedder` which skipped assignment when `get_default_embedder_path()` returned `""`. Changed to `if self.preferred_embedder is None` so the field is always populated consistently. (`opencontractserver/corpuses/models.py:426`)
- **Stale postProcessors in PipelineComponentsType**: Removed unused `postProcessors` field from the `PipelineComponentsType` TypeScript type to match the system settings GQL query. (`frontend/src/types/graphql-api.ts`)

### Added

#### Unified Rate Limiting for WebSocket and MCP (Closes #730, #745)

- Replaced `django-ratelimit` with a custom protocol-agnostic rate limiting engine (`config/ratelimit/`) supporting GraphQL, WebSocket, MCP, and Django views through a single shared infrastructure
- **Engine** (`config/ratelimit/engine.py`): Fixed-window counter algorithm using Django cache (Redis in production), with sync `is_rate_limited()` and async `ais_rate_limited()` APIs
- **Identity resolution** (`config/ratelimit/keys.py`): Unified IP extraction from both `HttpRequest` (GraphQL/views) and ASGI scope (WebSocket/MCP), plus key building with `user_or_ip`, `ip`, and `user` strategies
- **Rate categories** (`config/ratelimit/rates.py`): `RateLimits` singleton with 17 categories including new `WS_CONNECT` (10/m), `WS_HEARTBEAT` (120/m), and `MCP_GLOBAL` (100/m)
- **WebSocket rate limiting**: All 3 consumers (`UnifiedAgentConversationConsumer`, `NotificationConsumer`, `ThreadUpdatesConsumer`) now enforce connection-rate and per-message limits. Rate-limited messages receive a JSON `RATE_LIMITED` error; the connection stays open
- **MCP rate limiting**: Two-layer check — global cap (`MCP_GLOBAL`) plus per-tool limits mapped to existing categories (e.g., `search_corpus` → `READ_HEAVY`). ASGI scope threaded into tool handlers via `ContextVar`
- **Django view adapter** (`view_ratelimit`): Drop-in replacement for `django_ratelimit.decorators.ratelimit`, used in admin login views
- **Backward-compatible re-exports**: `config/graphql/ratelimits.py` remains a valid import path for all existing GraphQL files
- Comprehensive test suite (`opencontractserver/tests/test_unified_rate_limiting.py`) covering engine, keys, rates, and all protocol adapters

- Backend mutation test for the all-enabled-to-explicit toggle transition, verifying the mutation succeeds and the query reflects the change (issue #1036 item 7). (`opencontractserver/tests/test_pipeline_settings.py`)

### Changed

- **`add_annotations_from_exact_strings` API simplified to single-document** (breaking): Replaced per-item `(label, text, doc_id, corpus_id)` tuples with a flat `document_id`/`corpus_id` pair plus `AnnotationItem` TypedDict items containing only `label_text` and `exact_string`. Callers that previously annotated multiple documents in one call must now make separate calls per document. (`opencontractserver/llms/tools/core_tools.py`)

### Added

#### Action Library (Corpus Action Templates)

- **CorpusActionTemplate model** for reusable, agent-based action definitions that users can browse and add to individual corpuses (`opencontractserver/corpuses/models.py`)
- **5 default action templates** seeded via data migration: Document Description Updater, Corpus Description Updater, Document Summary Generator, Key Terms Annotator, Document Notes Generator — each with a dedicated `AgentConfiguration` and curated tool set (`opencontractserver/agents/migrations/0010_create_default_action_templates.py`)
- **Action Library UI**: "Add from Library" picker in Corpus Settings lets users browse available templates and add them to a corpus on demand. New corpora start empty — no auto-cloning (`frontend/src/components/corpuses/settings/CorpusActionsSection.tsx`)
- **`addTemplateToCorpus` mutation**: Clones a template into a `CorpusAction` for a given corpus with duplicate prevention (`config/graphql/corpus_mutations.py`)
- **`source_template` FK on CorpusAction**: Links cloned actions back to their source template for provenance tracking (`opencontractserver/corpuses/models.py`)
- **GraphQL query `corpusActionTemplates`**: Read-only query exposing available templates with optional `isActive` filter (`config/graphql/action_queries.py`)
- **`sourceTemplate` field on CorpusActionType**: Exposes template provenance in the existing corpus actions GraphQL type (`config/graphql/agent_types.py`)
- **`seed_action_templates` management command**: Idempotent command for seeding default templates on fresh databases (`opencontractserver/corpuses/management/commands/seed_action_templates.py`)

#### Optimize Vector Search and Index Scalability for Million-Scale Corpora

- **HNSW indexes on all Embedding vector columns** (384–4096 dimensions): Approximate nearest neighbor search reduces vector queries from O(n) sequential scan to O(log n). Created via `AddIndexConcurrently` to avoid table locks during index creation (`opencontractserver/annotations/models.py`, `opencontractserver/annotations/migrations/0063_add_hnsw_indexes_and_search_vector.py`)
- **Eliminated Python-side materialization** in `VectorSearchViaEmbeddingMixin.search_by_embedding()`: Previously materialized ALL matching rows into Python, sorted, and sliced. Now uses PostgreSQL `ORDER BY + LIMIT` so only top-k rows cross the wire. The unique constraint from migration 0059 guarantees no JOIN duplicates, removing the need for `DISTINCT ON` (`opencontractserver/shared/mixins.py`)
- **PostgreSQL full-text search** on Annotation: Added `search_vector` (`SearchVectorField`) with GIN index and a database trigger that auto-populates tsvector from `raw_text` on INSERT/UPDATE. Replaces `LIKE '%term%'` (`icontains`) with indexed tsvector matching for 100x+ faster text search at scale (`opencontractserver/annotations/models.py`)
- **Hybrid search with Reciprocal Rank Fusion (RRF)**: New `CoreAnnotationVectorStore.hybrid_search()` method runs vector similarity and full-text search in parallel, then fuses results using RRF (k=60). Semantic search GraphQL resolver now uses hybrid search for improved result quality (`opencontractserver/llms/vector_stores/core_vector_stores.py`, `config/graphql/search_queries.py`)
- **RRF utility function** (`opencontractserver/utils/search.py`): Generic `reciprocal_rank_fusion()` that merges any number of ranked lists
- **Search constants** (`opencontractserver/constants/search.py`): `HNSW_M`, `HNSW_EF_CONSTRUCTION`, `RRF_K`, `HYBRID_SEARCH_OVERSAMPLE_FACTOR`, `FTS_CONFIG`
- **pgvector PostgreSQL extension upgraded to 0.8.0** (installed via Docker, not the Python package): Enables iterative index scans for filtered ANN queries (prevents result loss when combining vector search with `WHERE` clauses like `embedder_path` filtering). Database default set via `ALTER DATABASE ... SET hnsw.iterative_scan = 'relaxed_order'` in `init.sql` (`compose/production/postgres/Dockerfile`, `compose/production/postgres/init.sql`)
- **PostgreSQL tuning for vector workloads**: Added `shared_buffers`, `maintenance_work_mem`, `effective_cache_size`, `work_mem`, `max_parallel_maintenance_workers`, `random_page_cost`, and `effective_io_concurrency` to all Docker Compose files. Added `shm_size` for parallel HNSW index builds (`local.yml`, `production.yml`, `test.yml`)
- **Pinned pgvector Python package** to `>=0.4.0` for `HnswIndex` support (`requirements/base.txt`)
- **Expanded valid embedding dimensions** in `CoreAnnotationVectorStore`: Search methods now accept 1024 and 2048 dimensions in addition to existing 384, 768, 1536, 3072, 4096
- **Added 2048-dimension support** to conversation and message vector search methods (`opencontractserver/conversations/models.py`)
- **GraphQL annotation mention search** now uses full-text search via `SearchQuery` on GIN-indexed `search_vector` instead of `raw_text__icontains` (`config/graphql/search_queries.py`)

- **IconPicker Lucide Rebuild (SUI Migration Step 3)**: Replaced the Semantic UI icon catalog with a curated Lucide icon catalog (~440 icons across 16 categories). New `IconPickerModal` component displays Lucide icons in a searchable, category-filtered grid with live preview. New `IconDropdown` component provides a compact trigger that opens the modal. Updated `iconCompat.ts` with dynamic fallback resolution so `resolveIcon()` supports all Lucide icons via `kebabToPascal` conversion against the full `lucide-react` export. Legacy SUI icon names remain backward-compatible through the existing `SEMANTIC_TO_LUCIDE` mapping. Includes 16 Playwright component tests and automated documentation screenshots. (`frontend/src/components/widgets/icon-picker/icons.ts`, `frontend/src/components/widgets/icon-picker/IconPickerModal.tsx`, `frontend/src/components/widgets/icon-picker/IconDropdown.tsx`, `frontend/src/utils/iconCompat.ts`)

### Changed

- Removed `django-ratelimit` dependency from `requirements/base.txt` — all rate limiting now handled by `config.ratelimit`
- `config/admin_auth/views.py`: Replaced `django_ratelimit.decorators.ratelimit` with `config.ratelimit.decorators.view_ratelimit`
- `opencontractserver/mcp/server.py`: Replaced custom `RateLimiter` class with unified `check_mcp_rate_limit`; added `ContextVar` for ASGI scope propagation
- `opencontractserver/mcp/permissions.py`: Removed `RateLimiter` class (moved to shared package)
- `opencontractserver/mcp/telemetry.py`: `get_client_ip_from_scope` now delegates to shared `config.ratelimit.keys`
- Removed `X-RateLimit-*` response headers that `django-ratelimit` previously set on GraphQL responses — the new engine communicates rate limit state through `RateLimitExceeded` errors (GraphQL), `RATE_LIMITED` WebSocket frames, or HTTP 429 status (views)
- Removed `RATELIMIT_ENABLE` dead-code setting from `config/settings/ratelimit.py` — the engine only checks `RATELIMIT_DISABLE`

- **Pipeline Component Management Redesign**: Separated component management from filetype default assignment in the Pipeline Configuration UI. New `enabled_components` JSON field on `PipelineSettings` tracks which components are available. Frontend splits into two sections: ComponentLibrary (flat filterable list with enable/disable toggles, search, stage filter chips) and FiletypeDefaults (MIME type rows with parser/embedder/thumbnailer dropdowns). Removed old stage-centric layout with MIME type filter buttons. (`opencontractserver/documents/models.py`, `config/graphql/pipeline_types.py`, `config/graphql/pipeline_queries.py`, `config/graphql/pipeline_settings_mutations.py`, `frontend/src/components/admin/SystemSettings.tsx`, `frontend/src/components/admin/system_settings/ComponentLibrary.tsx`, `frontend/src/components/admin/system_settings/FiletypeDefaults.tsx`)
- **Clean corpus landing page with Power User mode toggle**: Default corpus view is now a full-page landing without sidebar navigation, providing a cleaner experience for anonymous browsers and casual users. Users with edit permissions see a "Power User" toggle (`?mode=power` URL param) to access the full sidebar+tabs layout (`frontend/src/views/Corpuses.tsx`)
- **Recent discussions feed on corpus landing page**: New `RecentDiscussions` component shows 2-3 latest discussion threads below the "View Details" button, making community activity visible even to anonymous users (`frontend/src/components/corpuses/CorpusHome/RecentDiscussions.tsx`)
- **Inline discussions view**: New `?view=discussions` URL state enables viewing the full discussion thread list and thread detail directly from the corpus home, without switching to the Discussions tab (`frontend/src/components/corpuses/CorpusHome/CorpusDiscussionsInlineView.tsx`)
- `updateModeParam` navigation utility for managing the `?mode=` URL parameter (`frontend/src/utils/navigationUtils.ts`)
- Extended `CorpusDetailViewType` to include `"discussions"` alongside `"landing"` and `"details"` (`frontend/src/graphql/cache.ts`)
- Screenshot tests for new landing view states: clean view, discussion feed, empty discussions, and power user mode (`frontend/tests/CorpusHome.ct.tsx`, `frontend/tests/CorpusTabs.ct.tsx`)

#### Security Headers Middleware (CSP, Referrer-Policy, Permissions-Policy)

- Added `SecurityHeadersMiddleware` in `config/middleware.py` that attaches `Content-Security-Policy` and `Permissions-Policy` headers to every HTTP response; Referrer-Policy is handled by Django's built-in `SecurityMiddleware` via `SECURE_REFERRER_POLICY`
- Middleware positioned after `SecurityMiddleware` so it is the final authority on security headers in the response phase
- CSP directives configured via `SECURE_CSP_DIRECTIVES` dict in `config/settings/base.py` — covers `default-src`, `script-src` (with `blob:` for PDF.js worker fallback), `style-src`, `img-src`, `font-src`, `connect-src`, `worker-src`, `object-src`, `frame-ancestors`, `base-uri`, and `form-action`
- When `USE_AUTH0=True`, the Auth0 tenant domain is automatically added to `connect-src` and `script-src` with input sanitization via `validate_csp_domain()` (rejects domains containing spaces or semicolons to prevent CSP injection)
- Auth0 domain validation extracted into `config.middleware.validate_csp_domain()` for independent testability
- Permissions-Policy disables `camera`, `microphone`, `geolocation`, `payment`, `usb`, `magnetometer`, `gyroscope`, and `accelerometer` via `SECURE_PERMISSIONS_POLICY`
- Header values are pre-built once at middleware init for zero per-request overhead
- Local dev CSP (`config/settings/local.py`) scopes WebSocket `connect-src` to `localhost` instead of bare `wss:`/`ws:` scheme sources, with documented Auth0 domain interaction
- Tests in `opencontractserver/tests/test_security_headers_middleware.py` — unit tests, integration test against `/api/health/`, and `validate_csp_domain()` tests

#### Expand Corpus Import Test Coverage (Closes #999)

- Rewrote `test_corpus_import.py` with proper `TransactionTestCase` base class (previously `ImportCorpusTestCase` with no parent, never discovered by test runners)
- Fixed `FieldFile.save()` call signature in test setup helper (was passing `ContentFile` as `name` instead of `(name, content)`)
- Grouped read-only assertions into 2 test methods using `subTest` to reduce import pipeline executions from 15 to 5
- **Label integrity**: Validates all 107 labels (79 text + 28 doc) with correct color, icon, description, type, and labelset membership
- **Annotation validation**: Verifies raw text, page numbers, bounding box coordinates, token references, and label associations for all 6 annotations
- **Relationship verification**: Tests `import_relationships()` with single and multiple source/target annotations, plus structural flag preservation

- Expanded corpus forking test suite with field-level data integrity checks (Closes #998)

### Fixed

#### Missing embedding dimensions in VALID_EMBEDDING_DIMS

- `VALID_EMBEDDING_DIMS` was missing dimensions 1024, 2048, and 4096, causing validation failures for embedders that produce these common dimensions (e.g., some OpenAI and Cohere models). Added missing entries to `VALID_EMBEDDING_DIMS` and `DIM_TO_FIELD_MAP` in `opencontractserver/constants/search.py`.

#### Fix My Documents Corpus Not Navigable Due to Missing Slugs

- **Root cause**: Migration 0038 created personal corpuses using historical models which bypass `Corpus.save()` slug auto-generation, leaving `slug=NULL`. The frontend requires both `corpus.slug` and `creator.slug` to build navigation URLs (`/c/<user>/<corpus>`), so clicking "My Documents" logged "Cannot navigate to corpus without slugs" and did nothing.
- **Fix (model)**: `Corpus.get_or_create_personal_corpus()` now detects when a returned corpus lacks a slug and triggers `save()` to backfill it on access (`opencontractserver/corpuses/models.py:518-521`).
- **Fix (migration)**: Added data migration `0043_backfill_corpus_slugs` that backfills slugs for all existing corpuses and users missing them (`opencontractserver/corpuses/migrations/0043_backfill_corpus_slugs.py`).

#### Skip redundant document re-parsing during corpus import

- Set `processing_started` on standalone documents created via `create_document_from_export_data()` to prevent the post_save signal from triggering `ingest_doc` (`opencontractserver/utils/importing.py:323`)
- Imported documents already have PAWLS data from the export; re-parsing wasted resources and failed in environments without a parser service

#### Tighten JSON Field Validation for Malformed Input (Closes #1001)

- **Root cause**: `CustomJSONFieldFormTests.TestForm` used `NullableJSONField()` (a model field) instead of `UTF8JSONFormField` (a form field). Django's `Form` metaclass silently ignores model fields, so the form had zero fields and `is_valid()` always returned `True` — masking the fact that malformed JSON was never validated.
- **Fix**: Changed `TestForm.json_field` to `UTF8JSONFormField(required=False)` so form validation actually runs through Django's `forms.JSONField.to_python()`, which raises `ValidationError` on `json.JSONDecodeError` (`opencontractserver/tests/test_custom_fields.py:76`).
- **Re-enabled**: `test_form_with_invalid_json` now asserts that `'not json'` is correctly rejected (`opencontractserver/tests/test_custom_fields.py:90-92`).
- **Added**: `test_formfield_rejects_invalid_json` and `test_formfield_accepts_valid_json` integration tests on `NullableJSONFieldTests` to verify the model field's `formfield()` method produces a form field that properly validates JSON (`opencontractserver/tests/test_custom_fields.py:63-71`).

### Changed

#### Deployment: migration 0063 backfill may need a maintenance window for large deployments

- Operators with >1M annotations should consider running the search_vector backfill (Phase 4 of migration 0063) during a maintenance window. The migration now emits `RAISE NOTICE` progress messages so operators can monitor backfill progress in PostgreSQL logs.
- Production `maintenance_work_mem` reduced from `2GB` to `512MB`. The higher value is only needed during the initial HNSW index build; operators should temporarily increase it for that migration, then revert (`production.yml`).
- `hnsw.ef_search` increased from `40` to `64` to match `ef_construction`, improving recall for legal document search (`compose/production/postgres/init.sql`, migration 0063).

#### Annotation text search now uses PostgreSQL full-text search with English stemming

- The annotation mention search (`resolve_search_annotations_for_mention`) and the semantic search resolver now use `SearchQuery` on a GIN-indexed `search_vector` column instead of `raw_text__icontains`. This means text queries now match English-stemmed forms (e.g., searching "contract" also matches "contracting" and "contracted") rather than requiring exact substring matches. This is a semantic behavior change for users accustomed to exact substring matching on `raw_text`. See `config/graphql/search_queries.py`.

#### Triage and Clean Up TODO/FIXME Comments (Closes #971)

- Removed 62 TODO/FIXME/HACK annotations across 43 backend and frontend files
- Replaced vague TODOs with `NOTE(deferred):` comments explaining deferral reasoning
- Deleted stale comments referencing non-existent files, already-implemented features, and empty test stubs
- Fixed typo: "whould" → "should" in `test_permissioning.py`
- Removed `console.log` debug statement in `ModernDocumentItem.tsx`
- Deleted empty test stub file `test_doc_analysis_tasks.py`
- Consolidated redundant `logger.debug()` calls in `utils/files.py`

#### Extract Magic Numbers to Constants Files (Closes #970)

- Replaced hardcoded upload limit, truncation lengths, DPI, and title limits with named constants in `constants/document_processing.py` and `constants/llm_tools.py`
- Reused existing `MAX_PROCESSING_ERROR_LENGTH`/`MAX_PROCESSING_TRACEBACK_LENGTH` in `corpuses/models.py`

#### GraphQL Module Modularization (Closes #972)

- Split `graphene_types.py` (3,717→107 lines), `mutations.py` (6,229→405 lines), `queries.py` (4,408→54 lines) into domain-specific files
- Full backward compatibility via re-exports; no logic changes

#### Consolidate Duplicate String Truncation Utilities (Closes #976)

- Added `truncate()` helper in `opencontractserver/utils/text.py` and named constants in `constants/truncation.py`
- Replaced inline truncation across `core_tools.py`, `doc_tasks.py`, and `corpuses/models.py`

#### Break Up Large Frontend Components (Closes #977)

- Split 5 large components: StyledContainers (2,115→12), SystemSettings (2,616→1,108), CorpusChat (2,347→1,346), DocumentKnowledgeBase (3,363→2,322), ChatTray (2,215→1,772)
- Extracted shared chat WebSocket types into canonical `chat/types.ts`; renamed duplicate ConversationListView components; replaced `any` types with explicit typed properties

### Removed

#### Deprecated Semantic UI React Components and Icon Picker (PR #1009)

- Deleted 103 files (~20,900 lines) of deprecated frontend components, hooks, and tests that had been fully replaced by OS Legal styled equivalents
- Removed icon picker widget (`IconSelector.tsx`, `IconDropdown.tsx`, `IconPickerModal.tsx`, `icons.ts`, `styles.module.css`)
- Removed unused Semantic UI wrapper components: `DocTypeLabelDisplay`, `DocTypeLabels`, `LabelSelector`, `SemanticSidebar`, and related CSS
- Removed deprecated layout components: `AnnotatorSidebar.tsx`, `DropdownActionButton.tsx`, `CorpusCards.tsx`, `TreeItemDisplay.tsx`
- Removed dead modals: `SelectCorpusAnalyzerModal.tsx`, `SelectExtractFieldsModal.tsx`, `EditExtractModal.tsx`, `NewEditAnalysisModal.tsx`, `SelectDocumentFieldsetModal.tsx`
- Removed deprecated annotator hooks and display components: `useVisibleRelationships`, `useAnnotationDisplay`, `useAnnotationSelection`, `usePageAnnotations`, `RelationshipList`, `ActionBar`, `AnnotationSummary`, and others
- Removed legacy notification components: `NotificationBell`, `NotificationCenter`, `NotificationDropdown`, `NotificationItem`
- Removed deprecated thread components: `MentionPicker`, `ResourceMentionPicker`, `ModerationControls`, `ModeratorBadge`, `ReputationDisplay`, and associated hooks
- Removed orphaned CSS files: `DocTypeLabelDisplayStyles.css`, `DocTypeLabels.css`, `LabelSelector.css`
- Removed associated test files for deleted components

### Added

#### Replace Mock Data with Real User Query in @mention Dropdown (Closes #1002)

- **useMentionUsers hook** (`frontend/src/components/threads/hooks/useMentionUsers.ts`): Replaced hardcoded mock users with real `SEARCH_USERS_FOR_MENTION` GraphQL query. Added 300ms debounced input to reduce excessive API calls and minimum character threshold (2 chars). Hook now returns `{ users, loading, error }` instead of just `MentionUser[]`.
- **MentionPicker component** (`frontend/src/components/threads/MentionPicker.tsx`): Added loading and error state rendering. Shows "Searching users..." during query execution and "Failed to load users" on errors. Added `loading` and `error` optional props to `MentionPickerProps`.

#### Deep Linking and Context Menu for Text/PDF Annotators (Closes #958)

- Copy Link actions in PDF (`SelectionLayer.tsx`) and TXT (`TxtAnnotator.tsx`) context menus encode selections as `?tb=` deep link URLs
- URL-driven annotation selection from chat sources (`ChatTray.tsx`); delete button for processing documents (`ModernDocumentItem.tsx`)

#### Corpus Export Test Coverage (Closes #997)

- Added `test_exported_document_structure` to validate exported document data structure: top-level keys, PAWLS page schema, annotation structure with bounding boxes and token references, and PDF burn-in validity (`opencontractserver/tests/test_corpus_export.py`)
- Added `test_round_trip_consistency` to compare exported data against original import fixture: document title, content, PAWLS page dimensions and token counts, annotation count, raw text, label names (mapped through label lookups), and bounding box coordinates (`opencontractserver/tests/test_corpus_export.py`)
- Added `test_exported_label_names_match_fixture` to verify exported label name sets match the labels actually used in the import fixture (`opencontractserver/tests/test_corpus_export.py`)
- Loaded import fixture data in setUp for round-trip comparison, replacing the previous TODO placeholder (`opencontractserver/tests/test_corpus_export.py:62`)
- Cleaned up existing tests by removing verbose print statements and TODO comments

#### Expand burn_doc_annotations Test (Closes #1000)

- Added `test_burn_doc_annotations_with_text_labels` to exercise the text-label PDF burning code path with TOKEN_LABEL fixtures and bounding-box annotation data (`opencontractserver/tests/test_doc_tasks.py`)
- Validates output PDF contains highlight annotations with correct subtype, label text, and non-empty base64-encoded content
- Validates `doc_export` JSON contains expected `doc_labels` and `labelled_text` entries
- Renamed existing test to `test_burn_doc_annotations_doc_labels_only` for clarity

#### Test Coverage for Untested Backend Modules (Closes #975)

- Unit tests for feedback, shared utils, constants, types, and MCP extended modules (`opencontractserver/tests/`)

### Fixed

#### Code Review Fixes for Text Block Deep Linking (#958)

- Document resolution via corpus membership (`DocumentPath`) instead of `creator=owner`; simplified default path to return already-resolved doc
- Cross-document source click flash fix; `useClearTextBlockOnInteraction` hook consolidation; clipboard `.catch()` for non-HTTPS; dead code removal

#### Document Version Selector UI Cleanup (Closes #964)

- Removed unused query fields (`versionCount`, `hasVersionHistory`, etc.); added WAI-ARIA keyboard navigation; safe `v?` fallback during load
- Backend validation for invalid version numbers (≤ 0); isCurrent JSDoc; updated test mocks and new keyboard nav tests

#### Rollup Vulnerability (Closes #973)

- Pinned `rollup: "^4.59.0"` via yarn resolutions to fix 3 high-severity path traversal advisories
- **Result**: rollup updated from 4.53.1 to 4.59.0, eliminating all 3 rollup-related audit advisories

### Added

#### Worker Upload Management UI and Documentation (#955)

- **GraphQL queries**: `workerAccounts`, `corpusAccessTokens`, `workerDocumentUploads` resolvers with proper permission checks (superuser-only for accounts, superuser/corpus-creator for tokens and uploads) (`config/graphql/queries.py`)
- **ReactivateWorkerAccount mutation**: Allows superusers to re-enable previously deactivated worker accounts (`config/graphql/worker_mutations.py`)
- **Worker Account management page**: New admin page at `/admin/worker-accounts` for creating, listing, and activating/deactivating worker service accounts (`frontend/src/components/admin/WorkerAccountManagement.tsx`)
- **Worker Access Tokens section in Corpus Settings**: Corpus creators and superusers can view, create, and revoke access tokens scoped to their corpus. Includes one-time key display with copy-to-clipboard (`frontend/src/components/corpuses/settings/WorkerTokensSection.tsx`)
- **Documentation walkthrough**: End-to-end guide covering account creation, token management, document upload (with curl/Python examples), metadata format reference, rate limiting, error handling, and security model (`docs/worker_uploads/walkthrough.md`)
- **Component tests**: Playwright component tests for WorkerAccountManagement with automated documentation screenshots

#### Document Version Selector End-to-End Documentation (Closes #954)

- **User-facing guide**: `docs/features/document_versioning.md` — covers version creation workflow, visual status indicators (gray/blue/orange badges), Version History Panel usage, and Trash folder recovery
- **Documentation screenshots**: Added `docScreenshot` calls to capture five key UI states:
  - `versioning--badge--single-version` — gray badge for documents without history (`frontend/tests/VersionBadge.ct.tsx`)
  - `versioning--badge--latest-version` — blue badge showing version count (`frontend/tests/VersionBadge.ct.tsx`)
  - `versioning--badge--older-version` — orange badge for outdated versions (`frontend/tests/VersionBadge.ct.tsx`)
  - `versioning--history-panel--with-versions` — already captured in `frontend/tests/VersionHistoryPanel.ct.tsx`
  - `versioning--trash-folder--restore-ui` — deleted document recovery interface (`frontend/tests/TrashFolderView.ct.tsx`)

### Changed

#### Worker Upload Permission Expansion (#955)

- `CreateCorpusAccessTokenMutation` and `RevokeCorpusAccessTokenMutation` now allow corpus creators (not just superusers) to manage tokens scoped to their own corpora
- GlobalSettingsPanel refreshed with OS Legal design tokens and lucide-react icons, replacing Semantic UI dependencies

#### Auth0 Refresh Token Migration (#955)

- **DEPLOYMENT NOTE**: `useRefreshTokens: true` is now enabled in the Auth0 SDK configuration (`frontend/src/index.tsx`). Deployments using Auth0 **must** enable "Refresh Token Rotation" in the Auth0 dashboard before deploying this change, or silent authentication will fail for all users.

### Fixed

#### Document Version Structural Annotation Set Inheritance

- **Bug**: When a document was updated with new content (different hash), `import_document()` unconditionally inherited the old version's `structural_annotation_set`. This caused the parser's `_create_structural_annotation_set()` to short-circuit (early return at `pipeline/base/parser.py:299`), leaving freshly-parsed structural annotations orphaned — never migrated into a set.
- **Fix**: `opencontractserver/documents/versioning.py:224-231` — `structural_annotation_set` is now only inherited when the content hash is unchanged. When content changes, the field is set to `None` so the parser creates a fresh `StructuralAnnotationSet` during ingestion.
- **Tests**: `opencontractserver/tests/test_structural_annotation_portability.py` — replaced single test with two: one verifying `None` on changed content, one verifying inheritance on identical content.

### Added

#### Annotation Versioning and Document Version-Aware Deep Linking

- **Version-aware document resolution**: `documentInCorpusBySlugs` GraphQL query now accepts optional `versionNumber` parameter to resolve a specific historical version of a document (`config/graphql/queries.py`)
- **Corpus versions field**: New `corpusVersions(corpusId)` field on `DocumentType` returns all versions of a document in a corpus with version number, document ID, slug, creation date, and current status (`config/graphql/graphene_types.py`)
- **`CorpusVersionInfoType` GraphQL type**: New type for version selector data returned by `corpusVersions` field
- **`?v=N` URL parameter**: Deep links to documents now support a `?v=N` query parameter to view a specific version (e.g., `/d/user/corpus/doc?v=1&ann=123`)
- **`selectedDocVersion` reactive var**: New URL-driven state variable in `frontend/src/graphql/cache.ts` synced bidirectionally via CentralRouteManager Phases 2 and 4
- **CentralRouteManager version support**: Phase 1 passes version to GraphQL resolution, Phase 2 parses `?v=` into reactive var, Phase 4 syncs back to URL (`frontend/src/routing/CentralRouteManager.tsx`)
- **`DocumentVersionSelector` component**: Inline version badge and dropdown in document header that shows available versions and allows switching between them (`frontend/src/components/documents/DocumentVersionSelector.tsx`)
- **Navigation utilities**: `QueryParams` interface and `buildQueryParams` now support `version` field for URL construction (`frontend/src/utils/navigationUtils.ts`)
- **Routing documentation**: Updated `docs/frontend/routing_system.md` with version parameter documentation, examples, and reactive var listing

#### Worker Document Upload System

- **New Django app** `opencontractserver.worker_uploads` — enables external document-processing workers to upload fully ingested, annotated, and embedded documents to a target corpus via REST API
- **Service account model** (`WorkerAccount`): dedicated machine identity with auto-created Django User for permission compatibility. Created via `createWorkerAccount` GraphQL mutation (superuser only)
- **Corpus-scoped access tokens** (`CorpusAccessToken`): cryptographically random 256-bit tokens scoped to a single corpus, with configurable expiry and per-token rate limiting. Created via `createCorpusAccessToken` GraphQL mutation
- **Hashed token storage**: tokens are stored as SHA-256 hashes — plaintext shown only once at creation via `create_token()`. Auth backend hashes incoming keys before DB lookup (`opencontractserver/worker_uploads/models.py`, `auth.py`)
- **DRF authentication backend** (`WorkerTokenAuthentication`): validates `Authorization: WorkerKey <token>` headers, hashes token and checks validity, expiry, and account status (`opencontractserver/worker_uploads/auth.py`)
- **REST upload endpoint** (`POST /api/worker-uploads/documents/`): accepts multipart form data (file + JSON metadata), stages uploads in database, returns 202 Accepted immediately. Status polling via `GET /api/worker-uploads/documents/<id>/` and listing via `GET /api/worker-uploads/documents/list/`
- **Upload format** (`WorkerDocumentUploadMetadataType`): extends V2 export format with pre-computed embeddings (`embedder_path` + document/annotation vectors), target path/folder placement, and inline label definitions for auto-creation (`opencontractserver/types/dicts.py`)
- **Database-backed queue** (`WorkerDocumentUpload`): staging table with PENDING/PROCESSING/COMPLETED/FAILED status tracking, avoids Redis saturation for high-volume uploads (millions of documents)
- **Batch processor task** (`process_pending_uploads`): Celery task on dedicated `worker_uploads` queue using `SELECT ... FOR UPDATE SKIP LOCKED` for concurrent processing without conflicts. Configurable batch size via `WORKER_UPLOAD_BATCH_SIZE` setting. Self-reschedules when more work exists
- **Multi-queue architecture**: worker upload processing runs on dedicated `worker_uploads` Celery queue, preserving capacity on the default queue for regular user operations
- **Pre-computed embedding storage**: workers can include embeddings in upload metadata; stored directly via bulk_create without re-running embedder models. Supports all vector dimensions (384–4096)
- **Corpus creator ownership**: all documents, annotations, and labels created via worker uploads are owned by the corpus creator (not the service account), ensuring correct permission inheritance
- **GraphQL management mutations**: `createWorkerAccount`, `deactivateWorkerAccount`, `createCorpusAccessToken`, `revokeCorpusAccessToken` (all superuser-only) in `config/graphql/worker_mutations.py`
- **Celery task routing**: `CELERY_TASK_ROUTES` canonicalized in one place with guard comment (`config/settings/base.py`)
- **Settings-based Beat schedule**: `CELERY_BEAT_SCHEDULE` for worker upload drain (60s interval), replacing fragile data migration approach
- **File size limit**: `MAX_WORKER_UPLOAD_SIZE_BYTES` setting (default 256 MB) enforced at upload endpoint
- **Filename sanitization**: worker-supplied document titles are sanitized before use as filenames, stripping path traversal characters and null bytes

### Technical Details

- New files: `opencontractserver/worker_uploads/{models,views,auth,serializers,tasks,urls,apps}.py`, `config/graphql/worker_mutations.py`
- Migrations: `0001_initial.py` (models), `0002_setup_beat_schedule.py` (cleanup old DB schedule), `0003_hash_token_keys.py` (SHA-256 token hashing)
- Settings: `WORKER_UPLOAD_BATCH_SIZE` (default 50), `MAX_WORKER_UPLOAD_SIZE_BYTES` (default 256 MB), `CELERY_TASK_ROUTES` for queue isolation, `CELERY_BEAT_SCHEDULE` for periodic drain
- Tests: `opencontractserver/tests/test_worker_uploads.py` covering models, hashed token auth, REST endpoints, file size limits, batch processor, filename sanitization, null corpus creator guard, and GraphQL mutations

#### Corpus Export Format Specification and Validation Utility

- **Format specification**: `docs/architecture/corpus-export-format-spec.md` — complete reference for V1 and V2 corpus export ZIP format covering all data.json fields, PAWLs structure, referential integrity rules, security limits, and import behavior
- **Standalone validator**: `opencontractserver/utils/validate_export.py` — checks structural and referential integrity of export ZIPs without requiring Django or a database. Usable as CLI (`python -m opencontractserver.utils.validate_export corpus.zip`) or library (`validate_export()` / `validate_data_json()`)
- **Validation checks**: ZIP↔data.json file consistency, label definitions and type constraints, annotation token/page index bounds, annotation bounds non-negativity, structural set hash consistency, folder hierarchy (circular reference detection, path consistency), document path references, relationship label type enforcement (including structural relationships), V2 required top-level fields, conversation/message/vote cross-references, unknown version warnings
- **Test suite**: `opencontractserver/tests/test_validate_export.py` — 48 pure-Python tests covering all validation paths including CLI entry point

### Changed

#### Migrate from deprecated PyPDF2 to pypdf (Closes #938)

- Replaced `PyPDF2==3.0.1` with `pypdf` in `requirements/base.txt`
- Removed redundant `pypdf` entry from `requirements/local.txt` (now provided by base)
- Updated imports in `opencontractserver/utils/files.py`, `opencontractserver/utils/etl.py`, and `opencontractserver/tests/test_pdf_redaction.py`
- Removed unused `add_highlight_to_page` function from `opencontractserver/utils/files.py` (used deprecated `_addObject` API, never called)

#### Django 4.2 → 5.2 LTS Upgrade

- **Django version**: Upgraded from Django 4.2.24 to 5.2.11 (LTS)
  - `requirements/base.txt`, `requirements/local.txt`, `requirements/production.txt`
- **STORAGES migration**: Replaced deprecated `STATICFILES_STORAGE` and `DEFAULT_FILE_STORAGE` settings with the unified `STORAGES` dict (required since Django 5.1)
  - `config/settings/base.py` — LOCAL, AWS, and GCP storage backends all migrated
  - `config/settings/test.py` — test storage configuration migrated
  - `opencontractserver/tests/base.py` — test `@override_settings` migrated
  - `opencontractserver/tests/test_agent_search_tools.py` — all `@override_settings` decorators migrated
  - `opencontractserver/tests/test_storage_backends.py` — assertions updated to check `STORAGES` dict
- **Removed `USE_L10N` setting**: This setting was removed in Django 5.0 (localization is always enabled)
  - `config/settings/base.py:78`
- **Removed `SECURE_BROWSER_XSS_FILTER` setting**: This setting was removed in Django 5.0 (modern browsers handle XSS filtering natively)
  - `config/settings/base.py:503-504`
- **Replaced `pytz` with `datetime.timezone`**: Django 5.0+ uses `zoneinfo` instead of `pytz`
  - `opencontractserver/users/tasks.py` — replaced `pytz.utc.localize()` with `datetime.datetime.now(datetime.timezone.utc)`
  - Removed `pytz` from direct requirements in `requirements/base.txt`
- **Updated third-party packages for Django 5.2 compatibility**:
  - `graphene-django`: 3.2.2 → 3.2.3 (adds Django 5.1+ support; Django 5.2 not officially supported — tracked via TODO)
  - `django-stubs`: 4.2.7 → 5.2.0
  - `djangorestframework-stubs`: 1.8.0 → 3.15.4
  - `django-celery-beat`: 2.6.0 → 2.8.1 (adds Django 5.2 support)
  - `django-filter`: 24.3 → 25.1 (adds Django 5.2 support)
  - `django-model-utils`: 4.3.1 → 5.0.0 (adds Django 5.x support; no direct imports in codebase — transitive dependency)
  - `django-crispy-forms`: 2.4 → 2.5 (adds Django 5.2 support)
  - `django-cte`: 2.0.0 → 3.0.0 (adds Django 5.2 support, fixes ambiguous column names; LOUTER breaking change does not affect this project — no `_join_type` usage found)
  - `django-environ`: 0.12.0 → 0.13.0 (adds Django 5.2 support)
- **Removed `django-debug-toolbar`**: Was never wired into INSTALLED_APPS or MIDDLEWARE; removed unused dependency and associated INTERNAL_IPS config from `config/settings/local.py`
- **Replaced Collectfast with Collectfasta** (production static file collection):
  - `Collectfast==2.2.0` was archived/unmaintained (last release 2020), incompatible with Django 5.x `STORAGES` setting
  - Switched to `collectfasta>=3.2.0`, an actively maintained fork tested with Django 5.2.3
  - `requirements/production.txt` — package swap
  - `config/settings/production.py` — updated INSTALLED_APPS reference
  - `config/settings/base.py` — renamed `COLLECTFAST_STRATEGY` to `COLLECTFASTA_STRATEGY` and updated paths from `collectfast.strategies.*` to `collectfasta.strategies.*`

### Security

#### Dependency Security Updates

- **Django 4.2.24 → 4.2.28 (now 5.2.11)**: CVEs fixed by the 5.2.11 LTS release include multiple SQL injection vectors (CVE-2025-59681, CVE-2025-64459, CVE-2025-13372, CVE-2026-1312, CVE-2026-1287, CVE-2026-1207), directory traversal (CVE-2025-59682), DoS attacks (CVE-2025-64458, CVE-2025-64460), and user enumeration timing attack (CVE-2025-13473)
  - Updated in `requirements/base.txt`, `requirements/local.txt`, `requirements/production.txt`
- **cryptography 46.0.3 → 46.0.5**: Fixes CVE-2026-26007 — missing subgroup validation in ECDSA/ECDH public key loading for SECT curves, enabling signature forgery and private key leakage
  - Updated in `requirements/base.txt`
- **axios ^1.12.0 → ^1.13.5**: Fixes DoS vulnerability via `__proto__` key in `mergeConfig`
  - Updated in `frontend/package.json`
- **Removed unused `worker-loader`**: Webpack-specific package unused in Vite project; removal eliminates transitive `ajv@6.12.6` ReDoS vulnerability (via `worker-loader > schema-utils > ajv`)
  - Removed from `frontend/package.json`

### Fixed

#### TxtAnnotator Infinite Re-render Loop (Closes #933)

- **Unstable default parameter**: `chatSources = []` in `TxtAnnotator` component props created a new array reference on every render, triggering infinite re-renders via `useEffect` dependency arrays when the prop was not explicitly passed (`frontend/src/components/annotator/renderers/txt/TxtAnnotator.tsx:335`)
- Extracted `ChatSourceHighlight` interface and defined module-level `EMPTY_CHAT_SOURCES` constant as the default value, ensuring referential stability across renders

#### Follow-up Text Annotation Fixes (Closes #911)

- **Double-scroll bug**: `toggleSelectedAnnotation` in `AnnotatorSidebar.tsx:758` and `RelationshipList.tsx:106` called `scrollIntoView` for all annotation types, including text span annotations which already scroll via `TxtAnnotator`'s own `selectedAnnotations` useEffect. This caused two competing scroll animations. Fixed by guarding with `instanceof ServerTokenAnnotation` check.
- **Phantom ID tracking**: `TxtAnnotator.tsx:366` built `currentIds` from all visible annotations before verifying DOM elements existed. Annotations without rendered spans became "ghost" IDs tracked in `registeredAnnotationIdsRef` but never actually registered. Fixed by only adding IDs to the tracking set after confirming a DOM element was found and registered.
- **Page number display regression**: `HighlightItem.tsx` and `RelationHighlightItem.tsx` now use `(annotation instanceof ServerTokenAnnotation || annotation.page > 0)` to show page labels. PDF token annotations always display page labels (page is always meaningful), while span annotations only display them when `page > 0` (since `page=0` is a sentinel for "no page concept applies").
- **TypeScript type narrowing**: `HighlightItem.tsx:176` stored `instanceof` check in an intermediate boolean variable, preventing TypeScript's control-flow narrowing. Inlined the `instanceof` check directly in the conditional.

#### BaseChunkedParser Robustness and Consistency (Closes #926)

- **Config ValueError not wrapped**: `calculate_page_chunks` raises `ValueError` for invalid `max_pages_per_chunk`/`min_pages_for_chunking`, but the call in `_parse_document_impl` was unwrapped. Now caught and re-raised as `DocumentParsingError(is_transient=False)` (`opencontractserver/pipeline/base/chunked_parser.py`)
- **Small-document annotations unprefixed**: Single-chunk documents returned directly from `_parse_chunk_with_retry` without passing through `_reassemble_chunk_results`, resulting in unprefixed annotation/relationship IDs. Now all results consistently receive `c0_` prefixed IDs (`opencontractserver/pipeline/base/chunked_parser.py`)
- **Uncovered backoff cap branch**: `MAX_CHUNK_RETRY_BACKOFF_SECONDS` cap was never exercised by tests. Added test with `chunk_retry_limit=4` that verifies backoff values `[5, 10, 20, 30]` where the 4th retry hits the 30s cap (`opencontractserver/tests/test_chunked_parser.py`)
- **Theoretical race in concurrent test**: `slow_chunks_started.is_set()` assertion could fail on heavily loaded CI. Added `slow_chunks_started.wait(timeout=2)` before the assertion (`opencontractserver/tests/test_chunked_parser.py`)

#### Context Guardrails for LLM Conversation Management (Closes #907)

- **`truncate_tool_output` negative slice defense** (`opencontractserver/llms/context_guardrails.py`): Replaced fragile guard clause with explicit `char_budget = max(0, max_chars - len(notice))` to prevent negative slice indices when `max_chars` is smaller than the truncation notice length
- **Token double-counting across compaction cycles** (`opencontractserver/llms/context_guardrails.py`, `opencontractserver/llms/agents/pydantic_ai_agents.py`): Added `stored_summary_tokens` parameter to `compact_message_history()` so the stored summary is counted in `total_before` (threshold check) but not double-counted in `total_after` (the new summary replaces the old one)
- **Repeating prefix in compaction summaries** (`opencontractserver/llms/agents/pydantic_ai_agents.py`): Successive compaction cycles no longer accumulate duplicate `COMPACTION_SUMMARY_PREFIX` headers — the merge logic now strips the prefix from both old and new summaries before re-adding it once
- **Fragile sentence extraction in deterministic summary** (`opencontractserver/llms/context_guardrails.py`): Extended the first-sentence regex to split on double-newlines (paragraph boundaries) and newlines before markdown list markers (`-`, `*`, `•`, numbered lists), preventing entire bullet-list responses from being treated as a single sentence
- **Deprecated asyncio pattern in tests** (`opencontractserver/tests/test_context_guardrails.py`): Converted `TestPersistCompactionOptimisticLock` from `asyncio.run()` wrapper calls to native `async def` test methods, removing the unused `asyncio` import
- **Weak truncation test assertions** (`opencontractserver/tests/test_context_guardrails.py`): Strengthened `test_custom_max_chars` and `test_truncation_notice_contains_limit` to assert exact upper-bound length (`<= max_chars`) and verify content starts from the beginning of the input string; added `test_truncated_content_from_beginning_not_end` test
- **CHARS_PER_TOKEN_ESTIMATE docstring inconsistency** (`opencontractserver/constants/context_guardrails.py`): Clarified that the constant is intentionally 3.5 (not 4) to over-count tokens slightly for conservative compaction triggering
- **Missing integrity constraint documentation** (`opencontractserver/conversations/models.py`): Added comment and expanded `help_text` on `compacted_before_message_id` explaining why `BigIntegerField` (not `ForeignKey`) is safe — the `id__gt` filter remains correct even if the cutoff message is deleted
- **Unreachable defensive code** (`opencontractserver/llms/context_guardrails.py`): Added clarifying comment on the `recent_count < 1` guard explaining it is unreachable with default `MIN_RECENT_MESSAGES` but protects against callers passing `min_recent=0`
- **Missing compaction bookmark filter tests** (`opencontractserver/tests/test_context_guardrails.py`): Added `TestCompactionBookmarkDatabaseFilter` with two async tests verifying `get_conversation_messages()` applies `id__gt` filtering when a bookmark is set and skips it when `None`
- **New sentence extraction tests** (`opencontractserver/tests/test_context_guardrails.py`): Added `test_markdown_bullet_list_split` and `test_double_newline_paragraph_split` covering the improved regex

#### MCP Documentation Accuracy (Closes #924)

- **Missing `created` field in tool return docs**: `list_public_corpuses`, `list_documents`, and `list_annotations` all return a `created` ISO 8601 timestamp, but `llms-full.txt` omitted it from the documented return shapes
- **Incorrect annotation label shape**: `list_annotations` return docs showed `label` (string) but the actual response uses `annotation_label: { text, color, label_type }` (object) — updated to match `format_annotation()` in `opencontractserver/mcp/formatters.py`
- **Underdocumented `document://` resource**: The resource description only said "Document metadata and full extracted text" without listing the actual fields. Added field inventory including `text_preview` (first 500 chars), `full_text`, `corpus`, and `created` — critical for agents choosing between preview and full text under context window constraints
- **File**: `frontend/public/llms-full.txt`

#### BaseChunkedParser Cleanup (Closes #914)

- **Duplicate test line**: Removed redundant `PdfReader` assignment in `test_pdf_splitting.py:95`
- **Infinite loop guard**: Added input validation for `max_pages_per_chunk` and `min_pages_for_chunking` in `calculate_page_chunks()` (`opencontractserver/utils/pdf_splitting.py`); added `max_concurrent_chunks` validation in `_parse_document_impl` (`opencontractserver/pipeline/base/chunked_parser.py`)
- **Dead code / ID inconsistency**: Removed single-chunk fast-path short-circuit in `_reassemble_chunk_results()` that returned unprefixed IDs, creating inconsistency with multi-chunk results
- **Flaky test**: Replaced wall-clock timing assertion in `test_concurrent_failure_cancels_remaining` with a shorter sleep to reduce CI flakiness
- **Type safety**: Replaced `type: ignore[return-value]` in `_dispatch_concurrent` with explicit `cast()` call
- **Noisy logging**: Downgraded orphaned parent-child reference log from `warning` to `debug` level — these are expected on virtually every large hierarchical document
- **Backoff cap**: Added `MAX_CHUNK_RETRY_BACKOFF_SECONDS` constant (30s) to cap exponential backoff in per-chunk retries (`opencontractserver/constants/document_processing.py`)
- **Missing boundary test**: Added test for exact `min_pages_for_chunking` threshold (75 pages) and clarified docstring semantics
- **Memory trade-off documented**: Added comment explaining concurrent dispatch memory implications
- **Cross-chunk limitation documented**: Enhanced class docstring with follow-up improvement suggestion for section-aware chunk boundaries

### Added

- Unit tests for `HighlightItem` scroll behavior and page label display (`frontend/src/components/annotator/sidebar/__tests__/HighlightItem.scroll.test.tsx`)

### Security

#### Resolve Dependabot Security Advisories (pydantic-ai + ajv)

- **pydantic-ai 0.2.x → 1.x migration**: Upgraded from pydantic-ai 0.2.20 to >=1.56.0,<2 to resolve CVE in older version. Migration includes:
  - `End` import moved from `pydantic_ai.agent` to `pydantic_graph` (`opencontractserver/llms/agents/pydantic_ai_agents.py`)
  - All 3 `PydanticAIAgent` creation sites migrated from `system_prompt=` to `instructions=` to use the 1.x-recommended parameter that is always included in model requests regardless of message history
  - `griffe>=1.3.2,<2` pin removed (was a transitive workaround only needed for pydantic-ai 0.2.x)
  - Test file updated: `result.data` → `result.output`, `result_type` → `output_type`, `system_prompt` → `instructions` (`opencontractserver/tests/test_pydantic_ai_agents.py`)
  - `openai` bumped from ==1.102.0 to >=2.11.0,<3 (pydantic-ai 1.x requires openai >=2.11.0)
  - `pdf2image` pinned to >=1.16.0 (ancient 0.1.x versions have broken setup.py, caused cascading build failures in CI)
- **ajv ReDoS fix (CVE in ajv <8.17.1)**: Added scoped Yarn resolutions for `@rjsf/validator-ajv8/ajv` and `ajv-formats/ajv` to pin ajv 8.18.0, avoiding conflict with schema-utils which requires ajv 6.x (`frontend/package.json`)

#### IDOR Vulnerabilities Fixed in 4 GraphQL Mutations

- **HIGH**: Fixed information leakage allowing object ID enumeration via different error messages
  - `RemoveAnnotation` (`config/graphql/mutations.py`)
  - `RejectAnnotation` (`config/graphql/mutations.py`)
  - `ApproveAnnotation` (`config/graphql/mutations.py`)
  - `RemoveRelationship` (`config/graphql/mutations.py`)
- **Attack Vector**: Unauthorized users could distinguish between "object doesn't exist" and "object exists but you can't access it" by observing different error responses
- **Impact**: Allowed enumeration of valid annotation/relationship IDs
- **Solution**: All mutations now use `visible_to_user()` pattern with unified error messages; secondary permission checks also return the same unified message
- **Information leakage fix**: Outer exception handlers no longer return `str(e)` to clients; errors are logged server-side only
- **Test Coverage**: Added IDOR protection tests in `test_permission_fixes.py` and `test_voting_mutations_graphql.py`

#### QuerySet Permission Filtering Gaps Fixed

- `DocumentQuerySet.visible_to_user()` and `NoteQuerySet.visible_to_user()` inherited from `PermissionQuerySet` which had guardian permission checks commented out — only checking `is_public` and `creator`
  - `opencontractserver/shared/QuerySets.py` (classes `DocumentQuerySet`, `NoteQuerySet`)
- `AnnotationQuerySet.visible_to_user()` checked document/corpus visibility via `is_public` and `creator` only, missing guardian permission lookups for documents and corpuses
  - `opencontractserver/shared/QuerySets.py` (class `AnnotationQuerySet`)
- **Bug**: Code calling `Model.objects.filter(...).visible_to_user(user)` or `Model.objects.visible_to_user(user)` skipped guardian permission checks, making objects invisible to users with explicit share permissions
- **Impact**: Documents shared via `set_permissions_for_obj_to_user()` were invisible through the QuerySet chain code path; annotations on shared documents/corpuses were invisible; Notes on accessible documents were not visible
- **Fix**: All three QuerySets now override `visible_to_user()` with proper guardian permission table lookups. Documents and Annotations check guardian tables directly; Notes inherit from document + corpus permissions

### Fixed

#### Corpus Export/Import V2: Audit and Roundtrip Fixes

- **SPAN_LABEL and RELATIONSHIP_LABEL missing from label export**: `build_label_lookups()` in `opencontractserver/utils/etl.py` only exported TOKEN_LABEL and DOC_TYPE_LABEL labels. SPAN_LABEL and RELATIONSHIP_LABEL labels were silently dropped, causing annotation and relationship import to fail. Now all four label types are exported.
- **Relationship labels not gathered from Relationship model**: `build_label_lookups()` only queried labels from `Annotation` objects. Labels used exclusively on `Relationship` objects (RELATIONSHIP_LABEL type) were never collected. Added Relationship model queries to capture these labels.
- **Label lookup key mismatch for structural annotations and relationships**: Structural annotations and relationships reference labels by TEXT in exports, but the import label_lookup was keyed by PK strings. Created a text-keyed label lookup (`label_lookup_by_text`) in `_import_corpus()` for use by `import_structural_annotation_set()` and `_import_v2_relationships()`.
  - File: `opencontractserver/tasks/import_tasks_v2.py`
- **Document file_type not preserved**: Non-PDF documents (text/plain, etc.) lost their MIME type during export/import since `file_type` was not included in `OpenContractDocExport`. Added `file_type` to export data and import logic.
  - Files: `opencontractserver/types/dicts.py`, `opencontractserver/utils/etl.py`, `opencontractserver/utils/importing.py`
- **Document-level conversations not exported**: `package_conversations()` only exported corpus-level conversations (`chat_with_corpus=corpus`). Document-level conversations (`chat_with_document`) were completely missed. Now both types are exported.
  - File: `opencontractserver/utils/export_v2.py`
- **Conversation export missing permission filtering**: `package_conversations()` exported ALL conversations regardless of the exporting user's permissions. Added `visible_to_user()` filtering for both conversations and messages.
  - File: `opencontractserver/utils/export_v2.py`
- **Conversation fields missing from export/import**: `description`, `is_locked`, `is_pinned` were not exported or imported. Added to both export and import.
- **Message fields missing from export/import**: `parent_message` (threaded replies), `data` (JSON metadata) were not exported or imported. Added to both export and import with two-pass parent re-linking.
- **Timestamps silently discarded on conversation/message import**: Django's `auto_now_add=True` on `created_at` and `auto_now=True` on `updated_at` fields ignored values passed to `create()`. Fixed by using `QuerySet.update()` after creation to patch timestamps.
  - File: `opencontractserver/utils/import_v2.py`
- **include_conversations/include_action_trail not exposed in export mutation**: The V2 export task accepted these parameters but the GraphQL mutation never passed them, hardcoding `False`. Added both as optional mutation arguments.
  - File: `config/graphql/mutations.py`
- **DocumentPath version trees not reconstructed on import**: Exported DocumentPath data (paths, version numbers, folder assignments) was never used during import. Added `_reconstruct_document_paths()` to update auto-created DocumentPaths to match exported path structure.
  - File: `opencontractserver/tasks/import_tasks_v2.py`

### Technical Details

- All label types (TOKEN_LABEL, SPAN_LABEL, RELATIONSHIP_LABEL) are now exported in the `text_labels` dict with their actual `label_type` preserved for correct deserialization
- Conversation document hash (`chat_with_document_hash`) is exported alongside the document ID for cross-system re-linking
- Timestamp patching uses `Model.all_objects.filter(pk=obj.pk).update()` to bypass `auto_now`/`auto_now_add` behavior
- Comprehensive test coverage added: `TestLabelTypeExportCompleteness`, `TestDocumentFileTypeRoundTrip`, `TestConversationExportEnhancements`

### Added

#### Chunked Document Processing for Large PDFs

- **New `BaseChunkedParser` abstract class** (`opencontractserver/pipeline/base/chunked_parser.py`): Extends `BaseParser` to transparently split large PDF documents into page-range chunks for independent parsing and reassembly. Documents below a configurable page threshold are processed as a single request (zero overhead). Features:
  - Automatic PDF splitting via pypdf with configurable `max_pages_per_chunk` (default: 50) and `min_pages_for_chunking` (default: 75)
  - Optional concurrent chunk dispatch via `ThreadPoolExecutor` (`max_concurrent_chunks`, default: 3)
  - Per-chunk retry with exponential back-off before escalating to Celery-level retry
  - Correct reassembly of PAWLs page indices, annotation page references, `tokensJsons.pageIndex`, annotation/relationship IDs, and parent-child relationships across chunk boundaries
  - `_post_reassemble_hook()` for document-wide post-processing (e.g., image extraction on the full PDF)
- **New PDF splitting utility** (`opencontractserver/utils/pdf_splitting.py`): `get_pdf_page_count()`, `split_pdf_by_page_range()`, `calculate_page_chunks()` — pure functions for PDF page manipulation
- **New chunking constants** (`opencontractserver/constants/document_processing.py`): `DEFAULT_MAX_PAGES_PER_CHUNK`, `DEFAULT_MIN_PAGES_FOR_CHUNKING`, `DEFAULT_MAX_CONCURRENT_CHUNKS`, `DEFAULT_CHUNK_RETRY_LIMIT`
- **DoclingParser now extends `BaseChunkedParser`** (`opencontractserver/pipeline/parsers/docling_parser_rest.py`): Large documents are automatically split and parsed in chunks. Configurable via `PipelineSettings` (`DOCLING_MAX_PAGES_PER_CHUNK`, `DOCLING_MIN_PAGES_FOR_CHUNKING`, `DOCLING_MAX_CONCURRENT_CHUNKS`). Image extraction runs once on the full PDF after reassembly via `_post_reassemble_hook`.

#### Context Guardrails & Conversation Compaction (Closes #898)

- **Context guardrails constants** (`opencontractserver/constants/context_guardrails.py`): Centralized configuration for model context windows (OpenAI, Anthropic, Google), compaction thresholds, tool output limits, and token estimation parameters. Covers 20+ model variants with sensible defaults.
- **Token estimation** (`opencontractserver/llms/context_guardrails.py`): Fast heuristic token counter (~3.5 chars/token) for estimating conversation size without importing heavyweight tokeniser libraries. Intentionally over-estimates to trigger compaction conservatively.
- **Model context window lookup** (`context_guardrails.py`): Resolves model names to context window sizes via exact match then longest-prefix matching, with a 128K default fallback for unknown models.
- **Conversation compaction** (`context_guardrails.py`): When conversation history approaches the context window limit (default 75%), older messages are replaced by a concise summary while preserving recent turns (4–20 messages) verbatim. Supports pluggable summary functions for LLM-based summarization.
- **Tool output truncation** (`context_guardrails.py`, `opencontractserver/llms/tools/pydantic_ai_tools.py`): String outputs from tools are automatically truncated to 50K characters with a notice directing the LLM to use range parameters. Applied at the PydanticAI tool wrapper level so all tools benefit transparently.
- **Per-agent compaction configuration** (`CompactionConfig` dataclass): Added `compaction` field to `AgentConfig` allowing per-conversation overrides of threshold ratio, recent message counts, and tool output limits. Enabled by default.
- **Automatic compaction in message history retrieval** (`opencontractserver/llms/agents/pydantic_ai_agents.py`): `_get_message_history()` now checks conversation size against model limits and injects a compaction summary as a system message when needed, transparent to the agent framework.
- **Persisted compaction bookmarks** (`opencontractserver/conversations/models.py`): Added `compaction_summary` and `compacted_before_message_id` fields to the `Conversation` model. Compaction is computed once and persisted — subsequent reads skip old messages at the DB level via `id__gt` filter, making long conversations cheap to load.
  - Migration: `opencontractserver/conversations/migrations/0015_add_compaction_fields.py`
  - `CoreConversationManager.persist_compaction()` writes the bookmark with optimistic locking (concurrent requests are safely resolved)
  - `CoreConversationManager.get_conversation_messages()` honours the cutoff automatically
- **Comprehensive test suite** (`opencontractserver/tests/test_context_guardrails.py`): 30+ unit tests covering token estimation, model lookup, truncation, compaction triggers, summary generation, message proxy conversion, configuration defaults, and Conversation model field definitions. Uses `SimpleTestCase` for fast parallel execution.

### Changed

#### Pipeline Registry: Deduplicate and Filter Abstract Components

- **Removed `MultimodalMicroserviceEmbedder` backwards-compatibility alias**: The module-level alias `MultimodalMicroserviceEmbedder = CLIPMicroserviceEmbedder` in `opencontractserver/pipeline/embedders/multimodal_microservice.py` has been removed. Use `CLIPMicroserviceEmbedder` directly.
- **Fixed duplicate embedder entries in pipeline registry**: `_discover_subclasses()` in `opencontractserver/pipeline/registry.py` now deduplicates discovered classes by identity and skips abstract intermediate base classes via `inspect.isabstract()`, preventing aliases and abstract bases from appearing in the get-embedders query endpoint.

### Fixed

#### Prompt Injection via User-Controlled Content in Agent Prompts

- **Root cause**: Thread and document action prompt builders injected user-controlled content (message bodies, thread titles, document titles) directly into Markdown-structured LLM prompts without any sanitisation boundary. A user who can post a message to a moderated thread could craft content that overrides agent instructions.
  - File: `opencontractserver/tasks/agent_tasks.py` (lines 808-848)
  - File: `opencontractserver/llms/agents/core_agents.py` (lines 963-983, 1036-1054)
  - File: `opencontractserver/llms/agents/pydantic_ai_agents.py` (lines 1655-1669, 2301-2315)
- **Fix**: All user-generated content injected into agent prompts is now wrapped in `<user_content>` / `</user_content>` XML fence tags. An explicit `UNTRUSTED_CONTENT_NOTICE` instruction block is added to thread moderation prompts telling the LLM to treat fenced content as raw data and ignore any embedded directives. A size-threshold warning (`UNTRUSTED_CONTENT_SIZE_WARNING_THRESHOLD = 1000 chars`) logs a `[PromptInjection]` warning for abnormally large user content.
  - New file: `opencontractserver/utils/prompt_sanitization.py` — `fence_user_content()`, `warn_if_content_large()`, `UNTRUSTED_CONTENT_NOTICE`
  - New constant: `opencontractserver/constants/moderation.py` — `UNTRUSTED_CONTENT_SIZE_WARNING_THRESHOLD`
  - New tests: `opencontractserver/tests/test_prompt_sanitization.py`
  - Updated tests: `opencontractserver/tests/test_thread_corpus_actions.py` — added `test_async_thread_action_prompt_fences_user_content`

#### Prompt Injection Mitigation Follow-up (Closes #913)

- **Dead code fix**: `warn_if_content_large()` was called on truncated message previews (max 203 chars) but checks against a 1000-char threshold, making the warning ineffective. Moved the call to run on full content before truncation.
  - File: `opencontractserver/tasks/agent_tasks.py` (`_build_thread_action_system_prompt`)
- **Inconsistent monitoring**: Document and corpus titles embedded in system prompts in `core_agents.py` and `pydantic_ai_agents.py` now have `warn_if_content_large()` calls for consistent size monitoring.
  - File: `opencontractserver/llms/agents/core_agents.py` (`CoreDocumentAgentFactory`, `CoreCorpusAgentFactory`)
  - File: `opencontractserver/llms/agents/pydantic_ai_agents.py` (`PydanticAIDocumentAgent`, `PydanticAICorpusAgent`)
- **Null safety**: All title values passed to `fence_user_content()` and `warn_if_content_large()` now use `or "untitled"` fallback to prevent `TypeError` when `document.title` or `corpus.title` is `None`.
  - Affected files: `agent_tasks.py`, `core_agents.py`, `pydantic_ai_agents.py`
- **Documentation mismatch**: `UNTRUSTED_CONTENT_NOTICE` now describes the labeled tag variant (`<user_content label="...">`) matching the actual implementation, and explains that the label attribute does not change handling.
  - File: `opencontractserver/utils/prompt_sanitization.py`

#### Frontend: Most views show legacy corpus.description instead of versioned mdDescription (Closes #892)

- **Backend description sync**: `Corpus.update_description()` now keeps the plain-text `description` field in sync when `md_description` is updated via the versioned markdown system. A new `_markdown_to_plain_text()` static method strips markdown formatting for the plain-text field.
  - File: `opencontractserver/corpuses/models.py` (lines 249-272, `update_description` method)
- **New `useCorpusMdDescription` hook**: Reusable React hook that fetches markdown content from a corpus's `mdDescription` URL and returns the raw text for rendering with `SafeMarkdown`.
  - File: `frontend/src/hooks/useCorpusMdDescription.ts`
- **CorpusContextSidebar**: Now fetches and renders the versioned markdown description instead of the stale plain-text `description` field.
  - File: `frontend/src/components/threads/CorpusContextSidebar.tsx`
- **DocumentKnowledgeBase**: Corpus info display now fetches `mdDescription` content and renders it as markdown. Added `title`, `description`, and `mdDescription` fields to the `GET_DOCUMENT_KNOWLEDGE_AND_ANNOTATIONS` query's corpus selection.
  - File: `frontend/src/components/knowledge_base/document/DocumentKnowledgeBase.tsx`
  - File: `frontend/src/graphql/queries.ts` (line 3028)
- **CorpusHeader (settings)**: Now fetches and renders the versioned markdown description via `useCorpusMdDescription` hook with `SafeMarkdown`. Added `mdDescription` to prop chain through `CorpusSettings` and `Corpuses.tsx`.
  - File: `frontend/src/components/corpuses/settings/CorpusHeader.tsx`
  - File: `frontend/src/components/corpuses/CorpusSettings.tsx`
  - File: `frontend/src/views/Corpuses.tsx`
- **TypeScript type update**: Added `mdDescription` optional field to `RawCorpusType`.
  - File: `frontend/src/types/graphql-api.ts`

#### Edit Description Modal Does Not Save on Update (Issue #899)

- **Root cause**: The edit document CRUDModal in `App.tsx` had a no-op `onSubmit` handler that only closed the modal without calling the `UPDATE_DOCUMENT` mutation, so changes were silently discarded
  - File: `frontend/src/App.tsx` (lines 128-149, 398)
- **Fix**: Added `useMutation` hook for `UPDATE_DOCUMENT` in `App.tsx` with proper `onCompleted`/`onError` handlers and `refetchQueries: "active"` to refresh displayed data
- **Removed duplicate modals**: `Documents.tsx` rendered its own edit/view CRUDModals controlled by the same `editingDocument` reactive var as `App.tsx`, causing potential double-modal rendering. Removed the duplicates from `Documents.tsx` and consolidated into the global `App.tsx` handler
  - File: `frontend/src/views/Documents.tsx` (removed ~45 lines of duplicate modal + mutation code)

### Changed

#### Import/Export Pipeline Consolidation

- **DRY refactor of import/export code**: Extracted shared helpers into `opencontractserver/utils/importing.py`:
  - `prepare_import_labels()` - eliminates 4x duplicated label loading boilerplate
  - `create_document_from_export_data()` - eliminates 3x duplicated document creation
  - `import_doc_annotations()` - eliminates 3x duplicated doc+text annotation import loops
- **V1 import now delegates to V2 machinery**: `import_corpus()` in `import_tasks.py` delegates to `import_corpus_v2()` which handles both V1 and V2 formats through a unified `_import_corpus()` handler
- **V2 import fixed to use `corpus.add_document()`**: Previously created documents directly without corpus isolation; now properly uses the versioning API for correct DocumentPath records and corpus isolation
- **V2 import now sets permissions on annotations**: Previously skipped `set_permissions_for_obj_to_user` on annotations
- **V2 import now handles `content_modalities`**: Via the shared `import_annotations()` helper
- **Export finalization DRYed up**: New `finalize_export()` in `export_tasks.py` replaces 4x repeated save/timestamp/notification pattern in `package_annotated_docs`, `package_funsd_exports`, `on_demand_post_processors`, and `package_corpus_export_v2`
- **Removed duplicate `import_relationships` and `import_document_paths`** from `utils/import_v2.py` - relationship import handled inline in `_import_v2_relationships`, DocumentPaths created by `corpus.add_document()`
- **Deleted empty `opencontractserver/utils/export.py`**

### Added

#### Store Model Name in ChatMessage Metadata (#897)

- **Automatic model name persistence**: The LLM model name from `AgentConfig` is now stored in the `data` JSON field of every `ChatMessage` produced by an agent, enabling debugging, auditing, and reproducibility
  - `opencontractserver/llms/agents/core_agents.py` — all five `CoreConversationManager` message-writing methods now persist `data["model_name"]`:
    - `create_placeholder_message()` and `store_llm_message()` — unconditional write at creation time
    - `complete_message()`, `update_message()`, `mark_message_error()` — use `setdefault` to backfill without overwriting placeholder values
- **Tests**: Seven new async tests verifying model name storage across all message lifecycle paths
  - `opencontractserver/tests/test_core_agents.py` — covers explicit model name, default model name, all five methods, and `setdefault` preservation semantics

#### Nested Approval Gates for Corpus Agent Sub-Agents

- **Sub-agent approval propagation**: When a corpus agent delegates a question to a document sub-agent via `ask_document`, and the sub-agent encounters a tool requiring approval, the approval request now propagates up to the corpus agent level and is surfaced to the user via WebSocket (`ASYNC_APPROVAL_NEEDED`)
  - File: `opencontractserver/llms/agents/pydantic_ai_agents.py` (ask_document_tool closure)
- **Frontend sub-tool unwrapping**: CorpusChat's approval modal now displays the actual sub-tool name/arguments instead of the generic `ask_document` wrapper, with validation for malformed metadata
  - File: `frontend/src/components/corpuses/CorpusChat.tsx` (ASYNC_APPROVAL_NEEDED handler)
- **Comprehensive nested approval test suite**: 10 async tests covering approval propagation, metadata stripping, bypass flag lifecycle, malformed event handling, and schema safety
  - File: `opencontractserver/tests/test_nested_approval_gates.py`
- **Architecture documentation**: Added "Nested Approval Gates" section to LLM framework docs with flow diagrams and security notes
  - File: `docs/architecture/llms/README.md`

#### Expose Tool Usage in Chat UI

- **Tool Usage Badge** (`frontend/src/components/widgets/chat/ChatMessage.tsx:1180-1288`): Assistant messages that use tools now display a wrench icon badge ("X tools used") in the message header, visible in both document and corpus chat views. Users can quickly see AI tool usage without expanding the full timeline, improving agent transparency.
- **Tool Call Popover** (`ChatMessage.tsx:1222-1286`): Hovering over the badge opens a popover listing each tool call's formatted name, JSON input arguments, and output result. Keyboard accessible (Enter/Space to toggle, Escape to close) with full ARIA attributes.
- **Tool result content in timeline**: Backend now captures tool result/output content in timeline `tool_result` entries (previously only stored tool name)
  - `opencontractserver/llms/agents/timeline_schema.py:52` — added `result` field
  - `opencontractserver/llms/agents/timeline_utils.py:77-92` — captures result from metadata, truncated to 500 chars
  - `opencontractserver/llms/agents/pydantic_ai_agents.py:123-155` — `_extract_tool_result_summary()` extracts and truncates at source
- **Tool result entries for search tools** (`pydantic_ai_agents.py:642-657, 686-702, 807-813`): `similarity_search`, `search_exact_text`, and `ask_document` now emit `tool_result` timeline entries with result summaries (e.g., "Found 3 matching annotations"). Other tools use a generic extractor with "Completed" fallback.

#### Automated Documentation Screenshots

- **Screenshot capture utility** (`frontend/tests/utils/docScreenshot.ts`): Captures screenshots during Playwright component tests using an enforced `{area}--{component}--{state}` naming convention
- **CI workflow** (`.github/workflows/screenshots.yml`): Automatically runs component tests on PRs touching `frontend/` or `docs/`, then commits updated screenshots back to the PR branch
- **Initial screenshot coverage**: Landing page components (hero, stats bar, trending corpuses, call-to-action) and badge components (celebration modal, toast)

#### V2 Export Format

- **`OPEN_CONTRACTS_V2` export format**: New export type available in `StartCorpusExport` mutation that includes structural annotation sets, folder hierarchy, relationships, agent config, markdown descriptions, and conversations
- **`content_modalities` now exported**: Annotations with IMAGE or other modalities now survive export/import round-trips (`opencontractserver/utils/etl.py:build_document_export`)
- **Migration `0025_alter_userexport_format_add_v2`**: Adds `OPEN_CONTRACTS_V2` to UserExport format choices

#### Edge Case Tests for Personal Corpus (Issue #839)

- **Concurrent creation race condition test**: Verifies that 5 concurrent threads calling `get_or_create_personal_corpus()` all return the same corpus with no duplicates or errors
  - File: `opencontractserver/tests/test_personal_corpus.py` (`TestConcurrentPersonalCorpusCreation`)
- **Delete and recreate flow tests**: Verifies that after deleting a personal corpus, recreation produces a new corpus with correct attributes and permissions
  - File: `opencontractserver/tests/test_personal_corpus.py` (`TestDeleteAndRecreatePersonalCorpus`)
- **Embedding task queue failure tests**: Verifies graceful degradation when Redis/Celery is unavailable during embedding task queuing, including partial batch failure scenarios
  - File: `opencontractserver/tests/test_personal_corpus.py` (`TestEmbeddingTaskQueueFailure`)

### Fixed

#### MCP Telemetry in Async Context

- **`SynchronousOnlyOperation` in MCP server** (`config/telemetry.py`, `opencontractserver/mcp/telemetry.py`, `opencontractserver/mcp/server.py`): Added async telemetry functions (`arecord_event`, `arecord_mcp_tool_call`, `arecord_mcp_resource_read`, `arecord_mcp_request`) that use `sync_to_async` to safely run Django ORM lookups in a thread pool. Prevents "You cannot call this from an async context" errors on every MCP request.
- **Installation ID caching** (`config/telemetry.py:91-113`): Added process-lifetime cache for installation UUID to eliminate redundant database queries on every telemetry call, particularly beneficial for high-frequency MCP requests.

#### Security: LLM Prompt Injection Protection for Approval Bypass

- **Replaced `skip_approval` function parameter with `config._approval_bypass_allowed` flag**: The previous design exposed a `skip_approval` parameter in `ask_document_tool`'s function signature that a malicious LLM could set to `True` to bypass approval gates. Now uses a runtime flag on `AgentConfig` that only `resume_with_approval()` can set, wrapped in a `try/finally` block to guarantee reset
  - File: `opencontractserver/llms/agents/pydantic_ai_agents.py`

#### Inconsistent Approval Status Handling in CorpusChat

- **Added `updateMessageApprovalStatus` to CorpusChat**: Previously, `ASYNC_APPROVAL_RESULT` handler in CorpusChat only cleared pending state without updating message `approvalStatus`, unlike ChatTray and useAgentChat which both call `updateMessageApprovalStatus`. Now consistent across all components
  - File: `frontend/src/components/corpuses/CorpusChat.tsx`
- **Added message `approvalStatus: "awaiting"` on ASYNC_APPROVAL_NEEDED**: CorpusChat now marks messages as awaiting approval in both `chat` and `serverMessages` state arrays, matching ChatTray/useAgentChat behavior
  - File: `frontend/src/components/corpuses/CorpusChat.tsx`

#### Defensive Handling of Malformed Approval Events

- **Backend**: `ask_document_tool` now validates `pending_tool_call` is a dict with a non-empty `name` key before raising `ToolConfirmationRequired`; malformed events are logged and skipped
  - File: `opencontractserver/llms/agents/pydantic_ai_agents.py`
- **Frontend**: `_sub_tool_name` validation checks type is string and non-empty; `_sub_tool_arguments` validates type is object before use
  - File: `frontend/src/components/corpuses/CorpusChat.tsx`

#### Corpus Agent Action Failure: griffe/pydantic-ai Incompatibility

- **Pin `griffe>=1.3.2,<2`** (`requirements/base.txt`): griffe 2.0.0 (released 2026-02-09) removed the `**options` catch-all from all docstring parsers. pydantic-ai 0.2.x unconditionally passes `returns_named_value` and `returns_multiple_items` as parser options to all parsers (including numpy), causing `TypeError: parse_numpy() got an unexpected keyword argument 'returns_named_value'`. This broke all `run_agent_corpus_action` tasks during agent creation. Pinning griffe below 2.0 restores the `**options` parameter that absorbs these Google-specific options harmlessly.

#### SynchronousOnlyOperation in Vector Store Construction from Async Context

- **Wrap vector store construction in `sync_to_async`** (`opencontractserver/llms/vector_stores/pydantic_ai_vector_stores.py:392`): `create_vector_search_tool()` now wraps `PydanticAIAnnotationVectorStore(...)` in `sync_to_async` so the sync ORM calls inside `CoreAnnotationVectorStore.__init__` (embedder resolution via `get_embedder()`) run in a thread pool instead of triggering Django's `SynchronousOnlyOperation`.
- **Pre-resolve embedder_path in `PydanticAIDocumentAgent.create()`** (`opencontractserver/llms/agents/pydantic_ai_agents.py:1529`): Added async embedder pre-resolution using `aget_embedder()` before constructing the vector store, matching the existing pattern in `PydanticAICorpusAgent.create()`. This prevents the sync `get_embedder()` fallback from hitting the ORM in an async context.
- **Defensive `sync_to_async` fallback in both agent `create()` methods** (`pydantic_ai_agents.py`): If `aget_embedder()` fails and `embedder_path` remains `None`, the `PydanticAIAnnotationVectorStore(...)` constructor is wrapped in `sync_to_async` so the ORM calls inside `get_embedder()` run in a thread pool. Applied to both `PydanticAIDocumentAgent.create()` and `PydanticAICorpusAgent.create()`.
- **Fix async test** (`opencontractserver/tests/test_pydantic_ai_agents.py:412`): Wrapped `PydanticAIAnnotationVectorStore(...)` construction in `sync_to_async` in `test_pydantic_ai_vector_store_search`.

### Changed

#### Streamlined Agentic Corpus Action Configuration

- **Renamed `agent_prompt` to `task_instructions`** on `CorpusAction` model (`opencontractserver/corpuses/models.py`): Single, clearly-named field for describing what the agent should do. Migration `0041` handles the rename.
- **Goal-oriented system prompt assembly** (`opencontractserver/tasks/agent_tasks.py`): Agent corpus actions now auto-generate a structured system prompt with automation guardrails ("you MUST use tools"), execution context (trigger type, document metadata, corpus info), and the user's task instructions. Agents no longer receive raw `system_instructions` as the system prompt — the system wraps everything in a goal-oriented format that prevents conversational responses.
- **Document context injection**: Document-based agent actions now inject document title, ID, corpus title, and current description into the system prompt so agents don't waste tool calls loading basic metadata.
- **Thread context injection refactored**: Thread-based agent actions now use the same structured prompt pattern as document actions, with thread context, recent messages, and triggering message content all included in the system prompt rather than the user message.
- **`AgentConfiguration` is now optional for agent actions**: `CorpusAction` can be created with just `task_instructions` (no `agent_config` required). The DB constraint (`valid_action_type_configuration`) now allows lightweight agent actions. `AgentConfiguration` is still supported for custom persona/tool defaults.
- **Default tool selection by trigger type** (`opencontractserver/constants/corpus_actions.py`): When no tools are specified on `agent_config.available_tools`, the system auto-selects trigger-appropriate defaults (document tools for add/edit triggers, moderation tools for thread/message triggers).
- **`pre_authorized_tools` semantics changed** (`opencontractserver/tasks/agent_tasks.py`): `pre_authorized_tools` now only controls which tools skip approval gates. Tool availability is determined by `agent_config.available_tools` (if set) or trigger-appropriate defaults. See **Breaking Changes** below for migration guidance.
- **GraphQL API updated**: `CreateCorpusAction` and `UpdateCorpusAction` mutations use `taskInstructions` instead of `agentPrompt`. The `taskInstructions` field alone (without `agentConfigId`) is now sufficient to create an agent action.
- **Frontend updated**: `CreateCorpusActionModal` and `CorpusActionsSection` use "Task Instructions" labeling instead of "Agent Prompt".
- **Unified Quick Create flow**: `CreateCorpusActionModal` now supports inline agent creation for document triggers (add_document, edit_document) in addition to thread triggers. The modal auto-selects trigger-appropriate tools and default instructions.
- **Manual action trigger** (`config/graphql/mutations.py`): New `RunCorpusAction` mutation allows superusers to manually trigger agent actions on specific documents. Uses `transaction.on_commit()` to dispatch Celery tasks after the DB transaction commits, and `force=True` to bypass dedup checks for manual triggers.
- **ToolFunctionRegistry** (`opencontractserver/llms/tools/tool_registry.py`): Centralized singleton registry mapping tool names to sync/async function implementations. Replaces 3 manually-curated dicts in `_resolve_tools()`. Adding a new tool now requires edits in 2 files instead of 4+.

#### Mobile Navigation & URL-Driven Corpus Navigation

- **Detail view switching now pushes browser history**: `updateDetailViewParam()` in `frontend/src/utils/navigationUtils.ts` now pushes new history entries instead of replacing, so browser back/forward navigates between landing and details views. `updateTabParam()` retains replace semantics so tab switches don't accumulate history entries
- **Thread selection now URL-driven**: Discussions tab thread selection uses the existing `?thread=` query parameter (synced by CentralRouteManager) instead of the local `inlineSelectedThreadIdAtom` Jotai atom. Clicking a thread pushes `?thread=<id>` to the URL; browser back returns to the list
  - Files: `frontend/src/components/discussions/CorpusDiscussionsView.tsx`, `frontend/src/utils/navigationUtils.ts`, `frontend/src/views/Corpuses.tsx`
- **Tab-specific params cleared on tab switch**: `updateTabParam()` now removes `thread` and `message` params when switching tabs to prevent stale state persisting across tab changes
- **Removed `inlineSelectedThreadIdAtom`**: Replaced by URL-driven `selectedThreadId` reactive var from `frontend/src/graphql/cache.ts`; dead `onViewModeChange` callback removed from `CorpusDiscussionsView`

### Added

#### Mobile Menu Access in Corpus Home Views

- **Mobile navigation menu buttons**: Added `MobileMenuButton` (kebab icon, visible ≤600px) to `CorpusLandingView` (breadcrumb row) and `CorpusDetailsView` (header row), allowing mobile users to open the sidebar bottom sheet from the home tab — previously only accessible from non-home tabs
  - Files: `frontend/src/components/corpuses/CorpusHome/CorpusLandingView.tsx`, `frontend/src/components/corpuses/CorpusHome/CorpusDetailsView.tsx`, `frontend/src/components/corpuses/CorpusHome/styles.ts`
- **`updateThreadParam()` utility**: New navigation utility for setting/clearing the `?thread=` URL param with push semantics, following the same pattern as `updateTabParam()` and `updateMessageParam()`
  - File: `frontend/src/utils/navigationUtils.ts`

### Breaking Changes

- **`pre_authorized_tools` no longer controls tool availability**: Previously, `pre_authorized_tools` was used as both the tool set AND the approval gate — if set, it replaced `agent_config.available_tools` entirely. Now it only controls which tools skip the approval gate. **Migration**: If you relied on `pre_authorized_tools` to restrict which tools an agent can access, move those tool names to `agent_config.available_tools` instead. `pre_authorized_tools` should only list tools that are safe to run without human approval.

### Known Limitations

- **Orphaned QUEUED executions if Celery broker is unavailable**: The `RunCorpusAction` mutation creates a `CorpusActionExecution` with `QUEUED` status and dispatches the Celery task via `transaction.on_commit()`. If the Celery broker is down at commit time, the task dispatch silently fails and the execution record stays `QUEUED` indefinitely. This is a general characteristic of the `on_commit` + Celery pattern used throughout the codebase. Monitor for stale `QUEUED` records if broker reliability is a concern.

#### Edge Case Tests for Personal Corpus (Issue #839)

- **Concurrent creation race condition test**: Verifies that 5 concurrent threads calling `get_or_create_personal_corpus()` all return the same corpus with no duplicates or errors
  - File: `opencontractserver/tests/test_personal_corpus.py` (`TestConcurrentPersonalCorpusCreation`)
- **Delete and recreate flow tests**: Verifies that after deleting a personal corpus, recreation produces a new corpus with correct attributes and permissions
  - File: `opencontractserver/tests/test_personal_corpus.py` (`TestDeleteAndRecreatePersonalCorpus`)
- **Embedding task queue failure tests**: Verifies graceful degradation when Redis/Celery is unavailable during embedding task queuing, including partial batch failure scenarios
  - File: `opencontractserver/tests/test_personal_corpus.py` (`TestEmbeddingTaskQueueFailure`)

### Fixed

#### Security Hardening: Authentication & Permissioning Audit Remediation

- **Analysis callback DoS prevention** (`opencontractserver/analyzer/views.py`): Invalid callback tokens no longer mark analyses as FAILED; uses `hmac.compare_digest()` for timing-safe token comparison; unified error messages prevent analysis ID enumeration
- **User lock logic inversion** (`config/graphql/base.py`): Fixed `==` to `!=` in DRFDeletion and DRFMutation user lock checks — previously blocked the lock holder and allowed everyone else
- **IDOR prevention in base mutations** (`config/graphql/base.py`): DRFDeletion and DRFMutation now use `visible_to_user()` filtering before `.get()` to prevent object existence leakage
- **Open redirect prevention** (`config/urls.py`): `home_redirect` now validates the Host header against `ALLOWED_HOSTS` before constructing the redirect URL
- **Cross-corpus data leakage** (`config/graphql/graphene_types.py`): Document summary resolvers (`resolve_summary_revisions`, `resolve_current_summary_version`, `resolve_summary_content`) now verify corpus visibility before returning data
- **CSRF trusted origins** (`config/settings/production.py`): Fixed missing comma causing implicit string concatenation in `CSRF_TRUSTED_ORIGINS`
- **HSTS enforcement** (`config/settings/production.py`): Increased `SECURE_HSTS_SECONDS` from 60 to 518400 (6 days)
- **Analyzer visibility default** (`opencontractserver/analyzer/models.py`): Changed `Analyzer` and `GremlinEngine` `is_public` default from `True` to `False` to prevent accidental data exposure
- **IDOR prevention in GraphQL mutations** (`config/graphql/mutations.py`): Added `visible_to_user()` filtering to 11 previously unprotected `.objects.get()` calls in `StartDocumentExtract`, `DeleteAnalysisMutation`, `CreateColumn`, `CreateExtract`, `CreateCorpusAction`, `UpdateCorpusAction`, and `CreateNote`
- **IDOR prevention in conversation mutations** (`config/graphql/conversation_mutations.py`): `CreateThreadMutation` and `CreateThreadMessageMutation` now use `visible_to_user()` instead of fetch-then-check pattern
- **IDOR prevention in folder mutations** (`config/graphql/corpus_folder_mutations.py`): All folder mutations now verify corpus visibility before operating on folders; folder lookups scoped to validated corpus
- **IDOR prevention in voting mutations** (`config/graphql/voting_mutations.py`): `VoteMessageMutation` and `RemoveVoteMutation` now use `ChatMessage.objects.visible_to_user()` instead of fetch-then-check
- **IDOR prevention in badge mutations** (`config/graphql/badge_mutations.py`): `AwardBadgeMutation` now uses `Badge.objects.visible_to_user()` for badge lookup

### Fixed

#### Enable Relationships for Span-Based (Text) Annotations (Closes #281)

- **File type detection inconsistency**: Multiple frontend components checked for text file types using only `startsWith("text/")`, missing documents with `application/txt` MIME type. Created centralized `isTextFileType()` and `isPdfFileType()` utilities in `frontend/src/utils/files.ts` and updated all callers.
- **Label initialization race condition**: The `initialized.current` ref in `UISettingsAtom.tsx` (line 288) could be set to `true` after span label initialization, preventing relationship labels from auto-initializing on subsequent effect runs. Replaced with separate `spanLabelInitialized` and `relationLabelInitialized` refs.
- **Type restrictions blocking span annotations in relationship UI**: `RelationItem`, `RelationHighlightItem`, `HighlightItem`, and `annotationSelectedViaRelationship()` only accepted `ServerTokenAnnotation` (PDF annotations). Updated all to accept `ServerTokenAnnotation | ServerSpanAnnotation` union type, enabling the sidebar relationship display and creation flow for text documents.
- **Files changed**: `frontend/src/utils/files.ts`, `frontend/src/components/annotator/context/UISettingsAtom.tsx`, `frontend/src/components/annotator/sidebar/RelationItem.tsx`, `frontend/src/components/annotator/sidebar/RelationHighlightItem.tsx`, `frontend/src/components/annotator/sidebar/HighlightItem.tsx`, `frontend/src/components/annotator/utils.ts`, `frontend/src/components/annotator/hooks/AnnotationHooks.tsx`, `frontend/src/components/annotator/labels/EnhancedLabelSelector.tsx`, `frontend/src/components/annotator/labels/UnifiedLabelSelector.tsx`, `frontend/src/components/annotator/labels/label_selector/LabelSelector.tsx`, `frontend/src/components/knowledge_base/document/DocumentKnowledgeBase.tsx`, `frontend/src/components/widgets/chat/ChatMessage.tsx`

## [3.0.0.b4] - 2026-02-08

### ⚠️ Important Migration Notes

**Migration 0040 (`corpus_created_with_embedder`) backfills existing corpuses**

This release includes a data migration that:

- Adds a `created_with_embedder` audit field to all corpuses
- Backfills `preferred_embedder` on existing corpuses that don't have one set (uses current `DEFAULT_EMBEDDER`)
- Backfills `created_with_embedder` from `preferred_embedder`

This migration is safe and non-destructive. Existing corpuses with explicit `preferred_embedder` values are unchanged.

**Migration 0038 (`create_personal_corpuses`) is IRREVERSIBLE**

This release includes a data migration that creates personal "My Documents" corpuses for all users and moves standalone documents into them. This migration **cannot be rolled back** via `python manage.py migrate`. Attempting to reverse will raise `NotImplementedError`.

**Before deploying to production:**

- Ensure you have a database backup
- Test the migration in a staging environment first
- Plan for this being a one-way migration

If rollback is required after deployment, you must write a custom migration to handle your specific data preservation needs.

### Added

#### Document Processing Failure Indicators and Retry Controls (Issue #825)

- **Processing status display**: Document cards and list items now show distinct states for processing (spinner) vs. failed (error overlay with message) instead of a generic "Processing..." overlay for all locked documents
- **Retry button**: Failed documents display a retry button that triggers the `RetryDocumentProcessing` GraphQL mutation, allowing users to re-process documents without backend access
- **Context menu retry**: "Retry Processing" option added to the document context menu for failed documents
- **Permission-aware**: Retry controls only appear when the user has permission to retry (`canRetry` field from backend)
- **Error messages**: Processing error messages from the backend are displayed on the failure overlay (truncated for readability)
- Files: `frontend/src/components/documents/ModernDocumentItem.tsx`, `frontend/src/components/documents/DocumentItem.tsx`, `frontend/src/graphql/queries.ts`, `frontend/src/graphql/mutations.ts`, `frontend/src/types/graphql-api.ts`

#### Embedder Consistency Management (Issue #437)

- **Frozen embedder binding at corpus creation**: `preferred_embedder` is now auto-populated from `DEFAULT_EMBEDDER` when a corpus is created without an explicit embedder. This decouples existing corpuses from future changes to the global setting.
  - Files: `opencontractserver/corpuses/models.py` (save method)
- **Audit trail field `created_with_embedder`**: Records which embedder was active at corpus creation. Never changes, even after re-embedding.
  - Files: `opencontractserver/corpuses/models.py`, migration `0040_corpus_created_with_embedder.py`
- **Immutability guard on `preferred_embedder`**: `UpdateCorpusMutation` rejects changes to `preferred_embedder` after documents have been added to a corpus, preventing inconsistent embeddings.
  - Files: `config/graphql/mutations.py` (UpdateCorpusMutation.mutate)
- **`reEmbedCorpus` mutation**: Controlled migration path for changing a corpus's embedder. Locks the corpus, queues background re-embedding for all annotations, and unlocks when complete.
  - Files: `config/graphql/mutations.py` (ReEmbedCorpus), `opencontractserver/tasks/corpus_tasks.py` (reembed_corpus)
- **Fork with embedder override**: `forkCorpus` mutation now accepts optional `preferredEmbedder` argument to create the fork with a different embedder.
  - Files: `config/graphql/mutations.py` (StartCorpusFork)
- **Corpus-scoped search uses corpus embedder**: `resolve_semantic_search` now uses `corpus.preferred_embedder` for corpus-scoped queries instead of the global `DEFAULT_EMBEDDER`, ensuring consistent results.
  - Files: `config/graphql/queries.py` (resolve_semantic_search)
- **Startup system check**: Django system check warns at startup if `DEFAULT_EMBEDDER` has changed since existing corpuses were created, preventing silent search inconsistencies.
  - Files: `opencontractserver/corpuses/checks.py`, `opencontractserver/corpuses/apps.py`

#### Auth0 Authentication for Django Admin

- **Auth0 admin login support**: Django admin now supports Auth0 authentication when `USE_AUTH0=True`
  - Custom login view displays Auth0 "Sign in" button with password fallback
  - Custom logout view properly clears both Django session and Auth0 session
  - Backward compatible: password authentication always available
  - Files: `config/admin_auth/views.py`, `config/admin_auth/backends.py`
- **Admin claims synchronization**: Admin privileges can be set via Auth0 token claims

  - Supports `{namespace}is_staff` and `{namespace}is_superuser` claims
  - Claims synced on API requests with 5-minute cache TTL (configurable via `ADMIN_CLAIMS_CACHE_TTL` constant)
  - Immediate sync during admin login ensures fresh permissions for admin access
  - Handles boolean, string ("true"/"false"), and numeric (0/1) claim values
  - Configurable namespace via `AUTH0_ADMIN_CLAIM_NAMESPACE` env var
  - Files: `config/graphql_auth0_auth/utils.py:269-360`
  - **Required Auth0 Action** (Post-Login): Set up the following Auth0 Action to include admin claims in tokens:

    ```javascript
    exports.onExecutePostLogin = async (event, api) => {
      const namespace = "https://opencontracts.opensource.legal/";
      const appMetadata = event.user.app_metadata || {};

      // Add admin claims to access token
      if (appMetadata.is_staff !== undefined) {
        api.accessToken.setCustomClaim(
          `${namespace}is_staff`,
          appMetadata.is_staff
        );
      }
      if (appMetadata.is_superuser !== undefined) {
        api.accessToken.setCustomClaim(
          `${namespace}is_superuser`,
          appMetadata.is_superuser
        );
      }
    };
    ```

    Then set `app_metadata.is_staff` and `app_metadata.is_superuser` on users via Auth0 Management API or Dashboard.

- **Auth0AdminBackend**: Dedicated authentication backend for admin login via Auth0
  - Validates user exists, is active, and has `is_staff=True`
  - Files: `config/admin_auth/backends.py:18-88`
- **Security hardening**:
  - Open redirect prevention using `url_has_allowed_host_and_scheme()`
  - Host header injection prevention for Auth0 logout `returnTo` URL
  - CSRF protection on all login/logout endpoints
  - Files: `config/admin_auth/views.py:24-89`
- **Professional login template**: Standalone HTML template with Auth0 SDK integration
  - Loading states, error handling, graceful degradation
  - Uses Subresource Integrity (SRI) for CDN-hosted Auth0 SDK
  - **CSP Note**: Template uses inline JavaScript; if Content-Security-Policy is enabled,
    add `script-src 'unsafe-inline'` or implement CSP nonces
  - Files: `opencontractserver/templates/admin/auth0_login.html`
- **Comprehensive test coverage**: 50+ tests covering security edge cases
  - Open redirect prevention, boolean claim parsing, logout URL safety
  - Files: `opencontractserver/tests/test_admin_auth.py`

### Fixed

- **Admin token handling**: Admin login no longer accepts JWT tokens via query parameters (reduces CSRF/token leakage risk). Files: `config/admin_auth/views.py:146-179`
- **Admin claims demotion**: Missing or invalid admin claims now default to False to avoid privilege retention. Files: `config/graphql_auth0_auth/utils.py:331-411`
- **Token storage scope**: Admin Auth0 SPA client now uses in-memory token storage instead of localStorage. Files: `opencontractserver/templates/admin/auth0_login.html:249-257`

#### Runtime-Configurable Pipeline Settings (Superuser Only)

- **PipelineSettings singleton model**: Database-backed configuration for document processing pipeline
  - Stores preferred parsers, embedders, and thumbnailers per MIME type
  - Stores parser-specific kwargs and component settings overrides
  - Database is the single source of truth at runtime (no Django settings fallback)
  - Singleton pattern: only one instance exists, cannot be deleted
  - Files: `opencontractserver/documents/models.py:734-1140`
- **Encrypted secrets storage**: Secure storage for API keys and sensitive credentials
  - Uses Fernet symmetric encryption (key derived from Django SECRET_KEY)
  - Secrets are never exposed via GraphQL responses
  - GraphQL only returns list of components that have secrets configured
  - Methods: `set_secrets()`, `get_secrets()`, `update_secrets()`, `get_full_component_settings()`
  - Files: `opencontractserver/documents/models.py:1012-1139`
- **GraphQL query `pipelineSettings`**: Any authenticated user can read current pipeline configuration
  - Returns preferred components, parser kwargs, component settings
  - Includes `componentsWithSecrets` field (list of paths, not actual secrets)
  - Includes audit fields (modified, modified_by)
  - Files: `config/graphql/queries.py:4214-4250`
- **GraphQL mutation `updatePipelineSettings`**: Superusers can modify pipeline configuration at runtime
  - Validates component class paths exist in the pipeline registry
  - Tracks who made changes (modified_by field)
  - Changes take effect immediately for new document processing tasks
  - Files: `config/graphql/pipeline_settings_mutations.py:20-220`
- **GraphQL mutation `resetPipelineSettings`**: Superusers can reset to Django settings defaults
  - Restores all values from PREFERRED_PARSERS, PREFERRED_EMBEDDERS, etc.
  - Files: `config/graphql/pipeline_settings_mutations.py:223-302`
- **GraphQL mutation `updateComponentSecrets`**: Superusers can securely store API keys per component
  - Accepts component path and secrets dict, encrypts and stores in database
  - Supports merge mode (add to existing) or replace mode
  - Files: `config/graphql/pipeline_settings_mutations.py:305-411`
- **GraphQL mutation `deleteComponentSecrets`**: Superusers can remove secrets for a component
  - Files: `config/graphql/pipeline_settings_mutations.py:414-481`
- **Migration 0031**: Creates PipelineSettings table with encrypted_secrets field
  - Files: `opencontractserver/documents/migrations/0031_add_pipeline_settings.py`
- **Migration 0032**: Adds database index to `PipelineSettings.modified` for audit query performance
  - Files: `opencontractserver/documents/migrations/0032_add_index_to_pipeline_settings_modified.py`
- **Management command `migrate_pipeline_settings`**: Self-documenting component discovery and settings migration
  - `--list-components`: Introspects pipeline registry to show all components with settings schemas, env vars, defaults, and descriptions
  - `--sync-preferences`: Syncs PREFERRED_PARSERS, PREFERRED_EMBEDDERS, etc. from Django settings to database
  - `--component <name>`: Filters output to a specific component
  - Files: `opencontractserver/documents/management/commands/migrate_pipeline_settings.py`
- **Pipeline Configuration Guide**: Documentation covering first-time setup, upgrades, runtime configuration, and troubleshooting
  - Files: `docs/pipelines/pipeline_configuration.md`
- **Integration with doc_tasks**: `ingest_doc` and `extract_thumbnail` now read from PipelineSettings
  - Files: `opencontractserver/tasks/doc_tasks.py:355-374`, `opencontractserver/tasks/doc_tasks.py:686-721`
- **Integration with pipeline/utils**: `get_preferred_embedder` and `get_default_embedder` use PipelineSettings
  - Files: `opencontractserver/pipeline/utils.py:303-361`

### Changed

- Pipeline settings getters (`get_preferred_parser`, `get_preferred_embedder`, `get_parser_kwargs`, `get_default_embedder`) no longer fall back to Django settings at runtime. Database is the sole source of truth; initial values are populated from Django settings via `get_instance()`.
  - Files: `opencontractserver/documents/models.py:977-1092`
- Pipeline component settings are now DB-only at runtime (Django settings fallback removed from `PipelineComponentBase.get_component_settings()`)
  - Files: `opencontractserver/pipeline/base/base_component.py:180-217`
- Pipeline configuration UI now reads component settings schema from GraphQL instead of hardcoding config requirements.
  - Files: `frontend/src/components/admin/SystemSettings.tsx:69-129`
- Pipeline configuration UI now centralizes pipeline UI constants in `PIPELINE_UI` for sizing and validation values.
  - Files: `frontend/src/assets/configurations/constants.ts:174-187`

### Fixed

- Secrets modal now validates component existence, required secret fields, and payload size before mutation.
  - Files: `frontend/src/components/admin/SystemSettings.tsx:1500-1558`
- MIME filter accessibility labels now include stage context.
  - Files: `frontend/src/components/admin/SystemSettings.tsx:1076-1089`
- GraphQL `updateComponentSecrets` mutation now validates payload size before encryption attempt.
  - Files: `config/graphql/pipeline_settings_mutations.py:104-135`
- Pipeline components query now requires authentication; non-superusers only see configured components without settings schema details.
  - Files: `config/graphql/queries.py:1815-1949`
- Settings schema `_coerce_value` now logs warnings on coercion failures instead of silently swallowing errors.
  - Files: `opencontractserver/pipeline/base/settings_schema.py:416-419`
- `PipelineSettings.save()` now invalidates cache via `transaction.on_commit()` to prevent stale cache when DB write rolls back.
  - Files: `opencontractserver/documents/models.py:896-906`
- `PipelineComponentCard` memo now uses custom comparison to avoid unnecessary re-renders from object prop references.
  - Files: `frontend/src/components/admin/SystemSettings.tsx:1134-1140`
- Pipeline mutation error messages now consistently include component path for debugging.
  - Files: `config/graphql/pipeline_settings_mutations.py:644-656, 720-729`

### Removed

#### ModernBERT Embedders

- **⚠️ Breaking Change**: ModernBERT embedders have been removed from the codebase
  - `opencontractserver/pipeline/embedders/modern_bert_embedder.py` - removed
  - `opencontractserver/pipeline/embedders/minn_modern_bert_embedder.py` - removed
  - `model_preloaders/download_modernbert_model.py` - removed
  - Tests removed: `opencontractserver/tests/test_modern_bert_embedder.py`, `opencontractserver/tests/test_minn_modern_bert_embedder.py`
  - Documentation removed: `docs/embedders/modernbert_embedder.md`, `docs/embedders/minn_modernbert_embedder.md`
  - **Migration path**: Users currently using ModernBERT embedders must switch to alternative embedders:
    - `SentenceTransformerEmbedder` - General purpose sentence transformer embeddings
    - `OpenAIEmbedder` - OpenAI API-based embeddings (requires API key)
    - `VoyageAIEmbedder` - Voyage AI embeddings (requires API key)
  - Update PipelineSettings via admin UI or management command before upgrading

#### Personal Corpus ("My Documents") Feature

- **Personal corpus auto-creation**: Each user now automatically receives a personal "My Documents" corpus
  - Created via signal handler when user account is created
  - Uses database constraint to ensure one personal corpus per user (`one_personal_corpus_per_user`)
  - Personal corpus is private (`is_public=False`) and grants full permissions to owner
  - Files: `opencontractserver/corpuses/models.py:378-432`, `opencontractserver/users/signals.py:16-48`
- **All uploads default to personal corpus**: Documents without a specified corpus go to "My Documents"
  - Single file uploads via GraphQL now route to personal corpus
  - Zip file bulk uploads also default to personal corpus when no corpus specified
  - Files: `config/graphql/mutations.py:1807-1908`, `opencontractserver/tasks/import_tasks.py:514-585`
- **`Corpus.get_or_create_personal_corpus()` class method**: Idempotent method to get/create personal corpus
  - Thread-safe using `get_or_create` with atomic transaction
  - Grants full permissions on creation
  - Files: `opencontractserver/corpuses/models.py:390-432`
- **Data migration for existing users**: Migration creates personal corpuses for existing users and moves standalone documents
  - Creates "My Documents" corpus for all active users
  - Moves documents without any DocumentPath to their creator's personal corpus
  - **⚠️ IRREVERSIBLE MIGRATION**: This migration cannot be rolled back automatically. Attempting to reverse will raise `NotImplementedError`. Rolling back would delete DocumentPath records and orphan user documents from their corpus organization. If rollback is required, a custom migration must be written to handle data preservation.
  - Files: `opencontractserver/corpuses/migrations/0038_create_personal_corpuses.py`

#### Shared StructuralAnnotationSet

- **Reuse structural sets instead of duplicating**: `add_document()` now reuses the source document's structural set
  - Previously, adding a document to a corpus duplicated the entire StructuralAnnotationSet
  - Now shares the set, reducing storage and maintaining single source of truth
  - Files: `opencontractserver/corpuses/models.py:528-535`
- **Incremental embedding creation**: New Celery task checks for missing embeddings when document is added
  - `ensure_embeddings_for_corpus()` checks if embeddings exist for corpus's required embedders
  - Only queues embedding generation for annotations missing embeddings
  - Supports both DEFAULT_EMBEDDER and corpus's preferred_embedder
  - Files: `opencontractserver/tasks/corpus_tasks.py:712-850`

#### Inline Thread View with Corpus Context Sidebar

- **Added inline thread viewing**: Users can now view thread details inline within the Discussions tab instead of navigating away
  - Click a thread to view details in-place with a "Back" button to return to the list
  - Thread state tracked via `inlineSelectedThreadIdAtom` Jotai atom
  - Files: `frontend/src/components/discussions/CorpusDiscussionsView.tsx`, `frontend/src/atoms/threadAtoms.ts`
- **Added corpus context sidebar**: Displays corpus context alongside thread details
  - About section with corpus description (markdown rendered)
  - Documents section with collapsible table of contents
  - Quick stats grid (documents, threads, annotations, comments)
  - Collapsible sections with smooth animations via Framer Motion
  - Responsive behavior: hidden < 1024px, collapsible 1024-1200px, always expanded > 1200px
  - Sidebar expanded state persisted to localStorage via `threadContextSidebarExpandedAtom`
  - Files: `frontend/src/components/threads/CorpusContextSidebar.tsx`, `frontend/src/components/threads/ThreadDetailWithContext.tsx`
  - New styled components: `frontend/src/components/threads/styles/contextSidebarStyles.ts`
- **Added modernized discussion thread UI**: Comprehensive redesign following OS-Legal-Style design system
  - Typography-first design: Serif headings (Georgia), sans-serif body (Inter)
  - Teal accent color scheme (#0f766e) for interactive elements
  - Improved message cards, vote buttons, badges, and metadata displays
  - Mobile-responsive with proper breakpoints
  - Files: `frontend/src/components/threads/styles/discussionStyles.ts` (950+ lines)
- **Added agent mention rendering**: Discussion messages render styled agent mentions with custom colors
  - Runtime validation of badge configuration from GenericScalar fields
  - Hex color validation with fallback to default agent color
  - Tooltip display for agent mentions
  - Files: `frontend/src/components/threads/MarkdownMessageRenderer.tsx`
- **Added component tests for new features**:
  - Mention badge rendering tests: `frontend/tests/MentionRendering.ct.tsx`
  - Compact vote button tests: `frontend/tests/VoteButtonsCompact.ct.tsx`

### Technical Details

- Added `is_personal` BooleanField to Corpus model with database constraint
- Added composite index on `(creator, is_personal)` for efficient lookups
- Schema migration: `opencontractserver/corpuses/migrations/0037_add_is_personal_corpus.py`
- Comprehensive test suite: `opencontractserver/tests/test_personal_corpus.py` (14 tests)

#### AnnotationsPanel Shared Component

- **Created `AnnotationsPanel` reusable component**: Extracts shared filtering/display logic from annotations views
  - Provides filter tabs for type (All/Doc/Text) and source (All/Human/Agent/Structural)
  - Includes SearchBox, grid display with ModernAnnotationCard, empty state, and pagination
  - Can be used by both standalone Annotations view and corpus annotations tab
  - Files: `frontend/src/components/annotations/AnnotationsPanel.tsx`
- **Added `AnnotationsPanel` unit tests**: Comprehensive tests for filters, search, grid, empty/loading states
  - Files: `frontend/src/components/annotations/__tests__/AnnotationsPanel.test.tsx`
- **Added semantic search to corpus annotations tab**: Search box now uses vector similarity search
  - Debounced search triggers semantic search as user types (500ms delay)
  - Displays similarity scores on annotation cards
  - Supports infinite scroll for semantic search results
  - Shows appropriate empty state and loading messages for search mode
  - Files: `frontend/src/components/annotations/CorpusAnnotationCards.tsx`
- **Fixed semantic search similarity score calculation**: Scores now correctly display as percentages
  - CosineDistance returns distance (0=identical), converted to similarity (1=identical)
  - Results are sorted by similarity (highest first) and scores display correctly (e.g., 85% for close matches)
  - Files: `opencontractserver/shared/mixins.py`
- **Created lightweight `GET_ANNOTATIONS_FOR_CARDS` query**: Fetches only fields needed for ModernAnnotationCard display
  - Excludes heavy fields: `tokensJsons`, `json`, `page`, and unnecessary nested objects
  - Reduces payload from ~340KB to ~30KB for 2130 annotations (estimated 90% reduction)
  - Files: `frontend/src/graphql/queries.ts`

### Fixed

#### Annotations Panel Scroll Issue

- **Fixed corpus annotations tab scroll behavior**: Restructured AnnotationsPanel to scroll only the cards grid
  - Container uses flex column layout with `overflow: hidden`
  - FiltersSection has `flex-shrink: 0` to stay fixed at top
  - AnnotationsListContainer has `flex: 1` and `overflow-y: auto` for card scrolling
  - Filters (search, type tabs, source tabs) stay visible while cards scroll below
  - Files: `frontend/src/components/annotations/AnnotationsPanel.tsx`

#### Annotations Query Missing Pagination

- **Added initial page limit to annotations queries**: Previously loaded all annotations at once
  - Added `limit` and `cursor` fields to `GetAnnotationsInputs` interface
  - Set initial page size to 20 annotations for both Annotations.tsx and CorpusAnnotationCards.tsx
  - Infinite scroll loads more as user scrolls down
  - Files: `frontend/src/graphql/queries.ts`, `frontend/src/views/Annotations.tsx`, `frontend/src/components/annotations/CorpusAnnotationCards.tsx`

#### Corpus Annotations Tab Source Filter

- **Fixed structural annotations not visible in corpus tab**: Annotations tab now shows filter controls even when empty
  - Added source filter (Human/Agent/Structural) to `CorpusAnnotationCards`
  - Source filter syncs to GraphQL query variables: "structural" → `structural: true`, "human" → `structural: false, analysis_Isnull: true`, "agent" → `structural: false, analysis_Isnull: false`
  - Users can now toggle to see structural annotations that were previously hidden
  - Files: `frontend/src/components/annotations/CorpusAnnotationCards.tsx`
- **Added missing `usesLabelFromLabelsetId` to GetAnnotationsInputs interface**: Interface was missing a field used by the query
  - Files: `frontend/src/graphql/queries.ts:752`

### Changed

#### BREAKING: Removed Corpus.documents M2M Relationship (PR #840)

- **Removed `Corpus.documents` ManyToManyField**: DocumentPath is now the sole source of truth for corpus-document relationships
  - Migration `0039_remove_corpus_documents_m2m` validates no orphaned M2M entries before removal
  - All code paths now use `corpus.add_document()`, `corpus.remove_document()`, `corpus.get_documents()`, `corpus.document_count()`
  - GraphQL `CorpusType.documents` field now resolves via explicit DocumentPath-based resolver
  - Frontend queries updated to use `documentCount` field instead of `documents { totalCount }`
  - Files: `opencontractserver/corpuses/models.py`, `config/graphql/graphene_types.py`, `config/graphql/queries.py`
- **Removed deprecated Corpus methods**: `_create_text_document_internal()` and `create_text_document()` removed (use `import_content()` instead)
  - Removed deprecated `content` parameter from `add_document()` (use `import_content()` for content-based imports)
  - Files: `opencontractserver/corpuses/models.py`
- **Removed `sync_m2m_to_documentpath` management command**: No longer needed after M2M removal
  - Files: `opencontractserver/documents/management/commands/sync_m2m_to_documentpath.py` (deleted)
- **Added request-level caching to DocumentPathType**: Visible corpus IDs now cached per-request to prevent N+1 queries
  - Follows same pattern as `ConversationQueryOptimizer` and `DocumentRelationshipQueryOptimizer`
  - Files: `config/graphql/graphene_types.py:620-636`
- **Fixed stale frontend GraphQL queries**: Two queries still referenced removed `documents { totalCount }` connection field
  - `GET_EDITABLE_CORPUSES` in `AddToCorpusModal.tsx` now uses `documentCount`
  - `GET_MY_CORPUSES` in `queries.ts` now uses `documentCount`
  - Files: `frontend/src/components/modals/AddToCorpusModal.tsx`, `frontend/src/graphql/queries.ts`

#### Pipeline Configuration UI Redesign

- **Replaced JSON-based configuration with visual pipeline flow**: System Settings page redesigned for intuitive configuration
  - Visual pipeline stages: Document Upload → Parser → Thumbnailer → Embedder → Ready for Search
  - Clickable component cards replace JSON text editing
  - Per-stage MIME type selectors (PDF, TXT, DOCX)
  - Auto-expanding advanced settings for components requiring API keys
  - Collapsible advanced settings to reduce visual clutter
  - Files: `frontend/src/components/admin/SystemSettings.tsx`
- **New component icon system**: Custom SVG icons for each pipeline component type
  - Docling, LlamaParse, ModernBERT, OpenAI, and more
  - Semantic icons that are visually distinctive
  - Generic fallback icon for unknown components
  - Files: `frontend/src/components/admin/PipelineIcons.tsx`
- **Added accessibility attributes**: ARIA support for screen readers
  - `aria-pressed` on MIME type and component selection buttons
  - `aria-expanded` on collapsible settings sections
  - `aria-label` on interactive elements
  - Files: `frontend/src/components/admin/SystemSettings.tsx`

#### Window Resize Performance

- **Added debounce to window resize handler**: Prevents excessive re-renders during window resize
  - 150ms debounce delay on resize events
  - Properly cleans up timeout on unmount
  - Files: `frontend/src/components/hooks/WindowDimensionHook.tsx`

#### Annotations View Refactoring

- **Updated Annotations.tsx to use AnnotationsPanel**: DRY refactoring, keeps hero section, stats, semantic search, advanced filters
  - Files: `frontend/src/views/Annotations.tsx`
- **Deleted superseded AnnotationCards.tsx**: Functionality absorbed into AnnotationsPanel
  - Files: `frontend/src/components/annotations/AnnotationCards.tsx` (deleted)

#### Annotations Query Optimization

- **Switched to lightweight query for annotation cards**: Both `Annotations.tsx` and `CorpusAnnotationCards.tsx` now use `GET_ANNOTATIONS_FOR_CARDS`
  - Previous query fetched `tokensJsons` (huge JSON), `json`, full document paths (pdfFile, txtExtractFile, pawlsParseFile), full corpus details
  - New query fetches only: id, created, creator (id, email, username), corpus (id, slug, labelSet.title), document (id, slug, title), annotationLabel (id, text, color, labelType), analysis (id, analyzer.analyzerId), annotationType, structural, rawText, isPublic, contentModalities
  - Expected improvement: ~90% payload reduction, significantly faster load times
  - Files: `frontend/src/views/Annotations.tsx`, `frontend/src/components/annotations/CorpusAnnotationCards.tsx`

### Added

#### GraphQL Corpus Query Optimization

- **Added `documentCount` field to CorpusType**: Efficient document count using annotated subquery instead of N+1 queries
  - For list queries (`corpuses`), the resolver annotates `_document_count` via `DocumentPath` subquery
  - For single corpus queries, falls back to model's `document_count()` method
  - Files: `config/graphql/graphene_types.py:2028-2038`, `config/graphql/queries.py:836-869`
- **Added `annotationCount` field to CorpusType**: Efficient annotation count using annotated subquery
  - For list queries, `resolve_corpuses` annotates `_annotation_count` via Document→DocumentPath join
  - For single corpus queries, falls back to counting via DocumentPath query
  - Files: `config/graphql/graphene_types.py`, `config/graphql/queries.py`
- **Optimized LabelSet count resolvers**: Label counts now use corpus-annotated values when available
  - `resolve_label_set` on CorpusType copies annotated counts to LabelSet instance
  - `resolve_doc_label_count`, `resolve_span_label_count`, `resolve_token_label_count` check for annotations before querying
  - Files: `config/graphql/graphene_types.py:680-699, 2040-2056`
- **Optimized leaderboard `reputationGlobal` resolution**: `resolve_global_leaderboard` now attaches `_reputation_global` to user objects, avoiding N+1 queries when resolving `reputationGlobal`
  - Files: `config/graphql/queries.py`, `config/graphql/graphene_types.py`
- **Added query optimization tests**: Comprehensive tests for `documentCount`, `annotationCount`, and label set optimization
  - Files: `opencontractserver/tests/test_corpus_query_optimization.py`

### Changed

#### DiscoveryLanding GraphQL Query Optimization

- **Removed unused fields from landing page queries**: Eliminates ~39 N+1 queries per landing page load
  - Removed from `GET_DISCOVERY_DATA`: `chatMessages { totalCount }` (unused by ActivitySection), `totalMessages`, `totalThreadsCreated`, `totalAnnotationsCreated` (unused by CompactLeaderboard)
  - Replaced `documents { totalCount }` and `annotations { totalCount }` with `documentCount` and `annotationCount` (efficient subquery-backed fields)
  - Files: `frontend/src/graphql/landing-queries.ts`
- **Updated FeaturedCollections to use optimized count fields**: Uses `documentCount`/`annotationCount` instead of connection `totalCount`
  - Files: `frontend/src/components/landing/FeaturedCollections.tsx`
- **Updated TrendingCorpuses to use optimized count fields**: Uses `documentCount` instead of `documents.totalCount`
  - Files: `frontend/src/components/landing/TrendingCorpuses.tsx`
- **Updated RecentDiscussions to remove chatMessages dependency**: Display "View thread" instead of reply count
  - Files: `frontend/src/components/landing/RecentDiscussions.tsx`

#### Frontend Corpus Query Cleanup

- **Removed unused fields from GET_CORPUSES query**: Reduces payload and eliminates N+1 queries
  - Removed: `preferredEmbedder`, `appliedAnalyzerIds`, `documents.edges`, `annotations.totalCount`
  - Added: `documentCount` (efficient server-side count)
  - Files: `frontend/src/graphql/queries.ts:603-673`
- **Updated CorpusItem to use documentCount**: Uses new field instead of `documents?.edges?.length`
  - Files: `frontend/src/components/corpuses/CorpusItem.tsx:602-605`
- **Updated CorpusListView formatStats function**: Uses `documentCount` and removes annotation count display
  - Files: `frontend/src/components/corpuses/CorpusListView.tsx:303-306`
- **Added documentCount and annotationCount to TypeScript types**: Updated `RawCorpusType` interface
  - Files: `frontend/src/types/graphql-api.ts`

### Technical Details

- **Query reduction**: DiscoveryLanding page goes from ~39 N+1 queries to ~0 extra queries (all counts resolved via subqueries or removed)
- **Backward compatibility**: All new fields (`documentCount`, `annotationCount`) gracefully fall back to model methods for single corpus queries
- **Pattern**: Label counts are passed from corpus to label_set via instance attribute injection in `resolve_label_set`
- **Pattern**: Leaderboard reputation score is pre-attached to user objects via `_reputation_global` attribute

### Added

#### 2048-Dimensional Embedding Support

- **Added vector_2048 field to Embedding model**: Support for 2048-dimensional embeddings used by newer embedding models
  - Migration 0061 adds nullable `vector_2048` column to `annotations_embedding` table
  - Files: `opencontractserver/annotations/models.py:470`, `opencontractserver/annotations/migrations/0061_add_vector_2048.py`
- **Updated dimension handling across codebase**:
  - `Managers.py:_get_vector_field_name` returns "vector_2048" for 2048-dim vectors (lines 363-364)
  - `mixins.py:_dimension_to_field` returns embedding relation for 2048-dim (lines 37-38)
  - `mixins.py:get_embedding` retrieves 2048-dim vectors (lines 144-145)
  - Vector stores validate 2048 as supported dimension
  - Files: `opencontractserver/shared/Managers.py`, `opencontractserver/shared/mixins.py`, `opencontractserver/llms/vector_stores/core_vector_stores.py`, `opencontractserver/llms/vector_stores/core_conversation_vector_stores.py`

#### Multimodal Embedder Refactoring

- **Refactored MultimodalMicroserviceEmbedder into inheritance hierarchy**:
  - `BaseMultimodalMicroserviceEmbedder`: Abstract base class with shared multimodal embedding logic
  - `CLIPMicroserviceEmbedder`: CLIP ViT-L-14 model (768 dimensions) with backwards-compatible legacy settings
  - `QwenMicroserviceEmbedder`: Qwen embedding model (1024 dimensions)
  - Files: `opencontractserver/pipeline/embedders/multimodal_microservice.py`
- **Added model-specific settings**: `CLIP_EMBEDDER_URL`, `CLIP_EMBEDDER_API_KEY`, `QWEN_EMBEDDER_URL`, `QWEN_EMBEDDER_API_KEY`
  - Files: `config/settings/base.py:666-669`
- **Deprecated legacy settings**: `MULTIMODAL_EMBEDDER_URL` and `MULTIMODAL_EMBEDDER_API_KEY` still work but emit deprecation warnings
  - Users should migrate to `CLIP_EMBEDDER_URL` / `CLIP_EMBEDDER_API_KEY`

### Fixed

#### MicroserviceEmbedder Reliability

- **Fixed MicroserviceEmbedder production failures**: Added Content-Type header and 30s timeout to prevent silent failures
  - Files: `opencontractserver/pipeline/embedders/sent_transformer_microservice.py:522, 530`

### Added

#### Bulk Document Selection and Removal

- **Bulk document selection in folder toolbar**: New Select All / Deselect All functionality for corpus documents
  - Selection count display showing "X of Y" documents selected
  - Clear selection button to deselect all
  - Selection state persists across folder navigation for building cross-folder selections
  - Files: `frontend/src/components/corpuses/folders/FolderToolbar.tsx:780-827`
- **Bulk remove from corpus action**: Remove multiple selected documents from corpus in one operation
  - Dedicated danger button with document count indicator
  - Proper confirmation modal (replaces browser `window.confirm`)
  - Files: `frontend/src/components/corpuses/folders/RemoveDocumentsModal.tsx`, `frontend/src/components/corpuses/folders/FolderDocumentBrowser.tsx:367-371`
- **Mobile-responsive bulk actions**: Kebab menu includes selection controls for tablet/mobile viewports
  - Files: `frontend/src/components/corpuses/folders/FolderToolbar.tsx:976-993`
- **Loading state handling**: Select All button disabled while documents are loading to prevent incomplete selections
  - New `documentsLoading` reactive var syncs loading state from CorpusDocumentCards to FolderToolbar
  - Files: `frontend/src/graphql/cache.ts:379-384`, `frontend/src/components/documents/CorpusDocumentCards.tsx:193-201`

### Fixed

#### Embedder Error Handling and Response Parsing (PR #828)

- **Fixed silent embedding failures**: Added `EmbeddingGenerationError` exception class that triggers Celery task retries when default embeddings fail
  - Default embedding failures now properly raise and retry (up to 3 times with 60s delay)
  - Corpus-specific embedding failures are logged but don't fail the task (non-fatal)
  - Files: `opencontractserver/tasks/embeddings_task.py:165-273`
- **Fixed 1D vs 2D array response parsing**: Embedders now handle both array formats from embedding services
  - Some services return `[0.1, 0.2, ...]` (1D), others return `[[0.1, 0.2, ...]]` (2D batch format)
  - Previously caused "object of type 'float' has no len()" errors
  - Files: `opencontractserver/pipeline/embedders/sent_transformer_microservice.py:113-119`, `opencontractserver/pipeline/embedders/multimodal_microservice.py:195-201`
- **Fixed bytes-to-string decoding**: Added workaround for storage backends that return bytes even in text mode
  - Affects django-storages S3Boto3Storage with certain configurations
  - Previously caused "bytes not JSON serializable" errors
  - Files: `opencontractserver/tasks/embeddings_task.py:306-314`
- **Aligned error handling across embedders**: MicroserviceEmbedder now distinguishes 4xx (client) vs 5xx (server) errors like MultimodalMicroserviceEmbedder
  - Files: `opencontractserver/pipeline/embedders/sent_transformer_microservice.py:120-133`
- **Added comprehensive test coverage**: 18 new tests for error handling, bytes decoding, and array format parsing
  - Files: `opencontractserver/tests/test_embeddings_task.py`
- **Added TestEmbedder for fast, deterministic test embeddings**: Tests now use a fast in-memory embedder by default instead of the HTTP-based MicroserviceEmbedder
  - Returns deterministic fake vectors based on text hash (same text = same embedding)
  - Eliminates HTTP round-trips to vector-embedder service during tests (faster test execution)
  - Integration tests that need real service connectivity should explicitly instantiate MicroserviceEmbedder
  - Files: `opencontractserver/pipeline/embedders/test_embedder.py`, `config/settings/test.py:120-134`

#### Cache Eviction Consistency

- **Fixed folder document counts not updating after bulk removal**: Added `corpusFolders` cache eviction to `REMOVE_DOCUMENTS_FROM_CORPUS` mutation to match the pattern used by `MOVE_DOCUMENT_TO_FOLDER`
  - Files: `frontend/src/components/corpuses/folders/RemoveDocumentsModal.tsx:109-112`

#### Duplicate Tool Registration and Caller Tool Precedence

- **Fixed duplicate tool registration error in PydanticAI agent**: Resolved `pydantic_ai.exceptions.UserError` when caller-provided tools have the same name as default tools
  - Files: `opencontractserver/llms/agents/pydantic_ai_agents.py:2063-2082`
- **Caller-provided tools now take precedence over defaults**: When a caller passes a tool with the same name as a built-in default, the caller's tool configuration (description, requires_approval, etc.) is now used instead of silently dropping it
  - Allows callers to override tool behavior and configurations
  - Applies to both `PydanticAIDocumentAgent.create()` and `structured_response()`
  - Added info-level logging when caller tools override defaults
  - Files: `opencontractserver/llms/agents/pydantic_ai_agents.py:961-992`, `opencontractserver/llms/agents/pydantic_ai_agents.py:2063-2082`
- **Added comprehensive test coverage**:
  - `test_caller_tool_overrides_default_configuration` verifies caller's tool is used (not default)
  - `test_config_tools_deduplicated_in_structured_response` covers the config.tools path
  - Files: `opencontractserver/tests/test_duplicate_tool_registration.py`
- **Fixed PydanticAICorpusAgent consistency**: Now uses same caller-precedence pattern as document agent
- **Extracted `deduplicate_tools()` utility**: DRY refactor moves repeated deduplication logic to reusable function
  - Checks both `__name__` and `name` attributes for tool identification
  - Filters out `None` values to handle tools without names
  - Includes security documentation in docstring
  - Files: `opencontractserver/utils/tools.py`
- **Added documentation for tool precedence**: New section in LLM docs explaining when conflicts occur, which configuration wins, and security considerations
  - Files: `docs/architecture/llms/README.md`

### Added

#### Document Processing Pipeline Hardening (PR #824)

- **Document processing status tracking**: New `DocumentProcessingStatus` enum with PENDING, PROCESSING, COMPLETED, FAILED states
  - `processing_status` field on Document model with database index
  - `processing_error` and `processing_error_traceback` fields for failure diagnostics
  - Files: `opencontractserver/documents/models.py:24-32`, `opencontractserver/documents/models.py:141-159`
- **Typed parsing exceptions**: New `DocumentParsingError` with `is_transient` flag
  - Transient errors (network timeouts, service unavailable) trigger automatic retry
  - Permanent errors (invalid file, no parser) fail immediately
  - Files: `opencontractserver/pipeline/base/exceptions.py`
- **Automatic retry with exponential backoff**: Up to 3 retries with 60-300s backoff and jitter
  - Failed documents remain locked (`backend_lock=True`) to prevent broken state
  - Files: `opencontractserver/tasks/doc_tasks.py:287-435`
- **Manual retry via GraphQL**: New `RetryDocumentProcessing` mutation
  - Allows users to retry failed documents after infrastructure issues are resolved
  - Atomic state reset prevents race conditions from multiple retry clicks
  - Files: `config/graphql/mutations.py:2244-2330`
- **Failure notifications**: New `DOCUMENT_PROCESSING_FAILED` notification type
  - Notifies document creator when processing fails
  - Files: `opencontractserver/tasks/doc_tasks.py:113-146`, `opencontractserver/notifications/models.py`
- **Processing status constants**: Centralized in `opencontractserver/constants/document_processing.py`
- **24 unit tests**: Comprehensive coverage of new functionality
  - Files: `opencontractserver/tests/test_pipeline_hardening.py`

#### Bifurcated Conversation Permissions (CHAT vs THREAD)

- **New `conversation_type` field on Conversation model**: Distinguishes between personal agent chats and collaborative discussions
  - `CHAT` type: Restrictive permissions (creator + explicit permissions + public only)
  - `THREAD` type: Context-based permissions (inherits visibility from linked corpus/document)
  - Files: `opencontractserver/conversations/models.py:51-53`, `opencontractserver/conversations/migrations/`
- **Bifurcated `visible_to_user()` queryset method**: Different visibility logic based on conversation type
  - CHAT: Only creator, explicit guardian permissions, or public flag
  - THREAD: CHAT rules + context inheritance (READ on corpus AND/OR document)
  - AND logic when both corpus and document are linked (must have READ on both)
  - Files: `opencontractserver/conversations/models.py:127-238`
- **ConversationQueryOptimizer helper class**: Request-level caching to avoid N+1 queries
  - Caches visible conversation IDs per request
  - IDOR-safe `check_conversation_visibility()` method for mutations
  - Convenience methods: `get_threads_for_corpus()`, `get_threads_for_document()`, `get_chats_for_user()`
  - Files: `opencontractserver/conversations/query_optimizer.py`
- **ChatMessage visibility inheritance**: Messages inherit bifurcated permissions from parent conversation
  - Moderator access retained for corpus/document owners
  - Files: `opencontractserver/conversations/models.py:299-398`
- **22 new permission tests**: Comprehensive coverage of CHAT vs THREAD behavior
  - Files: `opencontractserver/tests/test_conversation_permissions.py`
- **Updated permissioning guide**: Documentation for bifurcated model with examples
  - Files: `docs/permissioning/consolidated_permissioning_guide.md`

#### Corpus Forking: Folder and Relationship Preservation

- **Folder hierarchy preservation during fork**: Forked corpora now maintain the complete folder structure
  - Folders cloned in tree-depth order to preserve parent-child relationships
  - Documents retain their folder assignments in the forked corpus
  - Uses `tree_queries` CTE with `.with_tree_fields()` for proper ordering
  - Files: `opencontractserver/tasks/fork_tasks.py:126-159`, `config/graphql/mutations.py:1180-1187`
- **Relationship preservation during fork**: Annotation relationships are now copied
  - Source and target annotations remapped to forked annotation IDs
  - Relationship labels preserved via label_map
  - Uses `prefetch_related()` for efficient M2M loading
  - Files: `opencontractserver/tasks/fork_tasks.py:286-356`
- **Fork task signature extended**: Added `folder_ids` and `relationship_ids` parameters
  - Files: `opencontractserver/tasks/fork_tasks.py:29-38`
- **Round-trip test suite**: Comprehensive tests validating fork data integrity across generations
  - Files: `opencontractserver/tests/test_corpus_fork_round_trip.py`

#### Corpus-Scoped MCP Endpoints for Shareable Links

- **New `/mcp/corpus/{corpus_slug}/` endpoint**: Scoped MCP endpoint for single-corpus access
  - All tools automatically operate within the specified corpus context
  - No need for explicit `corpus_slug` parameters in tool calls
  - Validates corpus exists and is publicly accessible before accepting requests
  - Returns 404 with helpful message for private/nonexistent corpuses
  - Files: `opencontractserver/mcp/server.py:371-651`, `config/asgi.py`
- **New `get_corpus_info` tool**: Returns detailed corpus information for scoped endpoints
  - Replaces `list_public_corpuses` for scoped context
  - Includes label set information, document count, and metadata
  - Files: `opencontractserver/mcp/tools.py:401-447`
- **Scoped tool wrappers**: Auto-inject corpus_slug into existing tools
  - Creates corpus-specific tool handlers that wrap global tool implementations
  - Files: `opencontractserver/mcp/tools.py:450-498`
- **TTL-based cache with eviction**: Scoped session managers cached with 1-hour TTL
  - LRU eviction at 100 entries to prevent unbounded memory growth
  - Cache invalidation logging for monitoring
  - Files: `opencontractserver/mcp/server.py:583-630`
- **Comprehensive test coverage**: 15+ tests for scoped endpoint functionality
  - Tests for validation, tool execution, cache behavior, error handling
  - Files: `opencontractserver/mcp/tests/test_mcp.py`
- **Updated MCP documentation**: Usage examples for both global and scoped endpoints
  - Files: `docs/mcp/README.md`

#### Unified Upload Modal with @os-legal/ui Design System

- **Consolidated `BulkUploadModal` and `DocumentUploadModal`** into single `UploadModal` component
  - Auto-detects upload mode: ZIP files → bulk mode, PDFs → single mode
  - Multi-step wizard for single mode (Select → Details → Corpus)
  - Simplified single-step flow for bulk ZIP uploads
  - Files: `frontend/src/components/widgets/modals/UploadModal/`
- **Custom hooks for upload state management**:
  - `useUploadState` - file list and selection state
  - `useUploadMutations` - GraphQL mutations with consistent `makePublic` handling
  - `useCorpusSearch` - debounced corpus search with permission filtering
  - Files: `frontend/src/components/widgets/modals/UploadModal/hooks/`
- **Modular sub-components**: `FileDropZone`, `FileList`, `FileDetailsForm`, `CorpusSelectorCard`, `StepIndicator`, `UploadProgress`
- **File size validation**: 100MB limit with user feedback (configurable via `UPLOAD.MAX_FILE_SIZE_BYTES` constant)
- **16 Playwright component tests** covering both modes, validation, and mobile responsiveness
- **Manual test documentation**: `docs/manual-test-scripts/upload-modal.md`
- **Upload constants**: Added `UPLOAD.MAX_FILE_SIZE_BYTES` and `DEBOUNCE.CORPUS_SEARCH_MS` to `frontend/src/assets/configurations/constants.ts`

#### Pre-extracted Image Content for Annotations

- **`image_content_file` FileField on Annotation model**: Stores pre-extracted image data as JSON files
  - Eliminates need to reload full PAWLs file (~10MB) for each image embedding request
  - Performance improvement: ~10-20x faster for image annotation embeddings
  - Files: `opencontractserver/annotations/models.py:109-114`
  - Migration: `opencontractserver/annotations/migrations/0060_add_annotation_image_content_file.py`
- **Batch image extraction utilities**: Efficient batch processing during annotation creation
  - `extract_and_store_annotation_images()` - extracts images from PAWLs and stores as JSON
  - `batch_extract_annotation_images()` - batch processes multiple annotations sharing PAWLs data
  - Files: `opencontractserver/utils/multimodal_embeddings.py:351-502`
- **Unique constraints on Embedding model**: Database-level prevention of duplicate embeddings
  - Migration: `opencontractserver/annotations/migrations/0059_add_embedding_unique_constraints.py`

#### Corpus-Specific Embeddings

- **Dual embedding strategy**: Creates both default (global search) and corpus-specific embeddings
  - Default embedder for cross-corpus search compatibility
  - Corpus-preferred embedder for corpus-specific semantic search
  - Files: `opencontractserver/tasks/embeddings_task.py:88-160`
- **Corpus ID propagation through ingestion chain**: Parser now receives corpus context
  - Enables corpus-specific embeddings during document ingestion
  - Files: `opencontractserver/tasks/doc_tasks.py:203-248`, `opencontractserver/pipeline/base/parser.py:130-143`

### Fixed

- **Corpus title not getting [FORK] prefix**: Fixed f-string that did nothing (`f"{corpus.title}"` → `f"[FORK] {corpus.title}"`)
  - Files: `config/graphql/mutations.py:1199`
- **tree_depth ordering error**: Removed explicit `order_by("tree_depth", "pk")` and rely on default `tree_ordering` from `with_tree_fields()`. The `tree_depth` field is CTE-computed and only available at SQL execution time, not during Django's `order_by()` validation.
  - Files: `config/graphql/mutations.py:1184`, `opencontractserver/tasks/fork_tasks.py:133`, `opencontractserver/utils/corpus_forking.py:46`, `opencontractserver/tests/test_corpus_fork_round_trip.py:389`
- **Fork fails with corpuses without label_set**: Added conditional handling to skip label set cloning when corpus has no label_set
  - Files: `opencontractserver/tasks/fork_tasks.py:56-136`
- **Document slug uniqueness violation during fork**: Clear slug before saving forked document so save() generates a new unique slug
  - Files: `opencontractserver/tasks/fork_tasks.py:186`
- **Annotation label mapping error**: Gracefully handle annotations without labels or when label_map is empty
  - Files: `opencontractserver/tasks/fork_tasks.py:279-285`
- **Test assertion bug**: Fixed comparison of count to queryset (`forked_labelset_labels.count() == original_labelset_labels.all()` → `.count() == .count()`)
  - Files: `opencontractserver/tests/test_corpus_forking.py:99`
- **Incorrect CorpusFolder permission setting in tests**: Removed `set_permissions_for_obj_to_user()` call for folders - CorpusFolder inherits permissions from parent Corpus, not individual permissions per the consolidated permissioning guide
  - Files: `opencontractserver/tests/test_corpus_fork_round_trip.py:277`
- **Critical: Infinite loop in corpus document copies**: Fixed chain reaction where corpus copies triggered re-ingestion
  - **Root Cause**: `add_document()` created corpus copies without setting `processing_started`, causing the ingestion signal to fire on each copy
  - **Impact**: Uploading one document created infinite chain of copies (doc → copy → copy of copy → ...)
  - **Fix**: Set `processing_started=timezone.now()` on corpus copies to prevent signal from firing
  - **Files**: `opencontractserver/corpuses/models.py:478-481`
- **Multimodal embeddings for structural annotations**: Fixed PAWLs loading from `structural_set.pawls_parse_file`
  - Structural annotations now correctly load images for embedding generation
  - Files: `opencontractserver/utils/multimodal_embeddings.py:136-166`
- **Embedding duplicate constraint violations with race condition handling**
  - **Root Cause**: Parallel Celery workers could create duplicate embeddings due to race conditions between check and create
  - **Fix**: Added `IntegrityError` catch in `store_embedding()` to handle race conditions atomically
  - **Fix**: Migration 0059 now cleans up existing duplicates before adding unique constraints (keeps best embedding per group)
  - **Fix**: Migration uses `atomic=False` to avoid PostgreSQL "pending trigger events" error
  - **Fix**: Removed `visible_to_user()` filtering from existence checks (constraints apply globally)
  - **Files**: `opencontractserver/shared/Managers.py:369-442`, `opencontractserver/annotations/migrations/0059_add_embedding_unique_constraints.py`

### Changed

- **Permission consistency**: Utility function `build_fork_corpus_task()` now uses `PermissionTypes.CRUD` (was `ALL`) to match mutation
  - Files: `opencontractserver/utils/corpus_forking.py:69`
- **`BulkUploadModal`** is now a thin wrapper: `<UploadModal forceMode="bulk" />`
- **`DocumentUploadModal`** is now a thin wrapper: `<UploadModal forceMode="single" />`
- **Image retrieval uses fast path**: Both REST API and embedding tasks check `image_content_file` first
  - Falls back to PAWLs loading only for legacy annotations without pre-extracted images
  - Files: `opencontractserver/llms/tools/image_tools.py:281-349`, `opencontractserver/utils/multimodal_embeddings.py:101-127`
- **`import_annotations()` accepts `pawls_data` parameter**: Enables batch image extraction during import
  - Files: `opencontractserver/utils/importing.py:58-150`
- **`StructuralAnnotationSet.duplicate()` copies image files**: Preserves pre-extracted images during corpus isolation
  - Files: `opencontractserver/annotations/models.py:705-745`
-

### Technical Details

- Documentation consolidated from separate remediation/edit plan files into `docs/architecture/corpus_forking.md`
- Removed obsolete files: `corpus_forking_edit_plan.md`, `corpus_forking_remediation_plan.md`
- Migrated from Semantic UI to `@os-legal/ui` design system (Modal, Button, Input, Progress, etc.)
- Uses `--oc-*` CSS design tokens for consistent theming
- Debounce cleanup on unmount to prevent memory leaks
- Sequential uploads to avoid server overload (documented trade-off)

### Removed

- **Deleted unused components**: `DocumentUploadList.tsx`, `DocumentListItem.tsx`

### Security

- **JWT authentication error message hardening** (CWE-209: Information Exposure Through Error Messages)
  - JWT errors now return generic messages (`"Invalid token"`) instead of exposing exception details
  - Detailed errors logged server-side only for debugging
  - Files: `config/rest_jwt_auth.py:80-90`
- **Sensitive data redaction in logs** (CWE-532: Insertion of Sensitive Information into Log File)
  - New `redact_sensitive_kwargs()` utility recursively redacts API keys, secrets, passwords, tokens, credentials
  - Applied to parser, embedder, and post-processor kwargs logging
  - Files: `opencontractserver/utils/logging.py`, `opencontractserver/tasks/doc_tasks.py`,
    `opencontractserver/pipeline/base/embedder.py`, `opencontractserver/pipeline/base/post_processor.py`,
    `opencontractserver/pipeline/parsers/llamaparse_parser.py`, `opencontractserver/pipeline/post_processors/pdf_redactor.py`

### Added

#### Image Annotation Display in UnifiedContentFeed

- **Modality badges for annotations**: Visual indicators showing TEXT, IMAGE, or MIXED modalities
  - Color-coded badges: Blue (text), Orange (image), Purple (mixed)
  - Integrated inline with annotation labels in HighlightItem
  - Files: `frontend/src/components/annotator/sidebar/ModalityBadge.tsx`
- **Image thumbnail previews**: Display image content directly in annotation feed
  - 80x80px thumbnails with hover effects and lazy loading
  - Automatic fetching only when IMAGE modality is present
  - Files: `frontend/src/components/annotator/sidebar/AnnotationImagePreview.tsx`
- **REST API endpoint for annotation images**: `/api/annotations/<id>/images/`
  - Permission-checked image retrieval using existing `get_annotation_images_with_permission()`
  - IDOR protection: Returns empty array for unauthorized access
  - Files: `opencontractserver/annotations/views.py`, `config/urls.py`
- **Unified JWT authentication utility**: Single entry point for token validation across all API surfaces
  - Automatic handling of both Auth0 (RS256/JWKS) and standard graphql_jwt (HS256) tokens
  - DRY architecture eliminates conditional Auth0/non-Auth0 switching in multiple files
  - Files: `config/jwt_utils.py` (NEW)
- **GraphQL content_modalities field exposure**: Added to AnnotationType schema
  - Enables frontend to filter annotations by modality
  - Files: `config/graphql/graphene_types.py`

### Fixed

- **Image annotations now clearly visible**: Image and mixed-modality annotations display properly in UnifiedContentFeed
  - Previously showed as empty text with no indication of content
  - Files: `frontend/src/components/annotator/sidebar/HighlightItem.tsx:163-167,225,249`
  - Files: `frontend/src/components/knowledge_base/document/unified_feed/ContentItemRenderer.tsx:218`
- **Structural annotations now return images**: Fixed image retrieval for structural annotations without direct document references
  - **Root Cause**: `get_annotation_images_with_permission()` returned empty array for structural annotations (no `document` field)
  - **Fix**: Load PAWLs data from `structural_set.pawls_parse_file` when document is None
  - **Impact**: Structural image annotations (e.g., figures, charts) now display thumbnails in UI
  - **Files Modified**:
    - `opencontractserver/llms/tools/image_tools.py:220-305` - Added `_extract_image_from_pawls()` helper
    - `opencontractserver/llms/tools/image_tools.py:278-305` - Updated `get_annotation_images()` to check structural_set
    - `opencontractserver/llms/tools/image_tools.py:434-492` - Updated `get_annotation_images_with_permission()` for structural permissions
  - **Test Coverage**: Added test for structural annotation image retrieval
    - Files: `opencontractserver/tests/test_annotation_images_api.py:253-321`
    - All 6 tests passing including new structural annotation test
- **Parser pipeline now populates content_modalities**: Text parser now correctly sets content_modalities field
  - **Text Parser**: Sets content_modalities to `["TEXT"]` for all text-only annotations
    - Files: `opencontractserver/pipeline/parsers/oc_text_parser.py:108`
  - **Backfill Command**: Created management command to populate existing annotations with missing content_modalities
    - Analyzes token references in PAWLs data to determine modalities
    - Fallback: Uses annotation label text as hint (e.g., "image", "figure", "chart")
    - Files: `opencontractserver/annotations/management/commands/populate_content_modalities.py`
    - Usage: `python manage.py populate_content_modalities [--dry-run] [--force]`

### Changed

- **Unified JWT authentication architecture**: Refactored authentication to use single shared utility
  - **REST API**: `config/rest_jwt_auth.py` now uses `jwt_utils.get_user_from_jwt_token()`
  - **WebSocket**: Unified `JWTAuthMiddleware` replaces separate Auth0/non-Auth0 middlewares
    - Files: `config/websocket/middleware.py` - Single middleware handles both token types
    - Files: `config/websocket/middlewares/websocket_auth0_middleware.py` - Now alias to unified middleware (deprecated)
  - **ASGI**: Simplified `config/asgi.py` to use single middleware instead of conditional switching
  - **Benefit**: DRY architecture - token validation logic centralized in one place

### Removed

- **NLM Ingest Parser**: Removed legacy NLM-Ingest PDF parser in favor of Docling (default) and LlamaParse
  - **Rationale**: Docling provides superior ML-based parsing with better structure extraction; NLM parser was rarely used
  - **Migration**: Users with `PDF_PARSER=nlm` should switch to `PDF_PARSER=docling` (default) or `PDF_PARSER=llamaparse`
  - **Files Removed**:
    - `opencontractserver/pipeline/parsers/nlm_ingest_parser.py`
    - `opencontractserver/tests/test_doc_parser_nlm_ingest.py`
    - `docs/pipelines/nlm_ingest_parser.md`
  - **Settings Updated**: Removed `nlm` option from `_PDF_PARSER_MAP` in `config/settings/base.py`

### Technical Details

- **Backend**: REST endpoint leverages existing permission-checked `image_tools.py` functions
- **Frontend hook**: `useAnnotationImages` conditionally fetches images only for IMAGE modality (performance optimization)
- **TypeScript types**: Added `contentModalities?: string[]` to annotation types
  - Files: `frontend/src/types/graphql-api.ts:147`
  - Files: `frontend/src/components/annotator/types/annotations.ts:92,145`
- **Test coverage**: 5 backend tests for REST endpoint with authentication and permission checking
  - Files: `opencontractserver/tests/test_annotation_images_api.py`

### Added

#### Corpus-Isolated Structural Annotations

- **StructuralAnnotationSet duplication per corpus**: Each corpus now gets its own copy of structural annotations when documents are added
  - Enables multi-embedder support (each corpus can use different embedding models)
  - Maintains consistent per-corpus vector spaces for similarity search
  - Files: `opencontractserver/annotations/models.py`, `opencontractserver/corpuses/models.py`
- **Extended content_hash format**: Changed from `{sha256}` to `{sha256}_{corpus_id}` (max 128 chars)
  - Migration: `opencontractserver/annotations/migrations/0056_alter_structuralannotationset_content_hash.py`

#### Multimodal Embedding Support

- **Image token extraction from PDFs**: Extract images from PDFs via Docling parser and store as unified tokens in PAWLs format
  - Storage path convention: `document_images/{doc_id}/page_{page}_img_{idx}.{format}`
  - Image tokens include position, dimensions, format, and storage path
  - Files: `opencontractserver/utils/pdf_token_extraction.py`
- **CLIP ViT-L-14 multimodal embedder**: 768-dimensional vectors in shared text/image embedding space
  - Enables cross-modal similarity search (text queries find relevant images)
  - Files: `opencontractserver/pipeline/embedders/multimodal_microservice.py`
- **ContentModality enum**: Type-safe modality tracking for embedders and annotations
  - Single source of truth: `supported_modalities: set[ContentModality]`
  - Convenience properties: `is_multimodal`, `supports_text`, `supports_images`
  - Files: `opencontractserver/types/enums.py`, `opencontractserver/pipeline/base/embedder.py`
- **Multimodal embedding utilities**: Weighted averaging for mixed text+image content
  - Default weights: 30% text, 70% image (configurable via `MULTIMODAL_EMBEDDING_WEIGHTS`)
  - Files: `opencontractserver/utils/multimodal_embeddings.py`
- **content_modalities field on Annotation model**: ArrayField tracking `["TEXT"]`, `["IMAGE"]`, or `["TEXT", "IMAGE"]`
  - Computed from PAWLs token analysis during annotation creation
  - Files: `opencontractserver/annotations/models.py`, `opencontractserver/annotations/utils.py`
- **LLM image tools for agents**: `list_document_images`, `get_document_image`, `get_annotation_images`
  - Permission-checked variants prevent IDOR vulnerabilities
  - Files: `opencontractserver/llms/tools/image_tools.py`, `opencontractserver/llms/tools/tool_registry.py`
- **Modality filtering in vector search**: Filter annotations by content type in similarity search
  - Files: `opencontractserver/llms/vector_stores/core_vector_stores.py`
- **Comprehensive documentation**: Architecture docs for multimodal embeddings and PAWLs format
  - Files: `docs/architecture/multimodal-embeddings.md`, `docs/architecture/pawls-format.md`

### Changed

#### Corpus Isolation Architecture

- **Removed content-based deduplication**: Each upload creates independent documents regardless of content hash
- **Removed source_document provenance**: `source_document_id` no longer set when adding documents to corpus
- **Structural annotations no longer shared**: Each corpus gets duplicated structural annotation sets
- **Updated documentation**: Rewrote `structural_vs_non_structural_annotations.md`, updated `document_versioning.md`, `documents_and_annotations.md`

#### Multimodal Support

- Extended PAWLs token format to support unified image tokens (`is_image=True`)
- Updated `BaseEmbedder` to use `ContentModality` enum instead of boolean flags
- Updated `PipelineComponentDefinition` in registry to store `supported_modalities`
- Enhanced embedding task to detect multimodal content and generate appropriate embeddings

### Fixed

#### Android Share URL Missing Entity Prefix (PR #795)

- **Bug**: Android native share was dropping entity type prefixes (`/c/`, `/d/`, `/e/`) from shared URLs
- **Root Cause**: `frontend/src/components/seo/MetaTags.tsx:50-56` was generating canonical URLs as `/{userSlug}/{entitySlug}` instead of `/{prefix}/{userSlug}/{entitySlug}`
- **Impact**: Links shared via Android browser resulted in 404s (e.g., `/john/my-corpus` instead of `/c/john/my-corpus`)
- **Fix**: Refactored `MetaTags.tsx` to use existing `buildCanonicalPath()` utility from `navigationUtils.ts`
- **Added**: Unit tests for MetaTags canonical URL generation (`frontend/src/components/seo/__tests__/MetaTags.test.tsx`)
- **Added**: Development warning for unexpected `entityType` values
- **Added**: Enhanced Cloudflare worker request logging for crawler debugging (`cloudflare-og-worker/src/index.ts`)

### Security

#### WebSocket Agent Permission Vulnerability Fixed (PR #792)

- **CRITICAL**: Fixed permission bypass in legacy WebSocket consumers
  - `config/websocket/consumers/corpus_conversation.py` - No permission checks
  - `config/websocket/consumers/document_conversation.py` - No permission checks
  - `config/websocket/consumers/standalone_document_conversation.py` - No permission checks
  - **Impact**: Any authenticated user could access ANY document/corpus via WebSocket
- **Solution**: Migrated to `UnifiedAgentConsumer` with three-layer permission model:
  - Consumer layer validates READ permission at WebSocket connect time (`config/websocket/consumers/unified_agent_conversation.py:93-187`)
  - Tool filtering layer removes write tools for read-only users (`opencontractserver/llms/agents/agent_factory.py:178-210`)
  - Runtime layer validates permissions before every tool execution (`opencontractserver/llms/tools/pydantic_ai_tools.py:20-111`)
- **Defense-in-depth**: Added `_check_user_permissions()` function that validates user has READ permission on document/corpus before any tool execution
- **Tool permission flags**: Added `requires_write_permission` flag to `CoreTool` (`opencontractserver/llms/tools/tool_factory.py:45-50`)
- **Write tools protected**: `add_document_note`, `update_document_summary`, `update_corpus_description`, `duplicate_annotations`

### Added

#### WebSocket Permission Escalation Tests (49 tests)

- **Test file**: `opencontractserver/tests/websocket/test_agent_permission_escalation.py`
- **Consumer-level permission tests** (13 tests): Validates connection-time permission checks for corpus, document, and combined contexts
- **Tool filtering tests** (13 tests): Verifies write tools filtered for read-only users and anonymous access
- **Runtime permission validation tests** (6 tests): Defense-in-depth `_check_user_permissions()` function
- **Permission escalation scenarios** (9 tests): Cross-user access, mid-session permission changes, resource substitution attacks
- **Integration tests** (8 tests): Full conversation flows with permission verification

#### MCP Telemetry Tracking

- **PostHog telemetry for MCP usage** (`opencontractserver/mcp/telemetry.py`): Track MCP tool calls and resource reads when telemetry is enabled
  - Records tool usage (`mcp_tool_call`): tool name, success/failure, error type
  - Records resource access (`mcp_resource_read`): resource type (corpus, document, annotation, thread), success/failure
  - Records general requests (`mcp_request`): endpoint, method, transport type, success/failure
  - Privacy-preserving: Uses salted SHA-256 IP hashing for unique user counting (raw IPs are never sent to PostHog)
  - Support for all MCP transports: streamable_http, sse, stdio
  - No query content or outputs are captured - only usage metadata
- **Telemetry integration in MCP server** (`opencontractserver/mcp/server.py`):
  - Context-based telemetry with per-request isolation via ContextVar
  - Automatic client IP extraction from ASGI scope (supports X-Forwarded-For, X-Real-IP)
  - Error telemetry for failed requests with error type classification
  - Records both successful and failed requests for error rate calculations
- **Comprehensive test coverage** (`opencontractserver/mcp/tests/test_mcp.py`):
  - Unit tests for IP hashing, context management, event recording
  - Integration tests for telemetry recording in tool/resource handlers
  - Context manager for test isolation (`isolated_telemetry_context`)

### Removed

#### Legacy WebSocket Consumers (Security Cleanup)

- **Deleted**: `config/websocket/consumers/corpus_conversation.py` (~330 lines)
- **Deleted**: `config/websocket/consumers/document_conversation.py` (~610 lines)
- **Deleted**: `config/websocket/consumers/standalone_document_conversation.py` (~530 lines)
- **Deleted**: `opencontractserver/tests/test_websocket_corpus_consumer.py` - Obsolete tests for deleted consumer
- **Deleted**: `opencontractserver/tests/test_websocket_document_consumer.py` - Obsolete tests for deleted consumer
- **Deleted**: `opencontractserver/tests/websocket/test_standalone_document_consumer.py` - Obsolete tests for deleted consumer
- **Updated**: `config/asgi.py` - Removed legacy WebSocket routes

### Changed

#### Frontend WebSocket Migration

- **Updated**: `frontend/src/components/chat/get_websockets.ts` - All WebSocket URLs now use unified endpoint
- **Updated**: `frontend/src/components/knowledge_base/document/utils.ts` - Document chat uses unified endpoint

### Technical Details

- Uses existing PostHog infrastructure from `config/telemetry.py`
- Respects `TELEMETRY_ENABLED` setting and TEST mode disable
- IP hashing uses `TELEMETRY_IP_SALT` setting to prevent rainbow table attacks
- ContextVar ensures proper isolation in concurrent async requests

### Changed

#### NavMenu Refactoring (PR #779)

- **Migrated to @os-legal/ui NavBar** (`frontend/src/components/layout/NavMenu.tsx`): Complete refactor from Semantic UI Menu to unified NavBar component
  - Single responsive component replaces separate NavMenu and MobileNavMenu
  - Built-in hamburger menu at 1100px breakpoint eliminates conditional rendering in App.tsx
  - Modern styling consistent with os-legal-style design system
- **Deleted obsolete files**: Removed `MobileNavMenu.tsx` and `MobileNavMenu.css` (~370 lines)
- **Simplified App.tsx** (`frontend/src/App.tsx:320-325`): Removed conditional menu rendering and `useWindowDimensions` dependency
- **Improved code quality** (`frontend/src/components/layout/NavMenu.tsx`):
  - Replaced inline SVG icons with lucide-react imports (Download, User, Settings, LogOut)
  - Refactored login button to use styled-components instead of inline styles
  - Added type-safe `getUserProps` helper to replace `as any` casts for user properties

### Added

#### NavMenu Component Tests

- **Playwright component tests** (`frontend/tests/NavMenu.ct.tsx`): 18 comprehensive tests covering:
  - Navigation items and active state highlighting
  - Authentication states (login button vs user menu)
  - User menu items (Exports, Profile, Admin Settings, Logout)
  - Superuser-only features (Badge Management nav item, Admin Settings menu)
  - Branding elements (logo, version badge, brand name)
  - Responsive behavior (hamburger menu, mobile navigation)
- **Test wrapper** (`frontend/tests/NavMenuTestWrapper.tsx`): Provides Auth0Provider, MockedProvider, MemoryRouter, and JotaiProvider context

### Fixed

#### Superuser Features in Non-Auth0 Mode

- **LOGIN_MUTATION missing isSuperuser** (`frontend/src/graphql/mutations.ts:49-65`): Added `isSuperuser` field to login query
  - Previously, superuser features (Badge Management, Admin Settings) were broken in non-Auth0 mode
  - Updated `LoginOutputs` interface to include `username`, `isUsageCapped`, and `isSuperuser` fields

### Fixed

#### WebSocket Connection Performance (Issue: Chat "Reconnecting" delay)

- **Auth0 JWKS caching** (`config/graphql_auth0_auth/utils.py:17-38`): Added in-memory cache for Auth0 JWKS with 10-minute TTL
  - Previously fetched JWKS from Auth0 on every token validation, causing 6-10 second delays
  - Now caches JWKS keys, reducing subsequent WebSocket auth to near-instant
- **CorpusChat double connection fix** (`frontend/src/components/corpuses/CorpusChat.tsx:1043-1056`): Skip forceNewChat useEffect on initial mount
  - `isNewChat` state already initialized from `forceNewChat` prop
  - Prevents redundant `startNewChat()` call that caused socket close/reconnect cycle
- **Notification WebSocket auth guard** (`frontend/src/hooks/useNotificationWebSocket.ts:312-318`): Skip connection attempt without auth token
  - Prevents 403 Access Denied errors when connecting before auth token is available
  - Eliminates unnecessary connection attempts and error spam in console

### Added

#### Extract View Refactoring (PR #772)

- **Route-based extract detail view** (`frontend/src/views/ExtractDetail.tsx:439-1063`, `frontend/src/components/routes/ExtractDetailRoute.tsx:1-58`): Complete refactor from modal-based to route-based architecture
  - Modern full-page layout with tabbed interface (Data, Documents, Schema)
  - Stats grid showing document count, column count, rows, and success rate
  - WebSocket-based real-time updates for running extracts (replaced polling)
  - Responsive design following existing patterns
- **WebSocket notification hook** (`frontend/src/hooks/useExtractCompletionNotification.ts:1-86`): Real-time extract completion detection
  - Listens for `EXTRACT_COMPLETE` notifications via WebSocket
  - Filters for specific extract ID and triggers refetch on completion
  - Eliminates need for polling (previously every 5 seconds)
- **Extracts list page** (`frontend/src/views/Extracts.tsx:1-410`): New landing page for extract management
  - Filter tabs (All, My Extracts, Running, Completed)
  - Search with debounced input and cleanup on unmount
  - CollectionCard components with status indicators
- **Extract list card** (`frontend/src/components/extracts/ExtractListCard.tsx:1-228`): Card component for extract listing
  - Status-aware styling (Running, Completed, Failed, Not Started)
  - Context menu with view and delete actions
  - Keyboard accessibility (Escape to close, Enter/Space to activate)
- **Shared utilities** (`frontend/src/utils/extractUtils.ts:1-70`): DRY utility functions using centralized constants
- **Extract landing route** (`frontend/src/components/routes/ExtractLandingRoute.tsx:1-35`): Route component for /extracts

### Removed

- **EditExtractModal component**: Replaced by route-based ExtractDetail view - modal approach deprecated
- **Obsolete test files** (`frontend/tests/EditExtractModal.ct.tsx`, `frontend/tests/EditExtractModalTestWrapper.tsx`): Removed tests for deleted modal component
- **Polling constants** (`frontend/src/constants/extract.ts`): Removed `EXTRACT_POLLING_INTERVAL_MS` and `EXTRACT_POLLING_TIMEOUT_MS` - replaced by WebSocket notifications

### Changed

- **openedExtract reactive var documentation** (`frontend/src/graphql/cache.ts:364-388`): Clarified that route components (like ExtractDetailRoute) can set this var, not just CentralRouteManager
- **Consolidated constants** (`frontend/src/assets/configurations/constants.ts:47-51`): Moved `EXTRACT_SEARCH_DEBOUNCE_MS` to centralized `DEBOUNCE` object
- **extractUtils refactor** (`frontend/src/utils/extractUtils.ts:32-55`): Now uses `EXTRACT_STATUS` and `EXTRACT_STATUS_COLORS` constants instead of hardcoded values

### Added

#### Corpuses Page Redesign

- **CorpusListView component** (`frontend/src/components/corpuses/CorpusListView.tsx`): Modern corpus listing page using @os-legal/ui components
  - Hero section with search and filter tabs (All, My Corpuses, Shared, Public)
  - Stats grid showing corpus, document, annotation, and shared counts
  - CollectionCard components with category badges, visibility status, and labelset information
  - Context menu for edit, view, export, fork, and delete actions
  - Infinite scroll support for large corpus lists
- **PostHog Analytics Integration** (`frontend/src/utils/analytics.ts`): Consent-based analytics tracking
  - Cookie consent banner integration
  - Automatic test/CI environment detection to prevent analytics in non-production
  - User identification and event tracking functions
  - Page view tracking for SPA navigation
- **Component tests** (`frontend/tests/CorpusListView.ct.tsx`): 12 Playwright component tests for CorpusListView
- **Unit tests** (`frontend/src/utils/__tests__/analytics.test.ts`): 20 Vitest tests for analytics utility functions

### Fixed

#### Routing Audit Follow-ups

- **Notification navigation fallback** (`frontend/src/components/notifications/NotificationDropdown.tsx:182-188`, `NotificationCenter.tsx:207-213`): Added fallback navigation to `/discussions` when corpus slug data is missing. Previously, users clicking notifications with missing slug data would see no response.
- **Network-only fetch policy optimization** (`frontend/src/routing/CentralRouteManager.tsx:621-632`): Changed thread resolution corpus query from `network-only` to `cache-and-network` since `authInitComplete` now ensures `clearStore()` completes before route queries run. This improves navigation performance when corpus data is already cached.
- **Unit test coverage** (`frontend/src/utils/__tests__/navigationUtils.test.ts:497-503`): Added missing test for `parseRoute("/discussions")` to prevent regression of discussions route parsing.

#### Type Safety and Bug Fixes

- **User email detection** (`frontend/src/components/corpuses/CorpusListView.tsx:345-349`): Fixed currentUserEmail logic to use `userObj` reactive variable from Apollo cache instead of inferring from corpus permissions - prevents filter failures when no corpus has CAN_REMOVE permission
- **TypeScript type casts** (`frontend/src/components/corpuses/CorpusListView.tsx`, `CorpusModal.tsx`): Removed 7 `as any` type casts by correcting `CorpusType.categories` type from `CorpusCategoryTypeConnection` to `CorpusCategoryType[]` to match backend GraphQL schema
- **N+1 query prevention** (`config/graphql/queries.py:820-825`): Added `prefetch_related("categories")` to `resolve_corpuses` to avoid N+1 queries when fetching corpus categories

### Changed

- **Deleted CorpusCards component** (`frontend/src/components/corpuses/CorpusCards.tsx`): Replaced by CorpusListView with @os-legal/ui components

#### LabelSet Detail Page Refactoring

- **LabelSetDetailPage component split** (`frontend/src/components/labelsets/LabelSetDetailPage.tsx`): Reduced from 2,064 lines to ~1,100 lines
  - Extracted 16 SVG icons to `detail/LabelSetIcons.tsx`
  - Extracted 40+ styled-components to `detail/LabelSetDetailStyles.ts`
  - Added barrel exports in `detail/index.ts`
- **Color constants centralized** (`frontend/src/assets/configurations/constants.ts`): Added `DEFAULT_LABEL_COLOR` and `PRIMARY_LABEL_COLOR`

### Security

- **Frontend permission checks** (`frontend/src/components/labelsets/LabelSetDetailPage.tsx:1189-1344`): Added defensive permission checks to all mutation handlers
  - `handleDeleteLabel` now verifies `canRemove` before deletion
  - `handleSaveEdit` now verifies `canUpdate` before updating
  - `handleSaveCreate` now verifies `canUpdate` before creation
  - `handleDelete` (labelset) now verifies `canRemove` before deletion
- **Color input sanitization** (`frontend/src/components/labelsets/LabelSetDetailPage.tsx:125-143`): Added `isValidHexColor` and `sanitizeColor` utilities
  - Validates hex color format (3 or 6 character, with or without #)
  - Prevents potential XSS via CSS color injection

### Technical Details

- New test file: `frontend/tests/LabelSetDetailPage.ct.tsx` with comprehensive component tests
  - Rendering tests for all tabs (Overview, Text/Doc/Span/Relationship Labels)
  - Permission-based UI visibility tests
  - Search functionality tests
  - Mobile navigation tests

### Added

#### Secure Zip Import with Folder Structure Preservation

- **Zip security utilities** (`opencontractserver/utils/zip_security.py`): Comprehensive security validation for zip file imports
  - Path traversal protection: Sanitizes all paths, rejects `..` sequences, drive letters, absolute paths
  - Zip bomb detection: Monitors compression ratios, enforces size limits (500MB total, 100MB per file)
  - Symlink rejection: Detects and skips symbolic links in zip entries
  - Resource limits: Max 1000 files, 500 folders, 20 levels deep (all configurable)
  - Hidden file filtering: Skips `.DS_Store`, `__MACOSX`, `Thumbs.db`, etc.
- **Security constants** (`opencontractserver/constants/zip_import.py`): Configurable limits via Django settings
- **Folder structure creation** (`opencontractserver/corpuses/folder_service.py:1268-1411`): `create_folder_structure_from_paths()` efficiently creates folder hierarchies, reusing existing folders
- **Import Celery task** (`opencontractserver/tasks/import_tasks.py:580-912`): `import_zip_with_folder_structure` task with three-phase processing:
  - Phase 1: Security validation without extraction
  - Phase 2: Atomic folder structure creation
  - Phase 3: Batched document processing with per-file error handling
- **GraphQL mutation** (`config/graphql/mutations.py:1890-2040`): `importZipToCorpus` mutation with rate limiting
  - Accepts base64-encoded zip file
  - Optional target folder placement
  - Returns job_id for async tracking
  - Requires corpus EDIT permission
- **Document upversioning on collision**: When importing a document to a path that already has a document, the new document becomes version 2 (or higher), with the previous version preserved in history
- **Comprehensive test suites**:
  - Security tests (`opencontractserver/tests/test_zip_security.py`): 49 tests for path sanitization, validation, edge cases
  - Integration tests (`opencontractserver/tests/test_zip_import_integration.py`): 17 tests for task and folder service
- **Design documentation** (`docs/features/zip_import_with_folders_design.md`): Complete specification including security model, API, error handling

#### Corpus Categories and Landing Page Redesign

- **CorpusCategory model** (`opencontractserver/corpuses/models.py`): New model for organizing corpuses by type (Legislation, Contracts, Case Law, Knowledge)
  - Admin-provisioned structural data - managed via Django Admin only
  - ManyToMany relationship with Corpus for flexible categorization
  - Default categories seeded via migration (`0035_seed_default_categories.py`)
- **CorpusCategoryType GraphQL type** (`config/graphql/graphene_types.py:1589-1633`):
  - Globally visible to all users (no individual permissions)
  - `corpusCount` field with N+1 query optimization via annotation
- **Landing page redesign** using @os-legal/ui component library:
  - `CompactLeaderboard` component - clean list-based leaderboard replacing grid cards
  - `CategorySelector` component for corpus categorization
  - Skeleton loading states and error handling throughout
- **TypeScript types** (`frontend/src/types/graphql-api.ts`): Added `CorpusCategoryType`, `CorpusCategoryTypeConnection`, `CorpusCategoryTypeEdge`
- **Array utilities** (`frontend/src/utils/arrayUtils.ts`): `arraysEqualUnordered` and `arraysEqualOrdered` for DRY comparison logic

### Fixed

#### Security and Performance

- **System user security** (`opencontractserver/corpuses/migrations/0035_seed_default_categories.py`): Defense-in-depth with unusable password for system user
- **N+1 query in corpusCount** (`config/graphql/queries.py:835-866`): Pre-annotated counts in `resolve_corpus_categories` resolver
- **Type safety** (`frontend/src/components/corpuses/CorpusModal.tsx`, `CorpusSettings.tsx`): Removed `as any` casts for categories field

### Changed

- **Permission model** (`config/graphql/graphene_types.py`): `CorpusCategoryType` no longer uses `AnnotatePermissionsForReadMixin` - categories are globally visible structural data
- **Documentation** (`docs/permissioning/consolidated_permissioning_guide.md`): Added section on CorpusCategory permissions

### Technical Details

- Categories are created by a `system` user with `is_active=False` and unusable password
- `corpusCount` respects user visibility: anonymous sees public corpuses only, authenticated users see corpuses they have access to
- Removed 632-line `TopContributors.tsx` component, replaced with ~280-line `CompactLeaderboard.tsx`

### Added

#### Moderation Dashboard and Rollback Features (Issue #742)

- **ModerationActionType GraphQL type** (`config/graphql/graphene_types.py:3071-3121`): Exposes ModerationAction audit records with computed fields:
  - `corpusId`: Links to parent corpus for filtering
  - `isAutomated`: Identifies agent vs. human moderation
  - `canRollback`: Indicates whether action can be undone
- **ModerationMetricsType** (`config/graphql/graphene_types.py:3109-3121`): Aggregated metrics for monitoring moderation activity:
  - Total/automated/manual action counts
  - Hourly action rate with threshold alerting
  - Actions grouped by type
- **New GraphQL queries** (`config/graphql/queries.py:1875-2043`):
  - `moderationActions`: Filterable query for audit logs (corpus, thread, moderator, action type)
  - `moderationAction`: Single action lookup by ID
  - `moderationMetrics`: Aggregated stats with threshold violations
- **RollbackModerationActionMutation** (`config/graphql/moderation_mutations.py:594-707`): Undo automated moderation actions:
  - Supports rollback of delete_message, delete_thread, lock_thread, pin_thread
  - Creates new audit record for the rollback
  - Permission-gated to moderators
- **DeleteThreadMutation and RestoreThreadMutation** (`config/graphql/moderation_mutations.py:267-363`): Complete thread lifecycle management for frontend
- **ModerationDashboard component** (`frontend/src/components/moderation/ModerationDashboard.tsx`): Full-featured moderation UI:
  - Metrics display with threshold alerts
  - Filterable action table (action type, automated only)
  - Rollback confirmation modal
  - Time range selector (1h, 24h, 7d, 30d)
- **Dynamic tool fetching** (`frontend/src/components/corpuses/CreateCorpusActionModal.tsx`): Replaces hardcoded moderation tools with GraphQL query to `availableTools(category: "moderation")`

### Fixed

#### Race Condition in Agent Thread Actions (Issue #742)

- **Fixed TOCTOU vulnerability** (`opencontractserver/tasks/agent_tasks.py:859-898`): Added `select_for_update()` with `transaction.atomic()` to prevent duplicate agent execution claims

#### Tool Validation for Inline Agents (Issue #742)

- **Added tool category validation** (`config/graphql/mutations.py:3875-3897`): CreateCorpusAction now validates that inline agent tools are from the MODERATION category when using thread/message triggers

### Added

#### MCP (Model Context Protocol) Interface Proposal (Issue #387)

- **Comprehensive MCP interface design** (`docs/mcp/mcp_interface_proposal.md`): Read-only access to public OpenContracts resources for AI assistants
- **4 resource types**: corpus, document, annotation, thread - with hierarchical URI patterns
- **7 tools for discovery and retrieval**: `list_public_corpuses`, `list_documents`, `get_document_text`, `list_annotations`, `search_corpus`, `list_threads`, `get_thread_messages`
- **Anonymous user permission model**: Operates as AnonymousUser with automatic filtering to `is_public=True` resources
- **Synchronous Django ORM implementation**: Uses `sync_to_async` wrapper pattern for MCP server integration
- **Performance optimizations**: Uses existing `AnnotationQueryOptimizer`, `prefetch_related` for threaded messages, and proper pagination
- **Robust URI parsing**: Regex-based URI parsing with slug validation to prevent injection attacks
- **Helper function implementations**: Complete `format_*` functions for corpus, document, annotation, thread, and message formatting

#### Markdown Link Generation Tool for Agent Responses (Issue #530)

- **New `create_markdown_link` agent tool** (`opencontractserver/llms/tools/core_tools.py:1990-2174`): Agents can now generate properly formatted markdown links for annotations, corpus, documents, and conversations
- **Supported entity types**:
  - **Annotations**: `[Annotation text](/d/user/corpus/doc?ann=123)` - Links to annotation with document context
  - **Corpus**: `[Corpus Title](/c/user/corpus-slug)` - Direct links to corpus
  - **Documents**: `[Document Title](/d/user/corpus/doc-slug)` - Smart routing (standalone or corpus-based)
  - **Conversations/Threads**: `[Discussion Title](/c/user/corpus/discussions/123)` - Links to discussion threads
- **Intelligent link generation**:
  - Automatically detects if documents belong to a corpus for proper URL structure
  - Truncates long annotation text (>100 chars) for readability
  - Uses entity titles when available, falls back to generic labels (e.g., "Annotation 123")
  - Validates entity existence, creator, and slug availability before generating links
- **Async support**: Both sync (`create_markdown_link`) and async (`acreate_markdown_link`) versions available
- **Tool registry entry** (`opencontractserver/llms/tools/tool_registry.py:364-380`): Registered as COORDINATION category tool with full parameter documentation
- **Comprehensive test coverage** (`opencontractserver/tests/test_llm_tools.py:2031-2417`):
  - 35+ test cases covering all entity types, edge cases, and error conditions
  - Tests for both sync and async implementations
  - Validation of error messages for missing entities, creators, slugs, and invalid types

#### Real-Time Notification System via WebSocket (Issue #637)

- **WebSocket notification consumer** (`config/websocket/consumers/notification_updates.py`): New `NotificationUpdatesConsumer` provides real-time notification delivery for all notification types (BADGE, REPLY, MENTION, THREAD_REPLY, moderation actions)
- **Frontend WebSocket hook** (`frontend/src/hooks/useNotificationWebSocket.ts`): `useNotificationWebSocket` hook manages WebSocket connection lifecycle with auto-reconnection, heartbeat monitoring, and graceful error handling
- **Signal broadcasting** (`opencontractserver/notifications/signals.py:33-100`): All notification creation signals now broadcast via WebSocket channel layer for instant delivery
- **ASGI routing** (`config/asgi.py:88-94`): Registered `ws/notification-updates/` WebSocket endpoint with authentication middleware
- **WebSocket URL helper** (`frontend/src/components/chat/get_websockets.ts:226-259`): `getNotificationUpdatesWebSocket` function constructs WebSocket URLs with proper protocol handling

### Changed

#### Badge Notifications Migrated from Polling to WebSocket (Issue #637)

- **useBadgeNotifications hook** (`frontend/src/hooks/useBadgeNotifications.ts`): Completely refactored from Apollo Client polling (30s intervals) to WebSocket-based real-time updates
- **Zero latency**: Badge awards now appear instantly instead of 0-30 second delay
- **Reduced server load**: Eliminated continuous polling requests from all connected clients
- **Backward compatible**: Maintains same interface (`newBadges`, `clearNewBadges`) with added `connectionState` for debugging

### Fixed

#### WebSocket Token Expiration Close Code Handling (PR #746)

- **Updated all WebSocket consumers** to check `scope['auth_error']` from middleware and use specific close codes:
  - `config/websocket/consumers/document_conversation.py:77-91`: Uses auth_error codes for expired/invalid tokens
  - `config/websocket/consumers/corpus_conversation.py:67-79`: Uses auth_error codes for expired/invalid tokens
  - `config/websocket/consumers/standalone_document_conversation.py:97-106`: Checks auth_error before falling back to anonymous handling
  - `config/websocket/consumers/unified_agent_conversation.py:119-127`: Uses auth_error codes for expired/invalid tokens
  - `config/websocket/consumers/thread_updates.py:77-88`: Uses auth_error codes for expired/invalid tokens
- **Removed unused `Union` import** from `config/websocket/middleware.py:2`
- **Fixed lazy import issue** in `config/graphql_auth0_auth/utils.py:124`: Moved `sync_remote_user` import inside function to avoid import error when `USE_AUTH0=False`
- **Added Auth0 test settings** in `config/settings/test.py:120-133`: Default Auth0 settings for test environment to allow importing Auth0 modules during testing

#### Impact

- Frontend can now distinguish between expired tokens (4001) and invalid tokens (4002) via WebSocket close codes
- Enables targeted token refresh vs full re-authentication based on close code
- Fixes issue #744 where token expiration wasn't properly signaled to clients

### Added

#### LlamaParse Document Parser Integration (Issue #692)

- **New LlamaParseParser** (`opencontractserver/pipeline/parsers/llamaparse_parser.py`): Full integration with LlamaParse API for document parsing with layout extraction
  - Supports PDF and DOCX file types
  - Extracts structural annotations (Title, Heading, Paragraph, Table, Figure, List, etc.) with bounding boxes
  - Generates PAWLS tokens from LlamaParse layout data for PDF annotation display
  - Supports multiple bounding box formats (fractional 0-1, absolute coordinates, array format)
  - Configurable via environment variables or Django settings
- **Environment variable configuration**:
  - `LLAMAPARSE_API_KEY` / `LLAMA_CLOUD_API_KEY`: API key for LlamaParse authentication
  - `LLAMAPARSE_RESULT_TYPE`: Output type ("json", "markdown", "text") - default: "json"
  - `LLAMAPARSE_EXTRACT_LAYOUT`: Enable layout extraction with bounding boxes - default: True
  - `LLAMAPARSE_NUM_WORKERS`: Parallel processing workers - default: 4
  - `LLAMAPARSE_LANGUAGE`: Document language - default: "en"
  - `LLAMAPARSE_VERBOSE`: Enable verbose logging - default: False
- **Parser selection via environment variable**:
  - `PDF_PARSER`: Set to "llamaparse" or "docling" (default) to select default PDF parser
  - Location: `config/settings/base.py:740-765`
- **Comprehensive test suite** (`opencontractserver/tests/test_doc_parser_llamaparse.py`):
  - Tests for successful parsing with layout extraction
  - Tests for markdown mode without layout
  - Tests for bounding box format conversion (fractional, absolute, array)
  - Tests for annotation creation and token generation
  - Tests for error handling (missing API key, API errors, empty results)
  - Tests for configuration via settings and kwargs override

#### Thread/Message Triggered Corpus Actions for Automated Moderation

- **Extended CorpusActionTrigger enum** with `NEW_THREAD` and `NEW_MESSAGE` triggers (`opencontractserver/corpuses/models.py:849-854`) to enable automated moderation of discussion threads
- **New moderation tools** (`opencontractserver/llms/tools/moderation_tools.py`): 9 tools for thread moderation including:
  - `get_thread_context`: Retrieve thread metadata (title, creator, lock/pin status)
  - `get_thread_messages`: Get recent messages for context
  - `get_message_content`: Get full content of a specific message
  - `delete_message`: Soft delete a message with audit logging
  - `lock_thread`/`unlock_thread`: Control thread access
  - `add_thread_message`: Post agent messages to threads
  - `pin_thread`/`unpin_thread`: Feature important threads
- **New MODERATION tool category** (`opencontractserver/llms/tools/tool_registry.py:42`) with 9 registered tools and proper approval requirements
- **Signal handlers** for thread/message creation (`opencontractserver/corpuses/signals.py`) using `transaction.on_commit` pattern to trigger corpus actions
- **New Celery tasks** (`opencontractserver/tasks/corpus_tasks.py`):
  - `process_thread_corpus_action`: Processes actions when threads are created
  - `process_message_corpus_action`: Processes actions when messages are posted
- **Agent thread action task** (`opencontractserver/tasks/agent_tasks.py:run_agent_thread_action`): Runs AI agents with thread context and moderation tools
- **Updated CorpusActionExecution model** (`opencontractserver/corpuses/models.py`) with optional `conversation` and `message` FKs for audit trail
- **Updated AgentActionResult model** (`opencontractserver/agents/models.py`) with nullable document FK and new `triggering_conversation`/`triggering_message` FKs
- **Frontend updates** (`frontend/src/components/corpuses/CreateCorpusActionModal.tsx`):
  - Added "On New Thread" and "On New Message" trigger options
  - Thread/message triggers automatically select agent action type
  - Info message explaining available moderation tools
- **Comprehensive test coverage**:
  - Backend tests: `opencontractserver/tests/test_thread_corpus_actions.py`
  - Frontend tests: `frontend/tests/create-corpus-action-modal.ct.tsx`
- **Database migrations**:
  - `opencontractserver/agents/migrations/0008_add_thread_message_triggers.py`: Adds nullable `triggering_conversation` and `triggering_message` FKs to AgentActionResult, makes `document` nullable
  - `opencontractserver/corpuses/migrations/0032_add_thread_message_triggers.py`: Adds nullable `conversation` and `message` FKs to CorpusActionExecution

#### Use Cases Enabled

- Automated content moderation (e.g., auto-delete messages with prohibited content)
- Thread management (e.g., auto-lock threads discussing prohibited topics)
- Automated responses (e.g., welcome messages for new threads)
- Content classification (e.g., auto-pin important announcements)

#### Proactive Apollo Cache Management System (PR #725)

- **New `CacheManager` service** (`frontend/src/services/cacheManager.ts`): Centralized Apollo cache management with debouncing, targeted invalidation, and auth-aware cache operations
  - `resetOnAuthChange()`: Full cache clear with optional refetch for login/logout transitions
  - `refreshActiveQueries()`: Soft refresh without clearing cache
  - `invalidateEntityQueries()`: Targeted invalidation for document/corpus/annotation CRUD operations
  - Debouncing: 1000ms for full resets, 500ms for entity invalidations
  - Debug utilities: `logCacheSize()`, `extractCacheForDebug()`
- **New `useCacheManager` hook** (`frontend/src/hooks/useCacheManager.ts`): React hook with memoized CacheManager instance and stable callback references
- **Comprehensive test suite** (`frontend/src/services/__tests__/cacheManager.test.ts`, `frontend/src/hooks/__tests__/useCacheManager.test.tsx`): 30+ tests covering debouncing, error handling, lifecycle, singleton management, and auth scenarios

### Technical Details

#### LlamaParse Parser Architecture

- Uses `llama-parse` library for API communication
- JSON mode with `extract_layout=True` provides bounding boxes as fractions of page dimensions (0-1)
- Converts LlamaParse layout elements to OpenContracts structural annotations
- Generates PAWLS tokens by splitting text into words and distributing across bounding box
- Element type mapping converts LlamaParse labels (title, paragraph, table, etc.) to OpenContracts annotation labels
- Falls back to text extraction mode when layout extraction is disabled

#### Markdown Link Tool Implementation

- Follows OpenContracts routing patterns from `docs/frontend/routing_system.md`
- Uses `select_related()` to minimize database queries (single query per entity)
- Handles both standalone documents and corpus-based document contexts
- Entity validation with clear error messages for IDOR prevention
- URL patterns match frontend `navigationUtils.ts` for consistency

### Fixed

#### Token Expiration Signal to Frontend (Issue #744)

- **Fixed `Auth0RemoteUserJSONWebTokenBackend.authenticate()` swallowing `JSONWebTokenExpired` exceptions** (`config/graphql_auth0_auth/backends.py:44-52`):
  - Previously, when a JWT token expired, the authentication backend caught all exceptions and returned `None`
  - The GraphQL layer then returned a generic "User is not authenticated" error
  - Frontend's `errorLink.ts` could not detect token expiration and trigger automatic refresh
  - Fix: Re-raise `JSONWebTokenExpired` so the GraphQL layer returns "Signature has expired"
  - Frontend now correctly detects expiration and triggers page reload for silent token refresh
- **Enhanced WebSocket middleware with auth error signaling** (`config/websocket/middleware.py:44-124`):
  - Added `scope["auth_error"]` dict with `code` and `message` fields
  - New close codes: `WS_CLOSE_TOKEN_EXPIRED` (4001), `WS_CLOSE_TOKEN_INVALID` (4002)
  - Consumers can now close connections with specific codes for frontend handling
- **Enhanced Auth0 WebSocket middleware** (`config/websocket/middlewares/websocket_auth0_middleware.py:52-130`):
  - Added consistent `scope["auth_error"]` handling for Auth0 tokens
  - Matches close code behavior with non-Auth0 middleware
- **New test coverage** (`opencontractserver/tests/test_token_expiration.py`):
  - Tests for `Auth0RemoteUserJSONWebTokenBackend` token expiration re-raising
  - Tests for WebSocket middleware auth error handling
  - Tests for WebSocket close code consistency

#### Independent Structural Annotation and Show Selected Controls (Issue #735)

- **Removed forced coupling between structural and showSelectedOnly controls** (`frontend/src/components/annotator/controls/AnnotationControls.tsx:200-207`):
  - Previously, enabling "Show Structural" would force "Show Only Selected" to be checked and disabled
  - Users can now toggle "Show Only Selected" independently when structural annotations are visible
  - All combinations now work:
    - Show all structural annotations: structural ON, selectedOnly OFF
    - Show only selected structural annotation: structural ON, selectedOnly ON
    - Hide all structural annotations: structural OFF
- **Updated checkbox onChange handler** (`frontend/src/components/annotator/controls/AnnotationControls.tsx:268`): Now correctly extracts `data?.checked ?? false` for consistency with other toggle handlers
- **Updated component tests** (`frontend/tests/FloatingDocumentControls.ct.tsx:263-371`):
  - Renamed test to reflect new independent behavior
  - Added new test verifying controls can be toggled independently
- **Note**: Users who previously had `showStructural: true` will notice different behavior: the "Show Only Selected" control now respects their actual preference instead of being forced to true

#### Cache Management Race Condition Fix (PR #725)

- **Auth state now set BEFORE cache clear** (`frontend/src/components/auth/AuthGate.tsx:69-92`, `frontend/src/views/Login.tsx:106-117`, `frontend/src/components/layout/useNavMenu.ts:64-90`):
  - Previously, cache was cleared before updating auth state, creating a window where queries could fetch with wrong auth context
  - Fixed by setting auth token/user/status first, then clearing cache
  - Refetched queries now correctly use the new auth context
- **AuthGate uses useCacheManager hook** (`frontend/src/components/auth/AuthGate.tsx:7,27`): Replaced direct `new CacheManager()` instantiation with proper hook usage, eliminating `as any` type assertion and ensuring memoization
- **Fire-and-forget logout cache clear** (`frontend/src/components/layout/useNavMenu.ts:69-79`): Logout no longer blocks on cache clear operation, improving perceived performance

### Technical Details

#### Cache Management Architecture

- **Race condition prevention**: Auth state updates are synchronous; cache clear is async. By setting auth first, any queries triggered during cache clear use the correct credentials.
- **Singleton pattern preserved for non-React contexts**: The singleton functions (`initializeCacheManager`, `getCacheManager`, etc.) remain exported for testing and non-React usage, with documentation clarifying when to use hooks vs singleton.
- **Dependency management**: `useCacheManager` hook returns stable callback references via `useCallback`, safe to include in effect dependencies.

### Fixed

#### Mobile Responsive Styling for Settings and Badge Widgets (Issue #690)

- **Badge component z-index optimization** (`frontend/src/components/badges/Badge.tsx:47,107`): Lowered z-index values from 9999/10000 to 200/201 to avoid conflicts with other UI elements while maintaining proper layering
- **Unified mobile behavior detection** (`frontend/src/components/badges/Badge.tsx:148-152`): Combined touch device detection with viewport width check to ensure mobile UX works consistently across real devices and test environments
- **Test wrapper extraction** (`frontend/tests/UserBadgesTestWrapper.tsx`, `frontend/tests/GlobalSettingsPanelTestWrapper.tsx`): Moved test wrappers to separate files following Playwright component testing best practices
- **Improved test reliability** (`frontend/tests/mobile-responsive.ct.tsx`): Fixed element disambiguation issues using proper locator strategies

#### Agent Chat Processing Indicator (PR #687)

- **Added visual feedback for agent processing** (`frontend/src/components/widgets/chat/ChatMessage.tsx:1342-1405`): When an agent starts processing a response, an animated "Agent is thinking..." indicator now displays instead of an empty message bubble
- **Processing indicator conditions**: Shows when assistant message is incomplete with no content and no timeline entries
- **Accessibility improvements**: Added ARIA attributes (`role="status"`, `aria-live="polite"`, `aria-label`) for screen reader support
- **Animation performance**: Added `will-change: transform, opacity` to animated dots for smoother rendering
- **Component tests**: Added comprehensive Playwright component tests (`frontend/tests/chat-message-processing-indicator.ct.tsx`) covering indicator visibility, accessibility, and state transitions

### Fixed

#### Trash View Error Prevention (Issue #691)

- **State synchronization fix** (`frontend/src/components/corpuses/folders/FolderTreeSidebar.tsx:363-369`):
  - Fixed trash folder click handler to use consistent state update pattern matching other folder navigation
  - Added `handleTrashClick` callback that properly delegates to `onFolderSelect` when provided (URL-driven state)
  - Removed direct Jotai atom manipulation that caused race conditions with CentralRouteManager
- **Defensive null handling** (`frontend/src/components/corpuses/folders/TrashFolderView.tsx`):
  - Added `safeFormatDistanceToNow()` and `safeFormat()` helper functions for robust date formatting
  - Added optional chaining for `creator`, `document`, and nested properties to prevent runtime errors
  - Added validation in `handleRestoreSingle()` and `handleRestoreSelected()` to check for valid document data
- **Type safety improvements** (`frontend/src/graphql/queries/folders.ts:92-104`):
  - Updated `DeletedDocumentPathType` interface to mark `creator` and `document` as potentially null
  - Ensures TypeScript catches potential null access issues at compile time

### Added

#### Agent Message Visual Differentiation (Issue #688)

- **Enhanced MessageItem component** (`frontend/src/components/threads/MessageItem.tsx:27-50, 59-66, 68-191, 211-245, 461-466, 530-550`):
  - Agent detection logic using `getAgentDisplayData()` helper function
  - `hexToRgba()` utility for generating color-tinted backgrounds from agent badge colors
  - Distinct visual styling for agent messages vs user messages:
    - **Background**: Subtle gradient using agent's badge color with low opacity (8% to 3%)
    - **Border**: Colored border matching agent's badge color instead of default gray
    - **Accent strip**: 4px colored left border (like highlighted messages) using agent color
    - **Avatar**: Bot icon instead of User icon, with agent-colored gradient background
    - **Box shadow**: Agent-colored shadow on avatar for visual consistency
- **Accessibility improvements**:
  - Updated `aria-label` to include "(AI Agent)" suffix for screen readers
  - Avatar `title` attribute identifies agent name and type
- Agent color sourced from `AgentConfiguration.badgeConfig.color` field (falls back to default blue #4A90E2)

#### Network Recovery on Screen Unlock (Issue #697)

- **New `useNetworkStatus` hook** (`frontend/src/hooks/useNetworkStatus.ts`): Monitors page visibility and network status changes to detect when the app resumes from background (e.g., screen unlock on mobile)
- **New `NetworkStatusHandler` component** (`frontend/src/components/network/NetworkStatusHandler.tsx`): Automatically refetches active Apollo Client queries when:
  - The page becomes visible after being hidden (screen unlock on mobile)
  - The network comes back online after being offline
- **WebSocket reconnection on resume**: Updated `useThreadWebSocket` and `useAgentChat` hooks to reconnect WebSockets when the page becomes visible
- **Toast notifications**: Informs users of connectivity changes ("Reconnecting...", "Connection restored", "You appear to be offline")

### Technical Details

#### Real-Time Notification System Architecture (Issue #637)

- **Security**: User-specific channel groups (`notification_user_{user_id}`) prevent IDOR and cross-user data leakage
- **Performance optimizations**:
  - User-specific WebSocket channels (not global broadcast)
  - Efficient `bulk_create()` for thread participant notifications
  - Exponential backoff on reconnection failures (2s → 4s → 8s → 16s, max 8x)
  - Heartbeat monitoring every 30s to detect stale connections
- **Signal integration**: All notification types (BADGE, REPLY, MENTION, THREAD_REPLY, moderation) automatically broadcast via `broadcast_notification_via_websocket()` helper
- **Error handling**: WebSocket broadcast failures don't break signal handlers - notifications still save to database
- **Connection lifecycle**: Auto-reconnection on network recovery, page visibility change, and authentication token refresh
- **Testing**: Comprehensive test suite (`opencontractserver/tests/test_notification_websocket.py`) covering authentication, IDOR prevention, concurrent connections, and signal integration
- **Network monitoring**: Integrated with `useNetworkStatus` hook for automatic reconnection on mobile screen unlock and network recovery

#### Network Recovery Implementation

- Uses `visibilitychange` event to detect page visibility changes
- Uses `online`/`offline` events to detect network status changes
- Configurable resume threshold (default 2s for NetworkStatusHandler, 1s for WebSocket hooks)
- Debounced refetch to prevent rapid repeated calls
- Graceful degradation: continues to work if events are not supported

#### Upload Modal Styling Improvements (Issue #696)

- **New styled components for upload modals** (`frontend/src/components/widgets/modals/UploadModalStyles.ts`): Comprehensive styled-components library with 25+ responsive components including `StyledUploadModal`, `DropZone`, `StepIndicator`, `FileListItem`, and more
- **Step indicator UI** for DocumentUploadModal showing progress through upload workflow (Select → Details → Corpus)
- **Modern gradient header** with icon and subtitle for both upload modals
- **Progress bar integration** showing real-time upload progress with success/error states

#### Mobile UI Improvements for Picker and Edit Message Modal (Issue #686)

- **Backend UpdateMessage mutation** (`config/graphql/conversation_mutations.py:455-619`):
  - New `UpdateMessageMutation` for editing existing thread messages
  - Validates CRUD permission on message or moderator status
  - Re-parses mentions when content is updated (with race condition protection - parsing happens before DB modifications)
  - Triggers agent responses for newly mentioned agents
  - Documented behavior: agents respond to ALL mentions, including re-mentions in edited messages
- **Frontend UPDATE_MESSAGE mutation** (`frontend/src/graphql/mutations.ts:2726-2760`): GraphQL mutation with TypeScript types
- **EditMessageModal component** (`frontend/src/components/threads/EditMessageModal.tsx`):
  - Full-screen modal on mobile for better touch interaction
  - Uses MessageComposer for consistent editing experience
  - Safe area insets for notched devices
  - Loading states and error handling
  - Custom unsaved changes confirmation modal (replaces browser `window.confirm()`)
  - Debounced content updates (150ms) for improved performance during typing
  - XSS protection documented: uses MarkdownMessageRenderer with `rehype-sanitize`
- **Message actions dropdown in MessageItem** (`frontend/src/components/threads/MessageItem.tsx:219-432`):
  - Desktop: Standard dropdown menu with Edit/Delete options
  - Mobile: Bottom sheet style for thumb-friendly interaction
  - Inline delete confirmation with mobile-optimized buttons
  - Backdrop overlay on mobile for visual focus

#### Improved Inline Reference Cards for Mentions (Issue #689)

- **Annotation mentions** now display the first ~24 characters of annotation text instead of cryptic IDs
  - Full annotation text accessible via hover tooltip
  - Falls back to label type if no raw text available
  - Location: `frontend/src/components/threads/MentionChip.tsx:212-229`
- **Document mentions** show document title with corpus context (e.g., "Document Title (in Corpus Name)")
  - Location: `frontend/src/components/threads/MessageComposer.tsx:361-375`
- **Corpus mentions** show corpus name instead of `@corpus:slug` format
  - Location: `frontend/src/components/threads/MessageComposer.tsx:351-359`
- **Shared constant** `MENTION_PREVIEW_LENGTH = 24` for consistent truncation across components
  - Location: `frontend/src/assets/configurations/constants.ts:6-8`
- **Text sanitization utility** for user-generated content to prevent XSS
  - Location: `frontend/src/utils/textSanitization.ts`
  - Unit tests: `frontend/src/utils/textSanitization.test.ts`
- **Component tests** for MentionChip covering all resource types and text truncation
  - Location: `frontend/tests/mention-chip.spec.tsx`

### Changed

#### Upload Modal Mobile Responsiveness (Issue #696)

- **DocumentUploadModal** (`frontend/src/components/widgets/modals/DocumentUploadModal.tsx`): Refactored to use new styled components with responsive grid layout for edit step
- **BulkUploadModal** (`frontend/src/components/widgets/modals/BulkUploadModal.tsx`): Complete visual overhaul with styled drop zone, file size display, and responsive layout
- **DocumentUploadList** (`frontend/src/components/documents/DocumentUploadList.tsx`): New drop zone styling with drag-active feedback and pulse animation
- **DocumentListItem** (`frontend/src/components/documents/DocumentListItem.tsx`): Improved file list items with proper touch targets (56px min-height, 64px on mobile), status icons, and delete button styling
- **Mobile-first breakpoints**: All upload modal components now have explicit breakpoints at 480px (mobile) and 768px (tablet)
- **Touch target compliance**: All interactive elements meet 44px minimum touch target size for mobile accessibility
- **Responsive action buttons**: Modal actions stack vertically on mobile for full-width tappable buttons
- **Custom scrollbar styling**: File list has styled scrollbars for visual polish

#### MentionChip Component Improvements (Issue #689)

- Extended `MentionChip` to support ANNOTATION type with green gradient styling
- Added default cases to all switch statements for TypeScript exhaustiveness checking
- Refactored `handleClick` to `handleActivation` accepting `React.MouseEvent | React.KeyboardEvent` union type (fixes unsafe `as any` assertion)
- Sanitized user-generated annotation text before display to prevent XSS

### Fixed

#### Mobile Layout for Picker Components (Issue #686)

- **Picker keyboard handling** (`MentionPicker.tsx:22-54`, `UnifiedMentionPicker.tsx:25-57`):
  - Added CSS environment variables (`env(safe-area-inset-bottom)`) for keyboard-aware positioning
  - Smooth slide-up animation for picker appearance
  - Max-height constraints using `min()` to prevent overflow on small screens
- **Touch targets** (`MentionPicker.tsx:83-108`, `UnifiedMentionPicker.tsx:96-108`):
  - Increased touch target size (52-60px min-height) for easier selection
  - Larger font size (15px) on mobile for readability
  - Mobile-specific border radius for rounded corners
- **MessageComposer mobile improvements** (`MessageComposer.tsx:48-93`):
  - Larger toolbar button touch targets (40x40px) on mobile
  - Increased gap between buttons for easier tapping

### Technical Details

#### Message Editing Tests (Issue #686)

- **New test for parent relationship preservation** (`opencontractserver/tests/test_conversation_mutations_graphql.py:1071-1168`):
  - Verifies that editing a reply message preserves its `parent_message` field
  - Ensures thread structure integrity when users edit replies
  - Part of comprehensive UpdateMessage mutation test suite

#### Upload Modal Architecture

- Styled-components with transient props (`$active`, `$selected`, `$status`) to prevent DOM attribute warnings
- CSS keyframe animations for drag-active pulse effect and fade-in modal transitions
- Gradient backgrounds using `linear-gradient(135deg, #667eea 0%, #764ba2 100%)` for visual consistency
- Semantic UI React components wrapped with styled-components for enhanced styling while preserving functionality

#### Permanent Deletion (Empty Trash) Functionality (PR #707)

- **Core deletion logic** (`opencontractserver/documents/versioning.py:617-760`):
  - `permanently_delete_document()`: Irreversible deletion with cascade cleanup
  - `permanently_delete_all_in_trash()`: Bulk deletion (empty trash) with partial success support
- **Cascade cleanup** deletes:
  - All DocumentPath records for the document in the corpus (entire history)
  - User annotations (non-structural) on the document
  - Relationships involving those annotations (uses Q objects to avoid duplicate counting)
  - DocumentSummaryRevision records for the document+corpus
  - The Document itself if no other corpus references it (Rule Q1)
- **Service layer** (`opencontractserver/corpuses/folder_service.py:1096-1181`): Permission-checked wrappers
- **GraphQL mutations** (`config/graphql/mutations.py:4069-4187`):
  - `PermanentlyDeleteDocument`: Delete single soft-deleted document
  - `EmptyTrash`: Delete all soft-deleted documents in corpus
  - Both enforce DELETE permission via django-guardian
- **Frontend UI** (`frontend/src/components/corpuses/folders/TrashFolderView.tsx`):
  - "Empty Trash" button with confirmation modal
  - Warning message explaining what will be permanently deleted
  - Auto-dismiss success/error messages with configurable durations
  - TypeScript type safety for all mutation responses
- **Comprehensive test suite** (`opencontractserver/tests/test_permanent_deletion.py`): 34 tests covering core logic, cascade cleanup, Rule Q1, permissions, GraphQL mutations, and edge cases

### Technical Details

- Partial deletions are allowed in bulk operations (each document deletion is atomic)
- Structural annotations are preserved (shared via StructuralAnnotationSet)
- Corpus-isolated deletion: Only affects target corpus, other corpus references preserved
- Composite index `[corpus, is_current, is_deleted]` on DocumentPath for efficient trash queries

#### Mobile-Friendly Corpus Modal

- **New CorpusModal component** (`frontend/src/components/corpuses/CorpusModal.tsx`): Purpose-built modal replacing CRUDModal for corpus create/edit/view operations with mobile-first design
- **13 comprehensive component tests** (`frontend/tests/corpus-modal.ct.tsx`): Full test coverage for all modal modes and interactions
- **Smart change detection for EDIT mode**: Only sends changed fields to backend using original value comparison (`CorpusModal.tsx:498-519`)
- **ARIA accessibility**: CloseButton includes `aria-label="Close modal"` for screen reader users

### Changed

#### Corpus Modal Architecture

- **Replaced CRUDModal with CorpusModal**: Simplified form handling with controlled inputs instead of complex JSON Schema Form library
- **Removed debug console.log statements** (`Corpuses.tsx`): Cleaned up 4 debug logging statements

### Technical Details

#### Corpus Modal Implementation

- Mobile-first responsive design: 16px input font prevents iOS auto-zoom, 48px min touch targets
- Proper TypeScript types: Icon type is `string | null` (not ArrayBuffer), slug field uses existing type from RawCorpusType
- isDirty computed by comparing current values against stored original values (not just tracking changes)

#### Social Media Preview (OG Metadata) System (PR #701)

- **Cloudflare Worker for social media previews** (`cloudflare-og-worker/`): Intercepts requests from social media crawlers (Facebook, Twitter, LinkedIn, Discord, Slack, etc.) and returns HTML with Open Graph meta tags for rich link previews
- **Public OG metadata GraphQL queries** (`config/graphql/queries.py:3235-3403`): New unauthenticated queries for fetching public corpus, document, thread, and extract metadata
  - `ogCorpusMetadata`: Returns title, description, icon, document count for public corpuses
  - `ogDocumentMetadata`: Returns title, description, icon for public standalone documents
  - `ogDocumentInCorpusMetadata`: Returns document metadata with corpus context
  - `ogThreadMetadata`: Returns discussion thread metadata (title, corpus, message count)
  - `ogExtractMetadata`: Returns data extract metadata
- **Worker architecture**: Modular TypeScript implementation with crawler detection, URL parsing, metadata fetching, and HTML generation
- **Comprehensive documentation** (`docs/architecture/social-media-previews.md`): Architecture overview, deployment guide, and testing instructions

### Fixed

#### New Corpus Modal Mobile Issues (Issue #702)

- **Mobile form data loss in CorpusModal** (`frontend/src/components/corpuses/CorpusModal.tsx:406-418`):
  - Fixed fields clearing when typing on mobile by tracking modal open transitions with `prevOpenRef` instead of resetting form on every render
  - The original `useEffect` was running on every `corpus` or `open` change, causing form state to reset during keyboard/focus events on mobile
- **Slow embedder loading** (`frontend/src/components/widgets/CRUD/EmbedderSelector.tsx:43-46`):
  - Changed Apollo query to `cache-first` policy since embedders rarely change
  - Prevents unnecessary network requests when reopening CorpusModal
- **Cramped mobile layout** (`frontend/src/components/corpuses/CorpusModal.tsx:327-333`, `frontend/src/components/widgets/file-controls/FilePreviewAndUpload.tsx:54-57,129-135`):
  - Reduced icon upload area max-width from 200px to 150px on mobile
  - Reduced ImagePreview height from 150px to 100px on mobile
  - Made EditBadge smaller and better positioned on mobile viewports

#### Production Deployment

- **Missing COLLECTFAST_STRATEGY for GCP storage backend** (`config/settings/base.py:436`): Added `collectfast.strategies.gcloud.GoogleCloudStrategy` for GCP deployments. Previously, `collectfast` was installed in production but `COLLECTFAST_STRATEGY` was only configured for AWS, causing `collectstatic` to fail with `ImproperlyConfigured: No strategy configured` error when using `STORAGE_BACKEND=GCP`.
- **GCS static files ACL incompatible with uniform bucket-level access** (`opencontractserver/utils/storages.py:38`): Changed `StaticRootGoogleCloudStorage.default_acl` from `"publicRead"` to `None`. GCS buckets with uniform bucket-level access enabled cannot use per-object ACLs; access must be controlled via IAM policies at the bucket level instead.

#### Social Media Preview Security & Performance Fixes (PR #701 remediation)

- **Prevented potential infinite loop in worker passthrough** (`cloudflare-og-worker/src/index.ts:23-42`): Added `passToOrigin()` helper function with `X-OG-Worker-Pass` header to prevent Cloudflare Worker from re-invoking itself on route-based deployments
- **Added rate limiting to public OG queries** (`config/graphql/queries.py`): All five OG metadata resolvers now have `@graphql_ratelimit(key="ip", rate="60/m", group="og_metadata")` to prevent abuse and DoS attacks
- **Fixed N+1 query in corpus document count** (`config/graphql/queries.py:3250-3255`): Changed from `corpus.documents.count()` to `Corpus.objects.annotate(doc_count=Count("documents"))` for single-query optimization
- **Fixed N+1 query in thread message count** (`config/graphql/queries.py:3359-3364`): Changed from `thread.messages.count()` to `Conversation.objects.annotate(msg_count=Count("messages"))` for single-query optimization
- **Added input validation for decodeURIComponent** (`cloudflare-og-worker/src/parser.ts:88-95`): Wrapped `decodeURIComponent()` in try-catch to handle malformed URLs gracefully instead of crashing the worker
- **Unified description truncation** (`config/graphql/queries.py`): Removed redundant Python-side `[:500]` truncation; description truncation now handled solely by the worker at 200 characters for consistency

### Added

#### Mobile UI/UX Improvements for Corpus Navigation

- **Mobile-first folder sidebar defaults**: Sidebar now collapses by default on mobile/tablet devices (≤768px) to maximize document viewing area
- **Mobile bottom-sheet mention pickers**: User, resource, and unified mention pickers now display as bottom sheets on mobile (≤600px) for thumb-friendly interaction
- **Discussions and Analytics quick access**: Added icon buttons to CorpusHome stat cards for direct navigation to Discussions and Analytics tabs
- **Sidebar auto-close behavior**: Folder sidebar automatically closes on mobile/tablet after folder selection for seamless navigation
- **Mobile sidebar backdrop overlay**: Semi-transparent backdrop behind mobile sidebar for visual focus and easy dismissal
- **Escape key accessibility**: Mobile sidebar can now be dismissed with Escape key for keyboard accessibility
- **TABLET_BREAKPOINT constant**: Added to `constants.ts` for consistent responsive breakpoint management across components

### Fixed

#### Mobile UI/UX Fixes

- **Settings button variable name bug** (`frontend/src/components/corpuses/CorpusHome.tsx:780`): Fixed `canUpdate` → `canEdit` reference error that prevented Settings button from displaying for users with update permissions
- **FAB z-index layering** (`frontend/src/views/Corpuses.tsx:1320`): Raised FAB z-index from 100 to 150 to ensure visibility above folder sidebar toggle (z-index: 101)
- **Explicit z-index layering**: Made mobile sidebar z-index layering explicit (backdrop: 98, toggle button: 99) to prevent fragile DOM-order-dependent behavior

#### Mobile Responsive Styling for Settings and Badge Widgets (PR #690)

- **UserSettingsModal responsive styling** (`frontend/src/components/modals/UserSettingsModal.tsx:14-80`):
  - Modal takes 95% width on mobile (≤768px) with reduced padding
  - Form groups stack vertically on small screens (≤480px) for single-column layout
  - Action buttons display full-width and stack vertically (Save above Close) on mobile
  - Added `styled-components` import and styled wrapper components
- **Badge component touch support** (`frontend/src/components/badges/Badge.tsx:23-41, 96-112, 145-199`):
  - Added tap-to-toggle tooltip on touch devices (detects via `ontouchstart`)
  - Created `MobileOverlay` backdrop for dismissing badge popups by tapping outside
  - Popup centers on mobile screens using fixed positioning instead of floating-ui
  - Increased touch target size (min-height 36px, larger padding)
  - Disabled hover transforms on touch devices using `@media (hover: none)`
- **UserBadges container responsive layout** (`frontend/src/components/badges/UserBadges.tsx:18-27, 37-48, 58-61`):
  - Reduced padding and gap on mobile viewports
  - Badges center-aligned on mobile for better visual balance
  - Empty state and header text sizes reduced on mobile
- **GlobalSettingsPanel responsive grid** (`frontend/src/components/admin/GlobalSettingsPanel.tsx:11-67, 82-104, 119-123, 137-139, 148-150, 163-168`):
  - Container padding reduced on mobile (2rem → 1rem → 0.75rem)
  - Settings grid switches to single column on small mobile (≤480px)
  - Card content padding reduced progressively on smaller screens
  - Touch-friendly card interactions with active state feedback (scale 0.98)
  - "Coming Soon" badge displays on its own line on very small screens

### Changed

#### Mobile UI/UX Refactoring

- **Hardcoded breakpoints replaced with constants**: Updated all hardcoded `768px` references in `FolderDocumentBrowser.tsx` and `folderAtoms.ts` to use `TABLET_BREAKPOINT` constant for maintainability
- **Improved breakpoint documentation**: Added detailed JSDoc comment in `folderAtoms.ts` explaining why `TABLET_BREAKPOINT` (768px) is used for sidebar collapse rather than `MOBILE_VIEW_BREAKPOINT` (600px)

## [3.0.0.b3] - 2025-12-11

### Added

#### v3.0.0.b3 Migration Tools (Issue #654)

- **New management command: `validate_v3_migration`**

  - Pre-flight and post-migration validation for dual-tree versioning and structural annotations
  - Checks: version_tree_id, is_current, DocumentPath records, XOR constraints, structural set uniqueness
  - Reports structural migration candidates
  - Options: `--verbose`, `--fix`
  - Location: `opencontractserver/documents/management/commands/validate_v3_migration.py`

- **New management command: `migrate_structural_annotations`**

  - Optional command to migrate structural annotations to shared StructuralAnnotationSet objects
  - Creates StructuralAnnotationSet by content hash (pdf_file_hash) for storage efficiency
  - Moves structural annotations/relationships from document FK to structural_set FK
  - Documents with same hash share StructuralAnnotationSet (O(1) storage vs O(n))
  - Options: `--dry-run`, `--document-id`, `--corpus-id`, `--batch-size`, `--verbose`, `--force`
  - Location: `opencontractserver/annotations/management/commands/migrate_structural_annotations.py`

- **Comprehensive migration test suite** (`opencontractserver/tests/test_v3_migration.py`)

  - DocumentVersioningMigrationTests: version_tree_id, is_current, DocumentPath creation
  - XORConstraintTests: Annotation/Relationship XOR constraint validation
  - StructuralMigrationCommandTests: Management command functionality, idempotency
  - RollbackAndEdgeCaseTests: Edge cases, error handling, data integrity
  - ValidationCommandTests: validate_v3_migration command testing
  - 25+ human-readable tests covering all migration scenarios

- **Migration documentation** (`docs/migrations/v3_upgrade_guide.md`)
  - Pre-upgrade checklist with backup recommendations
  - Step-by-step migration instructions for production and development
  - Optional structural annotation migration guide
  - Rollback procedure documentation
  - FAQ addressing common concerns (XOR constraint safety, storage savings, incremental migration)

#### Discovery Landing Page (New)

- **Beautiful, modern landing page** as the main entry point for the application

  - Replaces direct redirect to /corpuses with a unified discovery experience
  - Different content for anonymous vs authenticated users
  - Responsive design with mobile-first approach
  - Location: `frontend/src/views/DiscoveryLanding.tsx`

- **New landing page components** (`frontend/src/components/landing/`)

  - `HeroSection.tsx`: Animated hero with gradient backgrounds, floating icons, and global search
  - `StatsBar.tsx`: Community metrics display with animated counters (users, collections, documents, threads, annotations, weekly active)
  - `TrendingCorpuses.tsx`: Card grid of popular document collections with engagement metrics
  - `RecentDiscussions.tsx`: List of recent public discussions with badges for pinned/locked threads
  - `TopContributors.tsx`: Leaderboard-style display of top community contributors with reputation scores
  - `CallToAction.tsx`: Conversion section for anonymous users with feature highlights
  - All components feature modern UI/UX: glass morphism, smooth Framer Motion animations, skeleton loaders

- **GraphQL queries for discovery data** (`frontend/src/graphql/landing-queries.ts`)

  - `GET_DISCOVERY_DATA`: Unified query fetching corpuses, conversations, community stats, and leaderboard
  - `GET_TRENDING_CORPUSES`: Public corpuses with engagement metrics
  - `GET_RECENT_DISCUSSIONS`: Recent threads with pagination
  - `GET_COMMUNITY_STATS`: Platform-wide statistics
  - `GET_GLOBAL_LEADERBOARD`: Top contributors with badges

- **Route integration**

  - Root path (`/`) now displays DiscoveryLanding instead of redirecting to /corpuses
  - Location: `frontend/src/App.tsx:377-382`

- **Component tests** (`frontend/tests/landing-components.spec.tsx`)
  - HeroSection tests: rendering, authenticated/anonymous variants, search submission
  - StatsBar tests: stats rendering, loading state, null handling
  - TrendingCorpuses tests: corpus cards, loading skeletons, empty state
  - RecentDiscussions tests: discussion items, pinned badges, reply counts
  - TopContributors tests: contributor cards, reputation scores, leaderboard button
  - CallToAction tests: anonymous visibility, authenticated hiding
  - DiscoveryLanding integration tests: full page rendering, section visibility

#### Permission Audit Remediation - Query Optimizers

- **New `UserQueryOptimizer`** for centralized user profile visibility logic

  - Respects `is_profile_public` privacy setting
  - Private profiles visible via corpus membership with > READ permission
  - Inactive users filtered out (except for superusers)
  - IDOR-safe visibility checks
  - Location: `opencontractserver/users/query_optimizer.py`

- **New `BadgeQueryOptimizer`** for centralized badge visibility logic

  - Badge visibility follows recipient's profile privacy rules
  - Corpus-specific badges visible only to corpus members
  - Own badges always visible regardless of privacy
  - IDOR-safe visibility checks
  - Location: `opencontractserver/badges/query_optimizer.py`

- **New `DocumentActionsQueryOptimizer`** for document-related actions

  - Centralized permission logic for corpus actions, extracts, and analysis rows
  - Follows least-privilege model: `Effective Permission = MIN(document_permission, corpus_permission)`
  - Integrates with ExtractQueryOptimizer and AnalysisQueryOptimizer
  - Location: `opencontractserver/documents/query_optimizer.py`

- **Comprehensive permission test suites** (40 tests total)

  - `opencontractserver/tests/permissioning/test_user_visibility.py` - 16 tests for user profile visibility
  - `opencontractserver/tests/permissioning/test_badge_visibility.py` - 13 tests for badge visibility
  - `opencontractserver/tests/permissioning/test_document_actions_permissions.py` - 11 tests for document actions

- **Updated permissioning documentation**
  - Added Section 8: User Profile and Badge Visibility
  - Added Section 9: Document Actions Permissions
  - Added callouts for new privacy features
  - Updated Key Changes table with new optimizer rows
  - Location: `docs/permissioning/consolidated_permissioning_guide.md`

#### Corpus Engagement Analytics Dashboard (Issue #579)

- **New CorpusEngagementDashboard component** displaying comprehensive engagement metrics

  - Thread metrics: total threads, active threads, average messages per thread
  - Message activity: total messages, 7-day and 30-day message counts with bar chart visualization
  - Community engagement: unique contributors, active contributors (30d), total upvotes
  - Auto-refresh every 5 minutes with last updated timestamp
  - Mobile-responsive design with conditional layouts and grid systems
  - Location: `frontend/src/components/analytics/CorpusEngagementDashboard.tsx`

- **GraphQL integration for engagement metrics**

  - New query: `GET_CORPUS_ENGAGEMENT_METRICS` with TypeScript interfaces
  - Leverages existing backend `CorpusEngagementMetrics` model (already tested)
  - Location: `frontend/src/graphql/queries.ts:3873-3979`

- **Analytics tab in Corpus view**

  - New tab with BarChart3 icon next to Discussions tab
  - Conditionally rendered based on corpus ID availability
  - Location: `frontend/src/views/Corpuses.tsx:2209-2216`

- **Dependencies**
  - Added recharts@3.4.1 for data visualization (BarChart, ResponsiveContainer, Tooltip, Legend)
  - Added react-countup for animated number counters

#### Thread Search UI (Issue #580)

- **Backend pagination support for conversation search**

  - Updated `searchConversations` resolver to use `relay.ConnectionField` with cursor-based pagination
  - Supports `first`, `after`, `last`, `before` parameters for efficient result pagination
  - Returns paginated structure with `edges`, `pageInfo`, and `totalCount`
  - Location: `config/graphql/queries.py:1659-1748`

- **GraphQL queries and TypeScript types with pagination**

  - Updated `SEARCH_CONVERSATIONS` query to support paginated results
  - Added pagination parameters: `first`, `after`, `last`, `before`
  - Enhanced TypeScript interfaces with connection structure (edges, nodes, cursors, pageInfo)
  - Includes full thread metadata: chatMessages count, isPinned, isLocked, corpus/document references
  - Location: `frontend/src/graphql/queries.ts:3923-4059`

- **New search components** (`frontend/src/components/search/`)

  - `SearchBar.tsx`: Search input with clear button and Enter key support
  - `SearchFilters.tsx`: Filter by conversation type with clear filters button
  - `SearchResults.tsx`: Results display with pagination, reuses ThreadListItem component
  - `ThreadSearch.tsx`: Main search container with debounced query (300ms) and pagination
  - All components follow existing design patterns and are mobile-responsive

- **Embedded search in Corpus Discussions view**

  - Added tab navigation to switch between "All Threads" and "Search"
  - Search scoped to current corpus when embedded
  - Location: `frontend/src/components/discussions/CorpusDiscussionsView.tsx`

- **Standalone /threads route**

  - New dedicated search page accessible at `/threads`
  - Global search across all accessible discussions
  - Location: `frontend/src/views/ThreadSearchRoute.tsx`, `frontend/src/App.tsx:421`

- **Backend tests for paginated search**

  - Tests verify pagination structure (edges, pageInfo, totalCount)
  - Tests verify cursor-based pagination with multiple pages
  - Location: `opencontractserver/tests/test_conversation_search.py:609-743`

- **Frontend component tests** (18 tests, 100% passing)

  - SearchBar component tests (5 tests): input rendering, search icon, clear button, Enter key submission
  - SearchFilters component tests (5 tests): filter rendering, option counting, selected state, clear filters button
  - SearchResults component tests (4 tests): loading state, empty state, no results state, results rendering
  - ThreadSearch component tests (4 tests): search bar integration, filters toggle, corpus-scoped search
  - Location: `frontend/tests/search-components.ct.tsx`

- **Enhanced backend test coverage for conversation search** (Issue #580 - Coverage Improvement)
  - Added `GraphQLResolverEdgeCasesTest` class with 8 new comprehensive tests
  - Tests cover GraphQL resolver edge cases including:
    - Default embedder path fallback when no corpus/document ID provided
    - Error handling when DEFAULT_EMBEDDER_PATH is not configured
    - Reverse pagination with `last` and `before` parameters
    - Multiple result handling and pagination behavior
    - Message search with various filter combinations
  - Coverage improvements target previously untested code paths in `config/graphql/queries.py:1711-1722, 1797-1808`
  - Location: `opencontractserver/tests/test_conversation_search.py:2666-3050`

#### Structural Annotation Sets (Phase 2.5)

- **New `StructuralAnnotationSet` model** for shared, immutable structural annotations

  - Content-hash based uniqueness (`content_hash` field)
  - Stores parser metadata (`parser_name`, `parser_version`, `page_count`, `token_count`)
  - Stores shared parsing artifacts (`pawls_parse_file`, `txt_extract_file`)
  - Location: `opencontractserver/annotations/models.py`

- **Document → StructuralAnnotationSet FK** with PROTECT on delete

  - Multiple corpus-isolated documents can share the same structural annotation set
  - Eliminates duplication of structural annotations across corpus copies
  - Location: `opencontractserver/documents/models.py:119-127`

- **Annotation.structural_set FK** with XOR constraint

  - Annotations now belong to EITHER a document OR a structural_set (not both, not neither)
  - Database constraint: `annotation_has_single_parent`
  - Location: `opencontractserver/annotations/models.py`

- **Relationship.structural_set FK** with XOR constraint

  - Same pattern as Annotation for relationships
  - Database constraint: `relationship_has_single_parent`
  - Location: `opencontractserver/annotations/models.py`

- **Database migrations**

  - `opencontractserver/annotations/migrations/0048_add_structural_annotation_set.py`
  - `opencontractserver/documents/migrations/0026_add_structural_annotation_set.py`

- **Comprehensive test suite** (32 tests)
  - `opencontractserver/tests/test_structural_annotation_sets.py` (22 tests)
  - `opencontractserver/tests/test_structural_annotation_portability.py` (10 tests)

### Fixed

#### Permission Audit Remediation - GraphQL Resolver Fixes

1. **User profile visibility not respecting privacy settings**

   - **File**: `config/graphql/queries.py` - `resolve_user_by_slug`, `resolve_search_users_for_mention`
   - **Issue**: Resolvers returned users without checking `is_profile_public` or corpus membership
   - **Fixed**: Now uses `UserQueryOptimizer` for proper privacy filtering
   - **Impact**: Private user profiles no longer visible to unauthorized users

2. **Badge visibility not respecting recipient privacy**

   - **File**: `config/graphql/queries.py` - `resolve_user_badges`, `resolve_user_badge`
   - **Issue**: Badge awards were visible regardless of recipient's profile privacy
   - **Fixed**: Now uses `BadgeQueryOptimizer` which filters by recipient visibility
   - **Impact**: Badges of private users no longer leaked to unauthorized viewers

3. **Document actions missing permission checks**

   - **File**: `config/graphql/queries.py` - `resolve_document_corpus_actions`
   - **Issue**: Inline permission checks were inconsistent with least-privilege model
   - **Fixed**: Now uses `DocumentActionsQueryOptimizer` for centralized permission logic
   - **Impact**: Document-related data properly filtered by document AND corpus permissions

4. **Assignment resolver using incorrect visible_to_user signature**

   - **File**: `config/graphql/queries.py` - `resolve_assignments`, `resolve_assignment`
   - **Issue**: Called `Assignment.objects.visible_to_user(info.context.user)` but manager expected different signature
   - **Fixed**: Updated to use correct manager method call pattern
   - **Impact**: Assignment queries now properly filter by user visibility

5. **Unused local imports shadowing top-level imports**
   - **File**: `config/graphql/queries.py` - lines 2810, 2990
   - **Issue**: Local `UserBadge` imports inside resolvers were redundant and caused flake8 warnings
   - **Fixed**: Removed redundant local imports, using top-level import
   - **Impact**: Cleaner code, no shadowing warnings

#### Thread Search (Issue #580)

6. **Anonymous user null reference in searchConversations resolver**
   - **File**: `config/graphql/queries.py:1725`
   - **Issue**: Resolver accessed `info.context.user.is_anonymous` without checking if user was `None`, causing AttributeError in tests with anonymous users
   - **Fixed**: Added null check before accessing `is_anonymous` attribute
   - **Impact**: Anonymous user search queries now work correctly without AttributeError

#### Critical Production Code Fixes

2. **Missing parsing artifacts in corpus copies**

   - **Files**: `opencontractserver/corpuses/models.py:445-451`, `opencontractserver/documents/versioning.py:238-244`
   - **Issue**: When creating corpus-isolated document copies, essential parsing artifacts were not being copied
   - **Fixed**: Added copying of `pawls_parse_file`, `txt_extract_file`, `icon`, `md_summary_file`, `page_count`
   - **Impact**: Corpus copies now have all parsing data needed for annotation, search, and display

3. **Missing `is_public` inheritance in corpus copies**

   - **Files**: `opencontractserver/corpuses/models.py:451`, `opencontractserver/documents/versioning.py:244`
   - **Issue**: Public documents became private when added to a corpus (copy didn't inherit `is_public`)
   - **Fixed**: Added `is_public=document.is_public` to corpus copy creation
   - **Impact**: Document visibility is now correctly preserved across corpus isolation

4. **NULL hash deduplication bug**

   - **File**: `opencontractserver/corpuses/models.py:414-425`
   - **Issue**: All documents without PDF content hashes were incorrectly treated as duplicates
   - **Fixed**: Added null check: `if document.pdf_file_hash is not None:` before hash-based deduplication
   - **Impact**: Documents without hashes are now correctly treated as distinct documents

5. **Structural annotation portability**

   - **Files**: `opencontractserver/corpuses/models.py:456`, `opencontractserver/documents/versioning.py:248`
   - **Issue**: Structural annotations were not traveling with documents when added to multiple corpuses
   - **Fixed**: Corpus copies now inherit `structural_annotation_set` from source document
   - **Impact**: Structural annotations are shared (not duplicated) across corpus-isolated copies

6. **GraphQL corpus.documents field missing**

   - **Files**: `config/graphql/graphene_types.py:1179-1184`, `config/graphql/graphene_types.py:1297-1302`
   - **Issue**: After corpus isolation migration (removing M2M documents field), GraphQL queries for `corpus.documents` returned empty because no explicit field declaration existed
   - **Fixed**: Added explicit `DocumentTypeConnection` class and `documents = relay.ConnectionField()` declaration to CorpusType
   - **Impact**: GraphQL queries now correctly resolve documents via DocumentPath-based relationships

7. **Parser `save_parsed_data()` using old M2M relationship**

   - **File**: `opencontractserver/pipeline/base/parser.py:126-133`
   - **Issue**: `save_parsed_data()` used deprecated `corpus.documents.add()` M2M method which no longer exists
   - **Fixed**: Updated to use `corpus.add_document(document=document, user=user)` for corpus isolation
   - **Impact**: Parsers can now correctly associate documents with corpuses during processing

8. **Document mention resolver using old M2M relationship**

   - **File**: `config/graphql/queries.py:976-1015`
   - **Issue**: `resolve_search_documents_for_mention()` queried via `corpus__in` M2M relationship which no longer exists
   - **Fixed**: Updated to query via `DocumentPath` with `is_current=True, is_deleted=False` filters
   - **Impact**: Document mention autocomplete now correctly finds documents in corpuses

9. **BaseFixtureTestCase not adding documents to corpus**
   - **File**: `opencontractserver/tests/base.py:385-399`
   - **Issue**: Test setup created corpus but didn't add fixture documents to it via DocumentPath
   - **Fixed**: Added loop to call `corpus.add_document()` for each fixture document and update references to corpus copies
   - **Impact**: WebSocket and other tests now properly test with documents in corpus context

### Changed

#### Test Suite Updates for Corpus Isolation Architecture

- **Removed deprecated legacy manager tests**

  - **File**: `opencontractserver/tests/test_document_path_migration.py`
  - **Removed**: Test classes for deprecated `DocumentCorpusRelationshipManager` (20+ tests)
  - **Reason**: The backward compatibility M2M manager was removed in Issue #654 Phase 2
  - **Note**: `DocumentCorpusRelationshipManager` in `opencontractserver/documents/managers.py` remains as documentation but is unused
  - **Impact**: Improved test clarity by removing tests for code that never executes

- **Permission assignment order** in test setups

  - Moved permission assignment AFTER `add_document()` calls
  - Ensures permissions are assigned to corpus copies, not originals
  - Files: `test_visibility_managers.py`, `test_resolvers.py`, `test_permissioning.py`, `test_version_aware_query_optimizer.py`

- **Document count expectations**

  - Updated tests to account for both originals and corpus copies existing
  - Example: Owner sees 6 documents (3 originals + 3 corpus copies) instead of 3
  - Files: `test_visibility_managers.py`, `test_resolvers.py`

- **Document-to-corpus linking**

  - Changed from M2M `corpus.documents.add()` to `corpus.add_document()`
  - File: `test_custom_permission_filters.py:211-213`

- **Corpus document queries**
  - Updated tests to query corpus documents via DocumentPath, not M2M
  - File: `test_bulk_document_upload.py:305-313`

### Technical Details

#### Architectural Changes

The structural annotation set feature implements Phase 2.5 of the dual-tree versioning architecture:

1. **Content-based deduplication**: Structural annotations are tied to content hash, not individual documents
2. **Corpus isolation compatibility**: When a document is copied to multiple corpuses, all copies share the same structural annotation set
3. **Immutability guarantee**: Structural annotations in shared sets cannot be modified (protected by PROTECT on delete)
4. **XOR constraints**: Database-level enforcement that annotations belong to either a document or a structural set

#### File Changes Summary

**New Files:**

- `opencontractserver/tests/test_structural_annotation_sets.py`
- `opencontractserver/tests/test_structural_annotation_portability.py`
- `opencontractserver/annotations/migrations/0048_add_structural_annotation_set.py`
- `opencontractserver/documents/migrations/0026_add_structural_annotation_set.py`
- `docs/architecture/STRUCTURAL_ANNOTATION_SETS.md`
- `CHANGELOG.md`

**Modified Files:**

- `opencontractserver/annotations/models.py` - Added StructuralAnnotationSet model, updated Annotation/Relationship models
- `opencontractserver/documents/models.py` - Added structural_annotation_set FK
- `opencontractserver/corpuses/models.py` - Fixed add_document() to copy all artifacts + structural set
- `opencontractserver/documents/versioning.py` - Fixed import_document() to copy all artifacts + structural set
- `config/graphql/graphene_types.py` - Added DocumentTypeConnection and explicit documents field for CorpusType
- `config/graphql/queries.py` - Updated document mention resolver to use DocumentPath
- `opencontractserver/pipeline/base/parser.py` - Updated save_parsed_data() to use add_document()
- `opencontractserver/tests/base.py` - Updated BaseFixtureTestCase to add documents to corpus
- `opencontractserver/tests/test_visibility_managers.py` - Updated for corpus isolation
- `opencontractserver/tests/test_resolvers.py` - Updated for corpus isolation
- `opencontractserver/tests/test_bulk_document_upload.py` - Updated for corpus isolation
- `opencontractserver/tests/permissioning/test_permissioning.py` - Updated for corpus isolation
- `opencontractserver/tests/permissioning/test_custom_permission_filters.py` - Updated for corpus isolation
- `opencontractserver/tests/permissioning/test_version_aware_query_optimizer.py` - Updated for corpus isolation
- `CLAUDE.md` - Added Changelog Maintenance section

### Fixed (Continued)

10. **Query optimizer missing structural_set annotations**

- **Files**: `opencontractserver/annotations/query_optimizer.py:189-212, 273-301, 541-564, 624-643`
- **Issue**: `AnnotationQueryOptimizer.get_document_annotations()` and `RelationshipQueryOptimizer.get_document_relationships()` only queried by `document_id`, missing annotations/relationships stored in `structural_set` (which have `document_id=NULL`)
- **Impact**: GraphQL queries using query optimizer (most annotation/relationship queries) did NOT return structural annotations from structural sets - only vector store had the dual-query logic
- **Fixed**:
  - Added document fetch with `select_related("structural_annotation_set")` for efficiency
  - Built OR filter: `Q(document_id=X) | Q(structural_set_id=Y, structural=True)` to query BOTH sources
  - Updated corpus filtering to preserve structural_set items (which have `corpus_id=NULL`)
  - Applied same fix to both AnnotationQueryOptimizer and RelationshipQueryOptimizer
- **Tests Added**: `opencontractserver/tests/test_query_optimizer_structural_sets.py` (10 comprehensive integration tests)
- **Test Results**: All 42 structural annotation tests pass (10 new + 32 existing)

11. **Vector store returning duplicate results**

- **File**: `opencontractserver/shared/mixins.py:40-89`
- **Issue**: `search_by_embedding()` method returned duplicate results (2x, 4x, 6x expected counts) when annotations had multiple Embedding rows with the same `embedder_path`
- **Root Cause**: JOIN to Embedding table created cartesian product - if annotation had 2 Embedding rows, JOIN produced 2 result rows
- **Investigation**: Confirmed annotations have multiple Embedding rows due to dual FK relationship:
  1.  `Embedding.annotation` FK (one-to-many): annotation can have multiple embeddings
  2.  `Annotation.embeddings` FK (many-to-one): annotation points to single "primary" embedding
- **Fixed**: Hybrid deduplication approach in `search_by_embedding()`:
  1.  Order by `id, similarity_score` and apply PostgreSQL `DISTINCT ON (id)`
  2.  Materialize query to list
  3.  Sort in Python by `similarity_score`
  4.  Return top_k results
- **Rationale**: PostgreSQL `DISTINCT ON` requires the distinct field to be first in ORDER BY, conflicting with need to order by similarity_score. Hybrid approach ensures correctness.
- **Test Results**: All 9 version-aware vector store tests now pass (previously all 8 failing)

12. **Vector store excluding structural annotations from StructuralAnnotationSet**

- **File**: `opencontractserver/llms/vector_stores/core_vector_stores.py:168-196, 221-270`
- **Issue**: Version filtering excluded ALL structural annotations from structural sets, causing vector search to return 0 results
- **Root Cause - Filter Ordering Bug**:
  1.  `only_current_versions` filter applied `Q(document__is_current=True)` (line 170)
  2.  This creates `INNER JOIN` on document table
  3.  Structural annotations have `document_id=NULL` (stored in StructuralAnnotationSet)
  4.  NULL document_id fails the JOIN → structural annotations excluded
  5.  This happened BEFORE document/corpus scoping (lines 221-270)
  6.  Result: Scoping logic tried to include structural annotations, but they were already filtered out
- **Symptoms**:
  - Initial queryset: 1344 annotations
  - After version filter: 0 results (all structural annotations excluded)
  - WebSocket tests failed with no ASYNC_CONTENT (agent had no context)
- **Fixed**:
  - Modified version filter to preserve structural annotations:
    ```python
    active_filters &= Q(document__is_current=True) | Q(
        document_id__isnull=True, structural=True
    )
    ```
  - Logic: Annotations with document FK must have `is_current=True`, structural annotations (no document FK) pass through
  - Later scoping filters by `structural_set_id` to ensure only relevant structural annotations included
- **Comments Added**: Comprehensive inline documentation explaining:
  - Why structural annotations have `document_id=NULL`
  - Filter ordering and interaction between version filter and scoping
  - Two-phase filtering approach (version → scoping)
- **Test Results**:
  - Vector store now finds 336 annotations (was 0)
  - SQL shows correct filter: `(document.is_current OR (annotation.document_id IS NULL AND structural))`

13. **Agent tool execution failing due to list/QuerySet type mismatch**

- **Files**: `opencontractserver/llms/vector_stores/core_vector_stores.py:30-90`
- **Issue**: After deduplication fix (#10), `search_by_embedding()` returns list instead of QuerySet, breaking agent tool execution
- **Root Cause - Type Assumption**:
  1.  Deduplication fix materialized QuerySet to list for DISTINCT ON + Python sorting
  2.  Helper functions `_safe_queryset_info()` and `_safe_execute_queryset()` assumed QuerySet
  3.  Called `.count()` method on lists (which don't have `.count()` for length)
  4.  Agent's `similarity_search` tool failed silently
  5.  LLM called tool → tool execution broke → no second LLM call → no ASYNC_CONTENT
- **Symptoms**:
  - Only 1 LLM API call in cassettes (should be 2: tool call + final answer)
  - Agent produced ASYNC_START and ASYNC_FINISH but no ASYNC_CONTENT
  - Cassette files abnormally small (27KB vs expected 50-70KB)
- **Fixed**: Updated helper functions to handle both QuerySets and lists:

  ```python
  async def _safe_queryset_info(queryset, description: str) -> str:
      if isinstance(queryset, list):
          return f"{description}: {len(queryset)} results"
      # ... handle QuerySet

  async def _safe_execute_queryset(queryset) -> list:
      if isinstance(queryset, list):
          return queryset  # Already materialized
      # ... execute QuerySet
  ```

- **Test Results**:
  - Tool execution now succeeds ✅
  - Cassettes show 2 LLM calls (tool call + response) ✅
  - Cassette size increased to 55KB (proper content) ✅
  - WebSocket tests still fail (different issue: agent streaming layer - not tool execution)

### Known Issues

1. **Pre-existing annotation visibility limitation**: `AnnotationQuerySet.visible_to_user()` doesn't check object-level permissions (only checks `is_public` or `creator`). This was not introduced by these changes but is more apparent with corpus isolation.

2. **WebSocket conversation tests** (`ConversationSourceLoggingTestCase`): Tests fail with no ASYNC_CONTENT messages.
   - **Current Status**: Tests fail with `AssertionError: [] is not true : At least one ASYNC_CONTENT expected`
   - **Vector Store Issues RESOLVED**:
     1. ✅ Vector store deduplication (issue #10 above) - All 9 vector store tests pass
     2. ✅ Query optimizer structural_set support (issue #9 above) - All 42 structural annotation tests pass
     3. ✅ Vector store version filtering (issue #11 above) - Now finds 336 annotations (was 0)
   - **Remaining Issue**: Agent produces no streaming content despite finding annotations
     - Vector store successfully returns 336 annotations to agent
     - Agent runs but produces no ASYNC_CONTENT messages (only ASYNC_START and ASYNC_FINISH)
     - Likely cause: VCR cassette mocking issue or LLM API configuration
     - **NOT a vector store or structural annotation architecture issue**
   - **Next Steps**: Investigate VCR cassette recordings and LLM mocking configuration
   - **Impact**: Isolated to WebSocket tests - production vector search and retrieval works correctly

### Migration Notes

- Run migrations in order: annotations/0048 before documents/0026
- No data migration required - new fields are nullable
- Existing documents will have `structural_annotation_set=None` until parsed

### Performance Considerations

- Structural annotations are now shared (O(1) storage) instead of duplicated per corpus copy
- DocumentPath queries are indexed for efficient corpus document lookups
- Content-hash based deduplication prevents redundant parsing
