---
title: "Architecture — Parallel Execution: TermLink Substrate Layer"
status: design captured, pre-build
source: operator paste during session S-2026-0610-0915 on 2026-06-10
authoritative_owner: TermLink repository (Rust workspace)
notice: |
  This is a REFERENCE COPY. The authoritative version lives in the TermLink
  repository. Do not edit this file in place — propose changes upstream in
  TermLink and re-sync. The AEF-side scoping inception will define the
  re-sync mechanism as part of its artifact-placement decision.
companion: docs/architecture/parallel-execution-aef.md
landing_task: T-2302
landed: 2026-06-10
---

# Architecture — Parallel Execution: TermLink Substrate Layer

Status: design captured, pre-build. Companion: *Architecture — Parallel Execution:
AEF Orchestration Layer*. This document is the authoritative record for the
**substrate (TermLink/Rust)** half and **owns the collaboration seam** (§9).

This is an architecture *decision record*, not a summary: each decision below carries
the alternatives that were explored, why the losing options lost, and — where a
decision was not chosen but *forced* by a concrete substrate fact — the forcing chain
that compelled it. The intent is that a reader can reconstruct *why*, not just *what*,
and therefore knows which decisions are load-bearing and must not be casually
overturned.

Grounded in TermLink findings T-2007 (a source-cited read of the Rust workspace) and
two design sessions (the parallelization design session and the TermLink-capabilities
investigation).

---

## 1. Purpose and framing

Move AEF from single-agent / shared-tree / single-writer execution to multiple agents
executing tasks concurrently across ring20 (a homelab of several hosts, one or more
agents per host), coordinated through TermLink. The colloquial goal is "pump the
volume up."

A framing correction made early and worth preserving, because it reorders the whole
problem: it is tempting to treat "isolation" as the central question (shared tree vs
worktree vs branch) and "merging" as the hard part. That framing is wrong. **Worktrees
and branches do not buy safety; they buy the ability to *defer* a merge.** Safety comes
from *disjoint write-sets*, which is a decomposition property decided when tasks are
shaped, long before any merge. When two concurrent tasks' write-sets are disjoint by
construction, a three-way merge has nothing to resolve. Therefore isolation mechanism
and merge strategy are *downstream* of getting decomposition right, and most of the
substrate's job is not "isolate trees" but "provide the primitives that let the
orchestrator assign disjoint work and detect when disjointness is violated."

A second early correction: it was initially mis-stated that "TermLink runs workers, not
builders." This is false. TermLink launches agents with prompts and model routing; a
build agent is just an agent with a build prompt and write access to a tree. The
substrate can run a builder today. The gap is therefore **not capability but missing
scaffolding** for isolation and for routing governance-state writes — which is what
this document specifies.

## 2. Current substrate reality (T-2007)

What exists today, because several decisions below are forced by these facts:

- **Strict star topology.** Sessions never connect to one another; every cross-session
  path is routed through the hub. (Preserved deliberately — see §3.)
- **Pub/sub append-log as the primary surface.** A SQLite-backed, per-topic,
  append-only log with monotonic offsets, durable across hub restart, with
  per-subscriber persistent cursors. Any agent posts; any agent reads at its own
  cursor; every subscriber to a topic sees every post.
- **Push-only, broadcast delivery; no claim, no registry.** A sender chooses the
  recipient. There is no exclusive-delivery / claim / lock / lease / CAS verb anywhere.
  The hub keeps no idle/busy registry; liveness (LIVE/STALE/OFFLINE) is computed
  client-side by inspecting heartbeat timestamps.
- **Code-plane isolation exists, narrowly.** `termlink dispatch --isolate` creates a
  per-worker git worktree and can auto-commit / auto-merge at cleanup. This is the only
  place the runtime knows git exists at all.
- **No filesystem-write observation, anywhere.** An exhaustive search for inotify /
  fanotify / fs-watch found nothing at runtime. The hub cannot see what files an agent
  touches. (This single absence forces the conservative collision policy — see §4.)
- **Split transports.** Same-host uses an auth-bypassed Unix socket (UID trust, no TLS);
  cross-host uses authenticated TCP (HMAC challenge + cert pinning). Identical JSON-RPC
  semantics, non-equivalent trust models.
- **Durability is reader-side, not writer-side.** Channel logs and the inbox spool
  survive hub restart and replay to subscribers; but a spoke whose host loses the hub
  gets an immediate error and the outbound post is *discarded* — there is no
  client-side reconnect or outbound queue.
- **One observed scale incident.** T-1991: the agent-presence topic bloated from
  heartbeating at ~20 agents, slowing discovery; mitigated by a client-side cache.
  No connection cap, rate limiter, or backpressure governor exists in code. Relevant
  because the coordination/announcement pattern AEF wants generates exactly that
  traffic class.

## 3. Topology: strict star, no mesh

**Decision.** Keep the strict star. The hub mediates all coordination; spokes never
connect to one another.

**Alternatives explored.**

- *Agent-to-agent mesh / negotiated direct channels.* Proposed to address an efficiency,
  bottleneck, delay, and fragility concern: hub-routing every message is a serialization
  point and a single point of failure, so why not let agents that need to coordinate
  hold a direct (possibly permanent, expiry-timed) channel, with the hub as fallback?

**Why the mesh was rejected — three arguments, the third decisive.**

1. *Bottleneck* is real in principle but dormant at scale. The hub spawns per connection
   with no semaphore (bound only by OS file descriptors); the one real scale incident
   (T-1991) was a *retention* problem, not throughput. At tens of agents the hub is not
   the constraint, so a mesh relieves a bottleneck that does not exist while adding
   permanent complexity.
2. *Delay* is real but swamped by a decision made elsewhere. The agent reactiveness
   model (AEF layer) is deliberately cooperative — agents notice messages at their own
   yield points, which adds *seconds* of latency by design. Against that, the
   one-extra-hop of hub routing (microseconds) is noise. One cannot simultaneously want
   non-disruptive cooperative polling and worry about a routing hop three orders of
   magnitude smaller than the poll interval.
3. *Fragility* cuts the opposite way from intuition, and this is decisive. A star has
   *one* failure point — visible, diagnosable, and (because channel logs and the inbox
   spool are durable) *recoverable*: a hub crash is a pause, not data loss. A mesh
   distributes fragility across up to N² links, each with independent liveness and
   failure state, *no* central durable replay (a dropped direct message is gone), and —
   worst — silent partial-partition divergence, where A↔B and C↔B survive but A and C
   hold inconsistent coordination state with no authority to reconcile against. On a
   flaky homelab, partial partitions are the *normal* failure mode. Concentrated,
   recoverable fragility beats distributed, silent, unrecoverable fragility.

**The reusable insight.** The cure for "the hub is a single point of failure" is
**redundancy of the hub, not elimination of the hub** — durable, then restartable, then
(only if scale demands) replicated/federated. The analogy: a bank does not fix
"the vault is a single point of failure" by giving every teller a private safe and a
negotiation protocol; it replicates and reconciles the vault. Cross-hub routing already
exists in TermLink, so federation is the available scaling path. A mesh would throw away
the star's properties (ordering, durability, simplicity, agent independence) to solve a
problem those very properties make manageable.

**The efficiency the mesh was reaching for is already in the star.** The legitimate
desire — a cheap, permanent coordination channel between agents — is satisfied by a
*persistent shared topic*: append-and-tail is about as cheap as a direct socket, and it
stays durable, ordered, hub-visible, and survives either party dying. "A permanent
negotiated channel with an expiry timer" is just a named topic with configurable
retention (Days(N) / Forever / message-count). The hub *is* the shared medium. There is
nothing to "fall back" from — the topic is the channel, and the hub is primary.

## 4. Collision policy boundary: capability here, decision in the AEF layer

**Decision.** The substrate builds the *capability* for physical collision detection
(filesystem-write observation, §6 gap 4). It does **not** set the conservative-vs-
optimistic *policy*. Launch policy is conservative and lives in the AEF layer.

**The forcing chain (why optimistic is not available today).** The optimistic strategy —
parallelize unless overlap is obvious, catch collisions at merge — is only *safe* when
collisions are detected on a *physical* medium, the way Ethernet CSMA/CD detects them:
you cannot *not* notice a collision on the wire. TermLink has **no filesystem-write
observation**. The only available substitute is honour-system announcements (an agent
voluntarily posts "I am now touching path X" to a topic, and other agents must choose to
read it). An agent that forgets to announce, or whose announcement another agent reads
too late, produces an *undetected* collision that surfaces only at merge — as an ugly
governance-plane conflict in machine-maintained state. Therefore: optimistic-on-honour-
system is unsafe → conservative is forced *until and unless* real write-observation
exists. The asymmetry reinforces it: being too conservative wastes some potential
concurrency (a mild, performance-only loss); being too optimistic corrupts
machine-maintained state (severe). When one error is cheap and the other expensive, bias
to the cheap error.

**Consequence and the two-step that must not be collapsed.** Building write-observation
does **not** flip the policy. It creates the *option* to go optimistic later; the flip
is a separate, gated decision (owned by the AEF layer and the human, against criteria
defined in advance). The substrate's responsibility ends at making observation exist and
be sound. Conflating "shipped #4" with "now optimistic" is the specific error this
boundary exists to prevent.

## 5. The two planes

Parallel execution needs two planes with *opposite* isolation needs, and the central
AEF-specific difficulty is that they pull in opposite directions:

- **Code plane wants isolation.** Two agents editing source must not see each other's
  half-done work. Per-host isolated checkouts / git worktrees provide this.
- **Governance plane wants centralization.** Every agent would otherwise write the task
  ledger (`.tasks/`), audit jsonl, arc YAML, episodic memory, focus pointer. These are
  machine-maintained *edit hotspots*: two branches both touching `.tasks/` and
  `.context/audits/*.jsonl` conflict constantly, and those conflicts are nasty because
  the content is generated state you cannot eyeball-merge.

**Why git-worktree for the code plane (over full clones or branches-in-one-tree).** A
worktree shares the single object store and adds only a checked-out file copy per agent,
so space cost is *linear* in concurrent agents rather than exponential, while still
giving each agent an independent working directory. Branches-in-one-shared-tree was
rejected because it reintroduces single-writer contention on the index; per-agent full
clones were rejected on cost. On ring20 this gets cleaner still: hosts are physically
separate machines, so cross-host code-plane isolation is essentially free — each agent
has its own checkout on its own host, and the merge happens centrally.

**Why the governance plane centralizes through the hub append-log.** This is the first
lock the prior session identified as the thing everything hangs on:

*The question.* When N agents are live, where does the ledger get written?
- *Option (a) — scoped slices reconciled on merge:* each agent writes its own scoped
  ledger slice, reconciled when branches merge.
- *Option (b) — hub-routed:* all ledger mutations route through the hub; no spoke writes
  `.tasks/` or `.context/` directly.

*Decision: (b).* Option (a) reproduces exactly the ugly machine-state merge conflicts the
plane split exists to avoid — it just defers them to merge time. Option (b) makes them
impossible by construction: one writer, serialized. The cost of (b) is real and must be
designed for — a builder cannot simply run `fw task update` locally; it must message the
hub — which is why it is a constraint on how every worker is built, and why it was
explicitly the human's call before proceeding.

TermLink's existing append-log is exactly the (b) primitive: durable, monotonically
ordered, mutex-guarded, restart-safe. The hub-and-spoke dual-plane shape maps onto the
two planes almost too neatly — spokes do code work in isolated worktrees; the hub
serializes all ledger writes.

**Residual un-partitionable files.** Some files cannot be write-set-partitioned no matter
how cleanly tasks are decomposed — `Cargo.lock` is the canonical example (any two tasks
that add a dependency touch it). These are handled by **hub-owned regeneration after
merge**, not by agents and not by merge resolution. This is a deliberate carve-out from
the disjoint-write-set rule for files whose correct state is *derivable* rather than
*authored*.

## 6. Required primitives (the build manifest)

In priority order, each with the role it plays and, where relevant, the decision it
enables. This is the gap list from T-2007 — capabilities absent or partial relative to
the model above.

**Foundation — without these there is no safe assignment.**
1. *Exclusive-delivery / claim semantics.* Topics are broadcast: two subscribers to a
   "work" topic both see every post, so the naive design (post tasks to a queue, idle
   agents grab the next) would have two agents grab the same task. There is no
   lock/lease/CAS/claim verb. Needed for any safe handoff of a unit of work to exactly
   one consumer. **This is also why the orchestrator cannot be a passive pull-queue**
   (see the AEF layer) — until a claim primitive exists, exclusive assignment must be
   the orchestrator's explicit act.
2. *Hub-owned idle/busy registry.* No hub-tracked agent state, no "next idle worker for
   role X," no in-flight counter. Today heartbeats land in a topic and classification is
   client-side. The orchestrator needs a reliable picture of who is free.
3. *Pull/assign verb.* Every path is push (sender picks recipient). There is no
   "give me the next unit" RPC and no clean inverse for the orchestrator to hand a
   specific unit to a specific worker as a first-class operation.

**Resilience — ring20 hosts are not reliable.**
5. *Client-side reconnect + outbound queue for spokes.* Channel durability protects
   *readers*; a spoke that briefly loses the hub has its outbound post discarded. A
   worker that finishes and reports "complete" during a blip loses that report. Needed
   so the governance plane does not silently drop completion/ledger messages, and as the
   substrate half of the sender-side retry the AEF layer relies on.
7. *Hub-persistent presence + circuit-breaker state.* Both are in-memory today and reset
   on hub restart, so liveness inference resets to "everyone unknown" for one heartbeat
   interval after every restart. Channel logs and the inbox spool *do* survive, so
   message durability is intact; it is the *liveness* picture that is fragile.

**Contract — make the plane split a first-class concept.**
8. *Typed agent-launch surface aware of source-tree handoff* — `agent.checkout(ref)`,
   `agent.commit(scope)`, `agent.publish(branch)`. Today the only git awareness is the
   `dispatch --isolate` worktree wrapper; coordinating code plane vs governance plane is
   otherwise a shell convention sitting above TermLink. Making it typed turns the
   convention into a substrate concept the orchestrator can rely on.

**Keystone — the largest, and the one that changes what is possible.**
4. *Filesystem-write observation.* Nothing watches agent file I/O. This is what would let
   the system *physically* detect collisions and therefore makes a future optimistic mode
   safe (§4). It is `novel_mechanism: yes`, the biggest single build, and expected to
   split into sub-pieces. Its absence is *why* launch is conservative; its presence is
   the precondition (not the trigger) for optimistic.

**Supporting.**
6. *Symmetric authentication across transports* — so same-host (loopback TCP) and
   cross-host present one trust model rather than UID-trust UDS vs authenticated TCP.
9. *Broadcast-with-replay / current-value key* surfaced to a subscriber on registration —
   for late-joiner "room state" without replaying an entire log.
10. *Published throughput / connection-count budget with retention/compaction* — there is
    no connection cap or rate limiter, and T-1991 (agent-presence bloat) was found in
    production, not predicted. The coordination pattern AEF wants is exactly that traffic
    class, so retention/compaction must be designed in from the start, not bolted on.

## 7. Transport unification

**Decision.** Unify on one authenticated path: same-host clients dial loopback TCP
(`127.0.0.1`), cross-host use TCP, both under HMAC + cert pinning.

**Alternatives explored.** Keep the split (auth-bypassed UDS locally for speed, TCP
remotely). Rejected because it means two code paths, two trust models, and a long-lived
*privileged* sidecar listener that would otherwise sit on an unauthenticated local
socket. The cost of unifying is a microsecond-scale loopback-TCP latency over a Unix
socket and one port per listener — negligible at homelab scale. Note this is *"force
loopback TCP explicitly,"* not a default the code already takes; the default same-host
path is the auth-bypassed UDS, so this is an active configuration choice.

## 8. Open questions (substrate side)

- **Hub federation.** Whether to add multi-hub federation/failover depends on ring20's
  scale ceiling and the tolerable hub-restart pause — a human input the AEF layer also
  needs. Until answered, the design assumes a single supervised, durable hub (restart is
  a recoverable pause, not loss). Federation, if pursued, is *more hubs in a star
  federation with spoke failover*, never a spoke mesh (§3).
- **Write-observation mechanism (gap 4).** Whether file I/O can be observed cleanly on
  every ring20 host, by what mechanism (inotify/fanotify/ptrace/wrapper), at what cost,
  with what blind spots. A genuine two-sided unknown, co-discovered with the AEF layer.
- **Primitive build order.** The §6 ordering is a recommendation to confirm.

## 9. Collaboration seam (authoritative; the AEF layer references this)

The two layers are different repositories with a hard, directional dependency. The
boundary is treated the way task parallelism is treated: *a good contract makes the two
work-streams disjoint so they barely need to coordinate* — which is the same principle
as disjoint write-sets, applied one level up to the collaboration itself.

This was deliberately kept lightweight. A heavier shape was considered and rejected: a
standalone collaboration-protocol document with formal sign-off ceremony. For a homelab
where the "two parties" are agents the same operator orchestrates, that is enterprise
scaffolding for a workshop — the over-engineered version of solving a real problem. What
survives is the *split* (genuinely different repos with a real dependency) and the
*contract* (the insurance against a late, silent integration break), without the
ceremony.

**Hard dependencies — contracted up front, then worked independently.** For each, the
substrate commits the RPC signature and semantics early; the AEF layer builds against
that contract without ongoing back-and-forth as long as it holds: claim/exclusive
delivery (1), idle/busy registry (2), pull/assign verb (3), reconnect/outbound queue (5),
symmetric auth (6), persistent presence (7), typed git surface (8). The point of the
contract is to *minimize* coordination: once the interface is agreed, both sides run in
parallel against it.

**Soft dependencies — co-discovered, consulted, not pre-contracted.** These cannot be
contracted because nobody yet knows the answer, so forcing a contract would be wrong:
write-observation shape (4), un-partitionable-file regeneration mechanism, and the
conservative→optimistic flip criteria (which depend on 4's real behaviour). These are
the *only* places sustained dialogue is expected.

**Sign-off (producer ≠ judge).** The substrate does not declare a hard-dependency
primitive complete unilaterally — the AEF layer (the consumer that can see the need the
producer cannot) validates it is actually usable for dispatch before it is accepted.
Symmetrically, the AEF layer cannot treat a need as contracted until the substrate
confirms it is buildable. Neither side self-certifies the boundary.

**The collaboration rules, in full.** Contract the hard dependencies up front, then build
quiet; consult only on the three soft items; the consumer signs off the producer's output
and the producer confirms feasibility of stated needs; keep back-and-forth low by
default. **Rising consultation volume on a *hard* dependency is a smell** — it means the
contract was underspecified or the boundary is mis-cut; stop and re-contract rather than
grind. (Exactly the same signal as high coordination volume between two parallel tasks
meaning the tasks were not disjoint enough.) If the work runs sequentially (substrate
first, then AEF), the contracts are the handoff artifact, with a standing clause to
re-contract if one fails to hold.

## 10. Invariants (must not be violated)

Strict star; no peer-to-peer surfaces. Append-log durability and ordering preserved.
The substrate builds write-observation *capability* but does not set conservative/
optimistic *policy*; conservative remains the launch policy regardless of gap 4 landing.
Producer ≠ judge at the seam. Coordination efficiency is obtained via persistent hub
topics, never via dissolving the star.

## 11. Grill Me

This ADR is the authoritative substrate-side record (TermLink repo owns updates). It is
grillable in this repo against the AEF consumer's expectations. Substantive changes must
flow upstream into TermLink; this file is a reference copy.

**Entry point:** `/grill-with-docs` (skill: `.claude/skills/grill-with-docs/SKILL.md`)

**Primary grill targets** — flag any answer that cannot be defended in one sentence:

- **§3 mesh-rejection forcing chain.** Three arguments, third decisive (fragility). Drill
  the homelab failure mode: what happens at ring20 if the *one* hub host dies for a day?
  "Restart is a pause, not loss" — is that *actually* believed at zero-trust level, or is
  it a phrase that papers over a single point of failure we tolerate?
- **§4 collision policy boundary.** Capability here, policy in AEF. The "two-step that
  must not be collapsed" is structurally fragile — under build pressure, someone
  collapses it. Drill: what mechanism *prevents* the collapse (a doc note, or a code
  gate)?
- **§5 plane split.** Governance plane centralises through hub append-log (option b).
  Drill: a worker that finishes during a hub blip — the spec says channel durability
  protects readers but writers' outbound posts are discarded. Doesn't this leak the very
  "completion silently lost" failure the centralisation was meant to prevent?
  ([cross-ref §6 primitive 5: client-side reconnect + outbound queue is the answer; drill
  that it's *enough*]).
- **§6 primitive priority order.** Foundation (1,2,3) → resilience (5,7) → contract (8)
  → keystone (4) → supporting (6,9,10). Is keystone-last actually right, or does AEF's
  consumer pressure (the conservative→optimistic flip is the perception-moving event)
  force re-ordering keystone earlier? Drill: what evidence would justify re-ordering?
- **§7 transport unification.** Loopback TCP for same-host. Drill: the cost claim is
  "negligible at homelab scale" — what's the actual measured latency delta over UDS, and
  does it hold under the agent-presence-bloat traffic class (T-1991)?
- **§9 collaboration seam.** Producer ≠ judge is sound, but the test is concrete:
  *what evidence counts* as AEF "validated" a primitive? The ADR says "actually usable
  for dispatch" — that's a verbal test, not a wire test. Drill: name the artefact that
  proves consumer-validated.
- **§9 "rising consultation is a smell".** The §9 self-correcting clause. Drill: what's
  the threshold? How many consult rounds before "stop and re-contract" fires?

**Companion ADR:** [`parallel-execution-aef.md`](parallel-execution-aef.md) (this-repo
authoritative; grill the AEF consumer side there).

**Scoping inception that consumes this ADR (consumer side):**
[`T-2303`](../../.tasks/completed/T-2303-scoping-inception--parallel-execution-ar.md)
(IW-3 in particular: TermLink coordination timing + mechanism + contract artefact;
operator GO'd 2026-06-10 commit `989fc1e6e`. IW-3 resolved as "documented defer
with named trigger" — substrate engagement deferred to first downstream design
pressure that genuinely needs TermLink hub-state RPC; AEF-side downstream cluster
advances independently via [`T-2323`](../../.tasks/active/T-2323-aef-ic-1-yield-point-granularity-substra.md)
+ [`T-2324`](../../.tasks/active/T-2324-aef-ic-2-disjoint-write-set-policy.md)).
