---
title: Metadata Lifecycle & HMR
description: How metadata flows through Repository → Change Log → Cache → Registry — the canonical event stream that powers Studio HMR, REST writes, and future cloud editing.
---

# Metadata Lifecycle & HMR

This page documents the metadata data path introduced by [ADR-0008](/adr/0008-metadata-repository-and-change-log) and refined by [ADR-0005](/adr/0005-metadata-customization-overlay). It is the canonical event stream that powers Studio Hot Module Replacement (HMR), REST writes, and future cloud editing.

<Callout type="info">
**TL;DR** — Every write goes through a single `MetadataRepository.put()` call. The repository appends to a change log, emits a watch event with a monotonic `seq`, and broadcasts over SSE to the Studio UI. The Studio's `useMetadataHmr` hook displays the latest `seq` in the status badge and re-fetches affected views.
</Callout>

---

## Architecture (one diagram)

```
   ┌─────────────────────────────────────────────┐
   │                  Studio UI                  │
   │      (useMetadataHmr / HmrStatusBadge)      │
   └────────────────────┬────────────────────────┘
                        │ SSE  /api/metadata/hmr
                        ▼
   ┌─────────────────────────────────────────────┐
   │           MetadataManager (bridge)          │
   │   forwards Repository events → SSE channel  │
   └────────────────────┬────────────────────────┘
                        │ MetadataEvent{seq, op, ref, ...}
                        ▼
   ┌─────────────────────────────────────────────┐
   │            LayeredRepository                │
   │   ┌─────────────────────────────────────┐   │
   │   │  Top: SysMetadataRepository (org)   │   │ ← writable overlay
   │   │  Bottom: FS / InMemory (artifact)   │   │ ← read-only baseline
   │   └─────────────────────────────────────┘   │
   └────────────────────┬────────────────────────┘
                        │ append(change log) + emit watch event
                        ▼
                   sys_metadata table
                  (or filesystem artifact)
```

Reads walk top-to-bottom: the first non-null layer wins. Writes always route to the topmost writable layer (the overlay). Deletes from the layered API only remove the overlay row — the artifact baseline survives.

---

## The four primitives

| Primitive | Package | Purpose |
| :--- | :--- | :--- |
| **Repository** | `@objectstack/metadata-core` | CRUD + watch interface over a single metadata source. Implementations: `InMemoryRepository`, `FileSystemRepository`, `SysMetadataRepository`, `LayeredRepository`. |
| **Change Log** | `@objectstack/metadata-core` | Append-only log of every mutation, tagged with a monotonic `seq`. Watchers can replay from any `since`. |
| **Cache** | `@objectstack/metadata` | In-memory snapshot of the registry, keyed by `MetaRef`. Invalidated by change-log events. |
| **Registry** | `@objectstack/metadata` | Typed registry the Kernel and plugins query. Built from the cache. |

`MetaRef = (type, name, org)`. As of [ADR-0008 §0 amendment (2026-04-13)](/adr/0008-metadata-repository-and-change-log#0-2026-04-13-amendment--drop-project-and-branch-from-metaref), `project` and `branch` are removed from the runtime tuple. Project survives only as an artifact-packaging concept on the `objectstack.json` envelope; branching is left to Git.

---

## Write path: Studio → SSE → repository

When a user edits a view in Studio:

1. Studio calls `PUT /api/v1/metadata/views/case_grid` (REST).
2. `protocol.ts:saveMetaItem()` validates against `MetadataTypeRegistry.allowOrgOverride` (see [overlay whitelist](#overlay-whitelist)).
3. If allowed, the call lands on `MetadataRepository.put(ref, body, { parentVersion, actor })`.
4. The repository:
   - Verifies `parentVersion` matches the current head (`ConflictError` on mismatch).
   - Writes the new row + appends a change-log entry with the next `seq`.
   - Emits a `MetadataEvent { op: 'create' | 'update' | 'delete', ref, seq, source }`.
5. The `MetadataManager` bridge forwards the event over the SSE channel `/api/metadata/hmr` with the `seq` attached.
6. The Studio's `useMetadataHmr` hook receives the event, updates `lastSeq`, and triggers a refetch of the affected view.

The `seq` is the single source of truth for ordering. The Studio status badge (`HmrStatusBadge`) shows `Repo seq: #N` in its tooltip.

---

## Overlay whitelist (shared-DB tenancy invariant)

In shared-database multi-tenancy, **most metadata types must not be per-org customizable** — overriding them would break the physical schema. The whitelist lives in **one** place: `MetadataTypeRegistryEntry.allowOrgOverride` in `packages/spec/src/kernel/metadata-plugin.zod.ts`.

| Type | `allowOrgOverride` | Rationale |
| :--- | :---: | :--- |
| `view`, `dashboard`, `report`, `email_template` | ✅ | Pure rendering. Per-org customization is safe. |
| `object`, `field` | ❌ | Defines the table schema. Overriding would break existing data. |
| `flow`, `workflow`, `agent` | ❌ | Stateful execution; per-org overrides need a separate execution-scoping mechanism. |
| `permission`, `role`, `profile` | ❌ | Security; tenant-level controls already exist via role assignments. |
| `datasource` | ❌ | Connection strings; multi-tenant isolation is enforced at a higher layer. |

The runtime gate is implemented in `OVERLAY_ALLOWED_TYPES` (derived from the registry) and enforced by `SysMetadataRepository.put()`. Denied types return `403 not_overridable`.

See [ADR-0005](/adr/0005-metadata-customization-overlay) for the full design and amendments.

---

## Conflict handling

Every `put()` requires a `parentVersion` (or `null` for fresh creates). If the head has moved since the caller read it, `ConflictError(ref, expectedParent, actualHead)` is thrown. Studio surfaces this as a "metadata moved" toast and offers reload-and-retry.

The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON serialization of the body. Identical bodies produce identical hashes — `put()` short-circuits on no-op writes.

---

## HMR end-to-end (latency & ordering)

- **Latency.** A typical write-to-render round-trip is **&lt; 100ms** on localhost (REST `PUT` → DB write → SSE flush → Studio refetch → React re-render).
- **Ordering.** The `seq` is monotonic per repository. If the Studio tab disconnects and reconnects, it can resume from `lastSeq` to replay missed events. (Replay is implemented by `InMemoryRepository`/`SysMetadataRepository`; the SSE bridge passes `since` as a query param.)
- **Multi-tab.** Each tab has its own `seq` counter. Out-of-order delivery between tabs is impossible because they share the server change log.

---

## Cross-references

- [ADR-0005: Metadata customization overlay](/adr/0005-metadata-customization-overlay) — overlay semantics, whitelist, conflict rules.
- [ADR-0008 §0 amendment (2026-04): project/branch removal](/adr/0008-metadata-repository-and-change-log#0-2026-04-13-amendment--drop-project-and-branch-from-metaref) — why `MetaRef.project` and `MetaRef.branch` are dead.
- [ADR-0008: Metadata Repository & Change Log](/adr/0008-metadata-repository-and-change-log) — the four-primitive architecture.
- [IMetadataService Contract](/guides/contracts/metadata-service) — the runtime API plugins call.
- [North Star](/concepts/north-star) — the product shape these primitives serve.

---

## Status (as of 2026-04)

| Component | State |
| :--- | :--- |
| `InMemoryRepository`, `FileSystemRepository`, `LayeredRepository` | ✅ Shipped (`@objectstack/metadata-core`) |
| Change log + `seq` (per-org, monotonic) | ✅ Shipped |
| SSE bridge (`/api/metadata/hmr` with `seq`) | ✅ Shipped |
| Studio `useMetadataHmr` + `HmrStatusBadge` | ✅ Shipped |
| Console dev-mode HMR reloader (`MetadataHmrReloader`) | ✅ Shipped |
| `SysMetadataRepository` (overlay over `sys_metadata`) | ✅ Shipped (`@objectstack/objectql`) |
| `LayeredRepository(SysMeta + artifact)` composition | ✅ Shipped |
| `protocol.ts:saveMetaItem` routed through `SysMetadataRepository.put` | ✅ Shipped (PR-10d.6, flag removed) |
| `sys_metadata_history` table (durable, org-keyed change log) | ⏳ M1 |
| Cache + Registry refactor against `MetadataRepository` | ⏳ Post-M0 |

> **Cross-replica overlay sync is out of scope.** Single-instance deployments
> are the supported topology for the metadata overlay. If you run multiple
> replicas and need them to converge on the same overlay state, point them at
> the same database and rely on each replica re-reading on demand — there is
> no `LISTEN/NOTIFY` (or equivalent) push mechanism on the roadmap.

---

## FAQ — When does code-defined metadata enter the database?

**Short answer: never.** This is a deliberate design choice; the table below is the authoritative reference and supersedes any other description you may find.

| Where the metadata came from | Lands in `sys_metadata`? | Lands in history? |
|:---|:---|:---|
| `defineView(...)` / `defineFlow(...)` / any source file → compiled into `dist/objectstack.json` | ❌ Never. Loaded into the in-memory registry on boot; refreshed via HMR in dev. | ❌ The artifact's own version history *is* Git. The metadata layer does not duplicate it. |
| Editing a `.json` under `.objectstack/metadata/<type>/<name>.json` (FS overlay) | ❌ FS layer is independent of DB. | ✅ Appended to `.objectstack/.log/main.jsonl` by `FileSystemRepository`. |
| Studio inline edit, or `PUT /api/v1/metadata/...` (REST) on an `allowOrgOverride: true` type | ✅ Written by `SysMetadataRepository.put()` as an **overlay row** scoped to `organization_id`. | ⚠️ M0: in-process watch broadcast only — no durable history table yet. M1: appended to `sys_metadata_history` with a Postgres `SERIAL seq` (single-instance scope; no cross-replica push). |
| Deploying a new build (new `dist/objectstack.json`) | ❌ The artifact is loaded into memory, not synced into `sys_metadata`. | ❌ Use Git tags / your deployment platform's release log; that's where artifact "version history" lives. |

### Why artifact never enters the database

1. **Single source of truth.** Code → artifact → memory. If artifact also wrote to `sys_metadata`, every deploy would need a reconciliation step ("the row says v3.0.0 but the code says v3.0.1") that is impossible to get right under concurrent edits.
2. **Overlay stays small and auditable.** Per-org overlay = "what this organization deliberately changed at runtime". A clean table that's safe to dump, diff, or reset to factory.
3. **Immutable infrastructure.** Rolling back a deploy = switching image tags. No DB migration, no reverse `put`, no history replay required.
4. **Per-org multi-tenant economics.** If artifacts were materialised into `sys_metadata`, every org would carry tens of thousands of baseline rows. Overlay-only keeps each org's row count proportional to its actual customisation.

### What if I want auditable history of deployed artifact changes?

That's a **deployment log**, not a metadata change log. Track the envelope hash of each deployed `objectstack.json` against the org/environment in a deployment-history table. The internal items inside the artifact need no per-item history because their canonical history is the Git repo that produced them.

---
