# Migration: v0.7.x → v0.8.0

This guide is for operators upgrading a YantrikDB-server deployment
from a `0.7.x` release to `v0.8.0`.

## TL;DR by deployment shape

| Deployment | Action required | Downtime |
|---|---|---|
| **Single-node** | `docker pull` + restart | Seconds |
| **Single-node + encryption via env var** | Confirm `YANTRIKDB_ENCRYPTION_KEY_HEX` is set | Seconds |
| **Cluster on raft-lite (legacy)** | Reform cluster — see below | Minutes |
| **Cluster already on openraft** | Rolling restart | Minutes |
| **Python / SDK clients** | None — wire format unchanged | None |

## Wire / API compatibility

`v0.8.0` is **wire-compatible** with `v0.7.x` for all `/v1/*`
endpoints. SDK clients on the 0.1.x..0.7.x line do not require
changes.

The substrate batch in this release adds new modules and traits but
ships them as interfaces only — the consumer wiring (axum middleware
for admission, control-DB-backed `AuthProvider`, etc.) lands in
follow-up releases. Therefore this release does not change request
handling shape on the hot path.

The two bug fixes (#6, #7) only relax previously-incorrect rejections
— no caller that was working on `0.7.x` will break.

## Single-node operators

```bash
docker pull ghcr.io/yantrikos/yantrikdb:v0.8.0  # or :latest
docker compose down && docker compose up -d     # for compose users
```

After restart:
- `/v1/health/deep` now includes new `admission`, `runtime`, and
  `version` blocks alongside the existing `encryption` block. Older
  monitoring scrapes that read only the existing fields continue to
  work; new dashboards can consume the additions.
- The embedder cache is automatically active. No tuning required.

### If you had been blocked by issue #6 (encryption env var ignored)

Confirm the documented `-e YANTRIKDB_ENCRYPTION_KEY_HEX=<64-hex>` now
works. Server log on startup should include:

```
encryption: enabled via YANTRIKDB_ENCRYPTION_KEY_HEX env var
```

If you want to remove the env var and use TOML instead, the
`[encryption] key_hex = "<hex>"` form still works; precedence is now:

1. `YANTRIKDB_ENCRYPTION_KEY_HEX` env var (highest)
2. `[encryption] key_hex` in TOML
3. `[encryption] key_path` file
4. Auto-generated key (when `auto_generate = true`)

### If you had been blocked by issue #7 (snapshot in single-node)

`POST /v1/admin/snapshot` now accepts the per-database token of the
database being snapshotted in single-node mode. Example:

```bash
curl -X POST http://localhost:7438/v1/admin/snapshot \
  -H "Authorization: Bearer $YDB_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"database":"default","output_dir":"/var/lib/yantrikdb/snapshots"}'
```

The token must be valid for the same `database` named in the body.
For cluster operators, the cluster master token still works as before
and can snapshot any database.

## Cluster operators on raft-lite (Phase 1, pre-RFC 010 PR-4)

The substrate that ships in this release **includes** the openraft
adoption (RFC 010 PR-4) for the cluster commit path. Pre-Phase-2
nodes running raft-lite are not protocol-compatible with Phase-2
nodes running openraft.

If your cluster is running raft-lite (which is the case unless you
opted in to openraft via `cluster.raft_mode = "openraft"`), you have
two options:

### Option A: stay on raft-lite (default)

In your `yantrikdb.toml`, leave `cluster.raft_mode` unset (or set to
the default `disabled`). Rolling-upgrade the binaries one node at a
time. The new substrate code is dormant when `raft_mode != "openraft"`.

This is the recommended path if you don't have an immediate need for
the openraft correctness guarantees.

### Option B: migrate to openraft

This is a **breaking** topology change requiring a cluster reform.

1. **Backup first.** From the current leader:
   ```bash
   curl -X POST http://leader:7438/v1/admin/snapshot \
     -H "Authorization: Bearer $CLUSTER_MASTER_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"database":"default"}'
   ```
   Save the snapshot file off-host.

2. **Stop all nodes.** Bring the cluster down completely. Both
   raft-lite and openraft nodes cannot coexist in the same cluster.

3. **Configure openraft + mTLS.** RFC 014-A mandates cluster-transport
   mTLS for openraft mode. Update each node's `yantrikdb.toml`:

   ```toml
   [cluster]
   raft_mode = "openraft"
   # CRITICAL: openraft uses the HTTP gateway port (default 7438) for
   # peer-to-peer transport via /v1/raft/*, NOT the legacy
   # `cluster_port` (7440 — that's still bound by raft-lite for
   # backwards compat but unused by openraft).
   # The advertise_addr and every peer.addr MUST be a full URL with
   # `http://` scheme — they're consumed as URL prefixes by the
   # openraft HTTP client.
   advertise_addr = "http://192.168.4.141:7438"

   [[cluster.peers]]
   addr = "http://192.168.4.140:7438"
   role = "voter"

   [cluster_tls]
   cert_path = "/etc/yantrikdb/cluster.crt"
   key_path  = "/etc/yantrikdb/cluster.key"
   ca_path   = "/etc/yantrikdb/cluster-ca.crt"
   dev_mode  = true   # accept self-signed for dev/homelab
   ```

   > ⚠️ Common gotcha: writing `advertise_addr = "192.168.4.141:7440"`
   > (without `http://` scheme, with the legacy `cluster_port`) makes
   > the cluster *appear* to form (heartbeats discover the leader) but
   > log replication silently fails. All `/v1/raft/append_entries` POSTs
   > return Unreachable. See issue #28.

4. **Wipe the legacy raft state.** Remove `raft_log.sqlite` (and its
   `-wal`/`-shm` companions), `raft.json`, and `commit_log.sqlite*`
   from each node's data dir. Engine data (per-database directories
   and `control.db`) stays intact.

5. **Initialize the seed node.** On the first node, after it's up:

   ```
   yantrikdb cluster initialize-cluster \
     --leader http://seed:7438 \
     --master-token "$YDB_CLUSTER_MASTER_TOKEN"
   ```

   Confirm `GET /v1/cluster/raft` reports `state: leader`.

6. **Add other nodes** one at a time. **Recommended**: pre-stage engine
   state via cold-tar from the leader before adding (~30 sec for a
   ~400 MB data dir vs ~30 min for openraft 0.9's `full_snapshot`
   over the wire). Then:

   ```
   # On the new node, populate /var/lib/yantrikdb from leader's tar
   # (excluding raft_log.sqlite*, raft.json, commit_log.sqlite*).
   # Start the new node — it'll come up as a learner discovering
   # the leader via cluster_secret.

   # On any host with master token + network access to the leader:
   yantrikdb cluster add-learner \
     --node-id 4 --addr http://192.168.4.140:7438 \
     --leader http://192.168.4.141:7438 \
     --master-token "$YDB_CLUSTER_MASTER_TOKEN"

   yantrikdb cluster wait-caught-up \
     --node-id 4 --leader http://192.168.4.141:7438 \
     --master-token "$YDB_CLUSTER_MASTER_TOKEN"

   yantrikdb cluster promote-voter \
     --voters 2,4 \
     --leader http://192.168.4.141:7438 \
     --master-token "$YDB_CLUSTER_MASTER_TOKEN"
   ```

   `GET /v1/cluster/raft` on each node should now show both as
   voters with the same `last_applied_index`.

7. **Verify.** `GET /v1/cluster/raft` should now return openraft
   metrics (term, last_log_index, members, etc.).

If you run a 2-of-3 cluster, expect about ten minutes of downtime
during steps 2–7. For higher-availability rolling-upgrade, contact
maintainers — a no-downtime migration plan exists but requires
operator-side coordination.

## Cluster operators already on openraft

If you already set `cluster.raft_mode = "openraft"` in `0.7.x`, the
upgrade is a rolling restart. openraft handles the leader handoff
automatically.

```bash
# On each follower, in turn:
docker compose pull && docker compose up -d
# Wait for /v1/health to report healthy + leader visible
# Move to next follower
# Last: kill the leader to trigger a final handoff to a v0.8.0 node
```

## Removed

Nothing in this release. All deletions in the diff are local/working
files (e.g. converted `auth.rs` → `auth/mod.rs` module promotion);
the public API surface has only additions.

## Deprecated

Nothing in this release.

## Known limitations

- Substrate is **interface only**. The admission middleware, scope-check
  middleware, control-DB-backed `AuthProvider`, BM25/RRF/Reranker
  pipeline, and other consumer-side wiring will land in follow-up
  releases. Until then, the new modules are dormant; setting them up
  has no runtime effect on request handling.
- The Docker workflow on the engine repo (`yantrikos/yantrikdb`) has
  a pre-existing `permission_denied: write_package` failure for the
  `yql` and `yantrikdb-witness` images. That's unrelated to this
  release. Server image (`ghcr.io/yantrikos/yantrikdb:v0.8.0`) builds
  normally.

## Reporting issues

Open an issue at https://github.com/yantrikos/yantrikdb-server/issues
with:
- The output of `yantrikdb --version`
- Your `yantrikdb.toml` (with secrets redacted)
- Server log of a failed startup or the failing endpoint's response
