# Ralph Progress Log
Started: Wed Apr 29 04:23:03 AM PDT 2026
---
## Codebase Patterns
- `rivetkit-sqlite` statement routing classification should prepare exactly one statement with `sqlite3_prepare_v2`, read SQLite's decision through `sqlite3_stmt_readonly`, and capture prepare-time authorizer actions with `sqlite3_set_authorizer`.
- New public `rivetkit-sqlite` behavior tests belong under `rivetkit-rust/packages/rivetkit-sqlite/tests/` when they do not need private module access.
- Native SQLite VFS ownership is ref-counted through `NativeVfsHandle`; each `NativeConnection` holds a handle clone so the VFS unregisters only after the last connection closes.
- Envoy SQLite VFS names include the actor database startup generation, e.g. `envoy-sqlite-{actor_id}-g{generation}`, to avoid stale registration collisions.
- Tests that register multiple native SQLite VFS entries in one process should drop stale generations before replacement generations to avoid perturbing SQLite's global VFS registry.
- SQLite VFS file handles carry a reader or writer role; reader-owned handles must fail closed for mutating VFS callbacks instead of relying on TypeScript routing.
- Native SQLite work that can invoke VFS callbacks should run on `spawn_blocking`; VFS callbacks synchronously block on the transport runtime and can fail if SQL runs on an async runtime worker.
- The native SQLite connection manager keeps an idle writer open while `sqlite3_get_autocommit` is false; `COMMIT` or `ROLLBACK` must reuse that writer and close it once autocommit is restored.
- Native SQLite read-query routing must classify before installing the mandatory reader authorizer; statement classification uses a temporary authorizer and clears the connection-global authorizer when it finishes.
- Native SQLite single-statement work should route through `NativeDatabaseHandle::execute`; keep `exec` as the multi-statement compatibility path.
- TypeScript SQLite database wrappers should route single-statement work through native `SqliteDatabase.execute`; use `exec` only for multi-statement compatibility.
- TypeScript SQLite migration hooks should run inside native `writeMode` so setup queries use the writer connection and do not create readers.
- SQLite read-pool rollout config lives in `sqlite-storage::optimization_flags`; build `NativeConnectionManagerConfig` from `sqlite_optimization_flags()` and use `RIVETKIT_SQLITE_OPT_READ_POOL_ENABLED=false` for single-writer compatibility.
- Kitchen-sink SQLite real-world benchmark reporting should include read-pool route counters alongside VFS counters so parallel-read and read-write-transition workloads expose manager behavior.
- Native SQLite read-pool v1 closes readers before writes and does not pin per-reader head txids; TypeScript/NAPI wrappers must treat native execution as the routing policy boundary.

## 2026-04-29 04:27:40 PDT - US-001
- Implemented native SQLite statement classification with readonly detection, trailing-statement detection, authorizer action capture, and conservative reader eligibility.
- Added integration coverage for SELECT, read-only PRAGMA, mutating PRAGMA, INSERT RETURNING, CTE writes, VACUUM, ATTACH, BEGIN, SAVEPOINT, and multi-statement SQL.
- Files changed:
  - `rivetkit-rust/packages/rivetkit-sqlite/src/query.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/tests/statement_classification.rs`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `cargo check -p rivetkit-sqlite`
  - `cargo test -p rivetkit-sqlite`
- **Learnings for future iterations:**
  - SQLite reports raw `BEGIN` and `SAVEPOINT` as readonly, so authorizer transaction-control capture must block reader routing separately.
  - `sqlite3_prepare_v2` exposes unconsumed trailing SQL through the tail pointer; non-whitespace tail text should make reader routing ineligible.
  - Existing `rivetkit-sqlite` builds currently emit pre-existing Rust 2024 unsafe-op warnings from `src/vfs.rs`, but the package check and tests pass.
---
## 2026-04-29 04:33:10 PDT - US-002
- Implemented split native SQLite ownership with `NativeVfsHandle`, `NativeConnection`, and the existing `NativeDatabase` compatibility wrapper.
- Added generation-bearing envoy VFS names and tests for shared VFS context reuse plus unregister-after-last-connection cleanup.
- Files changed:
  - `rivetkit-rust/packages/rivetkit-sqlite/src/database.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/vfs.rs`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `cargo check -p rivetkit-sqlite`
  - `cargo test -p rivetkit-sqlite native_vfs_handle --lib`
  - `cargo test -p rivetkit-sqlite`
- **Learnings for future iterations:**
  - `sqlite3_vfs_register` duplicate-name behavior is not a good lifetime assertion; use `sqlite3_vfs_find` when tests need to inspect VFS registration state.
  - Keeping a `NativeVfsHandle` clone inside each `NativeConnection` makes close ordering fail-closed even if a connection outlives its manager wrapper.
  - `cargo test -p rivetkit-sqlite` may emit existing Rust 2024 unsafe-op warnings from `src/vfs.rs`; this session's full rerun passed.
---
## 2026-04-29 04:43:03 PDT - US-003
- Implemented native SQLite VFS reader/writer roles on main and auxiliary file handles, including output flag normalization from assigned role.
- Reader-owned VFS handles now reject mutating callbacks: xWrite, xTruncate, dirty xSync/xClose, xDelete for reader-owned aux files, and atomic-write file-control operations.
- Added inline VFS tests for reader fail-closed behavior, writer write behavior, reader aux creation denial, output flags, and reader-owned aux delete rejection.
- Files changed:
  - `rivetkit-rust/packages/rivetkit-sqlite/src/vfs.rs`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `cargo check -p rivetkit-sqlite`
  - `cargo test -p rivetkit-sqlite vfs_file --lib`
  - `cargo test -p rivetkit-sqlite role_flags --lib`
  - `cargo test -p rivetkit-sqlite reader_owned_aux_files_reject_delete --lib`
  - `cargo test -p rivetkit-sqlite`
- **Learnings for future iterations:**
  - VFS role enforcement belongs in `VfsFile`, not only connection setup, because SQLite mutating callbacks arrive through file handles.
  - Reader auxiliary-file creation is denied by default; only existing auxiliary paths can be opened read-only until a safe path class is explicitly documented in code.
  - `cargo test -p rivetkit-sqlite` still emits existing Rust 2024 unsafe-op warnings from VFS callbacks, but the full suite passes.
---
## 2026-04-29 04:54:49 PDT - US-004
- Implemented `NativeConnectionManager` with closed, read-mode, write-mode, and closing states, lazy read-only connection admission up to a max reader count, writer preference, read-to-write transition cleanup, and close-time VFS teardown.
- Added VFS-backed tests for read admission, writer preference, read-to-write transition state, and close ordering through VFS unregister.
- Files changed:
  - `rivetkit-rust/packages/rivetkit-sqlite/src/connection_manager.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/lib.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/vfs.rs`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `cargo check -p rivetkit-sqlite`
  - `cargo test -p rivetkit-sqlite connection_manager --lib`
  - `cargo test -p rivetkit-sqlite bench_large_tx_insert_100mb --lib`
  - `cargo test -p rivetkit-sqlite`
- **Learnings for future iterations:**
  - The connection manager is present as a native primitive but existing query/run/exec routing is intentionally unchanged until US-005 and later stories.
  - SQL executed through the native VFS should run on blocking threads, because VFS callbacks synchronously block on the transport runtime.
  - A full-suite run briefly failed the existing 100 MiB large-transaction test with a staged-delta decode error, but the single test and the full suite both passed on rerun.
---
## 2026-04-29 05:04:08 PDT - US-005
- Implemented exclusive write-mode routing for native SQLite run, query, exec, startup configuration, and batch-atomic verification through `NativeConnectionManager`.
- Added transaction-aware writer retention: raw `BEGIN` and `SAVEPOINT` keep the manager in write mode until `COMMIT` or `ROLLBACK` restores autocommit.
- Added manager tests proving pending readers wait behind manual `BEGIN` and `SAVEPOINT` write mode, alongside the existing writer-preference coverage.
- Files changed:
  - `rivetkit-rust/packages/rivetkit-core/src/actor/sqlite.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/connection_manager.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/database.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/vfs.rs`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `cargo check -p rivetkit-sqlite`
  - `cargo check -p rivetkit-core`
  - `cargo test -p rivetkit-sqlite connection_manager --lib`
  - `cargo test -p rivetkit-sqlite`
  - `cargo test -p rivetkit-core` was stopped after an unrelated actor-task log assertion failed and a separate actor-task test hung past 60 seconds; both reproduce outside SQLite-focused changes.
- **Learnings for future iterations:**
  - Per-connection SQLite PRAGMAs need to run when a writer connection is newly opened, not when reusing a transaction-held writer.
  - Raw transaction-control statements must be treated as write-mode state changes even when SQLite reports them as read-only.
  - The full `rivetkit-core` suite currently has non-SQLite actor-task test instability in `actor_task_logs_lifecycle_dispatch_and_actor_event_flow` and `save_tick_cancels_pending_inspector_deadline_and_broadcasts_overlay`.
---
## 2026-04-29 05:21:13 PDT - US-006
- Implemented read-only query routing through native read connections, including lazy reader opens, idle reader reuse, per-reader `PRAGMA query_only = ON`, and fallback to write mode only for classification-ineligible statements.
- Added a mandatory reader authorizer that denies transaction control, attach/detach, schema/temp/data writes, unsafe pragmas, and unsafe functions, with fail-closed behavior when reader execution rejects a statement.
- Moved native SQLite connection opens onto blocking threads because opening a VFS-backed connection can invoke callbacks that synchronously block on the transport runtime.
- Files changed:
  - `rivetkit-rust/packages/rivetkit-sqlite/src/connection_manager.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/database.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/query.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/vfs.rs`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `cargo check -p rivetkit-sqlite`
  - `cargo test -p rivetkit-sqlite native_database_routes_concurrent_readonly_queries_to_multiple_readers --lib`
  - `cargo test -p rivetkit-sqlite native_database_reuses_idle_reader_for_readonly_query --lib`
  - `cargo test -p rivetkit-sqlite native_database_reader_authorizer_denies_unsafe_functions --lib`
  - `timeout 240s cargo test -p rivetkit-sqlite`
- **Learnings for future iterations:**
  - Reader routing should treat classification errors as write-required, but errors after a statement is classified reader-eligible should fail closed instead of silently retrying on the writer.
  - `sqlite3_open_v2` can invoke VFS callbacks, so read and write connection opens need the same blocking-thread treatment as SQL execution.
  - A held reader plus a timed read-only query is a deterministic way to prove queries are using read-mode instead of waiting behind write-mode.
---
## 2026-04-29 05:28:16 PDT - US-007
- Implemented a native single-statement execute API that returns rows, columns, changes, last insert row id, and route metadata.
- Routed `NativeDatabaseHandle::query` and `run` through the native execute path while leaving `exec` as the multi-statement compatibility path.
- Updated core inspector database execution to use the native execute path through `ActorContext::db_execute`.
- Files changed:
  - `CLAUDE.md`
  - `rivetkit-rust/packages/rivetkit-core/src/actor/context.rs`
  - `rivetkit-rust/packages/rivetkit-core/src/actor/mod.rs`
  - `rivetkit-rust/packages/rivetkit-core/src/actor/sqlite.rs`
  - `rivetkit-rust/packages/rivetkit-core/src/registry/inspector.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/database.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/query.rs`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `cargo check -p rivetkit-sqlite`
  - `cargo test -p rivetkit-sqlite execute_single_statement --lib`
  - `cargo check -p rivetkit-core`
  - `timeout 240s cargo test -p rivetkit-sqlite`
- **Learnings for future iterations:**
  - `ExecuteRoute` metadata is assigned by the database routing layer; the low-level query helper only prepares, steps, and packages the supplied route.
  - The native execute helper rejects multi-statement SQL by checking SQLite's prepare tail. Use `exec` when multi-statement compatibility is required.
  - Inspector database execution should use `db_execute` so INSERT RETURNING and write statements go through the same native routing policy as user database calls.
---
## 2026-04-29 05:36:07 PDT - US-008
- Exposed native SQLite `execute` and forced-writer `executeWrite` through `rivetkit-napi` and the TypeScript native database wrapper.
- Removed TS-side per-query serialization from native, raw, and Drizzle database paths; single-statement calls now route through native `execute`, while multi-statement compatibility stays on `exec`.
- Added a native wrapper close gate so close waits for admitted calls and rejects new work, plus migration `writeMode` so migration hooks use writer execution.
- Files changed:
  - `rivetkit-rust/packages/rivetkit-core/src/actor/sqlite.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/database.rs`
  - `rivetkit-typescript/packages/rivetkit-napi/src/database.rs`
  - `rivetkit-typescript/packages/rivetkit-napi/index.d.ts`
  - `rivetkit-typescript/packages/rivetkit/src/common/database/config.ts`
  - `rivetkit-typescript/packages/rivetkit/src/common/database/mod.ts`
  - `rivetkit-typescript/packages/rivetkit/src/common/database/native-database.ts`
  - `rivetkit-typescript/packages/rivetkit/src/common/database/native-database.test.ts`
  - `rivetkit-typescript/packages/rivetkit/src/db/drizzle.ts`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `cargo check -p rivetkit-sqlite`
  - `cargo check -p rivetkit-core`
  - `cargo check -p rivetkit-napi`
  - `timeout 240s cargo test -p rivetkit-sqlite`
  - `pnpm --dir rivetkit-typescript/packages/rivetkit run check-types`
  - `pnpm --dir rivetkit-typescript/packages/rivetkit exec vitest run src/common/database/native-database.test.ts`
  - `pnpm --dir rivetkit-typescript/packages/rivetkit exec biome check src/common/database/native-database.ts src/common/database/native-database.test.ts src/common/database/mod.ts src/db/drizzle.ts`
  - `pnpm --dir rivetkit-typescript/packages/rivetkit run lint` is still blocked by pre-existing unrelated Biome errors in driver fixtures and tests.
- **Learnings for future iterations:**
  - Use `SqliteDatabase.execute` in TypeScript wrappers for single statements so native classification owns read/write routing.
  - A close gate is enough for TS wrapper lifecycle safety; write serialization belongs in the native connection manager.
  - NAPI-generated route metadata is typed as `string` in `index.d.ts`, so the TS wrapper should normalize it before exposing the public union.
---
## 2026-04-29 05:45:00 PDT - US-009
- Added central SQLite read-pool rollout flags for enabled/disabled state, max readers, and idle reader TTL, then wired `open_database_from_envoy` through `NativeConnectionManagerConfig::from_optimization_flags`.
- Added read-pool Prometheus metrics for reader gauges, wait histograms, routed reads, write fallbacks, manual transaction duration, reader opens/closes, rejected reader mutations, and mode transitions.
- Preserved disabled single-writer behavior by routing all statements through the writer when `RIVETKIT_SQLITE_OPT_READ_POOL_ENABLED=false`, with a regression test proving SELECT does not open readers.
- Files changed:
  - `engine/packages/sqlite-storage/src/optimization_flags.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/connection_manager.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/database.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/vfs.rs`
  - `rivetkit-rust/packages/rivetkit-core/src/actor/metrics.rs`
  - `rivetkit-rust/packages/rivetkit-core/tests/metrics.rs`
  - `examples/kitchen-sink/scripts/sqlite-realworld-bench.ts`
  - `engine/packages/sqlite-storage/AGENTS.md`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `cargo check -p sqlite-storage`
  - `cargo check -p rivetkit-sqlite`
  - `cargo check -p rivetkit-core`
  - `cargo check -p rivetkit-core --features sqlite`
  - `cargo test -p sqlite-storage optimization_flags`
  - `cargo test -p rivetkit-sqlite disabled_read_pool_routes_select_through_single_writer --lib`
  - `cargo test -p rivetkit-core --features sqlite sqlite_read_pool_metrics_render`
  - `timeout 240s cargo test -p rivetkit-sqlite`
  - `pnpm --dir examples/kitchen-sink test`
- **Learnings for future iterations:**
  - The read pool is enabled by default to preserve prior native parallel-read behavior; disabled mode intentionally keeps one writer connection open and reports readonly statements as write fallbacks.
  - Existing actor metrics already implement the SQLite VFS metrics trait, so read-pool internals can be exposed by extending that trait without adding a second metrics plumbing path.
  - Idle reader TTL cleanup is lazy on read admission; there is no background timer for reader expiry.
---
## 2026-04-29 05:49:07 PDT - US-010
- Implemented kitchen-sink SQLite real-world benchmark reporting for read-pool route and transition metrics, including routed reads, write fallbacks, mode transitions, reader opens, and reader closes in both console output and `summary.md`.
- Tightened the static benchmark test so the runner and actor workload catalogs remain in sync and read-pool metric reporting stays visible.
- Added a reusable examples agent note for kitchen-sink SQLite real-world benchmark catalog sync and summary reporting.
- Files changed:
  - `examples/CLAUDE.md`
  - `examples/kitchen-sink/scripts/sqlite-realworld-bench.ts`
  - `examples/kitchen-sink/tests/sqlite-realworld-bench.test.ts`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `pnpm --dir examples/kitchen-sink test`
  - `pnpm --dir examples/kitchen-sink exec tsx scripts/sqlite-realworld-bench.ts --help`
  - `pnpm --dir examples/kitchen-sink exec biome check --formatter-enabled=false --assist-enabled=false scripts/sqlite-realworld-bench.ts tests/sqlite-realworld-bench.test.ts`
  - `pnpm --dir examples/kitchen-sink run check-types` is the package-declared typecheck and currently prints `skipped - workflow history types broken`.
  - Direct `tsc --noEmit` remains blocked by pre-existing kitchen-sink/server, Drizzle dependency, and workflow declaration errors outside this story.
- **Learnings for future iterations:**
  - `sqlite_read_pool_mode_transitions_total` is label-bearing, so benchmark metric parsing should sum all series for a metric family instead of taking the first sample.
  - Scrape actor metrics once per workload and derive VFS plus read-pool snapshots from the same Prometheus text to keep reported counters comparable.
  - The kitchen-sink package intentionally stubs `check-types`; use its static tests and a `tsx --help` smoke parse for benchmark-script-only changes unless the broader TypeScript config is repaired.
---
## 2026-04-29 06:03:27 PDT - US-011
- Added lifecycle and fencing stress coverage for native SQLite reader pools, including shutdown close ordering, reader fence mismatch fail-closed behavior, generation-specific VFS names, raw manual transaction write-mode retention, and shared routing gates for inspector/user operations.
- Fixed a manual transaction self-deadlock by routing work through the held writer while the manager is already in write mode.
- Files changed:
  - `AGENTS.md`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/connection_manager.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/database.rs`
  - `rivetkit-rust/packages/rivetkit-sqlite/src/vfs.rs`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `cargo test -p rivetkit-sqlite native_database --lib`
  - `cargo test -p rivetkit-sqlite connection_manager --lib`
  - `cargo test -p rivetkit-sqlite actor_replacement_generation_uses_distinct_vfs_registration_name --lib`
  - `cargo test -p rivetkit-sqlite --lib -- --test-threads=1`
  - `cargo test -p rivetkit-sqlite`
  - `cargo check -p rivetkit-sqlite`
- **Learnings for future iterations:**
  - If `NativeConnectionManager` holds an idle writer for a raw transaction, `NativeDatabaseHandle::execute` must bypass reader classification and reuse that writer for later statements such as `COMMIT`.
  - Fence-mismatch tests need to clear the VFS page caches after setup so the stale reader is forced to fetch through the engine and observe the replacement generation.
  - Native VFS registration tests can affect later tests because SQLite's VFS list is process-global; drop the stale registration before the replacement registration during cleanup.
---
## 2026-04-29 06:05:43 PDT - US-012
- Documented the SQLite read-mode/write-mode connection manager invariant in internal VFS docs, including exclusive write mode, no reader/write overlap, and the native routing policy boundary.
- Moved the read-mode/write-mode manager tracker entry from recommended work into existing optimizations.
- Preserved the reusable invariant in the root agent notes for future SQLite changes.
- Files changed:
  - `AGENTS.md`
  - `docs-internal/engine/sqlite-vfs.md`
  - `docs-internal/engine/SQLITE_OPTIMIZATIONS.md`
  - `scripts/ralph/prd.json`
  - `scripts/ralph/progress.txt`
- Checks:
  - `cargo check -p rivetkit-sqlite`
- **Learnings for future iterations:**
  - Read-pool v1 intentionally avoids reader/writer overlap instead of pinning per-reader head txids or snapshots.
  - Internal SQLite docs are the right home for cross-layer invariants; keep the optimization tracker limited to benchmark and performance status.
  - Root `AGENTS.md` already has a SQLite Package section for short reusable constraints that should apply across future implementation work.
---
