# Security Policy

## Supported Versions

We provide security updates for the following versions:

- **Latest minor version:** Active support (e.g., v0.20.x)
- **Previous minor versions:** Bug-fix support only
- **Versions older than 3 minor versions:** No support

Example:
- v0.65.0 -- Full support (latest)
- v0.64.0 -- Full support
- v0.63.0 -- Full support
- v0.62.0 -- Full support
- v0.61.0 -- Full support
- v0.60.0 -- Full support
- v0.59.0 -- Full support
- v0.58.0 -- Full support
- v0.57.0 -- Full support
- v0.56.0 -- Full support
- v0.55.0 -- Full support
- v0.54.0 -- Full support
- v0.53.11 -- Full support
- v0.53.10 -- Full support
- v0.53.9 -- Full support
- v0.53.8 -- Full support
- v0.53.7 -- Full support
- v0.53.6 -- Full support
- v0.53.5 -- Full support
- v0.53.4 -- Full support
- v0.53.3 -- Full support
- v0.53.2 -- Full support
- v0.53.1 -- Full support
- v0.53.0 -- Full support
- v0.52.0 -- Full support
- v0.51.0 -- Full support
- v0.50.0 -- Full support
- v0.49.0 -- Bug-fix support only
- v0.48.0 -- Bug-fix support only
- v0.47.x and below -- No support

## Reporting a Vulnerability

**Do not** open a public issue or pull request for security vulnerabilities.

Instead, use [GitHub Security Advisories](https://github.com/MakazhanAlpamys/Soup/security/advisories/new) to report privately, or email **vpn.alpamys@gmail.com** with:

1. **Description**: A clear explanation of the vulnerability
2. **Steps to Reproduce**: How to trigger or demonstrate the issue
3. **Affected Versions**: Which Soup versions are impacted
4. **Suggested Fix** (optional): Any proposed solutions
5. **Contact Info**: Your email for follow-up (optional)

### What to Include

```
To: vpn.alpamys@gmail.com
Subject: Security Vulnerability Report: [Brief Title]

Description:
[Explain the vulnerability in detail]

Affected Component:
[e.g., data/loader.py, trainer/sft.py, etc.]

Steps to Reproduce:
1. [Step 1]
2. [Step 2]
3. ...

Impact:
[What could go wrong? Data exposure? RCE? DoS?]

Suggested Fix (optional):
[Your proposed solution, if any]
```

## Response Timeline

- **Initial Response**: Within 48 hours
- **Assessment**: 1-3 business days
- **Fix Development**: Varies by severity
- **Patch Release**: As soon as possible after fix verification
- **Public Disclosure**: Coordinated with reporter (typically 90 days after patch release)

## Severity Levels

- **Critical**: Remote code execution, data exposure, complete compromise (patch within 24-48 hours)
- **High**: Authentication bypass, privilege escalation, denial of service (patch within 1 week)
- **Medium**: Information disclosure, partial compromise (patch within 2 weeks)
- **Low**: Minor issues with limited impact (patch in next regular release)

## Security Best Practices

When using Soup, follow these practices to stay secure:

### 1. Keep Soup Updated

```bash
pip install --upgrade soup-cli
```

### 2. Protect API Keys

Never commit API keys or secrets to version control. Use environment variables:

```bash
export HUGGINGFACE_TOKEN=your_token_here
export WANDB_API_KEY=your_key_here
soup train
```

### 3. Validate Data

- Only use trusted datasets
- Verify checksums for large datasets
- Inspect data for malicious content before training

### 4. Model Permissions

- Be cautious when downloading models from untrusted sources
- Use model hub providers with verified publishers (HuggingFace, Meta, etc.)
- Keep track of which models you've fine-tuned and their base model sources

### 5. GPU/Compute Safety

- Run on isolated machines if training on sensitive data
- Clear cache and temporary files after training
- Don't share fine-tuned models containing sensitive information

## Known Vulnerabilities

We maintain a log of known security issues and their fixes. This will be updated as issues are discovered and resolved.

### Current Status

No known critical vulnerabilities in current releases.

### Security Hardening History

- **v0.65.0 — Eval Depth**: 5 LIVE parts ship `soup eval behavior / capability / checklist / irt-subset` + judge calibration with conformal abstention. **TOCTOU `O_NOFOLLOW` + `os.fstat` on SAME fd** — `load_checklist_spec` + `load_response_rows` + `_read_evidence_json` all use `os.open(path, O_RDONLY | O_NOFOLLOW)` + `os.fstat(fd)` + `os.fdopen(fd, ..., closefd=True)` so the shared `enforce_under_cwd_and_no_symlink` helper's lstat-on-raw + the downstream open both target the SAME file descriptor (HIGH H-NEW-1 / H-NEW-2 fixes — earlier double-lstat-on-path was a TOCTOU race the attacker could win by swapping the regular file for a symlink between the two calls). On Windows ``O_NOFOLLOW`` is absent but the OS does not follow symlinks in `os.open` by default and the containment check is the primary gate. **`load_battery_probes` namespace-package safety** (HIGH H1) — uses `importlib.resources.files("soup_cli") / "data" / "_fixtures" / ...` Traversable `/` operator + `as_file` context manager + `os.lstat + S_ISLNK` rejection + 4 MiB cap; earlier `Path(os.path.join(str(pkg_root), ...))` silently failed `is_file()` on `MultiplexedPath` from namespace-package installs. **Word-boundary regex agreement** (HIGH H4 + MEDIUM M2) — `behavior_battery._agreement_rate` and `checklist_dsl._mft_pass` use `re.search(rf"\b{re.escape(target)}\b", lower)` instead of substring `in` matching, so `"safe"` cannot match `"unsafe"` and `"and"` cannot match `"sand"`. **CLI `_validate_run_id` gate** (MEDIUM M5) — every `soup eval behavior / capability / checklist` command validates `run_id` for null bytes / oversize (>256-char) / empty / non-string BEFORE the value flows into report payloads. **16 MiB evidence file cap** (MEDIUM M6) — `_read_evidence_json` rejects oversize evidence files via `os.fstat` on the O_NOFOLLOW-opened descriptor. **`_MAX_ROWS` counts skipped lines toward the cap** (MEDIUM M-NEW-2) — earlier `irt.load_response_rows` only counted successfully-parsed rows, so a 1M+1 malformed-line file would stream to completion (just logging WARNINGs); now total iterations are bounded. **INV empty-string normalisation rejected** (MEDIUM M-NEW-3) — `checklist_dsl._inv_pass` returns False when all responses normalise to the empty string (was silently passing INV on `["", "  "]`). **Atomic file writes** via shared `paths.atomic_write_text` in `_write_json_output` (LOW L6 dedup) + `os.lstat + S_ISLNK` rejection at the target so a pre-placed symlink cannot redirect the write. **Named-test errors** (MEDIUM M4) — `parse_checklist_spec` raises `tests[idx].prompts must be a list/tuple` and `tests[idx].expected must be a list/tuple` instead of generic messages, so operators can grep the index. **Closed-allowlist validators**: `SUPPORTED_BATTERIES = {xstest, harmbench, jailbreakbench, elephant, syceval}` (battery), `CAPABILITY_BENCHMARKS` 7-entry frozenset (capability), `PROFILES = {full, fast, math, code}` (suite), `CHECKLIST_KINDS = {mft, inv, dir}` (DSL), `IRT_PROFILES = {full, small, tiny}` (IRT) — all bool-first / null-byte / oversize (≤32-char) / case-insensitive normalisation; per-list MappingProxyType immutability. **`ensure_judge_calibrated` production gate** — refuses to score with `None` report / `calibrated=False` / agreement below `min_agreement=0.7` / position bias above `max_bias=0.3`, raising `RuntimeError` (not silent). **`PairwiseJudgement` validators** — winner Literal {a, b, tie} + null-byte prompt_id + 256-char cap. **`_MAX_PROBES=100_000` cap** on `compute_behavior_diff`; **`_MAX_PROBES_PER_TEST=10_000`** + **`_MAX_TESTS=1000`** + **`_MAX_PROMPT_LEN=8 KiB`** caps on CheckList; **`_MAX_ROWS=1_000_000`** + **`_MAX_FILE_BYTES=256 MiB`** on IRT. **`_EPSILON=1e-3` clipping** in `fit_difficulty` keeps `info` scores numerically stable at the extremes. **No heavy top-level imports** — torch/transformers/peft never imported at module scope in any v0.65.0 module (source-grep regression tests). **Test count**: 10306 → 10577 (+271 net in `tests/test_v0650_part_{a-e}.py` + `tests/test_v0650_followups.py`). **Review-fix coverage** across 2 review waves (general-purpose consolidated code+security+TDD; python-reviewer agent context-thrashed on the 800+ KB CLAUDE.md release-notes history — matches v0.58.0 / v0.59.0 / v0.60.0 / v0.61.0 / v0.62.0 / v0.63.0 / v0.64.0 idiom): 0 CRITICAL + 6 HIGH (H1 Traversable `/` + as_file + lstat + 4 MiB cap; H2 shared `enforce_under_cwd_and_no_symlink`; H3 streaming via `os.fdopen` + bounded total iteration; H4 word-boundary regex; H-NEW-1 `O_NOFOLLOW` + fstat in checklist + irt; H-NEW-2 `O_NOFOLLOW` + S_ISLNK in `_read_evidence_json`) + 9 MEDIUM (M2 word-boundary in checklist; M4 named-test error; M5 `_validate_run_id` gate; M6 16 MiB evidence cap; M-NEW-2 `_MAX_ROWS` counts skipped; M-NEW-3 INV empty-string rejected; M3 + M7 + M8 documented design choices) + 7 LOW (L1 0.0/0.85/0.60/1.0 boundary tests; L4 + L-NEW-3 skipped-row WARNING + caplog test; L6 `_write_json_output` dedup helper + source-grep guard; L-NEW-1 ASCII word-boundary docstring note; L-NEW-2 divergent validator docstring note). **Known limitations**: (1) Live lm-eval-harness invocation deferred — `soup eval capability` emits the manifest for downstream `soup eval benchmark` chaining. (2) Live model-driven `soup eval behavior` deferred — without `--evidence`, emits a neutral OK report. (3) Behaviour battery probe sets are tiny redacted placeholders — operators pull real sets from upstream papers. (4) IRT model is 1PL Rasch only (2PL / 3PL deferred). (v0.65.0)

- **v0.64.0 — Pre-flight & Tooling**: 6 LIVE parts ship `soup tunability` + `soup plan` / `apply` + `soup env lock` / `status` / `check` + analytical hardware-fit calculator + `soup completions` + `soup license-advisor`. **TOCTOU symlink rejection on every new read path** — `_load_yaml_config` in `commands/plan.py`, `load_report` in `utils/tunability.py`, `read_lock` in `utils/env_lock.py`, `read_state` in `utils/terraform_plan.py` all reject `S_ISLNK` BEFORE any `open()` so a pre-placed `soup.yaml -> /etc/shadow` cannot redirect the read (HIGH H2/H3/H4 fixes mirroring v0.33.0 #22 / v0.43.0 / v0.55.0 / v0.62.0 policy). **`compute_dataset_sha` containment + symlink reject** (HIGH H5) — operator-supplied `data.train` path is `is_under_cwd`-checked + `os.lstat + S_ISLNK`-rejected before SHA-256; outside-cwd path returns the zero-hash sentinel (no file read), so a crafted `data.train: /etc/shadow` cannot smuggle file contents into the planning SHA. **Containment-before-existence ordering** (MEDIUM M8) on every new read path so a probing attacker cannot distinguish "file exists outside cwd" from "file missing" via the exception type (`ValueError("outside cwd")` fires BEFORE the `isfile` check). **`compute_config_sha` strict JSON** (MEDIUM M1) — no `default=str` fallback so two configs that differ only in an unserialisable field cannot silently collide (a `set` value now raises `TypeError` instead of being string-stringified). **`_activation_bytes` overflow clamp** at `_MAX_ACTIVATIONS_BYTES=1e18` (MEDIUM M4) so the analytical predictor stays finite even at schema max `seq_len=1_048_576 × batch=1024`. **Llama-family allowlist replaces `.startswith`** (MEDIUM M6) — `flag_downstream_risk` uses tight `_LLAMA_COMMUNITY_LICENSES = frozenset({llama-2, llama-3, llama-3.1, llama-3.2, llama-3.3, llama-community})` for the 700M-MAU gate so a hypothetical future `llama-permissive-2030` cannot false-trigger the block. **Windows + POSIX CUDA path-parse** (MEDIUM M7) — `_detect_cuda_version` strips both `\\` and `/` separators and a leading `v` prefix so `C:\Program Files\NVIDIA\CUDA\v12.1` extracts `12.1` correctly. **Narrow `except` policy** (HIGH H1) — `_detect_package_version` catches only `PackageNotFoundError` + `(OSError, ValueError)` instead of bare `Exception` so real bugs in metadata parsing surface loudly. **Closed-allowlist validators**: `SUPPORTED_SHELLS = {bash, zsh, fish}` (completions), `DEPLOY_TARGETS = {b2c, defense, embedded}` (license advisor), `_VALID_QUANT` 9-entry + `_VALID_PEFT` 4-entry + `_VALID_OPTIMIZERS` 10-entry (hardware-fit), `_VALID_SOURCES = {pip, conda, system, wheel, unknown}` (env lock), `_VALID_SEVERITIES = {ok, warn, block}` (license risk) — all bool-first / null-byte / oversize (≤32-char) / case-insensitive normalisation. **Atomic file writes** via shared `paths.atomic_write_text` in all 3 v0.64 writers (`tunability.write_report` / `terraform_plan.write_state` / `env_lock.write_lock`) + `os.lstat + S_ISLNK` rejection at the target so a pre-placed symlink cannot redirect the write (LOW L3 source-grep regression test). **Strict-bool validation on read** (MEDIUM M3) — `read_state` rejects non-bool `applied` rather than silent `bool()` coercion of `"yes"` / `1` / `[]`. **MAU upper cap** at `_MAX_MAU=10**13` (LOW L4) so a crafted `--mau=10**18` cannot overflow the gate. **PEP 604 → `Optional[str]` for Typer Py3.9 compat** (MEDIUM M5) — every Typer `Option` annotation uses `Optional[str]` instead of `str | None` so Typer's runtime introspection works on Python 3.9. **`Sequence` from `collections.abc`** not `typing` (LOW L5) so `isinstance(x, Sequence)` works at runtime on every Py 3.9 build. **Drift detection refuses on mutation** — `soup apply` exits 3 on config / dataset SHA drift between `soup.yaml` and `soup.tfstate`, surfacing the changed field name (`config_sha` / `dataset_sha` / `base` / `task`). **License block → exit 3** — `soup license-advisor --license llama-3 --mau 800_000_000` surfaces the Meta acceptable-use-policy MAU gate as a hard block, not a warning. **Test count**: 10035 → 10306 (+271 net in `tests/test_v0640_part_{a-f}.py` + `tests/test_v0640_followups.py`). **Review-fix coverage** across 1 consolidated wave (general-purpose code+security+TDD; python-reviewer agent context-thrashed on the 800+ KB CLAUDE.md release-notes history — matches v0.58.0 / v0.59.0 / v0.60.0 / v0.61.0 / v0.62.0 / v0.63.0 idiom): 0 CRITICAL + 6 HIGH (H1 narrow-except, H2/H3/H4 read-path symlink rejection, H5 compute_dataset_sha containment + symlink, H6 test-coverage gap) + 8 MEDIUM (M1 strict JSON, M3 strict-bool applied, M4 overflow clamp, M5 PEP 604 → Optional, M6 Llama allowlist, M7 Windows CUDA, M8 ordering, plus M2 _entry_from_dict default documented) + 4 LOW (L3 atomic_write_text source-grep, L4 MAU cap, L5 Sequence from collections.abc, plus end-to-end drift-refusal regression). **Known limitations**: (1) Live LoRA probe deferred to v0.64.1 — `_default_probe` is heuristic stub. (2) `soup apply` does NOT execute `soup train` in-process — prints the canonical invocation; Typer commands aren't safe to re-enter. (3) `soup env` does NOT recreate a venv — schema + drift detection ship; `--fix` action is v0.64.1. (4) Hardware-fit is analytical only — composes with v0.40.3 live CUDA probe but does NOT replace it. (5) `complete_target_modules` returns canonical Llama defaults — live HF-config introspection is v0.64.1. (6) License advisor takes the license id from the operator — no model-card scraping. (7) Deploy targets are a 3-entry allowlist (b2c/defense/embedded). (8) `compute_dataset_sha` returns zero-hash for outside-cwd paths rather than raising so `soup plan` runs before the dataset exists. (v0.64.0)

- **v0.63.0 — Production Trace Ecosystem**: 5 LIVE parts ship `soup ingest` (Langfuse / LangSmith / Helicone / OpenPipe / OTel / OpenAI Stored Completions) + `soup prune-prompt` + `soup data active-sample` + `soup ab` (Wald sequential SPRT) + `soup drift-alarm` (rolling KL + SSRF-hardened webhook). **Soup never makes the SaaS network call** — `soup ingest` parses JSONL exports the operator pulls from their dashboard; zero credential-handling threat surface. **`validate_webhook_url` SSRF parity with v0.51.0 / v0.30.0 OTLP**: scheme allowlist `{http, https}`, null-byte + control-char rejection (CRLF-injection defence), `0.0.0.0` explicitly rejected, plain HTTP only for loopback hosts (`localhost` / `127.0.0.1` / `::1`), RFC1918 / link-local / reserved / multicast IPs rejected via `ipaddress.ip_address`. **`post_webhook` never crashes drift check** — broad `except Exception` (with `# noqa: BLE001`) so telemetry/alerting failure cannot mask a real drift verdict. **Closed-allowlist validators**: `SUPPORTED_INGEST_SOURCES = {langfuse, langsmith, helicone, openpipe, otel, openai-stored}`, `SUPPORTED_METRICS = {latency, judge_score, retry_rate}` (mSPRT) — both bool-first / null-byte / oversize (≤32-char) / case-insensitive normalisation. **`TraceRecord.metadata` is `MappingProxyType`-wrapped post-init** (`__post_init__` via `object.__setattr__` because the dataclass is frozen) so a returned record cannot be mutated by callers (code-review MEDIUM fix v0.63.0). **DoS caps**: `_MAX_INGEST_LINES=1_000_000` on JSONL ingest, `_MAX_SCAN_ROWS=100_000` + `_MAX_ROW_CHARS=1_000_000` on `prune-prompt`, `_MAX_BUDGET=100_000` on `active-sample`, `_MAX_SAMPLES_PER_ARM=1_000_000` on mSPRT, `_MAX_WEBHOOK_URL_LEN=4096` + `_MAX_TEXT_LEN=1_000_000` on drift. **Path containment** via shared `is_under_cwd` (project-standard `os.path.realpath + commonpath`, NOT `Path.resolve + relative_to`) — `ingest_traces` / `prune_traces` / `sample_uncertain_rows` / `run_msprt` / `run_drift_check` all enforce cwd + null-byte rejection BEFORE any open. **Rich markup escape** on every user-controlled string surfaced through `rich.markup.escape` (`--source` / `--metric` / `--logs` / error messages). **`detect_common_prefix` never early-exits on the 100% match** (code-review HIGH fix v0.63.0) — binary-search-over-templates finds the longest threshold-meeting prefix even when a 100% prefix exists shorter than a threshold-majority prefix. **`prune_traces` row-cap uses `break` not `pass`** (code-review HIGH fix v0.63.0 — earlier draft left the DoS cap unenforced). **mSPRT decision math is Wald's classic SPRT for the point alternative**: LLR is a martingale under H0 (E[exp(LLR_n)] = 1) so Type-I error is controlled at every stopping time per the optional stopping theorem. Earlier draft used a malformed mixture-prior LLR with the wrong sign on the log term — drove LLR positive under H0 as n grew, producing unbounded false-positive rate (code-review CRITICAL fix v0.63.0). **`_is_private_or_link_local` precedence parens** (code-review MEDIUM fix v0.63.0) — safety-critical SSRF gate, future edits should not need to re-derive Python operator precedence to verify the logic. **Auth env-var resolution is read-only** — `resolve_auth_env` looks up `LANGFUSE_KEY` / `LANGSMITH_API_KEY` / `HELICONE_API_KEY` / `OPENPIPE_API_KEY` / `OPENAI_API_KEY` / `OTEL_EXPORTER_OTLP_HEADERS` from `os.environ`; Soup itself never makes the network call. **Test count**: 9816 → 10035 (+219 net in `tests/test_v0630_part_{a-e}.py` + `tests/test_v0630_followups.py`). **Review-fix coverage across 2 reviewer waves** (code-reviewer + tdd-guide; python-reviewer + security-reviewer agents context-thrashed on the 800+ KB CLAUDE.md release-notes history — matches v0.58.0 / v0.59.0 / v0.60.0 / v0.61.0 / v0.62.0 idiom; verified manually): 1 CRITICAL (mSPRT sign error) + 2 HIGH (detect_common_prefix early-exit + `pass`→`break` DoS cap) + 3 MEDIUM (TraceRecord MappingProxyType + `_AUTH_ENV` dedup + drift_alarm precedence parens) + 2 LOW (pooled_se dead-branch + mean_uncertainty NaN guard) + 8 follow-up tests (boundaries + source-grep regression guards for no-heavy-top-level-imports across all 5 new util modules). **Known limitations**: (1) No live SaaS API pull — operators export from the vendor dashboard / SDK first, Soup parses the JSONL. (2) `prune-prompt` operates on raw characters not tokens — UTF-8 multi-byte sequences may be cut mid-codepoint at the threshold boundary. (3) Active-learning K>2 RMs deferred to v0.63.1 — 3+ RM scores fall back to `max - min` range. (4) mSPRT single-metric per pass — operators monitoring (latency, judge_score, retry_rate) jointly should adjust alpha via Bonferroni and run three passes. (5) mSPRT assumes Gaussian-like data — binary metrics must be pre-aggregated to per-prompt rates. (6) drift-alarm uses whitespace tokeniser — sub-word BPE-level drift invisible (tokenizer-aware variant tracked for v0.63.1). (7) `post_webhook` swallows every exception so telemetry can NEVER crash the drift run. (8) Zero-variance arms in mSPRT → `continue` forever (proper SPRT behaviour: with no observed noise the test cannot bound Type-I error honestly). (v0.63.0)

- **v0.62.0 — RAG & Activation Steering**: 5 parts ship `data.format='raft'` + RA-DIT two-stage recipes + `soup steer train/apply/list` (CAA / ITI / RepE) + citation-faithful FT + GRACE codebook. Schema-only release (live training loops + decode-hook intervention + codebook lookup all deferred to v0.62.1 — mirrors v0.50.0 / v0.52.0 / v0.61.0 stub-then-live pattern). **Closed-allowlist validators**: `SUPPORTED_RA_DIT_STAGES = {retriever, generator}`, `SUPPORTED_STEERING_METHODS = {caa, iti, repe}`, `SUPPORTED_CITATION_STYLES = {bracket, inline, footnote}`. All follow the project's bool-first / null-byte / oversize (32-char method names, 128-char steering names, 512-char model ids) / case-insensitive normalisation policy with frozen `*Spec` dataclasses + `MappingProxyType`-wrapped metadata. **RAFT input validation**: `_check_raft_string` rejects non-string / empty / null-byte / >64 KiB per field on every RAFT row field (query / golden_doc / answer / each distractor); `distractor_docs` list-type-guarded + capped at 64 entries per row. **Steering name regex** `^[A-Za-z0-9][A-Za-z0-9._\-]{0,127}$` rejects path separators + shell metacharacters so the name is safe as a Registry artifact id + CLI flag + filename fragment. **Citation kernel ReDoS-safe**: `extract_citation_ids` regex `\[(?P<bracketed>[A-Za-z0-9][A-Za-z0-9._\-]{0,127})\]` uses bounded quantifiers (no nested optional groups); `score_citations` predicted text capped at 2 MB, expected_ids capped at 10k entries with per-entry null-byte + bool rejection. **`soup steer` TOCTOU defence**: `_validate_pairs_path` delegates to shared `enforce_under_cwd_and_no_symlink` (centralised TOCTOU helper, not a custom re-impl per review M1) — symlinks at the pairs JSONL path rejected before any open/read. **`soup serve --steer` Rich markup escape** on the exception-message error panel (security review M1 fix — defends against crafted `--steer` value injecting Rich markup via `{value!r}` repr). **`--base` length cap** at 512 chars + null-byte rejection (security review L1 — mirrors v0.40.5 reward_model policy). **`list_steers` SQLite context-manager** (`with RegistryStore() as store:` — fixes connection leak surfaced by code-review M2). **Cross-validator silent-no-op rejection**: `citation_style` / `citation_recall_threshold` without `citation_faithful=true` rejected; `citation_faithful=true` requires `data.format='raft'` AND `task in {sft, pretrain}` (task-gate review M3 fix mirroring v0.52.0 distill / classifier policy); `grace_codebook_size` / `grace_codebook_dim` without `grace_codebook=true` rejected, and `grace_codebook=true` requires BOTH knobs; `ra_dit_retriever_model` without `ra_dit_stage` rejected. **GRACE codebook bounds**: `grace_codebook_size` ∈ [1, 100_000], `grace_codebook_dim` ∈ [1, 16_384] — sanity caps so a misconfigured run cannot allocate a multi-GB codebook by accident. **`apply_edit` deferred-marker routing**: `grace` plans raise NotImplementedError with `v0.62.1` marker while legacy rome/memit/alphaedit retain the v0.61.1 marker (regression-guarded via `TestEditMarkerRegressionGuard`). **Deferred stubs validate first**: `apply_steering` / `build_steering_vector` / `apply_grace_codebook` all run input validators BEFORE raising NotImplementedError so callers passing junk get a meaningful TypeError / ValueError instead of a confusing deferred-live error. **`typing.Iterable` → `collections.abc.Iterable`** migration in `citation_faithful.py` (python-review M1 — `typing.Iterable` deprecated since 3.9 for annotation use). **`except Exception` → `except ImportError`** narrowed in `commands/steer.py::list_steers` (python-review M2). **Test count**: 9571 → 9786 (+215 net in `tests/test_v0620_part_{a-e}.py` + `tests/test_v0620_followups.py`). **Known limitations**: (1) Every live wiring (RAFT span-mask trainer, RA-DIT two-stage orchestrator, `apply_steering` decode-hook, `build_steering_vector` per-method kernels, `apply_grace_codebook` lookup/write) deferred to v0.62.1 with explicit v0.62.1 markers. (2) `citation_faithful` gated to {sft, pretrain} — DPO / GRPO / etc. rejected (task-gate review fix). (3) `soup steer apply` is preview-only — live decode-hook ships via `soup serve --steer` in v0.62.1. (4) `extract_citation_ids` is bracket-style only; inline / footnote variants schema-validated but use the same extractor today. (5) RA-DIT recipe pair does not auto-link the retriever's output to the generator's `ra_dit_retriever_model` — operators copy manually. (6) GRACE codebook has no Registry persistence yet — lives only in the trained checkpoint until v0.62.1 wires `apply_edit(method='grace')` into v0.26 Registry. (7) `soup serve --steer` validates name + strength but does not load the vector — decode-hook lands in v0.62.1. (8) **4-agent review wave**: code + security + tdd + python style. Direct reviewer agents hit context-window thrash on the 800+ KB CLAUDE.md release-notes history (matches v0.58.0 / v0.59.0 / v0.60.0 / v0.61.0 idiom); the `general-purpose` agent with explicit "do not crawl, read only these N files" prompts produced equivalent findings consolidated into 4 sequential review waves. (v0.62.0)

- **v0.61.0 — Unlearning & Knowledge Edit**: 5 parts ship `task='unlearn'` (NPO / SimNPO / RMU) + `soup eval unlearning` (TOFU / MUSE / WMDP) + `soup edit set` (ROME / MEMIT / AlphaEdit) + sequential edit governor + `soup edit diff`. Schema-only release (live trainer + kernel wiring deferred to v0.61.1 — mirrors v0.50.0 / v0.52.0 / v0.53.0 stub-then-live pattern). **TOCTOU lstat-before-write** on every operator-supplied path via the shared `paths.atomic_write_text` helper (write_unlearn_report / write_diff_report); `load_evidence_file` + `load_probes` cwd-contained + `os.lstat + S_ISLNK` rejection + size-cap (16 MiB evidence, 16 MiB probe file, 1000 probe rows, 4096-char per prompt with WARNING-log truncation per review MEDIUM M5). **Frozen dataclasses** for every report / plan surface (`UnlearnMetric` / `UnlearnReport` / `EditRequest` / `EditPlan` / `FactChange` / `DiffReport` / `NormBlowupPolicy` / `MethodRecommendation`); `EditGovernor` counters declared as real dataclass fields (review HIGH H1 — slots-safe, `asdict` / `replace` compatible). **Invalid evidence raises loudly** in `run_unlearn_eval` (review HIGH H3) — distinguishes "missing key → neutral OK" from "present-but-invalid → ValueError" so operator typos surface instead of silently scoring OK. **Validator policy** mirrors v0.41.0 / v0.51.0 / v0.52.0 / v0.53.0: bool-first / null-byte / oversize (≤32-char method names, ≤64-char benchmark / verdict, ≤512-char base_model, ≤4096-char forget_set/retain_set/subject/target) / case-insensitive normalisation; numeric bounds with `math.isfinite` rejection on NaN / Inf. **`registry_id` validated BEFORE deferred attach** (review HIGH H4) — defends against crafted null-byte / oversize crashing the v0.61.1 registry hook. **Exit-code policy**: `soup eval unlearning` exits 0 on OK / MINOR (operator can choose to gate), 2 on MAJOR (matches v0.56.0 diagnose strict-mode CI gate); `soup edit set` exits 0 with `--plan-only`, 2 on schema rejection, 3 on deferred-live `NotImplementedError` (review HIGH H5 — distinguishes "not yet shipped" from "validation rejection", matches v0.56.0 strict-mode policy). **`importlib.resources`** for TOFU fixture path (review MEDIUM M6 — zipapp / namespace-package safe; mirrors v0.53.8 #93 `_bundle_source_path` policy). **Test count**: 9446 → 9571 (+125 net in `tests/test_v0610_part_{a-e}.py`; 192 pass + 1 POSIX-only symlink test skipped on Windows). **Known limitations**: (1) Every live wiring (NPO/SimNPO/RMU loss kernels, ROME/MEMIT/AlphaEdit closed-form updates, `UnlearnTrainerWrapper.setup`) deferred to v0.61.1 with explicit v0.61.1 markers — schema accepts every flag today so soup.yaml written now will work the moment v0.61.1 ships. (2) MUSE + WMDP fixtures NOT bundled (only TOFU mini-set); operators wanting MUSE / WMDP today supply pre-computed evidence JSON via `--evidence`. (3) `soup edit diff` is shape-only — live model loading + before/after generation lands in v0.61.1. (4) Registry-attach for edited models is plan-only — v0.61.1 wires the `parent_adapter` lineage write. (5) `EditGovernor` is single-process (no SQLite persistence). (6) Norm-blowup detection depends on operator-supplied `norm_delta` — v0.61.1's live `apply_edit` will compute + thread it through. (7) Unlearn task gated to non-mlx (mlx-lm doesn't expose per-token logp scoring needed for NPO/SimNPO). (8) `apply_edit` does not consult `EditGovernor` automatically — v0.61.1 wires the optional `governor: EditGovernor` kwarg. (9) **5-agent review wave**: python + code + security + tdd + verification. Direct `code-reviewer` / `security-reviewer` agents hit context-window thrash on the 800+ KB CLAUDE.md release-notes history (matches v0.58.0 / v0.59.0 / v0.60.0 idiom); the `general-purpose` agent with explicit "do not crawl, read only these N files" prompts produced equivalent findings consolidated into Wave 1 (code) + Wave 2 (security) + Wave 3 (TDD). (v0.61.0)

- **v0.10.10**: Bearer token auth on Web UI, CORS restrictions, path traversal protection, SSRF prevention, max_tokens limits, supply-chain pinning (llama.cpp b5270), deprecated CLI secret flags
- **v0.12.0**: experiment_name path traversal validation, GaLore parameter type enforcement
- **v0.13.0**: Batch inference max_tokens capped at 16384, trust_remote_code warning
- **v0.14.0**: Plaintext loader UTF-8 encoding, MoE config validation (moe_aux_loss_coeff ge=0, moe_lora boolean only)
- **v0.14.3**: Data validate auto-detects format, Web UI `--show-token` flag + auth token documented
- **v0.15.0**: `rope_scaling_type` Literal constraint, `max_length` bounds (ge=64, le=1048576), FSDP config key allowlist, Liger Kernel exception handling narrowed
- **v0.16.0**: `embedding_loss` Literal constraint, `embedding_margin` gt=0 validation, ONNX export without trust_remote_code (with warning), TensorRT export subprocess list args (no shell injection), speculative decoding SSRF-protected (URL blocked) with warning panel, vLLM speculative model URL validation
- **v0.17.0**: Server data generation provider SSRF validation (scheme whitelist + localhost-only HTTP), audio model trust_remote_code warning panel, audio file path traversal protection (resolved paths confined to audio_dir), SGLang backend trust_remote_code warning panel
- **v0.18.0**: Ollama deploy GGUF path traversal protection + `.gguf` extension validation, model name validation (no path separators/null bytes), subprocess list args (no shell injection), Modelfile parameter key allowlist + value newline/null sanitization, overwrite warning panel
- **v0.19.0**: Custom eval JSONL schema validation + 10k task cap, regex scoring ReDoS guard, judge API SSRF protection + API key isolation, human eval local-only terminal UI + 10k prompt cap, leaderboard read-only SQLite queries
- **v0.20.0**: Ollama provider localhost-only validation (remote blocked), Anthropic provider API key from env only (never CLI arg), vLLM provider SSRF protection (scheme whitelist + localhost-only HTTP), output path traversal protection (`..` blocked), configurable rate limiting (`--requests-per-minute`)
- **v0.21.0**: Migrate input/output path traversal protection (resolve + relative_to(cwd)), Unsloth .ipynb AST-only parsing (no exec/eval), recipes output path traversal protection, NEFTune config bounded (ge=0.0, le=50.0)
- **v0.22.0**: Multi-adapter serving path traversal protection (resolve + relative_to(cwd)), adapter name validation (alphanumeric + hyphens only), unknown adapter returns 404 (not 500)
- **v0.23.0**: AWQ/GPTQ calibration data path traversal protection (resolve + relative_to(cwd)), AWQ/GPTQ output path stays under cwd, curriculum_buckets bounded (ge=1, le=20), AWQ/GPTQ trust_remote_code warning panel
- **v0.24.0**: HF download trust_remote_code=False + warning panel, HF download output path sanitized (Path.name), download --samples capped at 1M, dataset registry name validation (no path separators/null bytes), registry path traversal protection, loss_watchdog threshold le=100 + patience le=1000, freeze_layers le=1000
- **v0.24.1**: AWQ/GPTQ output path traversal validation moved before import check (previously unreachable when autoawq/auto-gptq not installed), Windows Unicode fix for Rich console output (replaced non-ASCII symbols with ASCII equivalents)
- **v0.24.2**: Chat proxy SSRF protection (localhost-only HTTP, HTTPS for remote), chat proxy max_tokens capped at 16384 + temperature/top_p bounded, chat proxy Bearer token auth required, XSS prevention (HTML-escape before markdown render), runs compare max 5 runs, config from-form validates via load_config_from_string, SSE read endpoints no auth (GET)
- **v0.25.0**: Tool-calling JSON-only parsing (no eval), RLVR math_verify regex-extracted numerics (no eval), code_exec 5s timeout + 512MB RLIMIT on POSIX + ephemeral cwd + socket patch + `python -I -S` + 10KB output cap, verifiable_domain Literal constraint, LoRA PEFT mutual exclusion (DoRA/VeRA/OLoRA), data augment path containment + caps, forgetting_detection bounds, checkpoint_intelligence bounds + symlink refusal, autopilot path containment (realpath + commonpath) + goal Literal + GPU/time budget bounds, MLX trainers no trust_remote_code
- **v0.26.0 — Registry**: name/tag validation (alphanumeric + `_-.` only, null-byte rejected, name ≤128 / tag ≤64 chars), artifact path containment (default `enforce_cwd=True` via `os.path.realpath + commonpath`, stored path is realpath), SQL LIKE wildcard escaping (`%` and `_` escaped with `ESCAPE '\\'` in `search()` and prefix `resolve()`), DB 600 perms on POSIX, lineage indirect-cycle detection (BFS ancestor walk before insert), CLI Rich markup escaped everywhere, `resolve()` raises `AmbiguousRefError` on ambiguous prefix (no silent None)
- **v0.26.0 — Eval Gate**: suite path via shared `utils/paths.is_under_cwd` containment, `regression_threshold` [0.0, 1.0], `every_n_epochs` [1, 100], `on_regression` Literal ("stop"/"warn"/"continue"), `GateTask.tasks`/`prompts` null-byte rejection, `judge_model` URL scheme allowlist (`ollama://`, `https://`, `http://localhost`/`http://127.0.0.1`) — SSRF hardening, callback fails-safe: structured errors treated as regressions under `on_regression="stop"`
- **v0.26.0 — Trace-to-Preference**: input/output path containment via shared `is_under_cwd`, trace line cap 100,000, `--format`/`--signal` Literal validation, PII warning panel before every run, JSON-only parsing (no eval), malformed JSON lines skipped silently
- **v0.26.0 — Quant-Lobotomy**: `--before`/`--after`/`--tasks` all containment-checked, `registry://` refs support optional `kinds` filter to avoid picking the wrong artifact, format Literal validated
- **v0.26.0 — Soup Cans**: Manifest format version pinned to 1; name alphanumeric+`_-.`; author max 128 chars, no null bytes/newlines; created_at must parse via `datetime.fromisoformat`; description max 4096; DataRef URL HTTPS-only; hf_dataset regex-validated; tar extraction uses `filter="data"` on Python 3.12+, fallback only on `TypeError`/`AttributeError` (not `TarError`); manual symlink/hardlink rejection + `commonpath` check; 100 MB size cap on pack + fork; dunder-key (`__*__`) and null-byte rejection in fork modifications to prevent prototype pollution; inspect/read_config refuse paths outside cwd
- **v0.27.0 — Multi-GPU Mastery**: `--gpus` bounds (reject bool, non-digit, zero, negative, values above `MAX_GPU_COUNT=128`); `--gpus auto` on 0-GPU host prints explicit yellow warning (no silent no-op); Rich markup escaped on `--config` path before embedding in the multi-GPU advice Panel; `accelerate launch` argv assembled via `shlex.quote` per element (copy-pasted command safe against crafted paths); `build_accelerate_argv` validates `num_processes >= 1`, `mixed_precision` Literal (`no/fp16/bf16/fp8`), `num_machines` bounded `[1, 256]`; ZeRO++ integer literals (`int(1e9)` not float) so DeepSpeed strict JSON validator accepts; `validate_fsdp2_compile_config` requires FSDP + CUDA + transformers + torch>=2.2/accelerate>=0.27; DeepSpeed-MII stub exits non-zero to prevent silent mis-start; `validate_pipeline_config` enforces `pipeline_stages >= 2` + CUDA + `gpu_count >= stages`; `pipeline_stages` Pydantic bounds `[1, 16]`; `parallelism` Literal `data|pipeline`; NCCL env (`NCCL_P2P_DISABLE`/`NCCL_IB_DISABLE`/`NCCL_NVLS_ENABLE`) applied via `os.environ.setdefault` only — user/launcher overrides are never stomped
- **v0.28.0 — Training Speed & Memory**: `quantization_aware: Union[bool, Literal["fp8"]]` rejects arbitrary strings (only `true` / `false` / `"fp8"`); FP8 path requires CUDA + Hopper+ SM capability + transformers backend; `gradient_checkpointing: Union[bool, Literal["selective","medium","full","auto"]]` rejects unknown tier strings and returns only HF-supported keys (no private markers leak into `TrainingArguments.gradient_checkpointing_kwargs`); `activation_offloading` Literal `cpu|disk`, scratch `save_dir` containment-enforced via shared `utils/paths.is_under_cwd` before disk writes, `torch.load(weights_only=True)` prevents arbitrary Python deserialization on reload, TOCTOU closed between `mkstemp` and `torch.save` by holding the fd open, best-effort cleanup on context exit (handles SIGKILL mid-backward); `kernel_picker.pick_best_kernel` raises `ValueError` when all candidates lack a finite `time_ms` (prevents silent promotion of an untimed combo); Cut CE architecture detector matches on last path component only (so `deepseek-ai/...-phi-...` org-prefix does not trigger a Phi patch on a DeepSeek model); `build_cross_doc_mask` numpy-vectorised to avoid O(seq_length²) pure-Python fill at `max_length` bound (1M); `@model_validator` requires `packing=true` when `packing_cross_doc_attn_mask=true` (prevents silent no-op); `SoupConfig._validate_v028_speed_memory_sft_only` rejects `use_cut_ce`/`quantization_aware="fp8"`/`kernel_auto_compose`/`activation_offloading` on non-SFT tasks — prevents legacy int8-QAT wrapper from crashing on the string `"fp8"` and prevents silent no-ops on DPO/GRPO/KTO/etc. (multi-trainer wiring tracked for v0.28.1)
- **v0.29.0 — HF Hub Deep Integration**: `HF_ENDPOINT` SSRF-hardened — scheme allowlist (http/https), null-byte rejection, `0.0.0.0` explicitly rejected, plain-HTTP only permitted for loopback (`localhost`/`127.0.0.1`/`::1`), RFC1918 / link-local / cloud-metadata (169.254.x) IPs rejected via `ipaddress.ip_address`; repo ID regex `[A-Za-z0-9][A-Za-z0-9._-]{0,95}` per component, ≤200 chars total, null-byte / whitespace / `..` / leading-`/` rejection (applied to `push --repo`, `train --push-as`, `data push --hf-dataset`, `deploy hf-space --model/--space`); collection slug `owner/slug-hash` regex-validated, ≤256 chars; HF token resolution single-sourced in `utils/hf.resolve_token` (env > cached login), explicit non-printable tokens rejected, `push --token` flag deprecated with yellow warning; `soup push --model` confined to cwd via `is_under_cwd` (prevents crafted `soup.yaml output:` from uploading system files); auto-push checkpoint `allow_patterns` restricts uploaded files to `*.safetensors`/`*.bin`/`*.pt`/`*.json`/`tokenizer*`/`trainer_state.json`/`training_args.bin`/`README.md` (keeps `.env` and source files out of auto-pushed branches); `prepare_hf_resume` enforces cwd containment and passes `local_dir_use_symlinks=False` (defeats symlink-based FS escape on older `huggingface_hub`); commit messages stripped to first line and capped at 200 chars (prevents multi-line injection into public HF commit history); `_render_eval_scorecard` neutralises `|`/`[`/`]`/`(`/`)`/`!`/newlines/tabs/`<`/`>` in task names and non-numeric scores; `data_lineage` HTML-escaped (defeats XSS on HF Hub README viewer); `render_space_template` validates `model_repo` via `validate_repo_id` before substitution into rendered `app.py` (crafted repo id cannot inject Python code); `HFPushCallback` uses sticky `_repo_failed` flag to short-circuit retries after hard failure (no log spam, no wasted API calls); `add_to_collection` prefers HfHubHTTPError 409 detection over string-match for duplicate handling
- **v0.30.0 — Inference Excellence**: OTLP endpoint SSRF-hardened matching v0.29.0 `HF_ENDPOINT` (scheme allowlist, `0.0.0.0` rejected, RFC1918 / link-local / cloud-metadata via `ipaddress.ip_address`, plain HTTP loopback-only); `pick_draft_model` rejects URL-scheme target names (`http://`/`https://`/`file://`), null bytes, names >200 chars; `validate_regex_pattern` length-capped at 2048 + null-byte rejection + must compile; `validate_json_schema` dict shape + 64KB serialised cap + required `type` field; `--json-schema` file path confined to cwd via shared `utils/paths.is_under_cwd`; `--structured-output json` requires `--json-schema` (fail-fast prevents silent no-op); FastAPI `/v1/adapters/activate/{name}` pattern `^[a-zA-Z0-9][a-zA-Z0-9\-]*$` enforced before handler runs; activate/deactivate state protected by `threading.Lock` (no race on concurrent hot-swap); `/v1/adapters` response omits filesystem paths (names + active flag only); CORS on transformers backend restricted to loopback origins (`^https?://(localhost|127\.0\.0\.1)(:\d+)?$`) since hot-swap endpoints mutate state without auth; `auto_quant.Candidate` name regex + score `[0.0, 1.0]` + finite-float check + non-negative latency; `pick_best` generator-safe (materialises to list) so error-message count is accurate; OTel span context uses `contextlib.ExitStack` so `__exit__` sees real exception info (spans correctly marked error on HTTPException); `record_latency` always runs in `finally` so tail-latency percentiles include failure paths; `build_tracer` idempotent — only installs provider when current is `ProxyTracerProvider`/`NoOpTracerProvider` (preserves operator-supplied instrumentation)
- **v0.32.0 — Training Stability & Auto-Tuning**: `--find-lr-output` containment via shared `utils/paths.is_under_cwd` (prevents writes outside cwd); `save_lr_finder_report` rejects NaN / Infinity floats in `lrs` / `losses` and serialises with `allow_nan=False` (keeps the report parser-safe); `compute_lr_schedule` rejects non-positive `start_lr`, inverted ranges, and `num_steps` outside `[2, 10_000]`; `pick_mixed_precision` rejects empty / null-byte / >200-char model names and resolves multi-version quirks (`qwen2.5` vs `qwen2`, `phi-3.5` vs `phi-3`) by longest-substring-first iteration so an added family can never accidentally make a more-specific entry dead code; `compute_warmup_steps` clamps to `[10, 1000]` with a `ratio==0.0` short-circuit matching HF Trainer's "no warmup" convention; `SpikeRecoveryStrategy` is `@dataclass(frozen=True)` (post-construction mutation cannot bypass validation), `max_attempts ∈ [1, 10]`, `lr_decay ∈ (0, 1)`, `min_lr > 0`; cross-validator `_validate_spike_recovery_requires_watchdog` rejects `loss_spike_recovery=true, loss_watchdog=false` at config-load (fails fast instead of never triggering); `convergence_window ∈ [5, 10_000]`, `convergence_rel_tol ∈ (0, 1]`, `recommend_action` reuses `detect_plateau` so plateau heuristic stays single-source-of-truth; `GradAccumMonitor.recommend()` caps doubled `accum` at `MAX_ACCUM=1024` so a runaway advisory loop cannot blow up DataLoader prefetch; `generate_config` validates BOTH the YAML output path AND the embedded `decisions["output"]` field via `is_under_cwd` (closes the gap where a crafted `decisions["output"]="../../etc"` would have silently propagated into the rendered YAML)
- **v0.34.0 — Observability & Dev UX**: `.crash` bundle generator (`utils/crash.py`) recursively redacts `hf_*` / `sk-*` / `Bearer …` token-shaped strings in any captured `config` and metric tail before serialisation, so a `.crash` file shared on a public GitHub issue cannot leak credentials; `output_dir` is reduced to `os.path.basename` so `$HOME` doesn't leak; `write_crash_bundle` uses `os.path.realpath + commonpath` for cwd containment (Windows-safe; raises `ValueError` not `PermissionError` so callers cannot silently swallow with `except OSError`); filename appends `secrets.token_hex(4)` so two crashes in the same UTC second don't collide; bundle truncated to `MAX_BUNDLE_BYTES=1_000_000`. `train.py` crash-write surfaces failures to the user (no silent missing-bundle). `profiling.py` `resolve_trace_path` rejects empty / `.` / `..` / `/` / `\\` / null-byte `run_id` (closes the `output_dir/profiles/../trace.json` escape) and uses `os.path.realpath + is_under_cwd`; profiles dir is created only on successful torch import (no stale empty dirs on torch-less CI). `tracker.get_run` LIKE-prefix match escapes `%` / `_` / `\\` and uses `ESCAPE '\\'` so a crafted `run_id` cannot widen the match (mirrors v0.26.0 registry policy). Lazy schema migration (`_ensure_schema`) tolerates the "duplicate column" race when two CLI processes start simultaneously on a fresh DB (fork-based multi-GPU training, TUI auto-refresh). `runs.py show/replay/clean` switched user `run_id` rendering to `markup_escape` and switched `clean` containment from broken `Path.resolve() + relative_to()` to project-standard `os.path.realpath + is_under_cwd`. `tui_app.py` lazy-imports `ExperimentTracker` and `markup_escape`s every DB-sourced string before passing into Textual widgets so a crafted base_model / experiment_name cannot inject `[bold red]…[/]` markup. `run_cost.estimate_run_cost_usd` rejects `bool` in `num_gpus` (bool is a subclass of int — same defence as v0.30.0 `Candidate.__post_init__`); duration clamped to `[0, 1 year]`; unknown GPU returns `None` so callers render `—` instead of fabricating `$0.00`. `log_level.parse_log_level` rejects non-string + null-byte input.
- **v0.33.0 — Live Wire**: RLVR `code_exec_reward` adds OS-level isolation (Linux best-effort `os.unshare(CLONE_NEWUSER|CLONE_NEWNET|CLONE_NEWPID)`, macOS `sandbox-exec` with default-deny `MACOS_SANDBOX_PROFILE` narrowed to a 3-name `mach-lookup` allowlist to prevent DNS / NSURLSession bypass of `(deny network*)`); `prune_checkpoints` switches to TOCTOU-safe `os.lstat + S_ISLNK` + `shutil.rmtree(onerror=_abort_on_symlink)` so a symlink encountered mid-walk aborts rather than escapes; `run_gate` wraps each task scorer in a typed `try/except` so backend failures produce `score=None, error=str(exc)` (never silent `score=1.0`); `_parse_judge_url` removes the bare `http://` catch-all (defence-in-depth after the Pydantic GateTask validator); `soup can run` requires `--yes` or explicit consent callback and raises `ValueError` (not `PermissionError`, which is an `OSError` subclass that broad `except` blocks would swallow); GGUF `rglob` result for ollama deploy is `realpath+commonpath` checked against extract_dir (prevents symlink escape from a crafted can); `DeployTarget.path` validator normalises mixed `\\`/`/` separators before splitting (closes a Windows `..` bypass); `CAN_FORMAT_VERSION` 1→2 (additive — v1 still loads); `soup can publish` validates `repo_id` via `utils/hf.validate_repo_id`, resolves token via `resolve_token`, sanitises commit messages (first-line, 200-char cap), uses HTTPS-only HfApi; `_write_spike_recovery_hint` adds `is_under_cwd` containment check on `args.output_dir` from raw HF `TrainingArguments`; `lookup_entry_by_output_dir` emits `ResourceWarning` when 1000-row scan limit is hit (no silent miss); `CrossDocCollator` no longer mutates input feature dicts (HF Dataset rows are cached and reused — mutation broke subsequent batches); `Candidate` rejects `bool` in `score`/`latency_ms` (was sneaking past `int` isinstance check); `evaluate_candidate` latency mean now divides by *completed* prompts (excludes crashed) so a broken candidate isn't artificially fast; `auto_quant.run_auto_quant_picker` soft-falls-back to highest-scored candidate when no candidate clears `min_score` (server still binds); `build_logits_processors` returns `[]` when neither `outlines` nor `lm-format-enforcer` is installed (server degrades to free-form rather than 500); MII server uses loopback-only CORS, max_tokens cap [1, 16384], stream rejection, generic 500 with no stack-trace leak; `os.execvp` auto-reexec uses list args (no shell), all forwarded flags pre-validated; `cleanup_extract_dir` uses `os.path.commonpath` (Windows-safe) instead of `startswith`; `_run_subprocess` catches `TimeoutExpired` and returns rc=124 (coreutils convention) instead of an unhandled traceback; new `eval_results` and `tensorrt` artifact kinds in `RegistryStore._VALID_KINDS`
- **v0.54.0 — `soup advise` (pre-flight decision)**: 3 Parts (Verdict engine / Probe runner / Cross-project learning). 5 review-agent waves (python / code / security / tdd / architect) ran; 16 findings fixed (0 CRITICAL + 5 HIGH + 7 MEDIUM + 4 LOW). (argv rewriter scope) `_rewrite_advise_argv` in `cli.py` is scoped to `argv[1] == "advise"` only (code-review HIGH fix — defends against false-positive rewrites when an unrelated `soup train --config advise.yaml` arg happens to contain the literal string `"advise"`). (`_write_last_verdict` atomicity + symlink reject) `~/.soup/advise_last.json` scratch file uses `tempfile.mkstemp` + `os.replace` for atomicity (defends against concurrent `soup advise` invocations racing to truncate-then-write) and rejects symlinks at the target path via `os.lstat + S_ISLNK` BEFORE any write (code-review HIGH fix — matches v0.33.0 #22 / v0.43.0 Part C / v0.44.0 Part B TOCTOU policy). (`_read_last_verdict` symlink reject) symlink rejection on the raw path BEFORE `open` (security-review MEDIUM fix — defends against a planted symlink leaking `/etc/passwd` content through `json.load` between successful write and next read). (`record_verdict` cross-process locking) `_append_with_lock` uses `fcntl.flock(LOCK_EX)` on POSIX and a separate `<target>.lock` sidecar file with `msvcrt.locking` on Windows (code-review HIGH fix — defends against torn-write race on Windows where two `soup advise --record` processes from different CI shells could seek to the same end-of-file offset before writing). The Windows sidecar avoids the `msvcrt.locking` seek-position pitfall (security-review MEDIUM fix — `seek(0)` on an append-mode file wouldn't match the locked range). (`load_history` per-line cap) `_MAX_HISTORY_LINE_BYTES=64 KiB` per-line cap on `load_history` reads (security-review MEDIUM fix — defends against a malformed or adversarially-crafted history file with a single multi-MiB JSON object causing OOM via unbounded `json.loads`). (`load_advise_dataset` containment + symlink reject) cwd-containment via shared `utils/paths.is_under_cwd` + `os.lstat + S_ISLNK` rejection on the RAW path BEFORE `realpath` (TOCTOU defence — matches v0.53.7 #106 policy); 1 GiB file cap + 1M row cap + UTF-8 BOM strip. (`history_path` env override containment) `SOUP_ADVISE_HISTORY_PATH` must resolve under `$HOME` / `$CWD` / `tempfile.gettempdir()`; out-of-bounds values fall through to default (mirrors v0.36.0 `SOUP_BATCH_CACHE_PATH` policy). Null-byte env values rejected. Dead-code branch `or not override` removed from the env-override guard (code-review MEDIUM fix). (`classify_task` per-row cap) per-row corpus chunks capped at 4096 chars BEFORE `findall` regex sweep (security-review LOW fix — defends against ~400 MiB allocation on adversarially-padded 200-row datasets at the `_MAX_FIELD_CHARS=1M` upper bound). The goal text is repeated 10× as SEPARATE list entries (NOT `goal * 10` string multiplication, which would have produced a single 40,960-char chunk bypassing the per-row cap — code-review MEDIUM fix). (`_TASK_KEYWORDS` ReDoS-safe) every regex is a word-boundary alternation with no nested optional quantifiers; the closed `_REASONING_RE` pattern is bounded literal. (`record_verdict` field validation) `accepted: bool` required (`isinstance(bool)` strict — rejects `"yes"` / `1`); `outcome: Optional[float]` rejects bool / non-finite (NaN, ±Inf) / out-of-`[-1, 1]`; `notes` rejects null-byte / oversize (> 4096 chars) / strips CRLF (defends against an attacker breaking the JSONL line layout via crafted notes containing newlines); `choice` + `task_category` re-validated against `CHOICES` + `TASK_CATEGORIES` allowlists at record time (defence-in-depth even though `Verdict` is frozen). (`load_history` history-row validation) `_row_to_entry` requires `isinstance(choice, str)` AND `choice in CHOICES` (defence-in-depth — defends against a hand-crafted history row with a non-string choice causing downstream type confusion); same guard on `task_category`; `confidence` / `outcome` reject `bool` before the float coerce. (Rich markup escape) all CLI output paths in `commands/advise.py` route user-controlled `goal` / `notes` / `reason` / `reverse_when` / project name / timestamp through `rich.markup.escape` (matches v0.43.0 Part B `Tournament` policy — defends against `[red]evil[/]`-style injection from a crafted dataset / history entry). Test surface: `tests/test_v0540.py` carries 132 new tests including 8-thread concurrent-record stress, oversize-history-line skip, symlink rejection on both load + write paths, null-byte goal reject, bool-delta `(invalid)` rendering, `path=` kwarg-bypasses-containment documentation regression guard. Known limitations: (1) Live model loading in `synth_probe_*` deferred to v0.54.1 — heuristic stubs ship with forward-compat `model` / `device` / `lr` / `timeout_seconds` kwargs already on the signature. (2) `record_verdict(path=...)` explicit kwarg is documented as trusted-internal-only (tests + `commands/advise.py`); operator-supplied paths must arrive via `SOUP_ADVISE_HISTORY_PATH` env which IS containment-checked. (3) Windows file locking uses a sidecar `.lock` file rather than locking the data file directly (POSIX `fcntl.flock` works on the data file); the sidecar persists across runs by design. (4) `langdetect` / Llama-Guard-3-1B integration left out of `[data-pro]` extras as in v0.53.10 known. (v0.54.0)
- **v0.53.10 — Quick wins + packaging + UX wiring**: 7 features. 4 review-agent waves (python / code / security / tdd / verification-loop) ran; 13 findings fixed (4 HIGH + 5 MEDIUM + 4 LOW). (#152 Rich markup injection via `base` / `local_path` / `cache_dir`) `utils/hubs.prefetch_model_from_hub` now wraps every user-controlled string with `rich.markup.escape` before embedding in `console.print` — a crafted `base` like `[bold red]evil[/bold red]` no longer renders as styled output (security-review HIGH). (#152 absolute-path leak in advisory) the SDK-returned `local_path` and the `cache_dir` are reduced to `os.path.relpath` (basename fallback on `ValueError` / `OSError`) before display so `$HOME`-prefixed paths from `modelscope.snapshot_download` no longer leak in the dim advisory line (code-review HIGH; matches v0.34.0 `crash.py` policy). (#152 slug-regex traversal defence) `[^A-Za-z0-9._-]+` substitution neutralises path-separator + `..` segments in `base` before the cache subdir join, with a final `is_under_cwd` realpath check on the resulting `cache_dir` so a crafted `base: "../../etc"` cannot escape `.soup_hub_cache/`. (#152 `cache_root` user override) when caller supplies `cache_root` to `prefetch_model_from_hub`, the resolved `cache_dir` is `is_under_cwd`-checked — an outside-cwd override raises `ValueError("resolved hub cache dir escapes the current working directory")` (security-review MEDIUM; regression test in `test_prefetch_outside_cwd_cache_root_raises`). (#154 PostHog endpoint precedence) `_resolve_posthog_target` now uses a sentinel default `_POSTHOG_ENDPOINT_DEFAULT` so a caller passing `endpoint=_POSTHOG_ENDPOINT` literally is NOT silently overridden by `SOUP_POSTHOG_ENDPOINT` (code-review HIGH — documented precedence contract is now correctly enforced). (#154 control-char + oversize key rejection) `SOUP_POSTHOG_KEY` env values are rejected on null byte, any byte `< 0x20` (control char / CRLF injection defence), or length > 256 — prevents an operator dropping `\nAuthorization: bypass` into the env from leaking a second header to the PostHog endpoint. (#154 endpoint SSRF re-validation) `_telemetry_endpoint_is_safe` delegates to `utils.hubs.validate_hub_endpoint` so the resolved endpoint goes through the same v0.51.0 HTTPS-only + loopback-HTTP-only + RFC1918 / link-local / cloud-metadata rejection policy used for HF / ModelScope hub endpoints. (#155 XSS-by-construction) Tool Outputs panel renders every cell via `document.createElement('td')` + `td.textContent = value` — NO `innerHTML` for any user-controlled field. A tool-call record containing `<img onerror=alert(1)>` will render as visible text, not HTML. (#156 callback best-effort policy) `on_step_end` is wrapped in a blanket `except Exception: # noqa: BLE001` per the v0.39.0 best-effort design — a misbehaving tool buffer / global singleton failure can NEVER take down a multi-hour training run. Cross-cutting: 0 CRITICAL findings; test surface `test_v05310.py` carries 45 new tests including source-grep regression guards (`apply_hub_to_cli_model` imported in every non-push command; `wait for v0.53.9` advisory removed from `data.py`), `prefetch_model_from_hub` cache_root containment, PostHog explicit-endpoint vs env precedence, oversize-key + empty-key rejection, and callback bool / empty-list short-circuits. (v0.53.10)
- **v0.53.9 — Live Dashboard + UX + Bench + Standalone CLIs**: 8 features. 3 review-agent waves (python / code / security / tdd) ran; 34 findings fixed (11 HIGH + 14 MEDIUM + 9 LOW). (#95 `set_auth_token` rotation race) module-level `_auth_token_lock: threading.Lock` now guards every read AND write of `_auth_token` in `ui/app.py`; `_verify_token` snapshots the expected `f"Bearer {_auth_token}"` under the lock before string comparison so concurrent rotation can never leave a window where some requests see the old token and some the new (security-review HIGH H2). 8-thread rotation stress test in `test_v0539.py`. (#95 QR token consumption) JS bootstrap in `ui/static/app.js` reads `?token=` from `URLSearchParams` on first load, populates `window._authToken`, caches in `sessionStorage`, and `history.replaceState`s the URL clean of the token so it does not sit in browser history (security-review HIGH H1 — token in QR URL was previously unreachable). (#95 CORS for `--public` LAN mode) when `host == "0.0.0.0"` the `CORSMiddleware` switches from a fixed `allow_origins=[f"http://{host}:{port}"]` (which never matches a real `Origin` header) to an `allow_origin_regex` covering loopback + RFC1918 ranges (`10.x.x.x` / `192.168.x.x` / `172.16-31.x.x`); Bearer token remains the actual security gate (security-review MEDIUM M5). (#15 `tokenizer.py` symlink rejection) `_validate_under_cwd` runs `os.lstat + stat.S_ISLNK` on the RAW user-supplied path BEFORE `os.path.realpath` resolution — `realpath`-first would silently follow symlinks and lstat the resolved regular-file target (security-review HIGH; matches v0.53.7 #106 policy). Post-`mkdir` re-check on the output directory defends against a symlink planted between containment check and write (security-review MEDIUM M1; matches v0.43.0 Part D `copy_bundle_to`). 50 MiB total + 8 KiB per-line corpus caps bound RAM on adversarial input (security-review HIGH). Panel + success messages use `.name` (basename only) to avoid leaking absolute paths (security-review MEDIUM M4; v0.34.0 `crash.py` policy). (#26 bench `--prompts-file` symlink) `os.lstat + S_ISLNK` on the raw path BEFORE realpath (security-review MEDIUM M2; parity with project TOCTOU policy). (#28 `_has_mlx_weights` symlink) per-entry `os.lstat` + `S_ISLNK` rejection so a symlinked `weights.npz` does NOT trigger MLX dispatch (code-review MEDIUM). (#98 `strip_reasoning` ReDoS bound) fast-path skip when the marker token is absent in `text.lower()` — bounds worst-case `re.sub` time on adversarial inputs without actual reasoning blocks; 1 MiB input cap; strips ONLY leading newlines after block removal so code outputs that begin with intentional spaces / tabs are preserved (security-review MEDIUM M3). (#94 SSE drain-race fix) `/api/train/stream` switched from destructive `buffer.drain()` to per-subscriber `snapshot_since(cursor)` so multiple concurrent dashboard tabs each receive every event (code-review HIGH); switched from `time.sleep(0.1)` in a sync generator to `asyncio.sleep(0.1)` in an `async def` generator so uvicorn's async loop is never blocked (code-review HIGH). (#94 `TrainEventBuffer` typing) closed allowlist + `TrainEvent`-only push validation; module-level singleton + test-hook `reset_global_buffer` is documented as test-only because reassignment under no lock is acceptable only when no training loop is active. (#100 `/api/tool-outputs`) `limit` Pydantic-bounded `[1, 1000]`; output preview already 4 KiB-capped by `ToolOutputsBuffer.record_call` (v0.44.0). Cross-cutting: 0 CRITICAL findings; test surface `test_v0539.py` carries 57 new tests including symlink rejection regressions, concurrent rotation stress, and SSE cursor rollover math. (v0.53.9)
- **v0.53.8 — Remote data + Hubs + Trackers (wave 2)**: 6 features wired live. 3 review-agent waves (python / code / security) ran; 16 findings fixed (3 HIGH + 8 MEDIUM + 5 LOW). (#130 hub dispatcher) `download_repo` / `upload_repo` reject `local_dir` / `folder_path` outside cwd via `utils.paths.is_under_cwd` (security-review HIGH — closes a gap where a public helper could write to arbitrary absolute paths); `_validate_repo_id_shape` rejects bool / null-byte / leading-slash / `..` segments / control characters (CR/LF, defence-in-depth against header injection) / oversize (>200 chars). `modelscope.push_model` does NOT accept `commit_message` — the sanitized message is omitted on that backend to avoid a silent TypeError at runtime (code-review HIGH). (#130 `commands/train.py` pre-fetch) cache subdir name sanitised via `re.sub(r"[^A-Za-z0-9._-]+", "__", cfg.base)` then strip-leading/trailing-dots to defeat Windows `..\\` traversal that would have escaped `.soup_hub_cache/` (security-review HIGH); resolved cache dir re-checked via `is_under_cwd` as defence-in-depth. `cfg.base = local_path` mutation replaced with `cfg = cfg.model_copy(update={"base": local_path})` so Pydantic field validators rerun (matches v0.33.0 #47 immutability policy; python-review + code-review HIGH). Pre-fetch idempotent — probes `cache_dir / config.json` and reuses on resume runs. `ImportError` print uses `rich.markup.escape` to defend against markup injection from a crafted dep name (security-review MEDIUM). (#90 telemetry) `_telemetry_endpoint_is_safe` re-runs the v0.51.0 hub-endpoint SSRF validator on every caller-supplied endpoint (HTTPS + private-IP / link-local / RFC1918 rejected) — defence-in-depth so a misconfigured `endpoint='https://10.0.0.1/'` cannot reach internal networks even when telemetry is opt-IN (security-review MEDIUM). Lazy `httpx` import + 1s hard timeout + silent-fail on every exception (no `traceback`). (#85 fsspec loader) `validate_remote_uri` runs BEFORE any `fsspec.open` connection (bucket regex, no userinfo / query / fragment); row-count capped at 1,000,000 to defend against pathological remote objects (security-review MEDIUM). (#69 `detect_space_sdk`) input length capped at 256 KB to match v0.39.0 Part E template policy — refuses to scan oversized requirements.txt that would degrade response time (security-review LOW). (#93 package-data) fixtures moved to `soup_cli/data/_fixtures/`; `_bundle_source_path` falls back to legacy `examples/data/` for editable installs. (#89 `tracker_missing_dep_message`) switched from `__import__(pkg)` to `importlib.util.find_spec` (non-executing probe) so swanlab can't initialise network threads during the absence check (code-review MEDIUM; `sys.modules[pkg] = None` sentinel still honoured for tests). Test surface: `tests/test_v0538.py` carries 66 new tests including 4 new cwd-containment + SSRF rejection regression guards. (v0.53.8)
- **v0.53.7 — Data Forge + Pipeline live (wave 1)**: 11 features (6 new + 5 v0.53.6 stub-to-live), 1 CRITICAL + 11 HIGH + 17 MEDIUM + 10 LOW review findings fixed. (#103 `/v1/tools/bash` REVERTED to HTTP 501) security review caught that `/bin/sh -c` spawns a child process outside the RLVR sandbox's OS-level isolation (`unshare(CLONE_NEWNET)` on Linux, `sandbox-exec` on macOS, socket monkey-patch); a caller could reach the cloud-metadata service from the child shell. Endpoint returns 501 with v0.53.8 marker until container/namespace work lands. Python + web_search remain LIVE. (#103 `/v1/tools/{python,web_search}` Bearer auth gate) optional `auth_token` kwarg on `_create_app` + `Bearer` gate on both endpoints; default `None` preserves loopback-only back-compat per v0.30.0 Part C CORS policy. (#103 `/v1/tools/web_search` deny-by-default) `WebSearchConfig.domain_allowlist` defaults to empty = deny-all (returns 403 `"web_search disabled"`); only operators who supply a domain list via constructor enable the endpoint. 5s `httpx` timeout, 5-result cap, snippet null-byte stripped. (#102 `/v1/messages` redaction) malformed Anthropic payloads (incl. validator-shape errors) mapped to generic `"Invalid request"` 400 body; detailed exception text logged server-side at DEBUG (matches v0.40.3 #33 trace-log policy). (#102 vLLM `/v1/messages` CORS) restricted to loopback origins on both transformers AND vLLM backends since it's a mutation route (matches v0.30.0 Part C transformers-backend policy that was previously vLLM-wildcard). (#102 SSE header injection) `model` + `msg_id` strings stripped of CRLF / NUL / oversize before SSE `data:` embedding — defends against `model="foo\ndata:{injected}\n\n"` injecting spurious SSE frames; `Cache-Control: no-store` + `X-Accel-Buffering: no` headers on `StreamingResponse` prevent intermediary caching of model outputs. (#106 `_node_seed` symlink TOCTOU) `os.lstat` runs on the RAW path BEFORE `realpath` resolves the symlink (matches v0.33.0 #22 `prune_checkpoints` policy); `realpath`-first ordering would have been a no-op since `S_ISLNK` on a resolved path is always False. (#106 `_save_checkpoint` atomic) switched from `path + ".tmp"` + plain `open()` to `tempfile.mkstemp(dir=path.parent, suffix=".tmp")` + `os.fdopen` + `os.replace` (matches v0.43.0 Part D `copy_bundle_to` / v0.48.0 `write_mix_recipe` policy); eliminates predictable `.tmp` symlink hazard. (#106 `failed_reason` redaction) raw exception messages run through `_redact_exc_message` which strips POSIX `/abs/path` and Windows `C:\abs\path` runs to `os.path.basename` and caps at 256 chars (matches v0.34.0 `crash.py` redaction policy); a `FileNotFoundError` no longer leaks the operator's `$HOME` into the world-readable `.checkpoint.json`. (#106 `_node_code` row JSON double-encode) row payloads serialised via `json.dumps(json.dumps(row))` before f-string embedding into the RLVR sandbox wrapper — defends against Python `repr()` escape edge cases on Unicode that could break sandbox invocation. (#106 resume rehydration) completed-node outputs rehydrated from per-node `.node-<name>.jsonl` sidecar files (NOT empty list — a v0.53.7 in-progress draft had a correctness defect that silently fed downstream consumers an empty list, causing zero-row output on resume). (#86 atomic Arrow write) `Dataset.save_to_disk` writes to a `.tmp_<name>` sibling directory then `os.replace`-ed (mirrors v0.48.0 `write_mix_recipe` pattern); mid-write Ctrl-C / OOM no longer leaves a partial Arrow directory that looks complete to subsequent runs. (#86 cache-hash gate) SFT + Pretrain wrappers verify `metadata.json::cache_key` matches the freshly-computed `make_preprocess_cache_key(base, max_length, format, train)` BEFORE consuming the Arrow shards; mismatch raises `ValueError("cache hash mismatch: re-run `soup data preprocess`")` so a user who pre-tokenized with one config and changed `max_length` cannot silently train on stale shards. Missing metadata.json falls back to yellow advisory + trusted mode. (#87 `resolve_prompt_strategy`) operator-supplied module imported via `importlib`; signature-validated `Mapping → Mapping`; `except` narrowed to `TypeError` only so the explicit ValueError from the signature-shape check propagates (a v0.53.7 in-progress draft had `except (TypeError, ValueError): pass` which swallowed its own validator); `assert isinstance(spec, str)` replaced with `raise TypeError` so the validator survives `python -O`. Per-row invocations swallow exceptions at DEBUG and fall through to the raw row (matches v0.33.0 #47 `CrossDocCollator` immutability policy). LIMITATION: this is a trusted-input surface — do NOT use `prompt_strategy` with operator-controlled YAML on shared CI hosts. (#88 markdown ingest) heading-section text capped per regex `^#{1,6}\s` so a crafted MD with a 1-byte heading followed by 1 GB body still respects the existing `data/ingest` MD branch size cap from v0.42.0 Part F. (#111 forge `--judge-provider`) lazy v0.20.0 provider instantiation reuses existing SSRF hardening per backend: Ollama localhost-only (rejects remote IPs), Anthropic env-only API key (`ANTHROPIC_API_KEY` — never CLI arg, matches v0.20.0 policy), vLLM scheme-allowlist + loopback HTTP. Per-call exceptions counted + logged at DEBUG (no silent silent-degrade). (#112 `--benchmark-file`) operator JSONL path runs through `is_under_cwd` + `os.lstat + S_ISLNK` rejection BEFORE the corpus is loaded (matches v0.46.0 `agent_forge.load_spec_file` policy). (#105 `instantiate_trainer_plugins`) `_instantiate_simple_plugin` returns None + WARNING when no `attr_candidates` match the imported module (matches v0.45.0 `register_plugin` policy — surface bad shapes loudly, not silently); `except TypeError` narrowed (a v0.53.7 in-progress draft swallowed all `TypeError` from the wrong layer). Cross-cutting: SSE generator return types annotated (`Generator[str, None, None]`); `JUDGE_PROVIDERS: frozenset[str]` PEP 585 element type; `extract_row_text` exposed publicly in `utils/data_score.__all__` (cross-module import of private helper eliminated). Test surface: `tests/test_v0537.py` carries 111 new tests; CI green across Ubuntu/macOS/Windows × Python 3.9/3.11/3.12. (v0.53.7)
- **v0.53.6 — Plugin + Agent + Anthropic API**: 6 features — 3 live, 3 deferred-live stubs. (#101 SoupPluginCallback live) New `soup_cli/monitoring/plugin_callback.py` ships `SoupPluginCallback(transformers.TrainerCallback)` that dispatches the four canonical hook events to every enabled plugin in the v0.45.0 registry. Per-hook invocation runs through `_safe_invoke` which swallows ALL exceptions at WARNING with `exc_info=True` — defends against one misbehaving plugin crashing a multi-hour training run (mirrors v0.44.0 / v0.45.0 plugin-loader policy). Hook snapshot is collected ONCE in `build_plugin_callback` and passed to the constructor so a plugin registered between the "is empty?" check and ctor cannot silently slip into the active hook list (code-review HIGH fix — race-window closed). `attach_plugin_callback(trainer, console=None) -> bool` short-circuits to False when no plugins are enabled (zero overhead on the hot path); inner `add_callback` failure is swallowed at DEBUG so a plugin-infrastructure bug never crashes training. Wired into all 13 transformer-backend trainers; source-level grep regression test parametrized over every trainer file. (#102 Anthropic `/v1/messages` live) New POST `/v1/messages` route reuses the v0.45.0 SSRF-hardened `from_anthropic` converter + the existing chat handler. Streaming requests return 501 BEFORE schema validation (defence-in-depth — defends against stream-only attackers leaking validator-error detail). All validation paths (`validate_anthropic_payload` / `from_anthropic` / `ChatCompletionRequest(**openai_payload)`) are wrapped to map to a generic `"Invalid request"` 400 body; detailed exception text is logged server-side at DEBUG (security-review MEDIUM fix — matches `utils/errors.py` policy "HTTP error responses return generic messages, details logged server-side"). The existing loopback-CORS restriction from v0.30.0 Part C and the trace-log-redaction from v0.40.3 #33 both apply to the new route. (#104 n-gram speculative decoding live) `_generate_response` gains `ngram_config: Any = None` kwarg; when set, emits `prompt_lookup_num_tokens=int(ngram_config.num_draft_tokens)` into `model.generate(...)`. The int cast is wrapped in `try/except (TypeError, AttributeError)` as defence-in-depth; primary validation lives upstream in v0.45.0 `validate_ngram_config` (bool-rejected, bounded `[1, 32]`). Mutually exclusive with `assistant_model` — real draft model wins. (#103 server-side tool endpoints STUB) Three new POST routes `/v1/tools/python` / `/v1/tools/bash` / `/v1/tools/web_search` return 501 with v0.53.7 marker. Tool names are hardcoded string literals (not user-controlled); no payload inspection. Live RLVR `code_exec` sandbox HTTP wrapper (python / bash) + `WebSearchConfig.domain_allowlist` enforcement (web_search) land in v0.53.7. (#106 `run_recipe` STUB + `soup data recipe --execute`) New `utils/recipe_run.run_recipe(dag, *, output_dir, ...)` type-checks every parameter (TypeError on non-`RecipeDAG` / non-string / empty `output_dir` / non-bool `resume` / non-string `judge_*`) BEFORE the `NotImplementedError("v0.53.7")`. CLI `--execute --output <dir>` enforces `is None` guard on `--output` (project policy since v0.40.6 — empty-string is a distinct operator error, NOT silent missing) + `is_under_cwd` containment at the CLI boundary BEFORE the `run_recipe` call (security-review MEDIUM fix — defends against future v0.53.7 live runner bypass; never want the live runner to be the first/only enforcement point per project policy). Regression test asserts the v0.53.7 marker does NOT surface on outside-cwd paths — guards against future live runner accidentally bypassing CLI containment. (#105 `instantiate_trainer_plugins` STUB) Validates name list through existing `validate_trainer_plugin_list` (closed allowlist, dedup, ≤8/run, canonicalisation — all v0.45.0 hardening) BEFORE raising `NotImplementedError`. Canonical names echoed in the error message are allowlist-validated lowercase strings (no injection surface). Test surface: 1 new test file (`tests/test_v0536.py`) carrying 53 new tests covering: per-hook exception swallow with caplog isolation, source-level grep parametrized over 13 trainer files (canonical import + invocation), generic `"Invalid request"` body redaction with detail logged at DEBUG, oversize `max_tokens` cap, n-gram kwarg omitted when config is None (regression guard against future "always emit" refactor), `run_recipe` type-rejection matrix (non-DAG / empty output_dir / non-str checkpoint_dir / non-str judge_provider / non-str judge_model / bool resume), `--execute --output ""` rejected with `"must be a non-empty path"`, outside-cwd `--output` rejected with `v0.53.7` marker NEVER surfacing on rejected paths. Known limitations: (1) vLLM `/v1/messages` parity deferred to v0.53.7 — Anthropic route is transformers-only. (2) Streaming `/v1/messages` returns 501 — true SSE event-shape ships in v0.53.7. (3) Plugin hook snapshot is at callback-construction time — a plugin registered mid-run does not retroactively receive hooks (by design — prevents partial-run inconsistencies). (4) n-gram is transformers-only — vLLM ships its own prompt-lookup path that needs a different kwarg. (5) Tool endpoints + recipe runner + trainer-plugin instantiation are stubs — live wiring in v0.53.7. (v0.53.6)

- **v0.53.5 — Adaptive Training (BETA → stable)**: six closes lifting the v0.48.0 BETA deferrals to live wiring. (#114 DynamicCurriculumCallback) new `soup_cli/monitoring/curriculum_callback.py` accepts `output_dir` through `is_under_cwd` containment + null-byte / oversize / non-string rejection (matches v0.40.5 `reward_model` policy) BEFORE any filesystem touch; per-step bucket-stats appended to `curriculum_history.jsonl` via `tempfile.mkstemp` + `os.replace` atomic write (mirrors v0.48.0 `write_mix_recipe` policy) so a crash mid-write cannot leave a half-row at the target. Rank-0 guard via `_is_rank_zero` helper — defends against multi-rank double-write on shared filesystems. Multi-rank coordination via `torch.distributed.all_reduce(SUM)` of per-bucket stats BEFORE `compute_bucket_weights` invocation — defends against per-rank divergent samplers (the DDP footgun the v0.48.0 schema gate explicitly warned about). (#115 multi-trainer expansion) `_validate_curriculum_dynamic_supported` cross-validator widened from `{sft, pretrain}` to every transformer-backend trainer; MLX backend still rejected with a distinct error message (matches v0.34.0 review-fix policy). New `attach_curriculum_callback(trainer, tcfg, output_dir, console=None) -> bool` shared helper follows the v0.40.6 `attach_relora_callback` pattern with `is None` guard on `tcfg.curriculum_dynamic` — defends against the schema-bypass footgun where `curriculum_dynamic=0` (falsy but explicit) would silently no-op a deliberate disable. (#116 `soup data mix --live`) new `soup_cli/utils/mix_proxy.py::proxy_run_for_weights` — argv-list `subprocess.run([sys.executable, "-m", "soup_cli.cli", "train", ...], timeout=…)` with NO shell (defends against shell injection via crafted dataset path / base-yaml path), tmp YAML staged under `tempfile.mkdtemp(prefix=".soup_mix_proxy.")` then cleaned up in `try/finally`. Weights validated through simplex (sum-to-1 ± 1e-6), finite (`math.isfinite`), `bool`-rejected, per-element `[0, 1]` bounds (matches v0.30.0 `Candidate` / v0.48.0 `MixCandidate` policy). `base_yaml_path` runs through `is_under_cwd` containment + null-byte rejection BEFORE the tmp YAML is rendered. `timeout_seconds ∈ [60, 30*60]` Pydantic-style bounds. Tracker-SQLite read uses existing v0.34.0 `ExperimentTracker.get_run` policy (LIKE escape, parameterised SQL). `SOUP_DB_PATH` env-override propagated via subprocess `env=` so per-candidate DB isolation is possible. (#117 skopt OptimizerProtocol) lazy `import skopt` — when the optional dep is absent, `_build_default_optimizer` falls back silently to the v0.48.0 Dirichlet sampler (mirrors v0.43.0 Part A tracker-package policy). The skopt path drives `skopt.Optimizer(GP)` through the existing v0.48.0 `OptimizerProtocol` — no API surface change. (#118 `MixOptimizationReport.elapsed_seconds`) headline elapsed now sums only successful-candidate wall-clock; failed-proxy time excluded. No security-surface change — observability fix only. Test surface: 1 new test file (`tests/test_v0535.py`) carrying 57 new tests covering cwd containment, symlink rejection, simplex + finite + bool validation, argv-list subprocess shape, atomic JSONL append, idempotent attach helper, MLX-rejection cross-validator, multi-trainer task gate. Known limitations: (1) `_pick_bucket` is step-mod round-robin (BETA); loss-percentile / curriculum-metric routing tracked for a follow-up patch. (2) `validate_distributed_curriculum` helper still requires callers to attest `rank_coordinated=True` for external invocations; the new callback wires `all_reduce` internally so the schema invocation passes. (3) `proxy_run_for_weights` is single-shot — concurrent proxy runs would race on `~/.soup/experiments.db`; the `SOUP_DB_PATH` env override is the per-candidate isolation hatch. (4) `scikit-optimize` is an optional dep, not bundled — silent fallback to Dirichlet sampler when absent. (5) `MixOptimizationReport.elapsed_seconds` is a behaviour change — operators printing the report's headline elapsed must note it now excludes failed-candidate time. (v0.53.5)

- **v0.53.4 — Long Context + Architecture**: six closes covering LongLoRA hardening, LLaMA Pro live wiring, and a CUDA-OOM-hint UX upgrade. (#11 OOM hint) `format_friendly_error` upgrades the CUDA-OOM and `OutOfMemoryError` patterns to point users at the explicit `--batch-size <half>` / `--grad-accum <double>` CLI flags before the legacy `quantization: 4bit` fallback — closes #11 with no functional change to the security surface. (#122 FlashAttention v3 incompatibility) New `soup_cli/utils/flash_attn.is_flash_attn_v3_available() -> bool` is a defensive probe (never raises, False on missing `flash_attn` / non-string `__version__` / unparseable / major < 3). `validate_longlora_compat` calls it AFTER the existing task / backend / architecture / ring-attention checks so the FA-v3 error only surfaces on otherwise-valid LongLoRA configs (avoids spurious confusion on unrelated misconfig). The check is loaded via a function-scoped import to keep `validate_longlora_compat` import-cheap and avoid CUDA-side effects at config load time on machines without `flash_attn` installed. (#120 LongLoRA arch allowlist) `soup_cli/utils/longlora.py` ships three new word-boundary regex helpers (`is_mistral_model`, `is_qwen_model`, `is_phi_model`) — same regex policy as v0.39.0 `is_gemma4_model` (rejects substring matches like `"my-mistralish-finetune"` or `"unmistral-7b"`). Shared `_check_model_name` input guard rejects `bool` BEFORE the `isinstance(str)` check (because bool is a subclass of int and would otherwise fall through silently — matches v0.53.3 `is_known_vlm_base` policy), rejects null bytes via explicit substring check, and returns `None` (→ helper returns False) for inputs >512 chars (avoids ReDoS-style overhead on adversarial input). New `is_supported_longlora_arch(model_name: object) -> bool` is the union accessor with defensive non-string surface (returns False rather than propagating TypeError, matches v0.53.3 / v0.52.0 model-detection policy). `validate_longlora_compat` also gained per-call null-byte rejection + bool/non-string TypeError on `task` and `backend` (matches v0.50.0 `validate_long_context_grpo_compat`); new `_truncate_for_message(value, limit=64)` helper bounds the `base` echo in error messages (security-review MEDIUM fix mirroring v0.53.3 `validate_vision_grpo_compat` redaction — defends against adversarial / long bases bloating stderr + log files). Mixtral is INTENTIONALLY excluded from the allowlist — regex matches `mistral` as a word-boundary token, NOT `mixtral`; documented at the docstring so a future contributor adding Mixtral support adds it explicitly. (#121 Llama 3.1 RoPE auto-detect) `apply_long_context_config` extended with `rope_scaling_type=None` auto-detect path — reads `model_config.rope_scaling` and runs `detect_llama3_rope_in_config` (v0.49.0 Part D helper) on it. If the existing block declares `llama3` (either via the legacy `type` key OR the newer `rope_type` alias), the auto-detect picks `"llama3"` + the upstream `LLAMA3_DEFAULT_*` constants; otherwise falls back to `"dynamic"`. Explicit caller pick still wins (any non-None value). Back-compat preserved by keeping the legacy default kwarg `rope_scaling_type="dynamic"`. The detect helper rejects non-Mapping config input via `TypeError` (no SSRF / file-read risk — the function is pure-Python data inspection). (#83 LLaMA Pro live block expansion) `soup_cli/utils/block_expansion.expand_model_blocks` lifts the v0.41.0 Part C `NotImplementedError` stub with a real implementation: clones the last `min(num_new_blocks, original_count)` decoder blocks via `copy.deepcopy` (full independent storage — no shared buffers), zero-inits each clone's residual projections (`mlp.down_proj.weight + bias` and `self_attn.o_proj.weight + bias`) so the appended block initially acts as identity per the LLaMA Pro paper §3.1, appends to `model.model.layers`, and updates `model.config.num_hidden_layers`. Validates `num_new_blocks` via `validate_expand_layers` (bool-guard + `[1, 64]`) BEFORE any model mutation. `_get_layers_module` uses explicit `is None` check (not falsy shortcut) to defend against `nn.Module.__bool__` overrides on subclasses (code-review HIGH fix). `_zero_init_block_residual` returns `bool` and the caller emits `warnings.warn` when neither standard projection path matches the cloned block (non-Llama-shaped arch — security-review LOW fix surfaces silent-degradation to operators training on Falcon-style models). Over-expansion silently clamps to `min(n, original_count)` rather than raising — matches the project's defensive-fallback policy for advisory operations. New `apply_llama_pro_freeze(model, num_new_blocks) -> int` is the canonical "train only new blocks" companion (global `requires_grad=False` pass, then unfreeze the tail N blocks; returns trainable parameter count). New shared helper `apply_block_expansion_if_configured(model, tcfg, console)` centralises the "if `expand_layers` is set, expand + optionally freeze + print" sequence — used identically by SFT and Pretrain trainers (matches v0.40.6 `peft_wiring` centralisation policy; defends against drift between trainer call sites which would otherwise produce subtle inconsistent behaviour). (#74 HF push surface QA) Manual QA of `soup push`, `soup train --push-as`, `soup data push`, `soup deploy hf-space` deferred to a contributor with private HF credentials — entry recorded in `tests/qa/v053_qa.md` with the full test plan + acceptance criteria. The HF push security surface (repo_id validation, token resolution, commit message sanitization, model card injection defence, Space template containment) is unchanged from v0.29.0 / v0.40.2 and remains covered by `test_hf_integration.py` + `test_v0402_part_a.py`. Test surface: 1 new test file (`tests/test_v0534.py`) carrying 49 new tests + 7 net updates to v0.49.0 / v0.41.0 / v0.10.x regression tests. Known limitations: (1) LongLoRA S² forward override still deferred to v0.49.1 — schema gate hardened, live monkeypatch is the next deliverable. (2) Mixtral excluded from LongLoRA allowlist (MoE attention forward signature differs). (3) Block-expansion zero-init covers Llama-shaped blocks only — non-standard arches still get appended + trainable, but lose the LLaMA Pro identity-init guarantee (and emit a runtime warning). (4) Llama 3.1 RoPE auto-detect only fires when caller passes `rope_scaling_type=None` (explicit pick wins). (5) #74 live QA against a private HF repo is the v0.53.5+ follow-up. (v0.53.4)
- **v0.53.3 — GRPO Plus partial wiring (#128 grpo_fp16, #129 vision-VLM probe)**: lifts two surgical v0.50.0 GRPO Plus deferred stubs while keeping the project's hardening invariants; the four larger items (#127 stability callback, #123 6 GRPO variant loss kernels, #126 PRMTrainerWrapper, #68 multi-objective preference live combine) are scope-deferred to v0.53.4. (#128 grpo_fp16 routing) New `_validate_grpo_fp16_amp_exclusive` SoupConfig cross-validator rejects the silent-mutex combo `grpo_fp16=True + auto_mixed_precision=True` at config load — both flags pick the mixed-precision dtype via different codepaths; combining them is a footgun where downstream behaviour depends on validator execution order. Cross-validator short-circuits when `task != 'grpo'` so the v0.50.0 stability task-gate diagnosis fires first (keeps the most actionable error at the front; code-review HIGH fix). New `GRPOTrainerWrapper._build_precision_kwargs(self) -> dict[str, bool]` returns the `{fp16, bf16}` HF kwargs per `(device, grpo_fp16)` matrix: non-CUDA (CPU / MPS / XPU) → both False (HF Trainer's fp16/bf16 kwargs are CUDA-specific, MPS / XPU use their own mixed-precision paths), CUDA + `grpo_fp16=True` → `fp16=True, bf16=False` (unsloth parity), default CUDA → `fp16=False, bf16=True` (legacy v0.50.0 path). Direct attribute access on `self.config.training.grpo_fp16` (no `getattr` fallback — Pydantic-guaranteed field). (#129 vision-GRPO base probe) New `soup_cli/utils/prm.KNOWN_VLM_REGEX` compiled regex with 10 word-boundary alternatives covering Qwen2-VL / Qwen2.5-VL / QVQ / Pixtral / InternVL / InternVL2_5 / InternVL3 / Llama-3.2-Vision (any size via `[a-z0-9._-]*vision` glob) / LLaVA / MiniCPM-V / Idefics / ShareGPT4V / Fuyu. Word-boundary idiom `(?:^|[^a-z0-9])…(?:[^a-z0-9]|$)` mirrors v0.39.0 `is_gemma4_model` / v0.44.0 `is_llama4_model` / v0.49.0 `is_llama_model` policy — rejects substring noise like `"my-pixtralish"`. New `is_known_vlm_base(name: object) -> bool` is defensive — returns False (never raises) on non-string / bool / empty / null-byte / `>_MAX_BASE_NAME_LEN=512`. Extended `validate_vision_grpo_compat` with optional `base: str | None = None` kwarg — `None` / empty-string skips the probe (back-compat for legacy v0.50.0 Part E callers); non-empty-non-VLM raises `ValueError` with friendly message naming the expected families (Qwen2-VL / Pixtral / InternVL / Llama-3.2-Vision / LLaVA / MiniCPM-V). Error message **truncates the echoed `base` to 64 chars** before serialisation (security-review MEDIUM fix mirroring v0.34.0 `crash.py` `output_dir` basename policy — defends against adversarial / long bases bloating error logs and from leaking unredacted user input into operator-facing tracebacks). `_validate_vision_grpo` in SoupConfig threads `base=self.base` so a YAML pairing `vision_grpo: true` with a non-VLM checkpoint is rejected at schema-load instead of surfacing as a cryptic `"module has no attribute 'vision_tower'"` runtime error. Test surface: 1 new test file (`test_v0533.py`) carrying 37 new tests covering: every `_build_precision_kwargs` matrix cell (CUDA + grpo_fp16 / default CUDA / CPU / MPS), every cross-validator branch (mutex rejection / task-gate priority / both-off pass), every regex alternative (Qwen2-VL / Pixtral / QVQ / Llama-3.2-Vision variants / negative matches), every defensive guard (bool / non-string / null-byte / 512-byte boundary), error-message truncation (security-review M regression), and end-to-end YAML load (happy + reject). Known limitations: (1) Scope-deferred — 4 larger v0.53.3 items moved to v0.53.4 because each requires deep TRL subclassing and warrants its own focused release; the v0.40.x stub-then-live cadence shipped 5 patch releases over 6 weeks, mirroring that here. (2) VLM allowlist is static name-regex only; a legitimate VLM published under an org whose checkpoint name lacks any of those tokens (e.g. a custom internal fork) is rejected at schema-load and operators must omit `vision_grpo: true` until a future release adds a runtime `model.config.vision_config` probe. (3) `_build_precision_kwargs` is GRPO-only — other RL trainers (PPO / RewardModel) follow their existing mixed-precision conventions. (v0.53.3)

- **v0.60.0 — Supply Chain Security**: 6 parts ship `soup adapters scan / sign / verify / check-safetensors`, namespace-pin TOFU SQLite, license-conflict matrix at merge, and `soup airgap-bundle`. **Merkle-root manifest is recursive** (review-fix HIGH) — `_enumerate_files` walks subdirs so nested `tokenizer/` / `processor/` files are signed; symlinked subdirs and files refused loudly via `os.lstat + S_ISLNK`; per-file realpath+commonpath check defends against hardlinks pointing outside the adapter dir. **`adapter_sign._load_signature` TOCTOU-hardened** — direct `os.lstat` then-open (no `isfile`-then-`open` race); explicit `S_ISLNK` and `S_ISREG` checks. **16 MiB cap on `.soup-signature.json`** prevents OOM on a crafted multi-GiB JSON. **`NamespacePinStore` containment** — public constructor refuses paths outside `$HOME / $CWD / $TMPDIR` (review-fix HIGH); SQLite path symlink-rejected before `sqlite3.connect`; POSIX `0o600` perms; `SOUP_NAMESPACE_PIN_DB` env override containment-checked. **Datetime parsing for backward-jump detection** — uses `datetime.fromisoformat` rather than lexicographic compare so non-UTC offsets decide correctly (python-review MEDIUM fix). **`allow_namespace_shift` bool rejected** (security-review CRITICAL idiom) — prevents `--allow-namespace-shift true` from becoming a free-for-all bypass. **Author override case-insensitive** so legitimate operator typing mismatch case doesn't fail the gate, but the case sensitivity is bounded to author normalization only. **`airgap_bundle` TOCTOU re-check** — `os.lstat + S_ISLNK` on parent dir AND final output path BEFORE `mkstemp` (review-fix HIGH); `os.path.realpath` (not `abspath`) for Windows 8.3 short-name safety; `os.path.lexists` (not `exists`) in tmpfile cleanup so a hostile symlink swap can't trick the unlink. **64 MiB manifest cap on inspect**; **`tarfile.data_filter`** set on the TarFile instance for any future `extractall` caller. **Deterministic dataset labeling** — sorted basename (not argv index) so reorderings produce identical manifests, fulfilling the "same inputs → same SHA-256 list" contract (review-fix HIGH). **License matrix**: closed allowlist of 33 SPDX-ish ids; conservative-by-design — when in doubt, flag; `--license-override <reason>` requires 8-char minimum to defend against `--license-override y` style bypass; reason capped at 4096 chars; null-byte rejected; TypeError on non-string (review-fix LOW for consistency with project policy). **`UNSAFE_EXTENSIONS` frozenset (8 entries)** locks the pickle-rejection allowlist behind `MappingProxyType`-style immutability; static-extension check is intentional (operators wanting protection against `evil.safetensors` containing actual pickle bytes should also enable v0.60.x magic-byte check, deferred). **`adapter_scan` thresholds** tuned against synthetic rank-1 trojans (`_RANK1_DOMINANCE_FAIL=200x`, `_ENERGY_TOP1_FAIL=0.95`, `_FROB_OUTLIER_FAIL_SIGMA=8.0`) with conservative-by-design false-positive bias; bool-as-int rejection on `ScanFinding.value` / `threshold`; non-finite values rejected. **Test count**: 9294 → 9446 (+143 net in `tests/test_v0600_part_{a-f}.py`; ~140 pass + 6 POSIX-only symlink tests skipped on Windows). **Known limitations**: (1) **Sigstore + ed25519 signing deferred to v0.60.1** — `UNSIGNED` backend live (offline Merkle-root tamper detection); the live signers raise `NotImplementedError` with explicit v0.60.1 marker (stub-then-live pattern, same as v0.59.0 attest). (2) **Namespace-pin not wired into HF download path** — helpers ship live but the integration into `utils/hubs.download_repo` lands in v0.60.1; operators can call `verify_namespace` manually via Python API today. (3) **License gate is opt-in** — `--license <id>` flag required to invoke the gate; auto-extraction from `adapter_config.json` / HF model-card metadata lands in v0.60.1. (4) **Airgap bundle does NOT yet auto-attach v0.59 repro-receipt** — train→bundle handoff wiring is v0.60.1. (5) **Strict-safetensors is extension-based only** — a pickle stream renamed to `.safetensors` bypasses the static scan; the actual safetensors loader will surface that loudly at load time, but magic-byte check is a v0.60.1 follow-up (`PK\x03\x04` / `\x80` pickle opcode detection on first 16 bytes). (6) **License override is NOT yet auto-emitted to audit-log** — v0.59 `audit_log.append_audit_event` integration lands in v0.60.1. (7) **NamespacePinStore SQLite is single-process** — concurrent `soup train` runs racing on the TOFU write may serialise via SQLite locking; WAL + cross-process file-lock tracked for v0.60.1. (8) **Adapter scan false-negative on multi-vector trojans** — sophisticated attacks distributing energy across many singular vectors will pass; per-arch threshold profiles tracked for v0.60.x. (9) **5-agent review wave: python-reviewer + general-purpose security + general-purpose code-review + general-purpose tdd-guide + manual smoke** (verification-loop). Direct `code-reviewer` / `security-reviewer` agent invocations hit context-window thrash on the 800+ KB CLAUDE.md release-notes history; the `general-purpose` agent with focused "do not crawl, read only these N files" prompts produced equivalent findings (matches v0.58.0 / v0.59.0 wave-3 idiom). (v0.60.0)

- **v0.59.0 — Governance & Provenance**: 6 parts ship `soup bom emit` (CycloneDX 1.6 ML-BOM + SPDX 2.3 + AI profile dual emitter), `soup attest emit` (in-toto v1 + SLSA-3 provenance v1), `soup train --annex-xi` (EU AI Act Annex XI/XII markdown auto-doc), `soup audit-log tail/rotate` (HIPAA/SOC2 JSONL with PII redaction), `soup train --repro-receipt` (SR 11-7 receipt), CodeCarbon hook schema + electricityMap SSRF validator. **TOCTOU lstat-before-write** on every atomic-write surface — `paths.atomic_write_text` shared helper centralises the v0.33.0 #22 / v0.43.0 / v0.55.0 / v0.56.0 / v0.57.0 / v0.58.0 TOCTOU defence so a future contributor cannot accidentally drop the symlink check. **Audit log POSIX `O_NOFOLLOW`** on `os.open` + `0o600` perms + symlink rejection at the rotation backup path. **PII redaction across every string field** (command / args / host_id / operator_id) via the v0.40.3 `_SECRET_RE` policy. **`SOUP_AUDIT_LOG_PATH` env containment** to `$HOME / $CWD / $TMPDIR` (mirrors v0.36.0 `SOUP_BATCH_CACHE_PATH` policy). **`validate_electricity_map_endpoint` SSRF parity** with v0.51.0 `validate_hub_endpoint` — scheme allowlist, loopback-only HTTP, RFC1918 / link-local / reserved / multicast IP rejection via `ipaddress.ip_address`, control-char + null-byte rejection, `0.0.0.0` explicitly rejected, IPv6 brackets handled by `urlsplit.hostname` (no manual `strip("[]")`). **Markdown injection defence** — `_md_escape` in Annex XI/XII renderers neutralises `|[](){}!<>` plus newline / CR / tab in every operator-controlled field (model_name / base_model / dataset_summary / top-domain names / modalities), defending against forged-heading + Markdown-link injection in downstream PDF/HTML renderers (mirrors v0.29.0 model-card v2 escape policy). **In-toto subject SHA-256 locked to 64-hex** via regex; **stage allowlist** `extract / train / eval / export / publish`; **builder_id capped at 256 chars** + null-byte rejected; **invocation_id capped at 256 chars** before JSON serialisation. **Test count**: 9193 → 9294 (+99 net in `tests/test_v0590.py`; 93 pass + 6 POSIX-only symlink tests skipped on Windows). **Known limitations**: (1) **Sigstore + ed25519 signing deferred to v0.59.1** — `UNSIGNED` backend live, `sigstore`/`ed25519` raise NotImplementedError with explicit v0.59.1 marker (stub-then-live pattern). (2) **Live CodeCarbon hook deferred to v0.59.1** — `measure_run_energy` returns None even with codecarbon installed; live `EmissionsTracker` lifecycle wiring inside trainer wrappers is the v0.59.1 deliverable. The SSRF-hardened endpoint validator ships now so v0.59.1 is purely additive. (3) **PDF rendering deferred to v0.59.1** — markdown body now; `reportlab` PDF integration later. (4) **Audit-log auto-instrumentation NOT shipped** — every command does NOT yet auto-emit one line via the Typer top-level callback; only the reader + write API ship live. (5) **Top-10 domain analysis NOT auto-populated** — `AnnexXIData.top_domains` accepts an operator-supplied tuple but the train-flag wiring currently passes empty. (6) **Annex XI / repro-receipt flags only fire on `LOCAL_RANK=0`** — single-node multi-GPU correct; multi-node `RANK`-aware guard tracked separately. (7) **Three review waves: python-reviewer + general-purpose security + general-purpose code-review + general-purpose tdd-guide.** verification-loop done manually via CPU smoke (`soup bom emit` / `soup attest emit` / `soup audit-log tail` / `soup train --help` shows `--annex-xi` + `--repro-receipt`; happy + failure modes all exercised). The direct `code-reviewer` / `security-reviewer` agent invocations hit context-window thrash on the full repo; the `general-purpose` agent with focused "do not crawl, read only these 9 files" prompts produced equivalent findings (matches the v0.58.0 wave-3 idiom). (v0.59.0)

- **v0.58.0 — `soup loop` data flywheel capstone**: 4 Parts ship `loop init / status / pause / resume / watch / canary / replay`. Three review waves fixed 1 CRITICAL + 7 HIGH + 9 MEDIUM + 2 LOW total before tag (python-review wave 1: BucketStats lock scope + TOCTOU + NUL-byte; code-review wave 2: watch-preserves-paused / budget-skip-no-manifest / canary-autoroll-persisted / `route()` math.ceil / `parse_budget("usd")` friendly error / `list_iterations` OSError swallow / module-top `replace`; security + tdd wave 3: `_check_dir` TOCTOU + boundary tests at `_MAX_STR_FIELD=512` and `_MAX_FILE_BYTES=1 MiB` + bool-rejection on `iteration_count` / `runs_today` / `monthly_budget_usd` / `spent_this_month_usd` + empty-string rejection on `canary_active` / `last_iteration_id` / `last_run_date`). **TOCTOU lstat-before-write** — `_check_path` and `init_state` use direct `os.lstat` (catching `FileNotFoundError` for the missing-file branch) instead of the `lexists` + `lstat` two-step that opens a race window; matches v0.33.0 #22 / v0.43.0 / v0.55.0 policy. **NUL-byte rejection on `_bucket_for_key`** — defence-in-depth on the SHA-256 input even though the request_key is internally-derived. **`BucketStats._lock` `compare=False`** — `threading.Lock` instances have no value-equality so the auto-`__eq__` would never return True; flag added per python-review MEDIUM. **`canary` command reloads after write** so the in-memory `LoopState` reflects the persisted `updated_at` (matches every other persist-then-read CLI in the project). **Atomic state-file writes via `tempfile.mkstemp + os.replace`** with POSIX `0o600` perms after rename (mirrors v0.26.0 registry.db policy). **1 MiB cap on the loop.yaml state file**; bool-as-int rejection on every numeric (matches v0.30.0 `Candidate` / v0.34.0 `estimate_run_cost_usd` policy); NUL-byte + oversize rejection on every string. **`subprocess.Popen` argv-list (no shell)** for `loop watch --detach`; `# noqa: S603` annotation documents the bandit suppression. **`CanaryPolicy` cross-field validation**: empty stable rejected, `canary == stable` rejected, traffic_pct ∈ [0, 100] with `math.isfinite` (NaN/Inf rejected), traffic without canary rejected, `sticky_on_rollback` must be `bool`. **Sticky-on-rollback policy** — a flaky verdict cannot ping-pong traffic between adapters; the operator must explicitly re-promote a canary after rollback (matches the v0.26.0 Quant-Lobotomy "no silent recovery" surface). **Test count**: 8998 → 9193 (+195 net in `tests/test_v0580.py`; 188 pass + 1 POSIX-only symlink test skipped on Windows). **Known limitations**: (1) **Stage callbacks ship as no-op stubs** — production wiring (v0.26 trace-to-pref + eval-gate + v0.30 multi-adapter deploy) is operator-driven via `WatchConfig` to keep the import graph one-directional; pre-wired versions tracked for v0.58.1. (2) **Soup Can per-iteration packaging deferred to v0.58.1** — iteration manifests under `.soup-loops/<iter-id>/iteration.json` are laid out so a v0.26 Soup Can wrapper hook can ship without re-shaping files, but the Registry-DAG append is the v0.58.1 deliverable. (3) **`--detach` is a single-process subprocess** — no `setsid` / nohup-style daemonization. Operators on Linux should pair with `systemd` or `tmux`; on Windows the subprocess survives the parent CLI exit. (4) **No automatic budget refill on UTC month rollover** — `spent_this_month_usd` is reset by the operator (or by writing a fresh `loop.yaml`); the daemon does not auto-detect month boundaries. (5) **Full 5-agent review wave completed across 3 sequential rounds.** The direct `code-reviewer` / `security-reviewer` agent invocations hit context-window thrash on the full repo (the 800+ KB release-notes history blew their context); the `general-purpose` agent with focused "do not crawl, read only these 7 files" prompts produced equivalent findings. verification-loop completed via manual CPU smoke covering init / status / pause / resume / watch --max-iterations / canary / replay end-to-end. All 188 of 189 tests pass (1 POSIX-only symlink test skipped on Windows); subprocess uses argv list, all paths cwd-contained + symlink-rejected, no top-level torch imports in any loop module. (v0.58.0)

- **v0.57.0 — `soup adapters` git-for-LoRA**: 4 Parts ship `adapters diff / merge / blame / branch / checkout / branches`. 5-agent review-fix wave landed 1 CRITICAL + 9 HIGH + 11 MEDIUM + 4 LOW fixes before tag. **TIES tied-sign defaults to +1** — first-cut `np.sign(0) == 0` would have silently zeroed every parameter whose adapters' signs balanced exactly; the fix elects positive on tie per the TIES paper. **`os.lstat + S_ISLNK` rejection added at 4 read/write boundaries**: `load_branch` (defends against `~/.soup/branches/<name>.json -> /etc/passwd` content leaking through JSON-parse error path), `delete_branch` (defends against silent deletion of victim files via planted symlinks), `merge_adapters` output `adapter_model.safetensors` + `adapter_config.json` writes, and `compute_adapter_diff` weights-file path (lstat BEFORE `is_file()` — defends against `safetensors -> /etc/passwd` escape from the directory-level containment check). **Atomic writes via `tempfile.mkstemp + os.replace` at every output path** — `_atomic_write_bytes` for adapter_config.json, sibling-tempfile + os.replace for safetensors, atomic diff `--output` write, atomic branch JSON pointer write. **`_count_dataset_rows` opens via realpath** captured at containment check (closes TOCTOU window between `enforce_under_cwd_and_no_symlink` and `open`). **`SOUP_BRANCHES_DIR` env override** rejects every C0 control char (CRLF / tab / null / 0x01-0x1f) before honouring the override (mirrors v0.51.0 hub-endpoint policy). **`SUPPORTED_STRATEGIES` migrated from `Tuple` to `frozenset`** (matches v0.41.0+ allowlist policy); `STRATEGY_ORDER` tuple preserved for canonical iteration. **Source adapter_config.json size-capped at 256 KB** before read (matches v0.53.0 `load_quant_config` policy). **`_MAX_ADAPTERS=16` per merge** + **`_MAX_LAYERS=10_000` per adapter** (DoS caps). **All operator-supplied paths cwd-containment-checked** via the shared `enforce_under_cwd_and_no_symlink` helper. **All Rich-rendered user-controlled fields pass through `rich.markup.escape`** in the new commands (legacy `adapters list/info/compare` Rich-escape backfill tracked for v0.57.1). **TypeError-then-FrozenInstanceError invariants on 4 frozen dataclasses** (`LayerDiff` / `AdapterDiffReport` / `MergeReport` / `BlamePlan` / `BlameShardWork` / `Branch`) — `pytest.raises(Exception)` tightened to `pytest.raises(FrozenInstanceError)` in 5 places (TDD-review HIGH). **Test count**: 8849 → 8998 (+149 net across 4 new test files; 4 POSIX-only symlink tests skipped on Windows). Known limitations: (1) **Live blame ablation runner deferred to v0.57.1** — `run_blame` raises `NotImplementedError` with explicit v0.57.1 marker; `soup adapters blame` emits the plan + budget check and exits clean. Same stub-then-live cadence as v0.27.0 MII / v0.37.0 multipack / v0.50.0 GRPO Plus / v0.56.0 diagnose. (2) **Merge canary verdict** — `MergeReport.verdict` is `'UNKNOWN'` stub; live canary-eval via v0.55 eval gate ships in v0.57.1. (3) **Branch pointers are local-only** — not yet wired into v0.26 Registry lineage DAG; cross-machine sharing requires copying the JSON pointer manually. (4) **`.bin` adapter format rejected** with friendly "re-save as safetensors" message (design choice — `safetensors` package is a hard dep and v0.4.0+ PyTorch tooling defaults to it). (5) **Legacy `soup adapters list/info/compare` (v0.22.0) still embeds adapter_config values directly into Rich markup** — pre-existing surface, not introduced by v0.57.0; backfill tracked for v0.57.1. (6) **`parse_budget` duplicated from `utils/data_mix`** — same `60s/5m/2h` syntax + `[60s, 24h]` bounds. Extraction to a shared helper is a code-review MEDIUM follow-up but the bounds may diverge between blame (long-running) and data_mix (per-candidate proxy) so deferred. (7) **TIES sign-tie default is +1** — paper convention; configurable tie-break (e.g. abstain) is out of scope. (v0.57.0)

- **v0.53.2 — Modality II live trainers**: lifts four v0.52.0 deferred stubs (#137, #135, #133, #132) into real trainer wrappers while keeping the project's hardening invariants. (#137 reasoning_effort + train_on_eot) `apply_reasoning_effort_prefix` follows v0.41.0 / v0.51.0 validator policy (bool-first, null-byte / empty / oversize / case-insensitive normalisation); messages list is treated as immutable (returns a new list — matches v0.33.0 #47 `CrossDocCollator` policy). `build_assistant_only_labels(train_on_eot=True)` reuses the existing v0.36.0 mask infrastructure — same null-byte / max_length / bool guards. (#135 EBFT / GDPO) `apply_ebft_loss` and `apply_gdpo_loss` enforce **finite-only inputs** (`torch.isfinite` guard on tensor inputs + `math.isfinite` on scalar params) — NaN / Inf would silently corrupt training otherwise. `dpo_margin` defaults to `None` (not `0.0`) per security-review M3 fix: silent zeroing in the `margin` variant when the operator forgot to set the margin would have looked like training success but produced a meaningless gradient. Both attach hooks (`attach_ebft_compute_loss`, `attach_gdpo_compute_loss`) are **idempotent** via a marker attribute on the wrapped method — re-attach is a no-op and a dedicated test class verifies the invariant (code-review M2 fix). (#133 DistillTrainerWrapper) **Separate trust_remote_code resolution for student and teacher** (security-review L2 fix): `model_requires_trust_remote_code(teacher)` runs independently of the student probe, otherwise a malicious teacher could piggy-back on the student's opt-in. Teacher is loaded with `device_map="cpu" if device == "cpu" else "auto"`, frozen via `requires_grad_(False)` + `.eval()` immediately after load — never participates in gradient computation. `_DistillTrainer.compute_loss` device-bridge: `teacher_device = next(teacher_ref.parameters()).device`, `teacher_inputs.to(teacher_device)` before teacher forward, `teacher_logits.to(student_logits.device)` before KL kernel — defends against HF Trainer's auto-CUDA promotion silently producing cross-device `index_select` crashes. **DataCollator correctness fix** (surfaced during Wave 3 CPU smoke): `DataCollatorForLanguageModeling` does NOT pad pre-tokenised `labels` — switched to `DataCollatorForSeq2Seq(label_pad_token_id=-100, padding=True)` so variable-length loss-masked rows batch correctly without runtime crash. (#132 ClassifierTrainerWrapper) `_normalise_label` caps multi-label entries at **1024 per row** (matches v0.52.0 schema cap; security-review HIGH fix — unbounded would allow OOM via crafted JSONL), dedups via set conversion, validates `label_names` map entries reject null bytes + empty strings. `problem_type` is set explicitly from `tcfg.classifier_kind` (not silently inferred from labels) so a multi-label-shaped row in a single-label config raises rather than mis-trains. Training Setup Panel renders `Head: num_labels=N, kind=...` for classifier-family tasks instead of meaningless LoRA r/alpha lines (code-review L3 cosmetic fix — Panel no longer mis-represents what the wrapper is doing). (Cross-cutting) `commands/train.py` task routing branches added for `distill` and `classifier` / `reranker` / `cross_encoder` — source-grep regression guards in the test suite use the **full instantiation expression** `DistillTrainerWrapper(cfg, **trainer_kwargs)` so comment-only mentions of the class name cannot satisfy the regression check (TDD-review hardening). Both new factories (`build_distill_trainer`, `build_classifier_trainer`) reject unknown kwargs via Python signature contract — dedicated `pytest.raises(TypeError)` tests cover the path (TDD-review L1 fix). Test surface: 1 new test file (`test_v0532.py`) carrying 120 new tests across 14 classes. Known limitations: (1) `#71` TinyLlama-1.1B-LoRA full ONNX export is host-RAM-bound (≥16 GB free RAM needed for the `onnx.load(load_external_data=True)` post-process step); tiny-gpt2 smoke proves pipeline integrity — recorded in `tests/qa/v053_qa.md`. (2) Distillation supports same-tokenizer pairs only — cross-tokenizer (Llama → Qwen) needs a projection or sequence-level loss, out of scope. (3) Classifier wrapper has no LoRA path — full head + base training; LoRA classifier finetuning is a follow-up. (4) EBFT / GDPO auto-attach only fires when the corresponding `*_variant` field is set; manual `attach_*` invocation from custom training loops is supported and idempotent. (5) `reasoning_effort` injection happens at data-prep time inside `build_format_row`; changing the level between runs requires re-rendering the dataset. (v0.53.2)

- **v0.53.1 — Quant Menu II + Export pipeline live**: lifts six v0.53.0 deferred stubs to live wiring while keeping the project's hardening invariants. New shared helper `soup_cli/utils/paths.enforce_under_cwd_and_no_symlink` consolidates the v0.33.0 #22 TOCTOU pattern (cwd containment via `os.path.realpath + os.path.commonpath` + `os.lstat + S_ISLNK` rejection) — used by `commands/merge.py`, `commands/export.py`, `utils/save_formats.py`, and `utils/gguf_quant.py` so the same boundary check fires at every CLI dispatch point. `merge_4bit` and `export_torchao` (`utils/save_formats.py`): cwd containment + symlink rejection on `merged_dir` / `model_dir` / `output_dir`; `load_quant_config` enforces `yaml.safe_load` only + 256 KB cap + extension allowlist (`.yaml`/`.yml`); **per-scheme closed kwarg allowlist** rejects dunder keys + unknown params before the splat into `torchao.<scheme>Config(**kwargs)` (security-review HIGH fix — `Int4WeightOnly` accepts `{group_size, inner_k_tiles}`, `NVFP4` accepts nothing extra). Corrected BNB-4bit skip-modules kwarg name from `llm_int8_skip_modules` to `bnb_4bit_skip_modules`. `export_advanced_gguf` (`utils/gguf_quant.py`): all three subprocess invocations (`convert_hf_to_gguf.py`, `llama-imatrix`, `llama-quantize`) use argv-list form with no shell, 30-min timeout, `sys.executable` for the convert script; `_run_convert_to_f16` realpath-verifies that `convert_hf_to_gguf.py` stays inside the `llama_cpp_dir` after resolution (security-review HIGH M5 fix — defends against a symlinked script escape). `_prepare_calibration_text` strips null bytes, collapses newlines to spaces, caps per-line at 8 KB + total at 50 MB (security-review M1), uses POSIX `O_NOFOLLOW` to refuse symlinks at the kernel level (security-review M3 — closes the TOCTOU window between the dispatch-time check and the actual `open()`); requires ≥ 1 usable row before invoking imatrix. `_safe_stderr` Rich-markup-escapes subprocess stderr before embedding in `RuntimeError` (security-review L4) so a crafted llama.cpp error cannot inject `[red]...[/]` into the operator-facing panel. UD-prefix stripped from flavour arg before passing to llama-quantize (`UD-Q4_K_XL` → `Q4_K_XL`). Calibration data path containment + symlink rejection fires at CLI dispatch in `commands/export.py::_export_gguf_advanced`. `detect_prequantized_format_from_path` (`autopilot/decisions.py`): cwd containment + `os.lstat + S_ISLNK` on `<model_dir>/config.json` (security-review HIGH H2 — out-of-cwd model paths silently return `None` to preserve soft-probe semantics so HF Hub repo IDs aren't rejected); null-byte rejection on `model_dir`. `commands/merge.py`: early `is_under_cwd(output)` check at CLI boundary (security-review M4) — consistent with the v0.20.0 / v0.40.2 containment-at-the-boundary policy. `deploy_measure.py`: cache file written atomically via `tempfile.mkstemp` + `os.replace` with `os.lstat + S_ISLNK` rejection on BOTH `load_cache` and `save_cache` (security-review M2 — was missing on the load side); env override `SOUP_DEPLOY_AUTOPILOT_CACHE` rejects null bytes + control chars before any path resolution and confines the override to home / cwd / tempdir; cache file gets best-effort 0o600 perms on POSIX (matches v0.26.0 registry.db policy); 1 MB cache-file cap. `_DEPLOY_MEASURE_BEFORE_GEN` / `_AFTER_FACTORY` module-level callables are documented as a non-public escape hatch (deferred until v0.46.1 live model-loader). Test surface: 4 new test files (`test_v0531_82.py` / `test_v0531_109.py` / `test_v0531_139.py` / `test_v0531_142.py`) carrying 112 new tests covering happy paths + failure modes + every security guard (POSIX symlink rejection, per-scheme kwarg allowlist, TOCTOU defences, `_MAX_CANDIDATES` cap, MINOR-verdict band, mxfp4 word boundary, BNB-alias detection, render-table markup escape). Known limitations: (1) `_DEPLOY_MEASURE_BEFORE_GEN` / `_AFTER_FACTORY` are a stop-gap until v0.46.1 ships first-party transformers / vLLM generator factories. (2) `#70` GGUF and `#72` AWQ/GPTQ manual QA smokes remain pending — require CUDA + llama.cpp build; recipes scripted in `tests/qa/v053_qa.md`. (3) BNB-4bit merge + TorchAO PTQ live happy-path is mock-covered only — CPU-only CI cannot execute the real BNB / torchao kernels. (4) `_prepare_calibration_text` accepts JSONL with `text` / `prompt` / `content` aliases + raw text fallback; other formats (parquet / markdown) are out of scope. (5) Cache key truncates `base_sha` to 16 hex chars at the call site (collision probability ≈ 1-in-2³² across ~4 billion entries). (6) Pre-quantized detection is heuristic — name regex + local `config.json` probe; HF Hub repo IDs without local download fall back to name-only matching. (7) `enforce_under_cwd_and_no_symlink` checks only the leaf path; deeper traversal relies on the per-file leaf check at each site. (v0.53.1)
- **v0.53.0 — Quant Menu II (UD GGUFs + KV cache + NVFP4 + LF parity + save formats)**: 6 schema-only Parts; live wiring deferred to v0.53.1. Every new validator follows the project's established hardening policy: closed allowlists (`UD_GGUF_FORMATS`, `IQ_GGUF_FORMATS`, `APPLE_ARM_GGUF_FORMATS`, `KV_CACHE_TYPES`, `MERGE_SAVE_FORMATS`, `TORCHAO_PTQ_SCHEMES`) as `frozenset` so registries cannot be mutated; `_GGUF_METADATA` / `_KV_CACHE_METADATA` / `_MERGE_METADATA` / `_TORCHAO_METADATA` wrapped in `MappingProxyType`; `_LOWER_INDEX` for GGUF lookup is also `MappingProxyType`-wrapped (replaces O(N) walk with O(1) lookup — code-review MEDIUM fix). All string validators reject non-string / bool / empty / null-byte / oversize with case-insensitive normalisation (matches v0.41.0 `validate_optimizer_name` / v0.51.0 `validate_hub_name` policy); `validate_torchao_scheme` is INTENTIONALLY case-sensitive (PyTorch class names — `torchao.quantize_` looks them up by exact name) with the asymmetry documented at both validators (security-review LOW fix). `validate_calibration_data_path` + `validate_quant_config_path` are shape-only at this release; their docstrings name the exact controls a v0.53.1 CLI dispatch contributor MUST add (`os.path.realpath` + `os.path.commonpath` cwd containment, `os.lstat` + `stat.S_ISLNK` symlink rejection before `open()`, existence check, `yaml.safe_load`-only for quant configs) — closes the security-review MEDIUM "documentation gap at trust boundary" finding. SoupConfig cross-validators: `_validate_fp8_attention_compat` (requires `quantization_aware='fp8'` BEFORE the MLX gate so the more actionable error fires first — code-review MEDIUM fix); `_validate_nvfp4_compat` (non-MLX + `modality='text'`; Blackwell SM ≥ 12.0 runtime check fires at trainer construction); `_validate_unsloth_bnb_4bit_compat` (requires `backend='unsloth'` + `quantization='4bit'`); `_validate_bnb_4bit_double_quant` (requires `quantization='4bit'` — rejects `none`/`8bit`/Quant-Menu); `_validate_llm_int8_alias` (asserts `quantization='8bit'`, deliberately disjoint from v0.41.0 `load_in_8bit` aliasing); `_validate_quantize_ref_reward` (extended ref-task allowlist `{dpo, ipo, simpo, orpo, bco, kto, preference, grpo, ppo}` per code-review HIGH fix — first-cut omitted grpo + kto + ppo which all have reference policies); `_validate_kv_cache_type_supported` (only `fp8` gated to non-MLX in v0.53.0; q8_0/bf16/f16 pass-through documented at validator site so v0.53.1 contributor sees the gate immediately). `requires_hopper` reads from `_KV_CACHE_METADATA` spec — single source of truth so adding a Hopper-only type means flipping the spec field only (code-review MEDIUM fix). All 7 new bool fields share `_validate_v053_bool_fields` `field_validator(mode='before')` that rejects bool-as-int with explicit `TypeError("v0.53.0 flag must be bool")` and passes `None` through to Pydantic's `default=False` rather than silently coercing it (python-review MEDIUM fix — `fp8_attention: null` in YAML now surfaces as a "valid boolean" ValidationError instead of masquerading as `False`). Known limitations: (1) Every live wiring is deferred to v0.53.1 — `export_advanced_gguf`, `apply_kv_cache_type`, `apply_fp8_attention`, `apply_nvfp4`, `merge_4bit`, `export_torchao` all raise `NotImplementedError` with explicit `v0.53.1` markers. (2) `validate_calibration_data_path` + `validate_quant_config_path` are shape-only this release; CLI dispatch in v0.53.1 MUST add cwd-containment + TOCTOU symlink rejection. (3) `kv_cache_type` MLX permissive policy: only `fp8` is rejected, the other three pass-through; v0.53.1 may narrow further. (4) Hopper SM-capability check is runtime-only — schema accepts `kv_cache_type='fp8'` + `fp8_attention=true` without GPU probe. (5) NVFP4 + Blackwell (SM ≥ 12.0) check is runtime-only. (6) `bnb_4bit_use_double_quant` only gated against `quantization`, not against `quantization_aware` — the latter combination is already rejected by v0.28.0 Quant-Menu + QAT cross-validator. (7) `llm_int8` is an assertion not an aliaser — diverges from v0.41.0 `load_in_8bit` design on purpose. (v0.53.0)
- **v0.52.0 — Modality II (TTS + Distillation + BitNet + EBFT-GDPO + MoE quant + reasoning_effort)**: 7 schema-only Parts; live trainer / loss / export wiring deferred to v0.52.1. Every new validator follows the project's established hardening policy: closed allowlist (`SUPPORTED_TTS_FAMILIES`, `CLASSIFIER_TASKS`, `DIVERGENCES`, `BITNET_QUANT_FORMATS`, `BITNET_EXPORT_FORMATS`, `EBFT_VARIANTS`, `GDPO_VARIANTS`, `MOE_EXPERT_QUANT_FORMATS`, `REASONING_EFFORT_LEVELS`, per-family `_FAMILY_EMOTIONS`) wrapped in `frozenset` / `MappingProxyType` so registries cannot be mutated at runtime; `validate_*` helpers reject non-string / bool / empty / null-byte / oversize / unknown inputs with case-insensitive normalisation (matches v0.41.0 `validate_optimizer_name` / v0.50.0 `grpo_variant` / v0.51.0 `hub` policy); float validators (`validate_distill_temperature`, `validate_ebft_temperature`) gate on `math.isfinite` to reject NaN AND `±inf` (matches v0.32.0 `save_lr_finder_report` policy). `field_validator(mode="before")` on `num_labels` (security-review HIGH fix) rejects `bool` before Pydantic's `ge=1` coercion silently treats `True` as `1`. Field validator on `reasoning_effort` routes through the shared `validate_reasoning_effort` helper so the schema and runtime validator agree on what's accepted (security-review MEDIUM fix). SoupConfig cross-validators: `_validate_tts_compat` (requires `task='tts'` + `modality='audio_out'` + non-MLX backend; per-family emotion allowlist via `_FAMILY_EMOTIONS`), `_validate_classifier_compat` (with lazy-import early-return — code-review HIGH fix — so SFT hot path doesn't pay import cost; requires `num_labels` on classifier tasks; rejects classifier-only fields outside the task family with named offenders), `_validate_distill_compat` (requires `teacher_model` when `task='distill'`; rejects distill-only fields outside the task), `_validate_bitnet_compat` (gates to non-MLX + text-modality + task ∈ {sft, pretrain, dpo}), `_validate_ebft_compat` + `_validate_gdpo_compat` (task-family gates), `_validate_moe_expert_quant_compat` (requires `moe_lora=true` to prevent silent no-op), `_validate_reasoning_effort_task_gate` (code-review HIGH fix — rejects `reasoning_effort` + `train_on_eot` outside the SFT-family task set with named offenders; mirrors v0.50.0 GRPO stability task-gate policy). Public `DIVERGENCES` frozenset is derived from `_DIVERGENCE_ALIASES` so adding a new alias updates both the accepted-input set and the error message in lockstep (review fix LOW). `validate_bitnet_export` enforces a closed-allowlist canonical form for `soup export --format <bitnet|tq1_0>`, both of which are CLI-registered with a yellow advisory panel + `Exit(0)` stub (no artifact written until v0.52.1 — the format flag is accepted so existing scripts pinned to v0.52.0 will not break). 6 new YAML recipes appended (5 TTS + Falcon-E BitNet) — every entry is exercised by `tests/test_v0520.py` for `load_config_from_string` round-trip + `_no_null_or_whitespace` model-id check (mirrors v0.51.0 review-fix LOW). Known limitations: (1) Every live trainer / loss / export path is deferred to v0.52.1 — `build_tts_trainer`, `build_classifier_trainer`, `build_distill_trainer`, `build_bitnet_trainer`, `export_bitnet_gguf`, `apply_ebft_loss`, `apply_gdpo_loss`, `apply_moe_expert_quant` all raise `NotImplementedError` with explicit `v0.52.1` markers; schema accepts every new task / quant / variant + the CLI stub for `soup export --format bitnet/tq1_0` prints a deferred-advisory panel and exits 0. (2) `modality='audio_out'` accepted on non-TTS tasks — design choice this release so future audio-output tasks (ASR / V2A) can reuse it; today's runtime trainer dispatch must check `task == 'tts'` to avoid silent routing into the deferred TTS path. (3) Oute emotion allowlist is a tight 6-entry subset (neutral / happy / sad / angry / calm / excited); operators wanting custom emotions will need a v0.52.1 patch to extend `OUTE_EMOTIONS`. (4) `is_bitnet_model` is best-effort heuristic over name prefixes (`bitnet`, `falcon-e`, `1bitllm`, `onebit`); a BitNet checkpoint published under an org without any of those prefixes returns False. This is detection, not gating — the trainer wrapper (v0.52.1) loads the model regardless of the heuristic. (5) `quantization='bitnet_1.58'` gated to task ∈ {sft, pretrain, dpo} — extending to GRPO / PPO / RewardModel requires upstream onebitllms RL kernels not yet shipped. (v0.52.0)
- **v0.51.0 — Model Catalog Expansion + Alternative Model Hubs**: 5 release Parts. New `soup_cli/utils/hubs.py` ships closed allowlist `SUPPORTED_HUBS = frozenset({hf, modelscope, modelers})` + three `MappingProxyType`-wrapped registries (`_HUB_DEFAULT_ENDPOINTS` / `_HUB_ENDPOINT_ENV` / `_HUB_PACKAGE`) so the registry cannot be mutated at runtime (matches v0.36.0 `_REGISTRY` policy). `validate_hub_name` rejects non-string / bool / empty / null-byte / >32-char / unknown with case-insensitive normalisation (matches v0.41.0 `validate_optimizer_name` policy). `validate_hub_endpoint` is the SSRF kernel — full parity with v0.29.0 `utils/hf.resolve_endpoint`: scheme allowlist (`http`/`https` only), null-byte rejection, **control-character / CRLF rejection** added in v0.51.0 as a defence-in-depth review fix (defends against URL-as-HTTP-header injection if the URL ever flows into a raw HTTP client), `0.0.0.0` explicitly rejected, plain HTTP only for loopback `{localhost, 127.0.0.1, ::1}`, RFC1918 / link-local / cloud-metadata IPs (169.254.x) rejected via `ipaddress.ip_address` for plain HTTP. `resolve_endpoint(hub, *, env=None)` looks up the per-hub env var (`HF_ENDPOINT` / `MODELSCOPE_ENDPOINT` / `MODELERS_ENDPOINT`) and runs the override through `validate_hub_endpoint`; default endpoints are baked-in HTTPS URLs. `is_hf` rejects `bool` explicitly (review fix HIGH — bool is a subclass of int and would have silently fallen through `hub.lower() == "hf"` → `False`, which happens to be correct by accident but violates the contract; matches v0.30.0 `Candidate` / v0.34.0 `estimate_run_cost_usd` policy). `TrainingConfig.hub: Literal["hf","modelscope","modelers"]` field gets a `field_validator(mode="before")` `_normalize_hub` that delegates to `validate_hub_name` so `hub: HF` in YAML normalises to `"hf"` (review fix HIGH — first-cut had Pydantic Literal exact-match while `validate_hub_name` was case-insensitive, breaking the v0.41.0 `validate_optimizer_name` / v0.50.0 `grpo_variant` / `rollout_backend` policy of agreement between schema and shared validator). SoupConfig `_validate_hub_supported` cross-validator rejects `hub != 'hf'` on `backend == 'mlx'` with a distinct error message (review fix HIGH — `mlx-lm` only downloads from HF Hub; without this gate a `backend: mlx` + `hub: modelscope` config would silently pass schema load and fail at runtime with a confusing `mlx-lm` error). 26 new YAML recipes appended to `soup_cli/recipes/catalog.py` — every entry is exercised by `tests/test_v0510.py` via `load_config_from_string` round-trip + `yaml.safe_load` (no Python tags / no template injection / no credential leak in the YAML strings) + a `_no_null_or_whitespace` model-id check that rejects empty path components (review fix LOW — first-cut allowed `"/name"` leading-slash IDs to pass). Two non-`<N>B` `size` strings (`"image"` / `"ocr"` / `"moe"` / `"medium"`) were normalised to `"N/A"` (review fix MEDIUM — `search_recipes(size=…)` would silently miss those entries, and the autopilot VRAM estimator could not parse them). Known limitations: (1) Live downloader / uploader / push integration deferred to v0.51.1 — `TrainingConfig.hub` schema lock-in ships now (Literal accept + MLX cross-validator + case-normalisation), but `soup data download --hub modelscope` and `soup push --hub modelers` still route through the existing HF Hub code path; the actual `modelscope-sdk` / `openmind-hub` adapters are the v0.51.1 deliverable. Same stub-then-live pattern as v0.27.0 MII / v0.37.0 multipack / v0.50.0 GRPO Plus. (2) Speculative / aspirational `base` model IDs in some Part A/C recipes — the catalog ships entries for `openai/gpt-oss-{20,120}b`, `THUDM/glm-5`, `Qwen/Qwen-Image`, `deepseek-ai/DeepSeek-OCR`, `PaddlePaddle/PaddleOCR-VL`, `google/embeddinggemma-300m` so users have ready-made recipes the moment those repos go live (matches the plan's "match Unsloth's day-zero coverage" directive). Recipes for not-yet-published repos will surface a clear HF Hub 404 when the user runs `soup train --recipe <name>`. (3) DNS-resolved private hostnames not blocked — `validate_hub_endpoint` only rejects literal RFC1918 / link-local IP addresses; a hostname like `corp-proxy.internal` that DNS-resolves to a private IP is accepted at validation time (mirrors the v0.29.0 `HF_ENDPOINT` policy — DNS resolution is intentionally not performed in this local-tool threat model). (v0.51.0)
- **v0.50.0 — GRPO Plus (RL parity)**: 22 features across 5 Parts shipped as schema-only (closed allowlists + Pydantic validators + NotImplementedError stubs for live wiring deferred to v0.50.1). All new validators follow the project's bool-rejection-before-int policy (matches v0.30.0 `Candidate`); closed-allowlist `validate_grpo_variant` / `validate_rollout_backend` reject non-string / bool / empty / null-byte / oversize / unknown inputs with actionable error messages and case-insensitive normalisation. `validate_grpo_delta` is bool-first / `math.isfinite` / `(0, 1]` bounded (matches v0.32.0 `save_lr_finder_report` / v0.41.0 Part B `lr_groups` policy). New `_VARIANT_METADATA` (Part A) and `_BACKEND_METADATA` (Part C) are `MappingProxyType`-wrapped frozen-dataclass registries (matches v0.36.0 `_REGISTRY` / v0.41.0 `_OPTIMIZER_PACKAGES` policy). Security-review fixes: (1) `grpo_delta` schema gets an explicit `field_validator(mode='after')` calling `math.isfinite` — Pydantic's `gt=0, le=1` bounds only incidentally reject NaN (since `NaN > 0` is False); the explicit validator prevents a future Pydantic change from regressing the guard. (2) `validate_long_context_grpo_compat` adds null-byte rejection on `task` AND `backend` strings + a `bool` guard on `use_ring_attention` (parity with `validate_grpo_variant` / `validate_rollout_backend`). (3) `validate_vllm_sleep_mode_compat` adds null-byte rejection on `backend`. Code-review HIGH fixes: (4) `_validate_grpo_stability_task_gate` now includes `grpo_fp16` in the GRPO-only-fields list — previously a user could silently set `grpo_fp16: true` on `task='sft'` and have it no-op. (5) `_validate_vllm_sleep_mode` now requires `task='grpo'` (sleep mode is a between-rollouts feature, meaningless on SFT) and rejects with a `task='grpo'` message. TDD-review HIGH fixes: (6) new `_reject_bool_on_grpo_numerics` field_validator on every Part D numeric field + `grpo_delta` explicitly rejects `bool` before Pydantic's `True→1` coercion (matches v0.30.0 / v0.41.0 Part B / v0.43.0 Part B policy). Known limitations: (1) Every live loss kernel / launcher (`apply_variant_loss`, `apply_vllm_sleep_mode`, `launch_rollout`, `build_prm_trainer`) raises `NotImplementedError` with explicit `v0.50.1` markers — same stub-then-live pattern as v0.27.0 MII / v0.37.0 multipack / v0.41.0 LLaMA Pro / v0.45.0 plugins / v0.48.0 curriculum / v0.49.0 LongLoRA. (2) `long_context_grpo` requires Tiled MLP (v0.56.0 Part A) to actually run; the schema gate ships now so v0.50.0 configs are stable. (3) `vision_grpo=true` does not check whether the base model is actually a VLM — upstream trainer surfaces that error loudly. (4) The 7 stability knobs schema-validate but none are wired into a live callback in this release; `replay_buffer_size`, `defer_rerolling`, and `skip_zero_advantage` are pure schema lock-ins. (v0.50.0)

- **v0.49.0 — Long Context & Architecture**: 4 release Parts ship YaRN RoPE scaling, Dynamic NTK hardening, LongLoRA S² shifted-sparse attention (schema-only gate), and full Llama 3.1 NTK-aware scaling. Security review hardened the public boundary of `soup_cli/utils/long_context.py::get_rope_scaling_config` — `target_length` / `original_length` / `yarn_factor` now reject `bool` / NaN / Inf / non-positive at entry so a direct caller bypassing Pydantic cannot emit `{factor: NaN}` into HF model configs (matches v0.30.0 `Candidate` / v0.34.0 `estimate_run_cost_usd` / v0.41.0 Part B `lr_groups` policy). `scale_inv_freq_llama3` rejects `bool` on every numeric parameter (`inv_freq` / `scale_factor` / `low_freq_factor` / `high_freq_factor` / `old_context_len`) — first-cut only guarded `inv_freq`. `yarn_get_mscale` now raises on non-finite input (review fix LOW — first-cut silently clamped NaN/Inf to identity, hiding the misconfig from callers). `detect_llama3_rope_in_config` uses explicit `is None` instead of the `or` idiom when probing `rope.get("type")` — falsy-but-set values no longer silently fall through to `rope_type` (matches v0.40.6 review-fix policy). New `soup_cli/utils/longlora.py` ships `is_llama_model` with a word-boundary regex `(?:^|[^a-z0-9])(?:code)?-?llama(?:-?\d+(?:\.\d+)?)?(?:[^a-z0-9]|$)` — substring `"llama"` inside an unrelated identifier (e.g. `my-llama-style-finetune`) does NOT match; null-byte rejection on `model_name` (matches v0.39.0 `is_gemma4_model` / v0.44.0 `is_llama4_model` policy); 512-char cap returns `False` rather than raising (bounded scan time). `_LLAMA_REGEX` has no nested quantifiers / overlapping alternation and is ReDoS-bounded by the 512-char cap. `validate_longlora_compat` emits distinct error messages per failure mode (mlx vs other non-transformers backends — matches v0.34.0 review-fix policy on distinct actionable rejections). `TrainingConfig` `field_validator(mode='before')` rejects `bool` on the four yarn fields before Pydantic's `gt`/`le` coercion silently treats `True` as `1.0`. SoupConfig `_validate_longlora_compat` invokes `validate_longlora_compat` at config load so a misconfigured `soup.yaml` fails fast with an actionable message rather than silently no-opping at trainer construction time (mirrors v0.39.0 ReLoRA / v0.48.0 curriculum_dynamic schema-gate policy). Known limitations: (1) LongLoRA live `LlamaAttention.forward` override deferred to v0.49.1 — `apply_longlora_forward_override` raises `NotImplementedError` with the v0.49.1 marker; the schema gate ships now so misconfigured runs cannot reach trainer construction. (2) LongLoRA architecture allowlist is Llama 1/2/3.x + CodeLlama only; Mistral / Qwen / Phi expansion tracked for v0.49.1+. (3) Llama 3.1 NTK auto-detect helper `detect_llama3_rope_in_config` ships but is not yet wired into `apply_long_context_config`; trainer wiring can pick it up when needed.
- **v0.48.0 — Adaptive Training (BETA)**: 2 release Parts ship a dynamic curriculum re-weighter and a Bayesian data-mixing optimiser. New `soup_cli/utils/curriculum_dynamic.py` ships frozen `DynamicCurriculumPolicy` with bounded fields (`num_buckets ∈ [1, 20]`, `recompute_every_n_steps ∈ [1, 100_000]`, `floor ∈ (0, 1/num_buckets]`, `temperature > 0`); bool-rejected on every numeric input (matches v0.30.0 `Candidate` / v0.34.0 `estimate_run_cost_usd` policy); `math.isfinite` on every float (matches v0.32.0 / v0.47.0 policy). `compute_bucket_weights` water-fill design (review fix HIGH — first-cut had a trailing `w/sum(w)` renorm that could push elements sitting exactly at the floor below the floor when accumulated float error left the sum slightly > 1.0; renorm removed because softmax already sums to 1.0 so water-fill output also sums to 1.0). `validate_distributed_curriculum` cross-validator rejects `enabled=True` with `world_size > 1` unless the caller attests an `all_reduce` hook is registered — DDP/grad-accum footgun: divergent per-rank stats without coordination silently desynchronise the sampler. SoupConfig cross-validators reject `curriculum_dynamic=true` on mlx backend (HF Trainer-callback specific) and on non-SFT/pretrain tasks with distinct error messages (matches v0.34.0 review-fix policy). `render_curve` and `parse_history_jsonl` enforce `_MAX_HISTORY_ROWS = 100_000` DoS cap (review fix MEDIUM — first-cut had no cap, an attacker-controlled JSONL with 10M rows would have OOM'd the process). New `soup runs curriculum-curve` CLI: `is_under_cwd` containment, `os.lstat + S_ISLNK` rejection (TOCTOU defence — mirrors v0.33.0 #22 / v0.43.0 Part C / v0.44.0 Part B / v0.45.0 Part E / v0.46.0 Part A policy), 50 MB file-size cap + 100k-line streaming cap (review fix HIGH — without these, `--history /path/to/giant.jsonl` would read the file into memory unbounded), null-byte rejection on tracker-supplied `output_dir` (defence-in-depth before `os.path.join`). New `soup_cli/utils/data_mix.py` ships frozen `MixCandidate` with simplex constraint (`sum(weights) == 1.0 ± 1e-6`), finite eval_loss (`math.isfinite`), `_MAX_LOSS = 1e6` sanity cap, bool / negative rejection on every numeric. `validate_datasets` enforces 2-32 entries (review fix MEDIUM — first-cut had a `not raw` empty-only guard that fell through to the realpath loop for single-entry inputs, surfacing a less-actionable error after path resolution), `is_under_cwd` containment, symlink rejection, dedup, null-byte / oversize / non-string rejection. `parse_budget` accepts digits + optional `s`/`m`/`h` suffix bounded to `[60s, 24h]`. `run_mix_optimizer` per-candidate proxy exceptions are logged at DEBUG + `opt.tell(_MAX_LOSS)` + `continue` (review fix MEDIUM — first-cut raised `RuntimeError` on the first proxy failure, breaking the documented `partial=True` contract; isolation policy mirrors v0.33.0 #47 `CrossDocCollator` and v0.40.3 `judge_filter_pairs`); `KeyboardInterrupt`/`SystemExit` re-raised; NaN proxy returns logged + skipped. `render_mix_recipe_yaml` rejects newlines / null bytes / >4096-char dataset paths (defends against YAML key injection — mirrors v0.46.0 Part A `render_recipe_yaml` policy). `write_mix_recipe` is atomic via `tempfile.mkstemp + os.replace`; `is_under_cwd` containment + ≤4096-char path + `os.lstat + S_ISLNK` rejection (TOCTOU) + overwrite-required gate. `load_mix_recipe` `os.lstat` is wrapped in `try/except OSError` (review fix HIGH — first-cut called `os.lstat` bare after `os.path.lexists`, leaving a TOCTOU race where path disappearance between the two calls would raise an unhandled `OSError` to the user); 256 KB file cap; `yaml.safe_load` only. New `soup data mix --optimize / --apply` Typer command with mutually-exclusive modes. Known limitations: (1) Live HF Trainer callback for dynamic curriculum deferred to v0.48.1 — schema gates, math kernel, and visualiser ship; the callback wiring with `torch.distributed.all_reduce` of per-bucket stats lands in v0.48.1. (2) Multi-trainer expansion (DPO/GRPO/etc.) deferred to v0.48.1 — schema rejects non-SFT/pretrain because per-sample loss semantics differ enough that bucket-level uncertainty does not transfer cleanly. (3) Live proxy training loop for `soup data mix` deferred to v0.48.1 — CLI ships with a synthetic offline proxy (quadratic penalty around uniform mixture). (4) `scikit-optimize` integration deferred to v0.48.1 — `OptimizerProtocol` ducktype is the integration point; default fallback is a deterministic Dirichlet sampler. (5) Both features ship `BETA:`-prefixed in CLI help and field descriptions until reference-benchmark validation lands in v0.48.1.
- **v0.47.0 — Data Forge**: 2 release Parts ship a synthetic data pipeline with full provenance + a data-quality scorecard. New `soup_cli/utils/data_forge.py` ships frozen `ForgePlan` / `ProvenanceRecord` / `ForgeRow` dataclasses + a closed `VALID_TASKS = ("sft", "preference", "tool")` allowlist. `chunk_document` paragraph splitter has `_MAX_DOC_CHARS = 4 MiB` cap, null-byte rejection, and bool-as-int reject on `max_chunk_chars` (matches v0.30.0 `Candidate` policy). `_validate_float_unit` uses `math.isfinite` to reject NaN AND ±Inf BEFORE the `[0, 1]` bounds check (review fix HIGH — first-cut accepted `float("nan")` because `nan < 0.0` is `False`, silently disabling active-pruning; mirrors v0.32.0 `save_lr_finder_report` and v0.41.0 Part B `parse_lr_groups` policy). `discover_documents` does `is_under_cwd` containment (review fix MEDIUM — first-cut only enforced containment from `build_forge_plan`, leaving direct callers unprotected; matches v0.42.0 `discover_*` policy), rejects symlinked directories via `os.lstat + stat.S_ISLNK`, restricts to a closed `_DOC_EXTENSIONS = {.txt, .md, .json, .jsonl}` allowlist, caps at `_MAX_DOCS = 10_000`, and skips dotfiles. `synthesise_forge_rows` runs `chunk → judge(prompt) → score_uncertainty → ForgeRow` with judge-exception swallow at DEBUG (matches v0.33.0 #47 `CrossDocCollator` and v0.40.3 `judge_filter_pairs` policy — no silent crash on a single bad judge call). `write_forge_dataset` + `write_provenance` are atomic via `tempfile.mkstemp` + `os.replace` (mirrors v0.43.0 Part D `copy_bundle_to`); both writers call `_check_write_path` which enforces `is_under_cwd` + ≤4096-char cap + `os.lstat + stat.S_ISLNK` rejection at the target (TOCTOU defence — mirrors v0.33.0 #22 / v0.43.0 Part C / v0.44.0 Part B / v0.45.0 Part E / v0.46.0 policy); rejects non-`ForgeRow` elements (review fix MEDIUM — first-cut would `TypeError` mid-write leaving partial state). New `soup_cli/utils/data_score.py` ships `BENCHMARKS` `MappingProxyType` (6 names — mmlu / gsm8k / humaneval / truthfulqa / arc / hellaswag) and frozen `ScoreReport` with `languages` exposed as `MappingProxyType` to prevent caller mutation. `ngram_set` caps `n ∈ [1, 32]`, text at `_MAX_TEXT_CHARS = 1 MiB`, rejects bool-as-int. **ReDoS-hardened PII regexes** — `_PII_PATTERNS` rewritten in the security review (review fix HIGH × 2): phone pattern flattened to remove nested `(?:\+\d{1,3}[\s.-]?)?(?:\(?\d{2,4}\)?[\s.-]?)?` (catastrophic backtracking on near-miss inputs) → flat alternation with digit-count post-filter ≥ 7; credit-card rewritten from `\b(?:\d[ -]?){13,19}\b` (exponential backtracking on a 13-digit-space pattern ending with `x`) to anchored `\b\d{4}[\s\-]?\d{4}[\s\-]?\d{4}[\s\-]?\d{1,7}\b` with hard digit caps; `_PII_SCAN_CAP = 50_000` chars enforced on every input via `detect_pii` pre-truncation. `_require_str` rejects null bytes (review fix MEDIUM — first-cut only checked length, leaving the data_score validators inconsistent with the data_forge `_validate_str`). `_require_unit_float` uses `math.isfinite` to reject NaN AND ±Inf (matches `data_forge` policy). `decontaminate_rows` documents that it uses one-sided **containment** ratio `|inter| / |b|` rather than symmetric Jaccard (review fix MEDIUM — first-cut docstring said "Jaccard" but formula was containment; renamed for accuracy so future maintainers don't accidentally swap the denominator). `compute_scorecard` per-row `try/except ValueError` blocks now log at DEBUG (review fix MEDIUM — first-cut was silent `pass`, violating v0.33.0 #47 and v0.40.3 logging policy). `load_jsonl_rows` + `write_jsonl_rows` enforce `is_under_cwd` containment, `_MAX_FILE_BYTES = 1 GiB` cap, `_MAX_ROWS = 1_000_000` row cap, `os.lstat + stat.S_ISLNK` rejection on both input and output, atomic write via `tempfile.mkstemp` + `os.replace`. `_read_rows` and `_write_rows` CLI helpers have full `List[Mapping[str, Any]]` / `Iterable[Mapping[str, Any]]` type annotations (review fix HIGH — first-cut had `def _read_rows(path: str):` with no return type, breaking downstream type checking). `decontaminate_texts` parameter is `Optional[Mapping[str, Sequence[str]]] = None` (review fix HIGH — first-cut had `Mapping[...] = None  # type: ignore[assignment]` masking the type error). `import math` and `import tempfile` moved to module top-level (review fix MEDIUM — first-cut had stdlib imports inside function bodies, violating the project's lazy-import policy which applies only to heavy ML deps). New `soup_cli/commands/data_forge.py` collapses the duplicate `discover_documents` call (review fix MEDIUM — first-cut called the helper twice, introducing a TOCTOU window where `plan.num_docs` could disagree with the actual doc list). New `soup_cli/commands/data_score.py` `--benchmarks` allowlist-validated against `BENCHMARKS` keys with Rich-escaped error messages. Known limitations: (1) Live judge providers (Ollama / Anthropic / vLLM via `--judge-provider`) deferred to v0.47.1 — `soup data forge` ships with a deterministic offline echo stub; stub-then-live pattern matches v0.27.0 MII / v0.37.0 multipack / v0.46.0 Part A. (2) Decontamination benchmark corpora not bundled — `soup data decontaminate --benchmarks mmlu` validates the flag but operates on an empty corpus; operator-supplied `--benchmark-file` lands in v0.47.1. (3) Llama-Guard-3-1B toxicity classifier + FineWeb-Edu educational classifier + full Presidio PII + `langdetect` / `fastText` ship behind `[data-pro]` extras in v0.47.1. (4) Provenance manifest stores absolute realpath in `source_doc` (security review M4) — operators sharing manifests should redact paths; kept as-is for audit-trail completeness, mirrors v0.34.0 `crash.py` design tension between traceability and `$HOME` leak prevention.

- **v0.46.0 — Deploy & Agent Autopilot**: 2 release Parts ship a deploy-target picker and an Agent Forge for spec-driven tool-calling SFT datasets. New `soup_cli/utils/deploy_autopilot.py` ships a 10-profile `MappingProxyType`-wrapped catalog with `DeployProfile` `@dataclass(frozen=True)` and closed allowlists on `runtime` (transformers / vllm / sglang / mlx / ollama / lm-studio / executorch), `quant` (none / 4bit / 8bit / gptq / awq / fp8 / mxfp4 / hqq:Nbit), `peft` (lora / dora / qlora / full). `_make` factory rejects non-kebab-case names, bool-as-int on `recommended_max_length`, out-of-bounds `[64, 1_048_576]`, null-byte / >512-char description+notes. `render_recipe_yaml` rejects empty / null-byte / newline / >200-char `base` and >4096-char `output_dir` (defends against a crafted `--base "evil\ntraining: { epochs: 9999 }"` injecting YAML keys into the rendered recipe). `render_deploy_script` uses `shlex.quote` on `model_path` and rejects newline / NUL / >4096-char. `write_recipe` / `write_deploy_script` enforce `is_under_cwd` containment, ≤4096-char path cap, and `os.lstat + stat.S_ISLNK` rejection at the write target (TOCTOU defence — mirrors v0.33.0 #22 / v0.43.0 Part C / v0.44.0 Part B / v0.45.0 Part E policy). `soup deploy autopilot` panel passes every profile field through `rich.markup.escape` (matches v0.43.0 Part B `Tournament` policy — defends against markup injection if a future profile carries `[blink]` in its description). New `soup_cli/utils/agent_forge.py` parses OpenAPI 3.x / MCP server manifests / GraphQL introspection JSON into a canonical `Endpoint` frozen dataclass — every parser routes synthesised `path` through `_validate_path` (non-empty single-line NUL-free ≤1024 chars), so a manifest with `name="evil\nhost"` is rejected with a warning (review fix HIGH — the v0.46.0 first-cut stored the raw `f"mcp://{name}"` without validation, allowing newline injection into the `Endpoint.path` field). `$ref` strings in OpenAPI are left opaque (no external resolution — defends against file-read SSRF that a fully-resolving parser would expose). `_MAX_ENDPOINTS=10_000`, `_MAX_SPEC_BYTES=5*1024*1024`, `_MAX_ROWS_PER_ENDPOINT=32`, `_MAX_DESCRIPTION=512`. `load_spec_file` enforces `is_under_cwd` containment + `os.lstat + stat.S_ISLNK` rejection BEFORE `realpath` resolution (review fix MEDIUM — the v0.46.0 first-cut ordered `isfile(real)` before the lstat check, which followed the symlink) + 5 MiB cap + `yaml.safe_load` only (never `yaml.load`). `write_dataset` is atomic via `tempfile.mkstemp` + `os.replace` (review fix HIGH — replaces a v0.46.0 first-cut direct `open(real, "w")` loop that would leave a partial JSONL file on mid-stream `TypeError`; mirrors v0.43.0 Part D `copy_bundle_to` policy). Symlink rejection at the dataset target. `endpoint_to_rows` rejects bool / out-of-bounds `examples_per_endpoint` (∈ [1, 32]). New `soup_cli/commands/agent.py`: `synth` Rich table passes every cell through `rich.markup.escape` (review fix HIGH — defends against spec-controlled markup injection through `ep.path`); `train` validates `--base` and `--output-dir` for NUL / newline / >4096-char BEFORE embedding into the rendered YAML recipe string (review fix CRITICAL — defends against YAML key injection where `--base $'evil\\ntraining: { epochs: 9999 }'` would smuggle injected training keys); `eval` enforces predictions path `is_under_cwd` containment + `os.lstat + stat.S_ISLNK` rejection + `_MAX_PRED_LINES=1_000_000` DoS cap (review fix HIGH — v0.46.0 first-cut had no line cap, a multi-GB predictions file would have iterated unbounded). Known limitations: (1) Live Quant-Lobotomy auto-measure deferred to v0.46.1 — autopilot writes the canonical PEFT+quant combo per profile but does not yet measure OK/MINOR/MAJOR via v0.26.0 Quant-Lobotomy Checker. (2) ExecuTorch packaging deferred to v0.54.0 — `iphone-16` / `pixel-9` recipes are plan-only. (3) `soup agent train` is plan-only — prints the planned `soup train` invocation rather than re-entering Typer in-process (same design as v0.44.0 `soup quantize`). (4) `soup agent eval` is heuristic — scores tool-name match + arguments-key validity only; live RLVR `code_exec` sandbox scoring deferred to v0.46.1. (5) `$ref` resolution in OpenAPI specs is intentionally not done (file-read SSRF defence); users wanting full resolution should run `openapi-spec-validator` upstream. (6) MCP / GraphQL non-HTTP sentinel methods (`invoke` / `query` / `mutation`) are stored on `Endpoint.method` without going through `_validate_method` (documented design intent — `_HTTP_METHODS` covers HTTP-only).
- **v0.45.0 — Plugin System & Ecosystem Wins**: 5 release Parts ship the plugin / hook system + 4 ecosystem-integration schemas. New `soup_cli/plugins/__init__.py` registry: kebab-case plugin name regex `^[a-z0-9][a-z0-9\-]{0,39}$`, semver-ish version regex, `_MAX_PLUGINS=64`, `_MAX_TEMPLATES_PER_PLUGIN=32`, `_MAX_MODEL_GROUPS_PER_PLUGIN=32`, `_MAX_NAME_ENTRY_LEN=128`. Re-registering with a different `(version, plugin object, templates, model_groups, description)` is rejected (review fix HIGH — first-cut omitted `description` from the conflict check, allowing silent description override). `load_plugins()` swallows per-plugin exceptions at WARNING level so one broken plugin cannot crash the CLI. `soup plugins enable / disable` raise `KeyError` for unknown names (caller-distinguishable from validation errors). All user-controlled output passes through `rich.markup.escape`. New `utils/anthropic_messages.py` enforces `_MAX_MESSAGES=1024`, per-message `_MAX_CONTENT_LEN=1MiB`, `max_tokens` cap mirrors v0.30.0 inference-server `_MAX_TOKENS_CAP=16384`; `tool` role with structured (list) content is concatenated into a single `tool_result` text block instead of silently dropped (review fix MEDIUM — first-cut emitted `""` for any non-str content); `from_anthropic` uses `.get("role")` defensively after `validate_anthropic_payload` (review fix MEDIUM — `KeyError` surface guard). New `utils/server_tools.py` ships closed `SUPPORTED_TOOLS = frozenset({python, bash, web_search})` allowlist + `WebSearchConfig` with `domain_allowlist` (≤64 entries, default empty = deny all) + `rate_limit_per_minute ∈ [1, 600]`. `is_domain_allowed` strips `:port` suffix before matching (review fix MEDIUM — without this, `Host: api.example.com:443` would silently fail to match the bare `api.example.com` allowlist entry, causing legitimate denials) AND rejects IPv6 literals (`[::1]` → False — IPv6 should never match a domain allowlist). `validate_domain` rejects null-byte / whitespace / `/` / oversize. `python` and `bash` reuse the v0.25.0 RLVR sandbox (5s timeout, RLIMIT_AS/CPU on POSIX, ephemeral cwd, socket patch); live HTTP endpoints land in v0.45.1. New `utils/ngram_spec.py` `NgramSpecConfig` is `@dataclass(frozen=True)` with `n ∈ [1, 8]`, `num_draft_tokens ∈ [1, 32]`, `prompt_lookup_max ∈ [0, 1MiB]`; bool rejection on every numeric field (matches v0.30.0 `Candidate` / v0.34.0 `estimate_run_cost_usd` / v0.42.0 `make_preprocess_cache_key` policy). New `utils/integrations.py` 15-entry catalog wrapped in `MappingProxyType`; `IntegrationSpec` is frozen so the catalog cannot be mutated at runtime. New `utils/trainer_plugins.py` 6-entry allowlist (`MappingProxyType`); `validate_trainer_plugin_list` accepts `Sequence[str]` (review fix MEDIUM — first-cut annotated as `Tuple[str, ...]`) but explicitly rejects bare `str` argument so a caller passing `"grokfast"` directly doesn't iterate per-character. New `utils/recipe_dag.py` parses Data Recipe DAG with closed `NODE_KINDS = frozenset({seed, llm_text, code, judge, validator, sampler})`; Kahn's topological sort uses `collections.deque` + sorted-on-arrival (review fix HIGH — first-cut had O(N² log N) `queue.sort()` inside the BFS body); cycle / self-loop / dangling-edge / duplicate-edge / duplicate-node-name / unknown-kind rejection with specific error messages. `_MAX_NODES=256`, `_MAX_EDGES=1024`, `_MAX_FILE_BYTES=1MiB`. `parse_recipe_yaml` uses `yaml.safe_load`. `load_recipe_yaml` enforces `is_under_cwd` containment AND `os.lstat + stat.S_ISLNK` symlink rejection at the recipe path (review fix MEDIUM — TOCTOU defence; mirrors v0.33.0 #22 / v0.43.0 Part C / v0.44.0 Part B policy; without this, a symlink under cwd pointing at `/etc/passwd` would resolve through `realpath` and load arbitrary files). `RecipeDAG` and `RecipeNode` are `@dataclass(frozen=True)`; `RecipeNode.config` is `MappingProxyType`-wrapped so caller mutation post-parse cannot affect the validated topology. `parse_recipe` does NOT mutate input dicts (matches v0.33.0 #47 `CrossDocCollator` immutability policy). Known limitations: (1) Plugin trainer-callback wiring deferred to v0.45.1 — `register_plugin` accepts hook objects and `discover_hooks` lists implementations, but no HF Trainer callback emits `pre_train` / `post_train` / `pre_step` / `post_step` yet. (2) Anthropic `/v1/messages` endpoint deferred to v0.45.1. (3) Server-side tool HTTP endpoints (python / bash / web_search) deferred to v0.45.1. (4) n-gram speculative-decoding live engine wiring deferred to v0.45.1. (5) External integrations are a catalog, not launchers — `IntegrationSpec` describes 15 ecosystem targets but no `soup deploy <target>` command auto-detects + invokes their CLIs (tracked for v0.46.0 Deploy Autopilot). (6) Advanced trainer plugins are an allowlist, not callbacks — `validate_trainer_plugin_list` accepts `[grokfast, spectrum, ...]` at schema validation, but no trainer wraps the upstream callbacks yet (live wiring per plugin in v0.45.1). (7) Data Recipe DAG runner deferred — `soup data recipe` validates topology and prints planned topological order; node-by-node execution against a local model is the v0.45.1 deliverable.
- **v0.44.0 — Live Dashboard & UX**: 21 features across 4 Parts. New `soup_cli/utils/qr_url.py` `build_phone_url` enforces a scheme allowlist (`http` / `https`), loopback-only HTTP (RFC1918 / link-local / cloud-metadata IPs implicitly rejected because `host not in _LOOPBACK_HOSTS` raises), IPv6 literal auto-bracketing per RFC 3986, and **token routing through the URL query string `?token=…`** (review fix HIGH — the v0.44.0 first-cut put it in the URL fragment `#token=…` which never reaches the server). Token shape regex `^[A-Za-z0-9_\-]{16,128}$` matches `secrets.token_urlsafe(N)` output. New `commands/fetch.py` enforces `is_under_cwd` on `--output` AND adds **`os.lstat + S_ISLNK` symlink rejection at the write target** regardless of `--force` (review fix HIGH — TOCTOU defence; without this, force=True would follow a pre-placed symlink and overwrite a file outside cwd; mirrors v0.33.0 #22 / v0.40.2 #51 / v0.43.0 Part C policy). Bundled-source path is `os.path.commonpath`-checked against `fetch_examples_dir()` (review fix HIGH — defends against a future catalog entry containing `..` that would escape the bundled directory). `fetch_examples_dir()` uses `os.path.realpath` not `abspath` (review fix MED — symlinked installs resolve to the real package root, not the symlink's parent). New `utils/checkpoint_trigger.py` `write_trigger` adds `os.lstat + S_ISLNK` rejection on the trigger target (review fix MED — symlink at `<output_dir>/.checkpoint_now` could redirect the write to any user-writable path). Trigger path stays under cwd via shared `is_under_cwd`. New `commands/llama.py` proxies to llama.cpp binaries with **`_LLAMA_ENV_ALLOWLIST` frozenset child-env filter** (review fix MED) — `HF_TOKEN` / `OPENAI_API_KEY` / `ANTHROPIC_API_KEY` / all `SOUP_*` are dropped before exec; only `PATH` / `HOME` / `USER` / `USERPROFILE` / `TMP` / `TEMP` / locale + llama.cpp-recognised `LLAMA_CPP_HOME` / `GGML_*` / `OMP_NUM_THREADS` are forwarded. Subprocess uses list args (no shell). Closed allowlist `_SUBCOMMAND_TO_BINARY: MappingProxyType` rejects unknown subcommands; `_validate_arg` rejects null-byte / newline / carriage-return / >1024-char per arg; `_MAX_ARGS=64` cap. New `utils/onboarding.py` `render_onboarding_yaml` enforces `is_under_cwd` on the `output` field at render time (review fix MED — without this, a crafted answer dict like `output: ../../etc/cron.d/x` would silently propagate into the rendered YAML). Literal allowlist on `task` (8 entries) and `quantization` (`4bit` / `8bit` / `none`); `epochs ∈ [1, 10]`; null-byte rejection on every string with field-named error message. New `utils/sweep_config.py` enforces a strict scalar allowlist (`str` / `int` / `float` / `bool`) on every param value (review fix LOW — without this, `params: {lr: [{__class__: evil}]}` would smuggle non-scalar YAML through to runtime); `_MAX_FILE_BYTES=256KB`, `_MAX_PARAM_KEYS=32`, `_MAX_VALUES_PER_KEY=64`, `_MAX_PARAM_KEY_LEN=128`. `SweepSpec.params` is `MappingProxyType[str, Tuple[Any, ...]]` for genuine `frozen=True` immutability (review fix HIGH — `frozen=True` on a `Dict[str, List[Any]]` field only prevents reassignment, not mutation). Same fix applied to `ConsolidationPlan.shard_files: Tuple[str, ...]` and `DelinearizePlan.weight_files: Tuple[str, ...]`. New `utils/gpu_monitor.py` `parse_nvidia_smi_csv` rejects null-byte in GPU name field; subprocess uses list args + 5s timeout + `OSError`/`TimeoutExpired` swallowed (never raises). `detect_apple_silicon` rewritten to use clean `platform.system()` / `platform.machine()` checks (review fix HIGH — the v0.44.0 first-cut had a parser-priority bug `if X if Y else Z:` that produced the right answer by coincidence on every platform but would silently break on any future branch refactor). New `utils/sse_train_stream.py` `TrainEvent` validates `type` against a closed `_VALID_TYPES = frozenset({metric, status, log, eval})`, rejects non-finite `ts`, rejects null-byte / >1024-char `message`. `to_payload` filters output to a closed `_ALLOWED_KEYS` allowlist (defence-in-depth against payload key drift from refactors). `format_sse_frame` emits the standard W3C SSE wire format `data: {json}\n\n` with `ensure_ascii=False`. New `utils/tool_outputs.py` `ToolOutputsBuffer` is thread-safe (`threading.Lock`) backed by `collections.deque(maxlen=_MAX_RECORDS=1000)` (review fix MED — replaces an O(N) `self.records[-N:]` list slice that triggered GC pressure on overflow); validates name (no null-byte, ≤128 chars), bool rejection on `started_ts` / `duration_ms` (matches project bool-as-int policy), `success` must be exactly `bool` not `int`, output preview truncated to 4096 chars. `ToolCallTimer.__exit__` returns `False` explicitly per project readability policy (does not suppress exceptions). New `utils/llama_server_timings.py` `parse_timings` tolerates missing keys (defaults to None), clamps `kv_cache_pct` to `[0, 100]`, rejects bool / non-finite / negative ints in numeric coercion. `format_kv_bar` validates `width ∈ (0, 200]`. New `utils/tail_latency.py` enforces `MAX_SAMPLES=1_000_000` DoS cap on every iteration; bool rejection + non-finite rejection + alpha bounds. New `soup_cli/ui/plugins/__init__.py` registry: tab name regex `^[a-z0-9][a-z0-9\-]{0,30}$`, `_MAX_TABS=32` cap, `list_tabs()` returns `MappingProxyType` defensive view, `load_plugins()` swallows per-plugin exceptions at WARNING level so one broken plugin doesn't crash the whole UI. `_NAME_DISALLOWED` in `utils/shortcuts.py` is `frozenset` (review fix LOW — was mutable `set` at module level). New `utils/ui_env.py` host validator restricts to `[a-zA-Z0-9.-:]` ≤253 chars + null-byte rejection; port `[1, 65535]`; key length cap 256 + null-byte rejection. Known limitations: (1) Live monitoring SSE endpoint deferred — schema + frame formatter ship, FastAPI route in v0.44.1. (2) Phone visibility flag wiring (`soup ui --public --auth-token` + QR print at startup) deferred to v0.44.1. (3) `soup merge-sharded-fsdp-weights` and `soup delinearize-llama4` are plan-only — torch-side runtime in v0.44.1. (4) `soup llama <subcommand>` does not bundle llama.cpp; the binary must be on PATH (`shutil.which`) and an actionable `FileNotFoundError` fires when missing. (5) `soup serve --reasoning-parser <name>` allowlist ships, runtime hook in v0.44.1. (6) Apple Silicon `soup monitor` prints a yellow advisory and exits with `nvidia-smi unavailable` rc=1 — `powermetrics` parser in v0.44.1.
- **v0.43.0 — Tracker & Eval Pro**: 18 features across 4 Parts. New `soup_cli/utils/trackers.py` ships an immutable `_REPORT_TO_BACKENDS` `MappingProxyType` allowlist (`wandb` / `tensorboard` / `mlflow` / `swanlab` / `trackio` / `none`); `validate_tracker_name` rejects non-string / empty / null-byte / >32-char inputs. PostHog telemetry is opt-IN (default OFF) via `SOUP_TELEMETRY=1` env var; `build_telemetry_payload` schema is closed-key (soup_version / command / python major.minor / os / arch / duration) — no model names, dataset paths, or config contents leak in the payload. Live PostHog network code deferred to v0.43.1. New `--tracker` flag on `soup train` mutually exclusive with `--wandb` / `--tensorboard` via `resolve_report_to`. `tracker_backend = tracker` capture in `commands/train.py` defends against the Typer parameter being shadowed by the local `ExperimentTracker()` instance (review fix HIGH #1 — silent `--tracker` drop regression). `nlg_metrics.py` is pure-math: BLEU `_lcs_length` allocates a fresh row per outer iter (review fix HIGH #2 — first-cut had a double-buffer re-zero that destroyed the last row). `effective_tokens_per_second` returns `None` on non-positive wall_clock (no fabrication). `eval/calibrate.py` `CalibrationReport` is `@dataclass(frozen=True)` with `FrozenInstanceError` test (matches v0.32.0 / v0.39.0 / v0.41.0 frozen-dataclass policy); `run_calibration` caps prompts at 10_000 (DoS defence). `eval/arena.py` `Tournament` enforces 256-model cap, 1M-match cap, `MappingProxyType` view on `ratings` so external callers cannot mutate Elo state (LOW review fix). `_validate_model_name` rejects null-byte / >128-char AND Rich markup metacharacters `[` / `]` (security review fix LOW #5 — leaderboard markup-injection defence). `update_elo` rejects bool / non-finite / NaN `k`. `eval/benchmarks_v0_43.py` ships `NEW_BENCHMARKS_V0_43 = frozenset({"ceval","cmmlu","aider_polyglot"})` + `MappingProxyType`-wrapped metadata; live Aider Polyglot runner deferred to v0.43.1. `utils/profiling_v0_43.py` `resolve_snapshot_path` rejects null-byte / `.` / `..` / path separators in `run_id`; `base_dir` rejects null-byte / absolute / `..` segments before realpath join (security review fix HIGH #1) — closes a Windows short-name path escape. `memory_snapshot_context` narrows the `RuntimeError` catch to the `record()` entry call only (review fix HIGH #2 — wide catch would have triggered `generator already executing` on user-body RuntimeError); yields `None` when torch / CUDA / `_record_memory_history` API missing. `nccl_bandwidth_check` rejects bool / non-finite / negative `measured_gb_per_sec`; reference table is `MappingProxyType`-wrapped. `utils/vscode_setup.py` `build_launch_json` rejects null-byte / newline / >512-char `config_path` so a crafted argument cannot inject Python args into the generated JSON. `write_vscode_launch` uses `os.lstat + S_ISLNK` symlink rejection at the target path regardless of `force` (security review fix MEDIUM #3 — TOCTOU defence; without this, force=True would follow a pre-placed symlink and overwrite a file outside cwd; mirrors v0.33.0 #22 / v0.40.2 #51 policy). `utils/demo_bundles.py` `copy_bundle_to` validates every JSONL line via `json.loads`, enforces 50 MB cap, stages writes to `<target>.tmp` with `os.lstat + S_ISLNK` rejection (security review fix HIGH #2) and `os.replace`-atomic rename on success — mid-stream rejection never leaves a partial file. New `soup data demo` Typer command: `--output` containment-checked via shared `is_under_cwd`. CLI output prints `_esc(written)` to defend against Rich markup injection through a crafted output path (review fix LOW). `_lcs_length` algorithmic correctness fix (review fix HIGH #2) + BLEU geometric-mean policy fix (returns 0 on any zero precision unless `smooth=True`) (review fix HIGH #1). All `math` imports moved to module-level (PEP 8 / project lazy-import policy: only torch / transformers / peft / trl / mlx are lazy). Known limitations: (1) MLflow / SwanLab / Trackio live integration deferred to v0.43.1 — schema accepts the names and threads into HF `TrainingArguments.report_to` but Soup does not bundle the upstream packages; HF Trainer raises a generic ImportError when the package is absent. (2) PostHog telemetry deferred — payload schema + `SOUP_TELEMETRY=1` env var ship, but no network code; `build_telemetry_payload` is purely a schema-locking stub (mirrors v0.27.0 MII / v0.37.0 multipack / v0.42.0 fsspec stub-then-live pattern). (3) Aider Polyglot live runner deferred — `aider_polyglot` is in the benchmark allowlist but live wiring requires the upstream `aider-chat` package + Docker. (4) `soup doctor --nccl` measurement CLI surface deferred — `nccl_bandwidth_check` accepts a measured value and classifies it, but does not yet measure via `torch.distributed.all_reduce`. (5) `examples/data/` fixtures shipped at the repo root (not as package data inside `soup_cli/`); `_bundle_source_path` derives `repo_root` via `str(importlib.resources.files("soup_cli"))` which is editable-install + wheel friendly but may break in zipapp / namespace-package installs.

- **v0.42.0 — Data Pipeline Pro**: closes the data-tooling gap with Axolotl + LlamaFactory in 18 features across 6 Parts. New `soup_cli/utils/data_pipeline.py` ships an immutable `_REMOTE_SCHEMES` `MappingProxyType` allowlist (s3 / gs / gcs / az / abfs / abfss / oci); `validate_remote_uri` rejects userinfo, fragments, AND query strings (the latter is SSRF-adjacent — fsspec backends interpret `?endpoint_url=…` as configuration overrides; a crafted `s3://bucket/x?endpoint_url=https://attacker.com` would bypass otherwise-trusted infrastructure). Bucket regex `^[a-zA-Z0-9][a-zA-Z0-9._\-]{0,62}$` (1-63 chars, leading alnum) matches RFC 3986 + S3/GCS naming. Path length cap 2048; null-byte rejection on every string-shaped input; bool rejected before int isinstance check on every numeric input (`buffer_size` / `shards` / `image_pixels` / `video_maxlen` / `num_datasets`). `parse_interleave` returns frozen `InterleaveSpec` dataclass; per-prob `math.isfinite` (rejects NaN AND ±inf, matches v0.32.0 / v0.41.0 policy); sum-to-1 ±1e-6; max 32 datasets. `validate_new_tokens` caps lists at 10_000 entries with no duplicates, per-token <= 256 chars, null-byte rejected; returns a defensive copy. `validate_prompt_strategy` enforces a `module.path:function_name` regex with per-side 128 char cap (total <= 260). `make_preprocess_cache_key` rejects bool / null-byte / non-string / non-positive on every parameter; uses `\x1f` unit-separator delimiter. `detect_ingest_format` rejects null-byte / empty / unsupported extensions. New schema validators on `DataConfig`: `video_dir` / `tokenized_path` use shared `is_under_cwd` containment so a crafted YAML like `data.video_dir: ../../etc` fails at config load (review-fix M1; distinct from `image_dir` / `audio_dir` legacy policy). `train_on_prompt` mutually exclusive with `train_on_responses_only` (matches v0.36.0 loss-mask exclusivity policy). `interleave` field has its own `field_validator` so `data.interleave: 99` fails at config load (review-fix L3) AND bare `interleave: probs` is rejected at schema load with the actionable "use {strategy: probs, probs: [...]} dict form" message instead of getting a confusing late error (review-fix HIGH #2). Image-pixel validator threads `info.field_name` so error messages name the actual field that tripped (review-fix M3). Bucket regex relaxed from `{1,62}` to `{0,62}` so 1-character bucket names (per S3/GCS spec) are no longer falsely rejected (review-fix HIGH #1). `_convert_video` validates `video` is non-empty string with null-byte rejection and 2048-char cap (review-fix H2). `_convert_prm` type-checks `prompt` (non-empty string) + every `completions[i]` (string) + every `labels[i]` (bool, not int — matches v0.30.0 `Candidate` policy) and caps at 10_000 steps. `_convert_multimodal` validates content-part `type` against the closed allowlist `{text, image, audio, video}`. New `soup data preprocess <config>` CLI: `--config` containment-checked via `is_under_cwd` BEFORE `load_config` (review-fix HIGH #1) so a crafted argument like `../../etc/passwd.yaml` cannot be opened. New `soup data ingest <file>` CLI: `os.lstat + stat.S_ISLNK` symlink rejection on the input file (TOCTOU defence, mirrors v0.33.0 #22 `prune_checkpoints` policy); `is_under_cwd` containment on input + output. Known limitations: (1) fsspec live loaders deferred to v0.42.1 (schema gate fires now, runtime ImportError advisory); (2) AOT preprocess live tokenize loop deferred to v0.42.1 (cache-key emit + path-plan only); (3) custom prompt-strategy runtime invocation deferred to v0.42.1 (regex shape validation only); (4) PRM live trainer wiring deferred to v0.50 per original roadmap; (5) markdown ingest does not split on headings (the docstring's heading-split claim was aspirational); (6) `_convert_video` / `_convert_multimodal` validate path/url null-byte and length but do NOT verify file existence (matches v0.40.5 `reward_model` policy — file-existence is the trainer's responsibility).
- **v0.41.0 — Optimizer & PEFT Zoo**: closes the optimizer-breadth gap with LlamaFactory + Axolotl. New `soup_cli/utils/optimizer_zoo.py` ships a closed `SUPPORTED_OPTIMIZERS` `frozenset` (HF-native + bnb-backed + 14 v0.41.0 additions: BAdam / APOLLO / Adam-mini / lomo / adalomo / grokadamw / schedule_free_adamw / schedule_free_sgd / muon / dion / came_pytorch / ao_adamw_{fp8,4bit,8bit}). `validate_optimizer_name` rejects non-string / empty / null-byte / >64-char inputs; lower-cases the name for deterministic lookup (matches v0.30.0 `pick_draft_model` policy). `_OPTIMIZER_PACKAGES` wrapped in `types.MappingProxyType` so the registry cannot be mutated at runtime (matches v0.36.0 `_REGISTRY` policy). `is_new_v0_41_optimizer` is non-string-safe (returns False rather than raising). New `soup_cli/utils/lr_groups.py` parses `training.lr_groups` accepting list-of-pairs / list-of-dicts / `{pattern: lr}` mapping; capped at `MAX_LR_GROUPS=32`; per-pattern non-empty string ≤256 chars + null-byte rejection + `re.compile` validation + best-effort ReDoS probe (`compiled.search("a"*128)` catches catastrophic-backtracking patterns); per-LR `(0.0, 1.0]` bounds + `math.isfinite` (rejects NaN AND `±inf`) + bool rejection (matches v0.30.0 `Candidate` policy). Duplicate patterns rejected. `LrGroup` is `@dataclass(frozen=True)` (matches v0.32.0 `SpikeRecoveryStrategy` policy). `lr_groups_from_schema` converts the canonical stored shape `List[Dict]` into runtime `List[LrGroup]` (closes the schema-to-runtime type gap that the code review caught). `build_optimizer_param_groups` rejects bool / non-positive `base_lr` at runtime (defence-in-depth). New `soup_cli/utils/loftq_init.py` exposes `validate_loftq_iter` (∈ [1, 10], bool rejected) and `validate_loftq_bits` (∈ {2, 4, 8}, bool rejected) with `build_loftq_config` lazy-importing `peft.LoftQConfig` (ImportError carries actionable `pip install --upgrade peft` hint). New `soup_cli/utils/block_expansion.py` validators reject bool on `expand_layers` / `freeze_trainable_layers` and use `field_validator(mode="before")` so Pydantic's `ge`/`le` does not silently coerce `True` to `1` (matches v0.30.0 / v0.34.0 / v0.36.0 / v0.40.6 bool-as-int hardening policy). `_count_layers` uses `hasattr(layers, "__len__")` instead of `try/except TypeError` so legitimate `__len__` bugs surface loudly. Schema `LoraConfig.init_strategy` Literal extended to `{"random", "pissa", "olora", "loftq"}`; cross-validator rejects loftq + use_dora / use_vera. `TrainingConfig.load_in_8bit` / `load_in_16bit` use `is True` policy (matches v0.34.0 / v0.39.0 / v0.40.6 `is None` over falsy guards) — explicit `False` is treated as "no preference" not as "off"; mutually-exclusive both-True rejected; combining alias=True with explicit Quant Menu format raises rather than silently overriding. The alias-driven `quantization` rewrite uses direct `self.quantization = ...` assignment (NOT `object.__setattr__` — code review caught that the latter would silently bypass any future field validator on `quantization`). `expand_model_blocks` raises `NotImplementedError` with a v0.41.1 marker on non-zero block counts (mirrors v0.27.0 MII / v0.37.0 multipack / v0.38.0 quant menu / v0.39.0 ReLoRA stub-then-live pattern). Known limitations: (1) LLaMA Pro live wiring deferred to v0.41.1; (2) Mixture-of-Depths (`use_mod=True`) live patch deferred to v0.41.1; (3) optimizer dependency check is advisory at trainer construction time, not at schema-load (CI environments often lack optional optimizer packages); (4) `load_in_8bit`/`load_in_16bit` rewrite `quantization` only when set to `True` — explicit `False` is intentionally a no-op so a YAML with `load_in_8bit: false` and `quantization: 4bit` still trains in 4-bit.
- **v0.40.6 — ReLoRA + surgical PEFT non-SFT**: closes the v0.39.0 known gap by extending the ReLoRA callback (v0.39.0 Part B) and the surgical PEFT patches (v0.39.0 Part D — Gemma4 `ClippableLinear` -> `nn.Linear` swap, fused-MoE 3-D expert dropout strip) from SFT-only to all 11 non-SFT transformer-backend trainers (DPO / GRPO / KTO / ORPO / SimPO / IPO / PPO / RewardModel / Pretrain / Embedding / BCO). New shared module `soup_cli/utils/peft_wiring.py` exposes three helpers: `apply_pre_lora_patches(model, base)` (Gemma4-gated, runs BEFORE `get_peft_model` so PEFT's `target_modules` matcher sees the swapped `nn.Linear`), `apply_post_lora_patches(model)` (3-D MoE expert dropout strip, runs AFTER LoRA injection — architecture-detected via `weight.ndim == 3` inside the helper, safe to call unconditionally), `attach_relora_callback(trainer, tcfg)` (returns `True/False`; uses `if relora_steps is None: return False` per project policy so a schema-bypassing caller passing `relora_steps=0` surfaces as a loud `ReLoRAPolicy` ValueError rather than a silent skip). SFT migrates to the same helpers in the same release (centralisation invariant): every trainer file calls only the helpers, eliminating the v0.39.0 inline copy in `sft.py`. `SoupConfig._validate_relora_supported_tasks` removes the `task != "sft"` rejection branch; MLX backend rejection retained with a distinct error message (callback is HF Trainer-specific). Source-level grep matrix in `tests/test_v0406_part_a.py` proves all 12 transformer-backend trainers (sft + 11 non-SFT) call `apply_pre_lora_patches` BEFORE `get_peft_model` BEFORE `apply_post_lora_patches`, plus behavioural unit tests for each helper (Gemma4 happy path + exception swallow, post-LoRA strip happy path + exception swallow, ReLoRA policy field forwarding, schema-gate matrix covering every transformer task plus the `preference` dispatcher with `preference_loss='dpo'`). Defence-in-depth carry-over: `peft_wiring` swallows broad `Exception` from each upstream patch at DEBUG level (matches v0.39.0 Part D best-effort design); `%s` formatting on `exc` (not `repr`) so `$HOME`-prefixed paths cannot leak (matches v0.34.0 `crash.py` redaction policy); the underlying `apply_gemma4_clippable_patch` and `strip_lora_dropout_for_3d_experts` already validate model_name (null bytes, length) and are duck-typed via v0.39.0 review fixes. Known limitations: (1) Multi-modal trainers (vision/audio paths in `sft.py`) inherit ReLoRA + surgical patches because they share the SFT trainer wrapper, but the surgical patches are best-effort (try/except DEBUG-logged) — a Gemma4 vision model is unlikely in practice; if encountered, the patch attempt may noisy-log without applying. (2) The schema gate now accepts every transformer-backend task with `relora_steps`, but real-world correctness on RLHF (PPO / RewardModel) is unverified — ReLoRA was originally validated on SFT/causal-LM training; rejection-sampling-style RL loops may interact unexpectedly with periodic LoRA pruning + optimizer reset. Tracked as a community QA item; the schema does not gate on this since the upstream paper does not preclude RL use. (3) `apply_post_lora_patches` swallows exceptions at DEBUG, consistent with the v0.39.0 best-effort design — a real PEFT-side breakage in `strip_lora_dropout_for_3d_experts` would silently no-op on a non-MoE model where the strip is also a no-op, so the silent fallback is acceptable.
- **v0.40.5 — Quant Menu non-SFT**: closes the v0.38.0 known gap by extending the seven Quant Menu formats (gptq / awq / hqq:Nbit / aqlm / eetq / mxfp4 / fp8) from SFT-only to all 11 transformer-backend trainers (DPO / GRPO / KTO / ORPO / SimPO / IPO / PPO / RewardModel / Pretrain / Embedding / BCO). `SoupConfig._validate_quant_menu_supported_tasks` removes the `task != "sft"` rejection branch; MLX backend rejection retained with distinct message; `modality != "text"` rejection retained for vision/audio (multi-modal Quant Menu is tracked for a follow-up). Each non-SFT `_setup_transformers` replaces its inline BNB-only branch with a call to `build_quantization_config_for_loader(tcfg=tcfg, base=cfg.base, console=console)` — same pattern as `sft.py:420-440`, no remaining `BitsAndBytesConfig(load_in_4bit=True ...)` literal in any non-SFT trainer (source-level invariant test in `tests/test_v0405_part_a.py`). The kbit-prep tuple is widened from `("4bit", "8bit")` to `("4bit", "8bit", "mxfp4")` so the BNB MXFP4 path runs through `prepare_model_for_kbit_training`. `_load_reward_model` (module-level helper in `ppo.py`) accepts an optional `tcfg=None` kwarg — when set, the reward model is loaded with the same Quant Menu config as the policy, defending against silent fp16 OOM on a GPTQ/AWQ/HQQ policy run. PPO call sites at `_create_reward_model` + `_setup_reward` both forward `tcfg=tcfg`. Defence-in-depth: new `TrainingConfig.reward_model` field validator rejects null bytes and caps length at 512 chars at config-load (matches the policy applied to `cfg.base`); the Quant Menu loader's per-call null-byte check in `_check_local_marker` remains as the runtime backstop. Known limitations: (1) vision/audio modality + Quant Menu still rejected by the modality gate — `_setup_vision_transformers` / `_setup_audio_transformers` retain inline `BitsAndBytesConfig` blocks because they need vision-specific kwargs the unified loader does not yet thread; (2) Autopilot's quantization picker still recommends only `4bit`/`8bit`/`none` — Quant Menu format awareness deferred; (3) `tcfg.reward_model` is null-byte and length-validated at schema load but not path-containment-checked (`is_under_cwd`) — consistent with how `cfg.base` is treated, both can be HF repo IDs or absolute local paths.
- **v0.40.4 — trust_remote_code multi-trainer + multipack live**: closes the v0.36.0 #63 known gap by extending the `--trust-remote-code` opt-in across every non-SFT trainer wrapper (DPO / GRPO / KTO / ORPO / SimPO / IPO / PPO / RewardModel / Pretrain / Embedding / BCO + the unified `PreferenceTrainerWrapper` dispatcher) and the 5 standalone commands (`soup diff`, `soup export`, `soup merge`, `soup infer`, `soup data generate`). Pattern (15 sites): each `__init__` resolves once via `model_requires_trust_remote_code(config.base) or False` + `resolve_trust_remote_code(...)` and stores `self._trust_remote_code` — every `from_pretrained` call site now reads the resolved value (no remaining `trust_remote_code=True` literal in any trainer file; source-level invariant test in `tests/test_v0404_part_a.py`). `commands/train.py` no longer carries the v0.36.0 `sft_kwargs` split; `trust_remote_code` is part of the unified `trainer_kwargs` dict that flows to every trainer regardless of task. `_load_reward_model` (module-level helper in `ppo.py`) accepts a `trust_remote_code: bool` parameter and resolves internally — design intent is that the helper is independently safe to call from outside `PPOTrainerWrapper`. `PreferenceTrainerWrapper` dispatcher forwards the raw bool to the inner DPO/SimPO/ORPO/IPO/BCO wrapper kwargs at both `_build_inner` and `_build_multi_objective` sites; the resolver fires inside the inner wrapper at construction time. `_export_onnx` / `_export_tensorrt` / `_export_awq` / `_export_gptq` and `_merge_adapter` helpers all gain a `trust_remote_code: bool = False` parameter threaded from the Typer flag. Multipack live HF Trainer wiring (#65) lands via a new `get_train_dataloader` override on `make_multipack_trainer_class` that installs `MultipackBatchSampler(real_batches=False)` (yields flat `list[int]` per pack — DataLoader-compatible) as the DataLoader's `batch_sampler=`. The override forwards `args.dataloader_drop_last` / `dataloader_num_workers` / `dataloader_pin_memory` from `TrainingArguments`. `_get_train_sampler` override stays as a defensive no-op fallback that ALWAYS delegates to super (review-fix: a multipack `list[list[int]]` from this method would cause a shape mismatch if any HF eval / prediction loop bypasses `get_train_dataloader`). The state-presence guard switched from falsy (`not max_seq`) to explicit `is None` + `not lengths` (defensively rejects only-None and empty-list cases — non-positive ints already rejected upstream by `attach_multipack_state`). Falls back to `super().get_train_dataloader()` when state is missing OR when `train_dataset` is unset (defence-in-depth so the subclass remains safe to instantiate). Known limitations: (1) `multipack: true` requires the dataset to expose `input_ids` (preferred) or `length` per row — un-tokenized text-only datasets trigger the v0.40.3 all-zeros WARNING and the `MultipackBatchSampler` will reject the run. (2) The DataLoader override does NOT thread FSDP / DeepSpeed parallelism env hints from `super().get_train_dataloader()`, so distributed `multipack: true` runs are still untested under FSDP / ZeRO; tracked for v0.40.5+ paired with v0.42.0 multi-GPU work. (3) `_live_lr_sweep_from_config` in `commands/train.py` still hardcodes `trust_remote_code=False` for the LR sweep's internal model load — defensive but means `--find-lr` cannot consume custom-code models even with the user opt-in (defence-in-depth, not a bypass). (4) Each non-SFT trainer's `__init__` repeats the resolver block (10 sites) — code-quality refactor candidate (single shared helper) deferred to a future patch to keep the v0.40.4 diff focused on the gap closure.
- **v0.40.3 — Stub-to-live**: New `soup_cli/utils/batch_probe.py:make_cuda_probe_fn` builds a CUDA probe closure that runs ONE forward+backward+step on a synthetic batch per candidate; `model.zero_grad(set_to_none=True)` runs BEFORE forward (defends against the synthetic backward accumulating into the live training model's grad buffers — matches v0.35.0 #45 `benchmark_kernel_combos` policy); intermediate `ids/attn/labels/outputs` are `del`-ed before `loss.backward()` so peak VRAM reflects a realistic training step; `bool` rejected on `batch_size` and `max_length`; `max_length < 8` rejected; `torch.cuda.OutOfMemoryError` returns False, other exceptions propagate; returns `None` (no-op) on non-CUDA / no-torch / missing model or tokenizer. New `soup_cli/utils/multipack_trainer.py:make_multipack_trainer_class` is `lru_cache`d so two calls with the same `base_cls` return the same subclass (consistent `isinstance`, pickle-safe); `attach_multipack_state` rejects `bool` on `max_seq_len`/`batch_size`/`seed` and rejects empty `lengths`; `lengths_from_dataset` logs WARNING when every row produces 0 (loud-fail mirrors v0.37.0 multipack arch allowlist — prevents silent NaN-loss footgun); `_get_train_sampler` override accepts `*args, **kwargs` for HF >= 4.41 signature compat. **Live wiring of the sampler into SFT / Pretrain trainer wrappers is deferred to v0.40.4** — adversarial 5th-pass review surfaced a `Sampler[int]` vs `list[list[int]]` shape mismatch with HF Trainer's DataLoader; the wrappers currently print a yellow advisory and fall back to the standard sampler when `multipack: true`. New `soup_cli/data/traces/quality.py:judge_filter_pairs` reuses v0.19.0 `JudgeEvaluator` SSRF protections; threshold rejects `bool` / NaN / out-of-`[0,1]`; `_MAX_BATCH=100_000` cap applied via lazy `itertools.islice` (never fully materialises a malicious / pathological generator); per-pair backend exceptions caught and logged at DEBUG (matches v0.33.0 #47 `CrossDocCollator` policy — never silently crash the harvest); `judge_provider` validated against `VALID_PROVIDERS` allowlist at the CLI boundary BEFORE constructor, with a Rich-escape error on mismatch. New `soup_cli/monitoring/trace_logger.py:TraceLogWriter` is thread-safe (single `threading.Lock` — multi-worker `--workers 4` documented as a single-process limitation); path containment via shared `is_under_cwd`; null-byte / empty / non-string path rejected; `cap_mb` bounds `[1, 10000]` with explicit `bool` rejection; rotation: when `current + extra > cap_bytes`, rename to `<path>.1` (one backup retained); symlink at the backup path is rejected via `os.lstat + stat.S_ISLNK` (matches v0.33.0 #22 TOCTOU policy) — defends against pre-placed `<log>.1 -> /etc/cron.d/x` overwrite. Secret redaction: prompt + response strings passed through `_SECRET_RE` matching `hf_*` (≥8), `sk-*` (≥16), and `Bearer …` (≥8) — replaces matches with `<redacted>` before serialisation (mirrors v0.34.0 `crash.py` policy). `--trace-log` constructor error messages in `commands/serve.py` are `rich.markup.escape`d before printing so a crafted path name cannot inject Rich markup. Unserialisable entries dropped silently; disk-full / OSError on write never crashes the request handler (passive log). Known limitations: live CUDA probe is wired in SFT only; multipack live wiring covers SFT+Pretrain only; `TraceLogWriter` retains exactly ONE backup file (operators wanting longer retention should use external rotation); custom HF Space templates from v0.40.2 still always create the Space with `space_sdk="gradio"` (tracked for v0.40.4+).
- **v0.40.2 — Quick polish + carry-overs**: New `soup_cli/utils/hf_space.py:render_custom_template_dir` enforces `is_under_cwd` containment on the template directory; `validate_repo_id` runs BEFORE `{MODEL_REPO}` substitution (matches v0.29.0 Part F policy); per-file 256 KB cap (matches v0.39.0 Part E template-size policy); only `app.py` / `README.md` / `requirements.txt` are read (closed allowlist — no path-from-user-data). Symlinks rejected via `os.lstat + stat.S_ISLNK` and non-regular files (FIFO / device) also rejected (matches v0.33.0 #22 prune_checkpoints TOCTOU policy) — defends against `<template_dir>/app.py -> /etc/passwd`. `_find_highest_local_checkpoint` reads `output_dir` after caller's `is_under_cwd` validation (in `prepare_hf_resume`) and silently drops non-directories + OSError. `prepare_hf_resume` skips the snapshot download when local `checkpoint-N >= remote checkpoint-N` (saves bandwidth and never overwrites a fresher local checkpoint). `commands/data.py:register_data` containment switched from `Path.resolve() + relative_to()` to shared `is_under_cwd` (Windows 8.3 short-name safety per CLAUDE.md project rule); same fix applied to `commands/bench.py` prompts-file containment. `commands/infer.py:--output` now containment-checked via `is_under_cwd` (late-evaluated after model+input validation so pre-existing `tmp_path` test contracts keep working). `commands/quickstart.py:--output` validates target dir via `is_under_cwd` before `mkdir(parents=True)`; rejects out-of-cwd targets with friendly message. `commands/runs.py:_filter_runs_by_cwd` uses `os.path.realpath + commonpath`, catches `(ValueError, OSError)` so cross-drive paths on Windows (`D:\runs` vs `C:\project`) drop silently rather than crash. `monitoring/display.py:format_gate_row` uses explicit `task.get("passed") is True` so a missing `"passed"` field renders neutrally instead of as a false-y red ✗. `commands/infer.py:_resolve_model_source` heuristic for HF-id-vs-local-path: only falls through to HF when value is NOT path-like (no `./`, `/`, `\\`, `~`, no Windows drive letter, non-empty); path-like-but-missing raises `FileNotFoundError` so users see actionable errors instead of confusing HF download attempts. Known limitation: custom HF Space templates always create the Space with `space_sdk="gradio"` regardless of the supplied `app.py` (no `--sdk` flag in this release; combine `--template streamlit-chat` with the inline registry for Streamlit Spaces). Tracked for v0.40.3+.
- **v0.40.1 — QA Hardening**: `soup_cli/utils/encoding.force_utf8_stdio` reconfigures Windows stdout/stderr to UTF-8 before any Rich Console is constructed; `os.environ.setdefault("PYTHONIOENCODING", "utf-8")` preserves user override; `(OSError, ValueError, AttributeError)` swallowed on detached streams; POSIX no-op. `SoupConfig._remap_root_level_misplaced_keys` (model_validator, mode='before') migrates root-level `lora:` into `training.lora` so nested validators (including `lora.init_strategy: Literal["random","pissa","olora"]`) actually fire — closes a footgun where the misplaced key was silently dropped. Caller's dict is never mutated (shallow-copy policy mirroring v0.33.0 #47 / v0.40.0 Part B). `PreferenceTrainerWrapper._build_multi_objective` replaces the v0.40.0 `NotImplementedError` stub with a primary-loss approximation; `validate_weight_compat` rejects BCO mixed with paired losses at runtime (data-format incompatible). `combine_losses` rejects empty weights, propagates NaN loudly (no silent zeroing), and rejects `bool` weight values (matches v0.30.0 `Candidate` policy). `_probe_cache_param_count` rejects empty / null-byte model names before path construction (mirrors v0.26.0 registry / v0.39.0 ReLoRAPolicy policy). `commands/doctor` flags `transformers ≥ 5.0.0` as INCOMPATIBLE via `_MAX_EXCLUSIVE` table; `_version_ge` parses leading-int chunks so `5.0.0.dev0` correctly trips the cap. `_detect_gpu_hw_without_torch_cuda` calls `nvidia-smi` via argv list (no shell), 5s timeout, `OSError` / `TimeoutExpired` caught; GPU label from `nvidia-smi` stdout is `rich.markup.escape`d before embedding in Rich-markup string (a real GPU name like `NVIDIA Quadro [T4]` cannot break or inject markup). `_detect_dual_python_interpreters` uses `os.path.realpath` (not `Path.resolve()`) for Windows 8.3 short-name compat. `_pick_quickstart_model` swaps TinyLlama-1.1B → SmolLM2-135M when `total_memory ≤ 6 GB` (prevents step-0 OOM on RTX 3050 4 GB / similar). `_live_lr_sweep_from_config` switched broken `load_local` import to `load_raw_data` (previously always silently fell back to a static placeholder curve). `commands/migrate` rejects `.jsonl` input (with first-line `{` sniff) with exit-2 friendly error; `.jsonl`-only suffix gate prevents false-positives on `.ipynb` notebooks. `commands/eval custom -o` is now honored independently of `--attach-to-registry`; loop-shadow regression where `output = generate_fn(...)` overwrote the CLI option fixed (variable renamed to `response`). `_load_jsonl` switched from `utf-8` to `utf-8-sig` so PowerShell `Out-File -Encoding utf8`-produced JSONL no longer fails first-row parse. Known limitation: `--trust-remote-code` opt-in surface still excludes 10 non-SFT trainers + 5 commands (v0.36.0 #63 carry-over).
- **v0.40.0 — Preference Variety**: New `task='bco'` (Binary Classifier Optimization) and `task='preference'` (unified dispatcher). New schema fields: `bco_beta` (gt=0), `preference_loss: Literal[dpo,simpo,orpo,ipo,bco]|None`, `preference_loss_weights: Optional[Dict[str,float]]`, `dpo_beta_schedule: Literal[linear,cosine,exponential]|None`, `dpo_beta_end: float, gt=0|None`, `dpo_ref_regen_epochs: int [1,1000]|None`. Cross-validators: `_validate_preference_dispatcher` rejects setting either `preference_loss` or `preference_loss_weights` outside `task='preference'` (closes ordering-dependency between Part B/D validators); `_validate_dpo_variants_supported_tasks` gates β-schedule + ref-regen to DPO-family tasks (`dpo`, `ipo`, or `preference` + `preference_loss in {dpo, ipo}`); rejected on mlx backend with distinct error message (matches v0.34.0 distinct-reason policy); `_validate_preference_loss_weights` enforces 2–5 entries (single-entry rejected with actionable message pointing at scalar `preference_loss`), key allowlist `{dpo, simpo, orpo, ipo, bco}`, explicit null-byte rejection on keys (matches v0.39.0 rank_pattern policy), per-value bounds `(0, 1]`, weights must sum to 1.0 (±1e-6), mutually exclusive with scalar `preference_loss`, rejected on mlx backend. `compute_beta_at_step` rejects `bool` on `step` and `total_steps` (project bool-as-int policy from v0.30.0). `BetaScheduleCallback` resolves `total_steps` lazily in `on_train_begin` so the schedule sees the real `state.max_steps` populated by HF Trainer (closes a first-cut silent-no-op bug where total_steps=0 emitted beta_end for every step). `RefModelRegenCallback._regenerate` uses `strict=True` on `load_state_dict` and logs at WARNING on mismatch (closes a first-cut silent partial-copy hazard where strict=False could produce a hybrid old-base + new-LoRA reference); epoch 0 regen suppressed (avoids copying untrained student); trainer `.beta` assignment swallow narrowed to `AttributeError` only. `PreferenceTrainerWrapper._make_inner_cfg` uses `model_copy` (not `model_dump`+`model_validate`) so re-validation never sees an inconsistent intermediate state and the caller's `cfg` is never mutated (mirrors v0.33.0 #47 immutability policy). `_split_dpo_rows_to_bco` skipped-row count emitted at DEBUG so production silent-degradation is inspectable (mirrors v0.33.0 #47 CrossDocCollator policy). Multi-objective live runtime weighted-loss combination is deferred to v0.40.1: `PreferenceTrainerWrapper.setup` raises `NotImplementedError` with a friendly message naming the deferred-version follow-up (mirrors v0.27.0 MII / v0.37.0 multipack / v0.38.0 quant menu / v0.39.0 ReLoRA stub-then-live pattern). Known limitation: `BCOTrainerWrapper._setup_transformers` still hardcodes `trust_remote_code=True` (v0.36.0 #63 known-gap family carry-over across non-SFT trainers).
- **v0.39.0 — LoRA Quality**: `LoraConfig.init_strategy: Literal["random","pissa","olora"]` rejects unknown strategies; PiSSA + DoRA / VeRA combinations rejected at config-load. `model_validator(mode="before")` aligns `use_olora=True` → `init_strategy="olora"` via dict-copy (no caller mutation; matches v0.33.0 #47 immutability policy). `rank_pattern`/`alpha_pattern: Optional[Dict[str, int]]` capped at 256 keys × value (0, 1024], rejects `bool` (subclass of `int` — matches v0.30.0 `Candidate` policy), null bytes in keys, empty keys; cross-validator rejects with `use_vera=True`. `ReLoRAPolicy` is `@dataclass(frozen=True)` (post-construction mutation raises `FrozenInstanceError`); bounds: `steps ∈ [1, 1e7]`, `warmup_ratio ∈ [0, 1]`, `prune_ratio ∈ (0, 1)` (strict — prevents zero-everything footgun). `magnitude_prune_tensor` strict `0 < prune_ratio < 1` rejection, non-Tensor input raises `TypeError`, empty / single-element tensor short-circuits (avoids `kthvalue(_, 0)` runtime crash). `_validate_relora_supported_tasks` cross-validator rejects `relora_steps` with `task != "sft"` and `backend=mlx` with distinct error messages (matches v0.34.0 distinct-reason policy); multi-trainer expansion deferred to v0.39.1. `is_gemma4_model` uses a word-boundary regex (`(?:^|[^a-z0-9])gemma-?4(?:[^a-z0-9]|$)`) so `"ungemma4ed"` / `"my-gemma4ish"` no longer over-match; null-byte rejection on `model_name`. `apply_gemma4_clippable_patch` weight-copy fallback logs at DEBUG instead of silent random-init; the patch is gated by `is_gemma4_model(cfg.base)` in `sft.py` before invocation so non-Gemma4 trainings never traverse the module tree. `apply_surgical_patches` rejects empty / null-byte `model_name` with `ValueError`. `templates/load_template` containment: filename re-validated via `_validate_name` (rejects `..`/`/`/`\\`/null/empty); `os.path.realpath + os.path.commonpath` containment check on the resolved path against `_templates_dir()` so a tampered `manifest.json` cannot read files outside the package directory (mirrors v0.26.0 registry policy). Tampered-manifest `ValueError` from `_validate_name` caught and falls back to inline (no propagating exception). 256 KB file-size cap. Inline `TEMPLATES` carries an explicit deprecation comment pointing at the canonical YAML registry; `tests/test_templates_yaml.py` asserts byte-equality of all 16 inline ↔ YAML pairs to prevent silent drift. Planned removal: v0.41.0+.
- **v0.38.0 — Quant Menu**: `TrainingConfig.quantization` Literal extended with `gptq` / `awq` / `hqq:1bit`..`hqq:8bit` (no `hqq:7bit` — HQQ doesn't support it) / `aqlm` / `eetq` / `mxfp4` / `fp8`; Pydantic rejects every other string at config-load. `validate_gptq_checkpoint` and `validate_awq_checkpoint` probe local paths for `quantize_config.json` / `quant_config.json`; HF repo IDs fall through; null-byte rejection + non-string `TypeError` on the ref. `_validate_prequantized_no_qat` rejects every pre-quantized format combined with `quantization_aware` (int8 QAT or `'fp8'`) — pre-quantized weights carry their own scale and QAT/FP8 prepare would silently corrupt them (mirrors LlamaFactory `quantization.py:117/199/211`). `_validate_bnb_quant_storage_only_with_4bit` rejects `bnb_4bit_quant_storage` on every non-BNB-4bit format (silent no-op otherwise); allowed dtypes: `Literal["uint8", "float16", "bfloat16", "float32"]`. `_validate_quant_menu_supported_tasks` restricts the new formats to `task='sft'` on `backend='transformers'` in v0.38.0 with distinct MLX-backend vs unsupported-task error messages (matches v0.34.0 distinct-reason policy). `check_quant_distributed_compat` hard-fails HQQ/EETQ/AQLM × {FSDP, ZeRO-3} (sourced from LlamaFactory `quantization.py:199/211` plus AQLM dequant constraints); warning-tier (not error) for BNB-4bit + FSDP without `bnb_4bit_quant_storage` so users see the silent perf cliff; unknown `quantization` raises `ValueError` (no silent pass) and the check is wired into `commands/train.py` startup. `parse_hqq_bits` rejects unsupported bit-rates and malformed `hqq:` strings before any kernel build.
- **v0.37.0 — Multipack**: `validate_multipack_architecture` raises `ValueError` on unknown arch (loud-fail vs Axolotl's silent-miss footgun); 18-arch frozen allowlist (Llama 3.x / Qwen 2/3 / Mistral / Gemma 2/3 / Phi 3/4 / DeepSeek V2/V3 / Mixtral / Falcon / StableLM / SmolLM2). FFD packer caps at `_MAX_FFD_ITEMS=1_000_000` (algorithm is O(N²) worst-case — defence against adversarial dataset DoS); `bool` rejection on every numeric input (`max_len`, per-element `lengths`, `batch_max_len`, `batch_size`, `seed`, `max_seq_length`) matches v0.30.0+ `Candidate` policy; generator-input materialisation prevents silent empty-bin output when validation exhausts the iterator. `MultipackBatchSampler` rejects empty `lengths`, non-positive `batch_max_len`/`batch_size`, items larger than `batch_max_len`. `build_multipack_sampler_for_lengths` rejects `tcfg.batch_size="auto"` with actionable message (must be resolved upstream). `_validate_multipack_packing_exclusive` cross-validator on `TrainingConfig` rejects both `multipack` and `packing` set; `_validate_multipack_supported_tasks` on `SoupConfig` restricts multipack to `sft`/`pretrain` on `transformers` backend with distinct error messages for MLX backend vs unsupported task (matches v0.34.0 distinct-reason policy). `build_4d_attention_mask` caps allocations at `_MAX_MASK_ELEMENTS=2³¹` cells (~8GB float32) — defence against `max_length=1M` × `batch_size=8` OOM; `tag_sub_sequences` capped at `_MAX_BOUNDARY_SEGMENTS=1_000_000`. Mask builder rejects non-floating dtypes (was silent `np.finfo` ValueError), non-2D `seq_pos_ids`, negative segment IDs; padding (id=0) tokens are fully masked, including diagonal, so softmax is well-defined. `select_packing_strategy` rejects non-bool `flash_attn_available`. `JinjaTemplateAnalyzer` parses chat templates via `Environment.parse` only — never renders, so a crafted soup.yaml cannot trigger SSRF / filesystem reads; 128KB template cap, null-byte rejection, `TemplateSyntaxError` re-raised as `ValueError`. `DEFAULT_MESSAGE_FIELDS` is a `frozenset` (runtime-immutable); `JinjaTemplateAnalyzer.message_fields` returns a defensive copy.
- **v0.36.0 — Correctness First**: `--trust-remote-code` opt-in replaces 9 unconditional `trust_remote_code=True` call sites across `soup train` / `chat` / `serve` / `data download` / `eval auto`; `KNOWN_SAFE_PREFIXES` allowlist (15 first-party orgs) suppresses warning panel for trusted repos; `model_requires_trust_remote_code` probes local `config.json` for `auto_map` (HF Hub repo IDs return `None`/unknown — HF still raises loudly when custom code is actually needed); `resolve_trust_remote_code` raises `ValueError` with actionable message when model needs custom code but the user did not opt in. Chat-template hardening: `DataConfig.chat_template` validator rejects null bytes, oversize (>64KB), AND filesystem-touching Jinja directives (`{% include %}`, `{% import %}`, `{% from %}`, `{% macro %}`, `{% extends %}` — both whitespace-control variants); empty string normalised to `None`; `_REGISTRY` wrapped in `MappingProxyType` (callers cannot mutate); `apply_chat_template_override` emits yellow advisory when active so users know `soup push` will persist the override into `tokenizer_config.json`. SFT silent f-string fallback (`f"{role}: {content}"`) replaced with hard `ValueError` — produced wrong loss labels for years on tokenizers without a chat template. Loss-mask fallback passes `add_special_tokens=False` to incremental tokenize calls so HF cannot double-prepend BOS at front of each render (consistent prefix-delta walk); narrowed exception catch in `_apply_template_with_mask` from `(TypeError, ValueError)` to `TypeError` only so a malformed messages list propagates instead of falling through to the loose path. `SOUP_BATCH_CACHE_PATH` env override containment-checked via `os.path.realpath + commonpath` against `~`/cwd/`tempfile.gettempdir()`; out-of-bounds values fall through to safe default; cache file gets best-effort `0o600` perms after atomic rename (matches v0.26.0 registry.db policy). `make_cache_key` rejects `bool` in numeric inputs (matches v0.30.0 `Candidate` policy). Documented limitation: non-SFT trainers (DPO/GRPO/KTO/ORPO/SimPO/IPO/PPO/RewardModel/Pretrain/Embedding) and `commands/{diff,export,merge,infer,generate}.py` still hardcode `trust_remote_code=True` — v0.36.x patch follow-up.

## Security Scanning

- All code is scanned with `ruff` for style and common issues
- Dependencies are regularly updated to patch known CVEs
- GitHub's dependency scanning alerts us to vulnerable dependencies
- We use GitHub Actions CI/CD for continuous integration

## Dependency Updates

We actively monitor and update dependencies:

- Major dependency updates: Tested in PR before merging
- Security patches: Applied immediately and released as patch versions
- Deprecated dependencies: Replaced proactively

## Coming Soon

- [x] Automated dependency scanning
- [ ] SBOM (Software Bill of Materials) for each release
- [ ] Third-party security audit (after 1.0.0 release)

## Questions?

If you have security questions (not vulnerability reports) or need clarification:

- Open a GitHub Discussion tagged `security`
- Open a [GitHub Issue](https://github.com/MakazhanAlpamys/Soup/issues) (non-vulnerability inquiries)
- Check our [CONTRIBUTING.md](CONTRIBUTING.md) for general support

## License

This Security Policy is provided under the Apache-2.0 license, same as the Soup project.

---

**Last Updated**: April 2026

For the latest version of this policy, visit: https://github.com/MakazhanAlpamys/Soup/blob/main/SECURITY.md
