# Improvements Log

This document tracks all skill improvements made as a result of testing iterations. Each entry links the improvement back to the test scenario and iteration that identified the need.

---

## Format

Each improvement entry should include:
- **Date**: When the improvement was made
- **Scenario**: Which test scenario identified the need
- **Iteration**: Which iteration discovered the issue
- **Issue**: What problem was observed
- **Improvement**: What was changed in the skills
- **Files Modified**: List of skill files changed

---

## Improvements

#### 2026-04-30: Iteration 001 - E-Commerce Order API (Rust / Axum)

- **Scenario**: ecommerce-order-api
- **Iteration**: 001-rust
- **Result**: ⚠️ PARTIAL — 86/91 tests passed after fixes (94.5%)
- **Score**: 8/10

**Rules Created** 🆕:
- None

**Rules Updated** 🔧:
- None (existing rules were sufficient; code did not follow them)

**Issues Encountered & Resolved**:
1. **Status transition too strict** — 🐛 CONTRACT VIOLATION
   - Problem: Code only allowed `pending → shipped → delivered`; test expects `pending → delivered` directly
   - Impact: 2 test failures (status update + persistence check)
   - Solution: Added `pending → delivered` as valid transition
   - Status: ✅ Fixed

2. **Missing composite indexes** — 📐 UNCLEAR EXISTING RULE
   - Problem: Container created without composite indexes; rule `index-composite` exists but wasn't followed
   - Impact: 1 test failure
   - Solution: Added composite indexes on (status, createdAt) and (customerId, createdAt)
   - Status: ✅ Fixed

3. **Missing type discriminator field** — 📐 UNCLEAR EXISTING RULE
   - Problem: Documents lacked `type` field; rule `model-type-discriminator` exists but wasn't applied
   - Impact: 1 test failure
   - Solution: Added `"type": "order"` to all order documents
   - Status: ✅ Fixed

4. **Missing schema version field** — 📐 UNCLEAR EXISTING RULE
   - Problem: Documents lacked `schemaVersion` field; rule `model-schema-versioning` exists but wasn't applied
   - Impact: 1 test failure
   - Solution: Added `"schemaVersion": "1"` to all order documents
   - Status: ✅ Fixed

**Test Results**:
- ✅ 86 tests passed (API contract, robustness, data integrity, cosmos infrastructure)
- ❌ 5 tests failed before fix (now addressed)

**Best Practices Applied**: 12 of 15 rules applied correctly on first attempt
**Lessons for Next Iteration**: Always include `type` discriminator, `schemaVersion`, and composite indexes from the start. These are fundamental rules that apply to every Cosmos DB container.

#### 2026-04-15: Batch #209 — Multitenant SaaS (Java / Skills Loaded)

- **Scenario**: multitenant-saas
- **Batch issue**: #209
- **Language**: Java (skills loaded)
- **Iterations evaluated**: 5 (PRs #215, #216, #219, #217, #218)
- **Result**: ⚠️ PARTIAL — No consistent failures, but 40% startup failure rate limits confidence
- **Mean score**: 6.4/10 (σ=4.9) — High variance driven by 2 startup failures (0% pass rate) vs. 100% pass rate in successful iterations

**Summary of Aggregate Results**:

| Metric | Value |
|--------|-------|
| 2026-04-18 | iot-device-telemetry | Batch #232 (skills, java) | Aggregated 5 iterations | See batch-results/ |
| Pass Rate (mean) | 60.0% |
| Pass Rate (std dev) | 54.8% |
| Always-fail tests | **0** |
| Always-pass tests | 1 (1%) — `build_startup::build_compilation` |
| Flaky tests | 74 (99%) |
| Startup failures | 2/5 iterations |

**Consistent Failures**: None. Per the batch evaluation recipe, no new rules are required when there are zero always-fail tests.

**Startup Failure Analysis**:

The sole driver of all test failures is startup: when `app_startup` fails (iterations 2 and 4), every subsequent test returns `missing` (0% pass rate). When startup succeeds (iterations 1, 3, 5), all 73 tests pass at 100%. The `app_startup` test shows a 60% pass rate with outcomes: `passed, failed, passed, failed, passed`.

The startup failures are caused by the Java Cosmos DB SDK's SSL/Netty behavior when connecting to the local emulator. The pattern (odd iterations pass, even fail) indicates that 40% of agent-generated implementations use an SSL initialization approach that fails in CI, while 60% use a working approach.

**Classification: SDK/framework quirk (startup-only)** — Not a functional Cosmos DB skill gap. When startup succeeds, skill coverage is perfect across all categories.

**Category Results (successful iterations only)**:

| Category | Pass Rate (when startup works) |
|----------|-------------------------------|
| 2026-04-18 | iot-device-telemetry | Batch #232 (skills, java) | Aggregated 5 iterations | See batch-results/ |
| API Contract | 100.0% |
| Cosmos Infrastructure | 100.0% |
| Data Integrity | 100.0% |

**Issues Encountered**:

1. **Java Cosmos SDK SSL initialization flakiness (40% startup failure rate)** — 🔧 SDK QUIRK
   - Problem: 2/5 agent-generated Java implementations use an SSL bypass that fails at startup (likely `SSLContext.setDefault()` without Netty override)
   - Impact: All tests return `missing` — 0% pass rate for those iterations
   - Classification: SDK/framework quirk — existing `sdk-emulator-ssl.md` lacks clear programmatic Java CI bypass guidance
   - Status: ⚠️ Noted — below the always-fail threshold per batch evaluation recipe; no new rule created at this time

**Rules Created** 🆕: None (no always-fail tests)

**Rules Updated** 🔧: None (no always-fail tests)

**Lessons for Next Iterations**:
1. Java startup reliability needs improvement — the existing `sdk-emulator-ssl.md` rule may need a clearer programmatic Java bypass section for CI environments where filesystem certificate import is not feasible
2. When startup succeeds, Java agents correctly implement all multitenant-saas Cosmos DB patterns (HPK, type discriminators, tenant isolation, composite indexes, analytics)
3. Consider adding more Java iterations to determine if startup can reach consistent 100% with improved rule guidance

**Files Modified**: None (no rule changes required for zero always-fail tests)

---

#### 2026-04-02: Batch #191 — Gaming Leaderboard (Python / Skills Loaded)

- **Scenario**: gaming-leaderboard
- **Batch issue**: #191
- **Language**: Python (skills loaded)
- **Iterations evaluated**: 5 (PRs #197, #198, #199, #200, #201)
- **Result**: ⚠️ PARTIAL — No consistent failures, but near-consistent cascade delete/update failures reveal a rule gap
- **Mean score**: 7.6/10 (σ=3.7) — High stochasticity driven by iteration 2's catastrophic 25.5% pass rate vs. 90–100% in others

**Summary of Aggregate Results**:

| Metric | Value |
|--------|-------|
| 2026-04-18 | iot-device-telemetry | Batch #232 (skills, java) | Aggregated 5 iterations | See batch-results/ |
| Pass Rate (mean) | 81.3% |
| Pass Rate (std dev) | 31.4% |
| Always-fail tests | **0** |
| Always-pass tests | 26 (27%) |
| Flaky tests | 70 (73%) |

**Consistent Failures**: None. Per the batch evaluation recipe, no new rules are strictly required when there are zero always-fail tests. However, two near-consistent failure patterns at 20% pass rate revealed a genuine gap in the existing rule set.

**Near-Consistent Failure Analysis** (lowest pass-rate flaky tests):

| Test | Pass Rate | Pattern |
|------|-----------|---------|
| 2026-04-18 | iot-device-telemetry | Batch #232 (skills, java) | Aggregated 5 iterations | See batch-results/ |
| `TestUpdateDeleteConsistency::test_deleted_player_removed_from_leaderboard` | 20% | `failed, error, failed, failed, passed` |
| `TestUpdateDeleteConsistency::test_deleted_player_scores_not_in_history` | 20% | `failed, error, failed, failed, passed` |
| `TestUpdateDeleteConsistency::test_updated_region_reflected_in_regional_leaderboard` | 40% | `passed, error, failed, failed, passed` |

**Root Cause Analysis**:

- `test_deleted_player_removed_from_leaderboard` and `test_deleted_player_scores_not_in_history` (20%): Agents correctly implement the DELETE endpoint returning 204, but fail to cascade-delete related documents in other containers (score history documents, leaderboard entry documents). The player is deleted but "ghost" entries remain in derived containers, causing leaderboards to still show deleted players and score history to return 200 instead of 404.

- `test_updated_region_reflected_in_regional_leaderboard` (40%): Agents update the player document's region field but fail to cascade-update leaderboard entries. The old regional leaderboard entry (with the previous region as the partition key) is not removed, and no entry is created in the new region's partition.

**Classification**: **Unclear existing rule** — `model-denormalize-reads.md` discusses keeping denormalized data updated when source changes (e.g., "When category changes, update products using Change Feed"), but has **zero guidance on cascade deletes** when the source document is deleted, and no guidance on fields that change the partition key of derived documents.

**Rules Updated** 🔧:

1. **`model-denormalize-reads.md`** — Added "Cascade Delete and Update of Denormalized Documents" section (HIGH impact)
   - Explicitly documents that deleting a source document requires deleting all derived/related documents in all containers
   - Documents that updating a field used as partition key in derived documents requires delete-and-recreate (not just update)
   - Added Python and C# code examples for correct cascade delete and cascade update patterns
   - Added checklist: "Every DELETE endpoint must cascade to all containers holding derived data"

**Issues Encountered**:

1. **Cascade delete of derived documents not implemented** — ⚠️ NEAR-CONSISTENT (80% fail rate excluding error iteration)
   - Problem: Agents implement DELETE returning 204 but orphan score and leaderboard documents in other containers
   - Impact: Deleted players appear in leaderboards; score history returns 200 instead of 404
   - Classification: Unclear existing rule (denormalize rule existed but omitted cascade delete guidance)
   - Solution: Updated `model-denormalize-reads.md` with explicit cascade delete pattern

2. **Cascade update of partition-key field not implemented** — ⚠️ NEAR-CONSISTENT (60% fail rate excluding error iteration)
   - Problem: Agents update player region but don't move leaderboard entries between regional partitions
   - Impact: Player appears in wrong regional leaderboard after region change
   - Classification: Unclear existing rule (same gap as above)
   - Solution: Same rule update covers this case (delete-and-recreate in new partition)

3. **High iteration-to-iteration variance** — σ=31.4% — considered too high for confident assessment
   - Iteration 2 scored only 25.5% (1/10) while iterations 3–5 scored 92–100%
   - All 70 "flaky" tests show `error` in iteration 2, suggesting that iteration had a structural implementation failure unrelated to Cosmos DB skill gaps
   - Recommendation: Run additional iterations to reduce variance and confirm the cascade patterns

**Files Modified**:
- ✅ `skills/cosmosdb-best-practices/rules/model-denormalize-reads.md` — UPDATED (added cascade delete/update section)
- ✅ `skills/cosmosdb-best-practices/AGENTS.md` — Recompiled (81 total rules)

---

#### 2026-03-12: New Rules — Parameterized TOP and Composite Index Directions

- **Scenario**: gaming-leaderboard
- **Iteration**: iteration-001-python (testing-v2 framework, PR #4)
- **Issue 1**: Copilot generated `SELECT TOP @top ...` with `@top` as a query parameter. Cosmos DB's `TOP` keyword requires a literal integer — parameterized values cause a 400 Bad Request at runtime. All 10 leaderboard endpoint tests returned HTTP 500 due to this.
- **Issue 2**: Copilot created a composite index with `(/bestScore, descending)` but the rank-neighbor query used `ORDER BY c.bestScore ASC`. Cross-partition `ORDER BY` requires an index with a matching sort direction. All 6 rank endpoint tests returned HTTP 500.
- **Improvement**: Created two new rules:
  1. `query-top-literal.md` — "Use Literal Integers for TOP, Never Parameters". Shows that `TOP` does not support `@param` syntax and the value must be interpolated as a validated `int`.
  2. `index-composite-direction.md` — "Composite Index Directions Must Match ORDER BY". Stresses that every composite index must have directions that exactly match the query's `ORDER BY`, and that both ASC and DESC variants should always be defined.
- **Files Modified**:
  - `skills/cosmosdb-best-practices/rules/query-top-literal.md` (NEW)
  - `skills/cosmosdb-best-practices/rules/index-composite-direction.md` (NEW)
  - `skills/cosmosdb-best-practices/AGENTS.md` (recompiled)
- **Pass Rate**: 52.9% (18/34) — 18 passed (health, player CRUD, score submission), 16 failed (leaderboard + rank queries hit the two issues above)

#### 2026-03-11: New Rule — Python Async SDK Missing aiohttp Dependency

- **Scenario**: gaming-leaderboard
- **Iteration**: iteration-001-python (testing-v2 framework, PR #2)
- **Issue**: Copilot generated code using `from azure.cosmos.aio import CosmosClient` but did not include `aiohttp` in `requirements.txt`. The `azure-cosmos` package does not automatically install `aiohttp` — it's an optional dependency for async operations. This caused `ModuleNotFoundError: No module named 'aiohttp'` at startup, preventing the app from running and all tests from executing.
- **Improvement**: Created new rule `sdk-python-async-deps.md` (Rule 4.15) — "Include aiohttp When Using Python Async SDK". Shows correct and incorrect requirements.txt examples, with the exact error message reproduced.
- **Files Modified**:
  - `skills/cosmosdb-best-practices/rules/sdk-python-async-deps.md` (NEW)
  - `skills/cosmosdb-best-practices/AGENTS.md` (recompiled)
- **Also**: Added "Step 8: Verify dependencies" to issue template instructions, and a Python import verification step to the CI workflow, to catch missing dependencies before startup.

#### 2026-03-02: Iteration 003 - Gaming Leaderboard Scenario (Python / FastAPI)

- **Scenario**: gaming-leaderboard
- **Iteration**: 003-python
- **Result**: ✅ SUCCESSFUL - All endpoints functional, all tests passed
- **Score**: 9/10
- **Key Achievement**: Highest score yet. Skills feedback loop validated — COUNT-based ranking (Rule 9.2) and ETag concurrency (Rule 4.7) from iteration-001 applied correctly from the start. Found and fixed a bug in the skill's own Python ETag example.

**Rules Fixed** 🔧:

1. **sdk-etag-concurrency.md** — FIXED Python example (HIGH)
   - The existing Python code example used `match_condition=etag` (passing the ETag string value)
   - Python SDK requires `match_condition=MatchConditions.IfNotModified` from `azure.core`
   - The string approach raises `TypeError: Invalid match condition`
   - Added explicit warning about the `MatchConditions` enum requirement
   - Added proper import: `from azure.core import MatchConditions`

**Issues Encountered & Resolved**:

1. **Python SDK ETag API Incorrect in Skill** — 🐛 BUG IN SKILL → FIXED
   - Problem: Rule 4.7's Python example used `match_condition=etag` (string), but SDK requires `MatchConditions.IfNotModified` enum
   - Impact: Any agent following the Python ETag example would get a runtime TypeError
   - Solution: Fixed the example in `sdk-etag-concurrency.md` with correct import and enum usage
   - Status: ✅ Fixed and recompiled

2. **Pydantic Model Consistency** — ⚠️ CODE QUALITY (not skill issue)
   - Problem: Response model field names didn't match constructor arguments in main.py
   - Impact: 500 errors on API responses
   - Root cause: Agent-generated models and routes had naming inconsistencies
   - Status: ✅ Fixed during development

3. **OFFSET/LIMIT Usage** — ⚠️ MINOR
   - Problem: Some queries use `OFFSET 0 LIMIT @limit` for bounded results
   - Impact: Minimal — always single-partition, small limits (≤100)
   - Status: ⚠️ Acceptable for this use case but noted

**Test Results**:
- ✅ POST /scores — Score submission with ETag concurrency, 4 leaderboard entries
- ✅ GET /leaderboards/global?period=all-time — Correct rank order
- ✅ GET /leaderboards/global?period=weekly — Weekly partition key
- ✅ GET /leaderboards/regional/US?period=all-time — Country-filtered
- ✅ GET /players/{id}/rank — COUNT-based ranking + nearby ±10
- ✅ GET /players/{id} — Player profile with aggregated stats
- ✅ 404 for non-existent players
- ✅ 404 for non-existent player rank
- ✅ Higher score correctly re-ranks player (Alice 1500→3000, moved to #1)

**Best Practices Applied Successfully**:
1. ✅ **Materialized Views** (Rule 9.1) — Leaderboard container as denormalized view
2. ✅ **COUNT-based Ranking** (Rule 9.2) — From iteration-001 feedback, applied from start
3. ✅ **ETag Concurrency** (Rule 4.7) — With retry loop, from iteration-001 feedback
4. ✅ **Synthetic Partition Keys** (Rule 2.7) — `"global_2026-W10"`, `"US_all-time"`
5. ✅ **Composite Index** (Rule 5.1) — `(bestScore DESC, lastUpdatedAt ASC)`
6. ✅ **Exclude Unused Index Paths** (Rule 5.2) — Custom policies on all 3 containers
7. ✅ **Singleton CosmosClient** (Rule 4.17) — Created once in lifespan
8. ✅ **Parameterized Queries** (Rule 3.5) — All queries use parameters
9. ✅ **Field Projections** (Rule 3.6) — Specific fields, not SELECT *
10. ✅ **Single-Partition Queries** (Rule 3.1) — All queries target one partition
11. ✅ **Type Discriminators** (Rule 1.11) — "player", "score", "leaderboardEntry"
12. ✅ **Schema Versioning** (Rule 1.10) — `schemaVersion: 1` on all documents
13. ✅ **Denormalized Reads** (Rule 1.2) — Player info embedded in leaderboard entries
14. ✅ **load_dotenv(override=True)** (Rule 4.12) — Correct local dev config
15. ✅ **SSL Disabled for Emulator** (Rule 4.6) — Conditional on endpoint detection

**Best Practices NOT Applied**:
- ❌ Preferred regions, availability strategy, circuit breaker (production only)
- ❌ Rich diagnostics logging (Python SDK has limited diagnostics compared to .NET/Java)
- ❌ Throughput configuration (autoscale/provisioned)

**Score Improvement Trend**:
| Iteration | Language | Score | Key Issues |
|-----------|----------|-------|------------|
| 2026-04-18 | iot-device-telemetry | Batch #232 (skills, java) | Aggregated 5 iterations | See batch-results/ |
| 2026-04-01 | gaming-leaderboard | Batch #191 (skills, python) | Aggregated 5 iterations | See batch-results/ |
| 001 | .NET | 7/10 | O(N) ranking, missing ETag |
| 002 | Java | 7/10 | OFFSET/LIMIT, API build errors |
| 003 | Python | 9/10 | Only skill bug (fixed), model consistency |

**Lessons Learned**:
1. **Skills feedback loop works** — Rules from iter-001 (ranking, ETag) applied correctly in iter-003
2. **Python SDK has different API surface** — `MatchConditions` enum vs string/option patterns in .NET/Java
3. **Skill examples must be tested** — The Python ETag example was wrong since rule creation
4. **Score trend confirms improvement** — 7/10 → 7/10 → 9/10 shows skill refinement over iterations

**FILES MODIFIED**:
- 🔧 `skills/cosmosdb-best-practices/rules/sdk-etag-concurrency.md` — FIXED Python example
- 🔧 `skills/cosmosdb-best-practices/AGENTS.md` — Recompiled (67 total rules)
- ✅ `testing/scenarios/gaming-leaderboard/iterations/iteration-003-python/ITERATION.md` — NEW

---

#### 2026-02-17: Iteration 001 - Gaming Leaderboard Scenario (.NET / ASP.NET Core)

- **Scenario**: gaming-leaderboard
- **Iteration**: 001-dotnet
- **Result**: ✅ SUCCESSFUL - All endpoints functional, data persists correctly
- **Score**: 7/10
- **Key Achievement**: Identified two significant gaps — O(N) rank lookups and missing ETag concurrency — and created new rules for both

**New Rules Created** ⭐:

1. **pattern-efficient-ranking.md** (HIGH)
   - Documents count-based, cached rank, and score bucket approaches for efficient ranking
   - Prevents O(N) full partition scan anti-pattern found in `GetPlayerRankAsync`
   - Covers three solutions: COUNT queries, Change Feed pre-computed ranks, score buckets
   - Applicable to all leaderboard/ranking scenarios across languages

2. **sdk-etag-concurrency.md** (HIGH)
   - Documents ETag-based optimistic concurrency for read-modify-write operations
   - Player stat aggregation had race condition: concurrent score submissions cause lost updates
   - Includes .NET, Java, and Python examples with retry logic
   - Covers when to use vs. skip ETag checks

**Issues Encountered & Resolved**:

1. **O(N) Rank Lookup** — ⚠️ DESIGN ISSUE → RULE CREATED
   - Problem: `GetPlayerRankAsync` reads ALL entries in a leaderboard partition to find one player's rank
   - Impact: At 500K players, this consumes thousands of RU and takes seconds
   - Solution: Created `pattern-efficient-ranking.md` with COUNT-based approach
   - Status: ✅ Rule created to prevent this in future iterations

2. **Missing Optimistic Concurrency** — ⚠️ DESIGN ISSUE → RULE CREATED
   - Problem: `Player._etag` exists but is never used in `UpdatePlayerStatsAsync`
   - Impact: Concurrent score submissions can overwrite each other's stat updates
   - Solution: Created `sdk-etag-concurrency.md` with retry pattern
   - Status: ✅ Rule created to prevent this in future iterations

3. **OFFSET/LIMIT Instead of Continuation Tokens** — ⚠️ PARTIAL
   - Problem: Used `OFFSET 0 LIMIT @limit` instead of continuation tokens
   - Impact: RU cost increases with page depth
   - Status: ⚠️ Existing `query-pagination.md` rule should be strengthened with anti-pattern warning

**Test Results**:
- ✅ POST /api/scores — Score submission with player/leaderboard updates
- ✅ GET /api/leaderboard/global — Weekly global top N
- ✅ GET /api/leaderboard/global/all-time — All-time global rankings
- ✅ GET /api/leaderboard/regional/{country} — Regional weekly rankings
- ✅ GET /api/leaderboard/regional/{country}/all-time — Regional all-time rankings
- ✅ GET /api/leaderboard/player/{playerId} — Player rank + nearby players
- ✅ GET /api/players/{playerId} — Player profile with cumulative stats
- ✅ 404 for non-existent players
- ✅ 400 for missing required fields
- ✅ Empty results for countries with no data

**Best Practices Applied Successfully**:
1. ✅ **Materialized Views** — Leaderboard container as denormalized view (excellent)
2. ✅ **Synthetic Partition Keys** — `leaderboardKey` = `"global_2026-W07"`, `"US_all-time"`
3. ✅ **Singleton CosmosClient** — DI registration, Direct mode for production
4. ✅ **Composite Index** — `(bestScore DESC, lastUpdatedAt ASC)` on leaderboards
5. ✅ **Enum Serialization** — `JsonStringEnumConverter` with System.Text.Json
6. ✅ **Parameterized Queries** — All queries use `QueryDefinition.WithParameter`
7. ✅ **Projections** — Specific field selection instead of `SELECT *`
8. ✅ **Single-Partition Queries** — All queries target a single partition key
9. ✅ **Type Discriminators** — `"player"`, `"score"`, `"leaderboardEntry"` type fields
10. ✅ **Denormalized Reads** — Player stats embedded, leaderboard entries denormalized

**Best Practices NOT Applied**:
- ❌ Preferred regions, availability strategy, circuit breaker
- ❌ Diagnostics logging (only Debug-level RU logging)
- ❌ Azure Monitor / Application Insights integration
- ❌ Throughput configuration (autoscale/provisioned)
- ❌ Custom indexing policies on players/scores containers
- ❌ Schema versioning

**Lessons Learned**:
1. **Rank computation is a non-trivial problem** — Full partition scans are a natural but incorrect first approach
2. **Skills effectiveness is visible** — Materialized views, composite indexes, enum serialization all applied correctly because skills were loaded
3. **Production hardening gap** — Skills cover these topics but agent prioritized functionality over operational readiness
4. **ETag concurrency is commonly overlooked** — Read-modify-write patterns need explicit guidance

**FILES MODIFIED**:
- ✅ `skills/cosmosdb-best-practices/rules/pattern-efficient-ranking.md` — NEW (HIGH)
- ✅ `skills/cosmosdb-best-practices/rules/sdk-etag-concurrency.md` — NEW (HIGH)
- ✅ `skills/cosmosdb-best-practices/AGENTS.md` — Recompiled (57 total rules, up from 55)
- ✅ `testing/scenarios/gaming-leaderboard/iterations/iteration-001-dotnet/ITERATION.md` — NEW

---

#### 2026-02-17: Iteration 002 - Gaming Leaderboard Scenario (Java / Spring Boot 3)

- **Scenario**: gaming-leaderboard
- **Iteration**: 002-java
- **Result**: ✅ SUCCESSFUL - All endpoints functional, data persists correctly
- **Score**: 7/10
- **Key Achievement**: Skills feedback loop validated — COUNT-based ranking (Rule 9.2) from iteration-001 applied correctly. Schema versioning (Rule 1.5) also added. Java/Spring Data Cosmos API knowledge gaps identified.

**New Rules Created**: None (iteration-001's new rules were sufficient)

**Recommended Rule Updates** 📝:

1. **sdk-emulator-ssl.md** — UPDATE RECOMMENDED (HIGH)
   - `COSMOS.EMULATOR_SSL_TRUST_ALL` system property does NOT work with Java's Netty-based Cosmos SDK
   - Netty uses OpenSSL, bypassing Java's `SSLContext` entirely
   - Correct approach: extract emulator certificate, import into JDK truststore or custom truststore
   - Run with `-Djavax.net.ssl.trustStore=<path> -Djavax.net.ssl.trustStorePassword=changeit`

2. **query-pagination.md** — STRENGTHEN RECOMMENDED (MEDIUM)
   - Same OFFSET/LIMIT gap as iteration-001
   - Add explicit anti-pattern warning: "OFFSET/LIMIT RU cost scales linearly with offset depth"

**Issues Encountered & Resolved**:

1. **Wrong `CosmosTemplate.runQuery()` Signature** — ❌ BUILD ERROR → FIXED
   - Problem: Agent used `runQuery(SqlQuerySpec, Class<T>, String containerName)` — container name as 3rd arg
   - Correct: `runQuery(SqlQuerySpec, Class<?> domainType, Class<T> returnType)` — container inferred from `@Container` annotation
   - Impact: 3 compilation errors in LeaderboardService.java
   - Status: ✅ Fixed manually. SDK API documentation/skills gap identified.

2. **`Iterable` vs Stream API** — ❌ BUILD ERROR → FIXED
   - Problem: `CosmosTemplate.runQuery()` returns `Iterable<T>`, agent called `.stream()` on it
   - Correct: `StreamSupport.stream(iterable.spliterator(), false).toList()`
   - Status: ✅ Fixed. Common Java pattern not documented in skills.

3. **Emulator SSL with Netty** — ❌ RUNTIME ERROR → FIXED
   - Problem: `COSMOS.EMULATOR_SSL_TRUST_ALL` doesn't work with Netty/OpenSSL
   - Solution: Extracted emulator cert via PowerShell TcpClient, imported into custom truststore
   - Run JAR with `-Djavax.net.ssl.trustStore` and `-Djavax.net.ssl.trustStorePassword`
   - Status: ✅ Fixed. Rule update recommended for `sdk-emulator-ssl.md`.

4. **Validation Error Returns 500** — 🐛 BUG FOUND (NOT FIXED)
   - Problem: `GlobalExceptionHandler` handles `IllegalArgumentException` → 400, but Spring's `MethodArgumentNotValidException` falls to generic `Exception` → 500
   - Impact: POST /api/players with empty body returns HTTP 500
   - Status: ⚠️ Not fixed (documentation/testing only). Spring Boot-specific knowledge gap.

5. **`partitionKey` Null in JSON** — 🐛 BUG FOUND (NOT FIXED)
   - Problem: LeaderboardEntry `partitionKey` field serializes as `null` in REST responses
   - Impact: Cosmetic — queries work correctly via Cosmos DB partition key routing
   - Status: ⚠️ Likely Jackson/Spring Data Cosmos annotation interaction issue

**Comparison with Iteration 001 (.NET)**:

| Aspect | Iter-001 (.NET) | Iter-002 (Java) | Delta |
|--------|----------------|----------------|-------|
| 2026-04-18 | iot-device-telemetry | Batch #232 (skills, java) | Aggregated 5 iterations | See batch-results/ |
| 2026-04-01 | gaming-leaderboard | Batch #191 (skills, python) | Aggregated 5 iterations | See batch-results/ |
| Rank computation | ❌ O(N) scan | ✅ COUNT-based | ✅ Improved |
| Schema versioning | ❌ Missing | ✅ Applied | ✅ Improved |
| Build success | ✅ First try | ❌ 3 fixes needed | ❌ Regression |
| Validation errors | ✅ Returns 400 | ❌ Returns 500 | ❌ Regression |
| Composite index | ✅ Declared | ❌ Not available | ❌ Spring Data limitation |
| SSL handling | ✅ Easy callback | ❌ Truststore import | ❌ Harder |
| Error handling | ❌ No global handler | ✅ GlobalExceptionHandler | ✅ Improved |

**Skills Feedback Loop Validation** ✅:
- `pattern-efficient-ranking.md` (created in iter-001) → Applied correctly as COUNT-based ranking
- `sdk-etag-concurrency.md` (created in iter-001) → Partially applied (Spring Data implicit ETags)
- Rules created from previous iterations ARE improving subsequent iterations

**Best Practices Applied**:
- ✅ Materialized views (Rule 9.1), COUNT-based ranking (Rule 9.2)
- ✅ Synthetic partition keys (Rule 2.6), high cardinality keys (Rule 2.4)
- ✅ Singleton client (Rule 4.16), Gateway/Direct auto-detect (Rule 4.6)
- ✅ Session consistency (Rule 7.2), query metrics (Rule 8.4)
- ✅ Parameterized queries (Rule 3.5), projections (Rule 3.6)
- ✅ Type discriminators (Rule 1.6), schema versioning (Rule 1.5)
- ✅ contentResponseOnWriteEnabled (Rule 4.9)
- ❌ Preferred regions, availability strategy, circuit breaker, diagnostics, custom indexing

**FILES MODIFIED**:
- ✅ `testing/scenarios/gaming-leaderboard/iterations/iteration-002-java/ITERATION.md` — NEW

---

#### 2026-02-02: Iteration 001 - Multi-Tenant SaaS Scenario (.NET / ASP.NET Core)

- **Scenario**: multitenant-saas
- **Iteration**: 001-dotnet
- **Result**: ✅ **BUILD SUCCESSFUL** / ⚠️ **PACKAGING ERROR**
- **Score**: 7/10
- **Key Achievement**: Created valid Newtonsoft.Json dependency rule based on Microsoft docs

**New Rules Created** ⭐:

1. **sdk-newtonsoft-dependency.md** (MEDIUM)
   - Documents explicit Newtonsoft.Json >= 13.0.3 requirement
   - Covers security vulnerabilities in 10.x versions
   - Explains requirement even when using System.Text.Json
   - Provides version conflict troubleshooting
   - Based on Microsoft Learn official documentation section

**Issues Encountered**:

1. **Type Name Conflict** (Build Error) - ✅ RESOLVED
   - Problem: `User` class conflicts with `Microsoft.Azure.Cosmos.User`
   - Solution: Type alias `using CosmosUser = MultiTenantSaas.Models.User;`
   - Lesson: Avoid common SDK class names

2. **Missing Newtonsoft.Json** (Build Error) - ✅ RESOLVED → RULE CREATED
   - Problem: SDK requires explicit Newtonsoft.Json >= 13.0.3
   - Solution: `dotnet add package Newtonsoft.Json`
   - **Validation**: Microsoft docs have entire section "Managing Newtonsoft.Json Dependencies"
   - **Merit**: Documented pain point, security implications, non-obvious requirement
   - **Action**: Created comprehensive rule with troubleshooting

3. **Packaging Error** (Agent Error) - ✅ IDENTIFIED
   - Problem: Created zip with files at wrong directory levels
   - Result: Duplicate Program.cs files when extracted
   - Error: `error CS8802: Only one compilation unit can have top-level statements`
   - Impact: Prevented proper archival and testing

**Agent Methodology Issues** (Learning Experience):

1. ❌ **Premature Diagnosis**
   - Initially assumed "emulator SSL issue" without evidence
   - Created incorrect rule, then removed after user review
   - Should have: Extracted zip, read compiler error, diagnosed properly

2. ❌ **Poor Packaging**
   - Created source-code.zip with incorrect directory structure
   - Didn't verify extraction and build process
   - Prevented endpoint testing

3. ✅ **Corrective Actions**
   - Removed incorrect emulator SSL rule
   - Researched Microsoft docs to validate Newtonsoft.Json issue
   - Created proper rule with security guidance

**Best Practices Applied Successfully**:

1. ✅ **Hierarchical Partition Keys** - `[/tenantId, /projectId]` (EXCELLENT design)
   - Perfect for multi-tenant isolation
   - Overcomes 20GB limit per tenant
   - Enables efficient project-scoped queries

2. ✅ **Singleton CosmosClient** - DI registration with Direct mode
3. ✅ **Parameterized Queries** - All repositories use QueryDefinition
4. ✅ **Type Discriminators** - Polymorphic data in shared container
5. ✅ **Embedded Data** - Comments embedded in Tasks

**Files Modified**:
- **New Rule**: `skills/cosmosdb-best-practices/rules/sdk-newtonsoft-dependency.md`
- **Regenerated**: `skills/cosmosdb-best-practices/AGENTS.md` (55 rules, up from 54)
- **Documented**: `testing/scenarios/multitenant-saas/iterations/iteration-001-dotnet/ITERATION.md`

**Key Lessons**:

1. **Validate with official docs** - Microsoft Learn confirmed Newtonsoft.Json is a real issue
2. **Test packaging thoroughly** - Always extract and build from zip before archiving
3. **Read error messages** - Compiler errors tell you exactly what's wrong
4. **Don't assume root causes** - Evidence-based diagnosis prevents wasted effort
5. **User review is valuable** - Caught the premature conclusion about SSL

**Positive Outcomes**:

- Discovered and documented legitimate Newtonsoft.Json dependency rule
- Excellent hierarchical partition key design for multi-tenancy
- Demonstrated proper Cosmos DB best practices in code
- Learned better packaging and testing methodology

---

#### 2026-01-29: Iteration 001 - AI Chat/RAG Scenario (.NET / ASP.NET Core)

- **Scenario**: ai-chat-rag
- **Iteration**: 001-dotnet
- **Result**: ✅ **SUCCESSFUL** - Vector search implemented with emulator, containers created successfully
- **Score**: 8/10
- **Key Achievement**: **IDENTIFIED CRITICAL GAP** - Zero vector search rules in skills, created 4 new comprehensive rules
- **Rules Applied**: VectorEmbeddingPolicy, VectorIndexes, VectorDistance queries, autoscale, singleton client

**Critical Discovery**:

**ZERO Vector Search Rules Existed**
- Problem: Agent initially used wrong SDK version (3.44.0-preview.0) lacking VectorIndexes support
- Root Cause: No vector search rules existed in AGENTS.md to guide implementation
- Impact: Agent made incorrect assumptions about SDK capabilities
- Documentation: User provided official Microsoft Learn links proving SDK 3.45.0+ supports vector search
- Outcome: Created comprehensive vector search rules covering all languages

**New Rules Created** (All with examples in .NET, Python, JavaScript, Java):

1. **vector-enable-feature.md** (CRITICAL)
   - How to enable vector search feature on account via Portal or Azure CLI
   - Requirement to wait 15 minutes for feature activation
   - SDK version requirements per language
   
2. **vector-embedding-policy.md** (CRITICAL)
   - VectorEmbeddingPolicy configuration (path, dataType, dimensions, distanceFunction)
   - Cannot be modified after container creation
   - Examples: cosine, dotProduct, euclidean distance functions

3. **vector-index-type.md** (CRITICAL)
   - VectorIndexes configuration (QuantizedFlat vs DiskANN)
   - **CRITICAL**: Must exclude vector paths from regular indexing (ExcludedPaths)
   - Index type selection guide (QuantizedFlat < 50K vectors, DiskANN for larger)
   
4. **vector-distance-query.md** (HIGH)
   - VectorDistance() system function usage for similarity search
   - Parameterization best practices (query plan caching)
   - ORDER BY VectorDistance() for ranking results
   - Hybrid search patterns (vector + filters)

**Issues Encountered & Resolved**:

1. **Wrong SDK Version** (Initial Implementation)
   - Problem: Used Microsoft.Azure.Cosmos 3.44.0-preview.0 which lacks VectorIndexes property
   - Error: Agent assumed SDK didn't support VectorIndexes
   - Solution: Updated to SDK 3.45.0 (release version) with full vector search support
   - Status: ✅ RESOLVED - proper SDK version documented in rules

2. **Missing ExcludedPaths for Vectors**
   - Problem: Initially didn't exclude vector paths from regular indexing
   - Impact: Would cause high RU consumption and latency on vector inserts
   - Solution: Added vector paths to ExcludedPaths in indexing policy
   - Status: ✅ RESOLVED - now documented as CRITICAL in vector-index-type.md

3. **Configuration Parsing Issue** (Azure vs Emulator)
   - Problem: `configuration["CosmosDb:UseKey"]` string comparison failed
   - Error: Always evaluated to false, used DefaultAzureCredential even with UseKey=true
   - Solution: Changed to `configuration.GetValue<bool>("CosmosDb:UseKey", false)`
   - Status: ✅ RESOLVED - proper boolean parsing

4. **Azure Cloud Authentication Issues**
   - Problem: DefaultAzureCredential authentication hung during container initialization
   - Observation: Worked previously, then failed silently with no error logs
   - Workaround: Reverted to emulator for testing
   - Status: ⚠️ UNRESOLVED - Azure authentication intermittent, needs investigation

**Test Results**:

✅ Database `ai-chat-rag-db` created in emulator
✅ Container `sessions` created with partition key `/userId` (embedded messages pattern)
✅ Container `documents` created with partition key `/category` and vector search:
  - VectorEmbeddingPolicy: 1536 dimensions, Cosine distance
  - VectorIndexes: QuantizedFlat type
  - ExcludedPaths: `/embedding/*` (optimized for inserts)
✅ Application started successfully on http://localhost:5054
✅ REST API endpoints tested:
  - POST /api/chat/sessions - Created session successfully
  - Session ID returned: deeb93bc-0063-44c8-8cd5-872ed245ed55
✅ Vector search configuration validated

**Lessons Learned**:

1. **CRITICAL**: Agent kits MUST have comprehensive coverage of all major features (vector search was completely missing)
2. Always verify SDK versions in documentation - preview versions may lack features
3. Official Microsoft documentation is authoritative - trust it over assumptions
4. ExcludedPaths for vector properties is critical for performance (not optional)
5. Configuration parsing in .NET requires proper type conversion (GetValue<bool>)
6. Testing iterations reveal real gaps that wouldn't be found through code review alone

**Rule Enhancement Impact**:

From **0 vector search rules** to **4 comprehensive rules** covering:
- Feature enablement and prerequisites
- Embedding policy configuration  
- Vector index types and performance optimization
- Query patterns and best practices
- All with multi-language examples (.NET, Python, JavaScript, Java)

**FILES MODIFIED**:
- ✅ `rules/_sections.md` - Added Section 10: Vector Search
- ✅ `rules/vector-enable-feature.md` - NEW (CRITICAL)
- ✅ `rules/vector-embedding-policy.md` - NEW (CRITICAL)
- ✅ `rules/vector-index-type.md` - NEW (CRITICAL)
- ✅ `rules/vector-distance-query.md` - NEW (HIGH)
- ✅ `AGENTS.md` - Recompiled with 54 total rules (was 50)

**Priority for Future Testing**:
- HIGH: Test vector search with other languages (Python, Java, JavaScript)
- HIGH: Investigate Azure DefaultAzureCredential intermittent failures
- MEDIUM: Add vector search to other scenarios (multitenant-saas, etc.)
- MEDIUM: Create rule for vector index performance tuning

---

#### 2026-01-29: Iteration 003 - IoT Telemetry Scenario (Python / FastAPI)

- **Scenario**: iot-device-telemetry
- **Iteration**: 003-python (WITH skills)
- **Result**: ✅ **SUCCESSFUL** - End-to-end tested, database and containers created in emulator
- **Score**: 9/10
- **Key Achievement**: **VALIDATED Rule 4.9** and successfully tested complete application with database creation
- **Rules Applied**: 30+ rules including hierarchical partition keys (production), TTL, autoscale, composite indexes

**Issues Encountered & Resolved**:

1. **Pydantic Dependency Issue** (Python 3.13 compatibility)
   - Problem: pydantic 2.5.3 doesn't have prebuilt wheels for Python 3.13
   - Error: "Failed building wheel for pydantic-core"
   - Solution: Updated to pydantic 2.10.0 and pydantic-core 2.27.0
   - Status: ✅ RESOLVED

2. **Environment Variable Override** (**Rule 4.9 Validated**)
   - Problem: System `COSMOS_ENDPOINT` environment variable overrode `.env` file
   - Error: "Local Authorization is disabled. Use an AAD token"
   - Solution: Added `python-dotenv` with `load_dotenv(override=True)` before pydantic-settings
   - Status: ✅ RESOLVED - Rule 4.9 accurately predicted this scenario

3. **ThroughputProperties API Usage**
   - Problem: Python SDK doesn't accept dict for autoscale throughput
   - Error: "TypeError: offer_throughput must be int or an instance of ThroughputProperties"
   - Solution: Changed from `{'maxThroughput': 1000}` to `ThroughputProperties(auto_scale_max_throughput=1000)`
   - Status: ✅ RESOLVED

4. **Emulator Hierarchical Partition Key Limitation**
   - Problem: Local emulator doesn't support `kind='MultiHash'` for hierarchical partition keys
   - Error: "The 'kind' value 'MultiHash' specified in the partition key definition is invalid"
   - Solution: Used single partition key `/deviceId` for emulator, documented production recommendation
   - Status: ✅ RESOLVED with documentation for production

**Test Results**:

✅ Database `iot-telemetry-db` created in emulator
✅ Container `devices` created with partition key `/id` (autoscale 1000 RU/s)
✅ Container `telemetry` created with partition key `/deviceId` (autoscale 4000 RU/s, TTL enabled)
✅ Application started successfully on http://0.0.0.0:8000
✅ Gateway connection mode working with emulator
✅ Environment variables loaded correctly from .env file

**Lessons Learned**:

1. Always test end-to-end with database creation verification (not just code review)
2. Python 3.13 is new - check package compatibility before using
3. Use `python-dotenv` with `override=True` to prevent system env var issues
4. Python SDK API differs from other SDKs (ThroughputProperties class vs dict)
5. Local emulator has limitations (hierarchical partition keys not supported)

**Rule Enhancement Recommendations**:

None - Rule 4.9 worked exactly as designed and helped identify/resolve the environment configuration issue.
Potential future rules:
- Python SDK-specific rule about ThroughputProperties class usage
- Emulator limitations documentation (hierarchical keys, etc.)
  - Solution: Recommend logging endpoint at startup (already in rule) + manual .env loading if override needed
  - Priority: HIGH - affects all FastAPI/Pydantic v2 projects

**Python SDK Observations**:
- ✅ Hierarchical partition key requires dict format: `{'paths': [...], 'kind': 'MultiHash', 'version': 2}`
- ⚠️ No built-in bulk executor (unlike .NET/Java)
- ⚠️ Python SDK is not truly async (HTTP is synchronous under the hood)

**Lessons Learned**:
- Rule 4.9 is critical and accurately predicts real-world issues
- Logging endpoint at startup is essential for catching config issues
- Pydantic v2 environment variable handling differs from python-dotenv
- Python 3.13 requires newer dependency versions

**FILES TO MODIFY** (Proposed):
- `rules/sdk-local-dev-config.md` (enhance with pydantic-settings example - HIGH priority)
- `rules/partition-hierarchical.md` (add Python dict format example - MEDIUM priority)

---

#### 2026-01-29: Iteration 002 - IoT Telemetry Scenario (Java / Spring Boot)

- **Scenario**: iot-device-telemetry
- **Iteration**: 002-java (WITH skills)
- **Result**: ✅ **BUILD SUCCESS** - Complete Spring Boot 3 implementation after resolving compatibility issues
- **Score**: 8/10
- **Key Achievement**: Validated Cosmos DB best practices in Java ecosystem, identified framework version requirements gap
- **Rules Applied**: 26+ rules from AGENTS.md including:
  - 1.2: Denormalized device info in telemetry readings (deviceName, location)
  - 1.3: Embedded device summary for read efficiency
  - 1.5: Schema versioning (schemaVersion field)
  - 1.6: Type discriminators (type="device", type="telemetry")
  - 2.3: Hierarchical partition key (deviceId + yearMonth)
  - 2.5: Partition key aligned with query patterns
  - 3.1: All queries single-partition
  - 3.5: Parameterized queries throughout
  - 4.1: Async APIs with Project Reactor (Mono/Flux)
  - 4.13: Singleton CosmosClient
  - 5.1: Composite indexes for time-range queries
  - 6.1: Autoscale throughput for variable IoT workload
  - TTL: 30-day automatic expiration at container level

**Issues Encountered and Resolved**:

1. **Java Version Mismatch** (CRITICAL - NEW RULE NEEDED)
   - Problem: Spring Boot 3.2.1 requires Java 17+, pom.xml initially had Java 11
   - Error: "bad class file...has wrong version 61.0, should be 55.0"
   - Solution: Updated `<java.version>` to 17, set JAVA_HOME to Java 17 installation
   - Impact: Build blocker - developers would fail immediately without this knowledge
   - **GAP IDENTIFIED**: AGENTS.md doesn't document Java/Spring Boot version requirements

2. **SDK API Evolution** (MEDIUM impact)
   - Problem: `setMaxItemCount()` method not public in azure-cosmos 4.52.0
   - Solution: Removed explicit page size settings (SDK uses sensible defaults)
   - Impact: Code compiles after removal, no functionality loss

3. **Hierarchical Partition Key API** (HIGH - DOCUMENTATION NEEDED)
   - Problem: Constructor signature changed in current SDK version
   - Old (non-working): `new CosmosContainerProperties(name, List<String>)`
   - New (working):
   ```java
   PartitionKeyDefinition partitionKeyDef = new PartitionKeyDefinition();
   partitionKeyDef.setPaths(Arrays.asList("/deviceId", "/yearMonth"));
   partitionKeyDef.setKind(PartitionKind.MULTI_HASH);
   partitionKeyDef.setVersion(PartitionKeyDefinitionVersion.V2);
   ```
   - Impact: Build failure without proper API usage
   - **GAP IDENTIFIED**: Rule 2.3 mentions hierarchical keys but lacks Java SDK-specific API example

**Proposed Skill Improvements**:

1. **NEW RULE: `sdk-java-spring-boot-versions.md`** (CRITICAL impact)
   - Category: SDK Configuration (Section 4)
   - Problem: Framework version requirements not documented in skills
   - Content: Document Spring Boot 3.x → Java 17+, Spring Boot 2.7.x → Java 11/17, SDK compatibility matrix
   - Rationale: Prevents immediate build failures, critical for developer onboarding
   - Priority: HIGH - foundational requirement

2. **ENHANCE RULE 2.3**: Add Java SDK-specific hierarchical partition key code
   - Current state: Conceptual explanation only
   - Enhancement: Add code example showing `PartitionKeyDefinition` with `MULTI_HASH` and `V2`
   - Rationale: API has evolved, developers need current syntax
   - Priority: MEDIUM - affects hierarchical partition key adoption

**Observations**:
- ✅ All Cosmos DB best practices successfully applied in Java/Spring Boot ecosystem
- ✅ Async/reactive programming with Project Reactor properly implemented
- ✅ Build artifacts generated correctly (iot-telemetry-api-1.0.0.jar - 18.4 MB)
- ⚠️ Framework version requirements are external to Cosmos DB but critical for successful implementation
- ⚠️ SDK API documentation should be version-specific

**Lessons Learned**:
- Spring Boot version dictates minimum Java version (non-negotiable dependency)
- Azure Cosmos SDK API evolves between versions - check current documentation
- Skills should include framework compatibility matrices for complete guidance
- Hierarchical partition key API requires explicit V2 definition in Java SDK

**FILES TO MODIFY** (Proposed):
- `rules/sdk-java-spring-boot-versions.md` (new - CRITICAL)
- `rules/partition-hierarchical.md` (enhance with Java API example - MEDIUM)
- `rules/_sections.md` (if new rule added)
- `SKILL.md` (update quick reference)
- `AGENTS.md` (regenerate after rule changes)

---

#### 2026-01-29: Iteration 001 - IoT Telemetry Scenario (.NET)

- **Scenario**: iot-device-telemetry
- **Iteration**: 001-dotnet (WITH skills)
- **Result**: ✅ **SUCCESS** - Excellent implementation with all best practices applied
- **Score**: 9.5/10
- **Key Achievement**: Demonstrated comprehensive application of best practices for time-series IoT data
- **Rules Applied**: 30+ rules from AGENTS.md including:
  - 1.2: Denormalized `latestReading` in Device (avoids joins)
  - 1.3: Embedded `TelemetrySummary` for related data
  - 1.6: Type discriminators (`type` field)
  - 2.1: Hierarchical partition key prevents 20GB limit
  - 2.2: Synthetic key distributes writes (no hot partition)
  - 2.3: Hierarchical partition key (`deviceId` + `yearMonth`)
  - 2.5: Partition key aligned with query patterns
  - 2.6: Synthetic partition key for time-series
  - 3.1: All queries single-partition
  - 3.5: Parameterized queries throughout
  - 4.1: Async APIs throughout
  - 4.4/4.6: Correct mode per environment (Gateway for emulator, Direct for production)
  - 4.11: 429 retry configuration
  - 4.13: Singleton CosmosClient
  - 5.1: Composite indexes for time-range queries
  - 5.2: Excluded unused paths (temperature, humidity, batteryLevel)
  - 6.1: Autoscale for variable IoT workload
  - TTL: 30-day automatic expiration
- **Observations**:
  - ✅ All best practices from AGENTS.md were successfully applied
  - ✅ Hierarchical partition key perfectly suited for time-series data
  - ✅ Database initialized successfully with optimal container configurations
  - ✅ Clean build with no errors
  - ✅ Comprehensive documentation linking every decision to best practices
- **No Issues Found**: The iteration ran smoothly with no bugs or gaps requiring new rules
- **Lessons Learned**:
  - Indexing policy must include "/" path - learned to use `/*` with exclusions rather than overly specific inclusions
  - TCP connection settings only apply to Direct mode (not Gateway for emulator)
  - The skill kit is comprehensive for IoT time-series scenarios

**NO NEW RULES NEEDED**: All patterns and issues were covered by existing rules. This validates the skill kit's completeness for time-series IoT workloads.

**FILES MODIFIED**: None - no skill improvements needed

---

#### 2026-01-28: Cross-Iteration Review - New Rules from Lessons Learned

Based on reviewing all iteration ITERATION.md files and identifying patterns that could become rules:

**NEW RULES CREATED:**

1. **`pattern-change-feed-materialized-views.md`** (HIGH impact)
   - **Source**: Iterations 001, 002, 003 all noted cross-partition queries for admin endpoints
   - **Issue**: Status/date queries require expensive cross-partition fan-out
   - **Solution**: Document Change Feed pattern to maintain separate container optimized for admin queries
   - **Benefit**: Converts cross-partition queries into single-partition lookups

2. **`sdk-java-content-response.md`** (MEDIUM impact)
   - **Source**: Iteration 003-java discovered createItem returned null
   - **Issue**: Java SDK doesn't return document content after write operations by default
   - **Solution**: Set `contentResponseOnWriteEnabled(true)` at client or request level
   - **Benefit**: Prevents confusion when created items appear null

3. **`sdk-local-dev-config.md`** (MEDIUM impact)
   - **Source**: Iteration 004-python encountered system env vars overriding .env file
   - **Issue**: Developers with Azure CLI configured have COSMOS_ENDPOINT pointing to cloud, not emulator
   - **Solution**: Use `load_dotenv(override=True)`, environment-specific configs, and log endpoint at startup
   - **Benefit**: Prevents accidental connections to production during local development

**EXISTING RULE ENHANCED:**

4. **`sdk-emulator-ssl.md`** - Now covers ALL SDKs
   - **Previous**: Java-only SSL certificate guidance
   - **Enhanced**: Added .NET, Python, Node.js examples for Gateway mode and SSL handling
   - **Benefit**: Single reference for all SDK emulator configuration

**COMPILE SCRIPT UPDATED:**
- Added `pattern-` prefix for Design Patterns category (section 9)

**FILES MODIFIED:**
- `rules/pattern-change-feed-materialized-views.md` (new)
- `rules/sdk-java-content-response.md` (new)
- `rules/sdk-local-dev-config.md` (new)
- `rules/sdk-emulator-ssl.md` (enhanced)
- `rules/_sections.md` (added section 9)
- `SKILL.md` (updated quick reference)
- `scripts/compile.js` (added pattern- prefix)
- `AGENTS.md` (regenerated - now 53 rules)
- `testing/README.md` (added Continuous Improvement / Feedback Loop section)

---

#### 2026-01-28: Iteration 003 - Java SDK Validation (ecommerce-order-api)

- **Scenario**: ecommerce-order-api
- **Iteration**: 003-java (WITH skills)
- **Result**: ✅ **SUCCESS** - Critical enum serialization test passed
- **Improvement Validated**: Rule 4.10 works across both .NET and Java SDKs
- **Score**: 9.0/10
- **Key Rules Applied**:
  - 4.10: `@JsonValue`/`@JsonCreator` for enum serialization ✅
  - 4.1: Singleton CosmosAsyncClient via Spring @Bean ✅
  - 4.2: Async APIs with Project Reactor (Mono<T>) ✅
  - 4.3: ThrottlingRetryOptions configured ✅
  - 1.1: Embedded OrderItems ✅
  - 2.1: High-cardinality partition key (customerId) ✅
  - 3.1/3.5: Single-partition queries with SqlParameter ✅
  - 5.2: Composite indexes ✅
- **Observations**: 
  - The skill kit successfully guided Java-specific enum serialization
  - Gateway mode required for emulator (Direct mode has SSL issues)
  - `contentResponseOnWriteEnabled(true)` needed to get item back after create
- **Tests**: 5/6 passing (date range query has format issue, not enum-related)

**NEW RULE CREATED**: `sdk-emulator-ssl` - Configure SSL for Cosmos DB Emulator in Java
- **Issue**: Java SDK fails with SSL handshake errors when connecting to emulator
- **Solution**: Import emulator certificate into JDK truststore + use Gateway mode
- **Files Modified**: 
  - `rules/sdk-emulator-ssl.md` (new file)
  - `SKILL.md` (added to quick reference)
  - `AGENTS.md` (regenerated)

#### 2026-01-28: Iteration 002 - Skills Successfully Applied (ecommerce-order-api)

- **Scenario**: ecommerce-order-api
- **Iteration**: 002-dotnet (WITH skills)
- **Result**: ✅ **SUCCESS** - All tests passed, including the critical enum serialization test
- **Improvement Validated**: Rule 4.10 (enum serialization) was applied correctly, fixing the bug from iteration-001
- **Score**: 9.1/10 (vs 6/10 in baseline iteration-001)
- **Key Rules Applied**:
  - 4.10: JsonStringEnumConverter configured ✅
  - 4.1: Singleton CosmosClient ✅
  - 4.4: Direct connection mode ✅
  - 1.1: Embedded OrderItems ✅
  - 2.1/2.4: High-cardinality partition key (customerId) ✅
  - 3.1/3.2: Single-partition queries with projections ✅
  - 5.2: Composite indexes ✅
- **Observations**: 
  - The skill kit successfully prevented the enum serialization bug
  - Agent applied 18+ rules from the skill kit
  - Cross-partition queries (status/date) are unavoidable given partition key choice - could add guidance for Change Feed patterns

#### 2026-01-27: Testing Framework - Skills Must Be Loaded First

- **Scenario**: ecommerce-order-api
- **Iteration**: 001-dotnet
- **Issue**: Iteration 001 was run WITHOUT loading the skill kit first, making it a baseline test rather than a skill kit test
- **Improvement**: Updated testing framework documentation to emphasize skills MUST be loaded before each iteration
- **Files Modified**: 
  - `testing/README.md` - Added "CRITICAL: Install Skills FIRST" section
  - `testing/scenarios/_iteration-template.md` - Added "Skills Verification" section
  - `iteration-001-dotnet/ITERATION.md` - Marked as baseline (no skills)

#### 2026-01-29: Iteration 002 - AI Chat/RAG Scenario (Python / FastAPI / Azure Cloud)

- **Scenario**: ai-chat-rag
- **Iteration**: 002-python
- **Result**: ✅ **SUCCESSFUL** - Complete working implementation with vector search in Azure
- **Score**: 10/10 - Full repository layer, test data, end-to-end vector search validated
- **Environment**: Azure Cosmos DB (Cloud), DefaultAzureCredential auth
- **Key Achievement**: **VALIDATED** all 4 vector search rules work correctly in Python/Azure

**Implementation Summary**:
- ✅ Complete repository pattern (chat_repository.py, document_repository.py)
- ✅ Vector search using VectorDistance() queries working end-to-end
- ✅ Test data: 10 documents with 1536D embeddings + 3 chat sessions
- ✅ Vector similarity search returning ranked results
- ✅ Data visible and queryable in Azure Portal

**Technical Details**:
- SDK: azure-cosmos 4.14.5 (requires >= 4.7.0 for vector search)
- Vector Index: QuantizedFlat (optimal for < 50K vectors)
- Distance Function: Cosine similarity
- Partition Keys: /userId (sessions), /category (documents)

**Issues Encountered**:

1. **Windows Unicode Console Error**
   - Issue: Checkmark characters (✓) caused UnicodeEncodeError on Windows console
   - Solution: Replaced all ✓ with [OK] in logging output
   - Impact: Minor - cosmetic logging fix
   - Files: config.py, cosmos_service.py, main.py

2. **SDK Version Requirements**
   - Issue: Vector search requires azure-cosmos >= 4.7.0
   - Solution: Updated requirements.txt to specify minimum version
   - Installed: 4.14.5 (latest stable)
   - Impact: Critical for vector search functionality

3. **Missing Repository Layer**
   - Issue: Initial skeleton had endpoints but no data access implementation
   - Solution: Implemented complete repository pattern based on Microsoft samples
   - Pattern: upsert_item() for documents, read-modify-write for embedded messages
   - Impact: Required for functional application

4. **Vector Search Query Syntax**
   - Issue: Needed correct pattern for VectorDistance() queries
   - Solution: Referenced GitHub samples for proper query structure:
   ```python
   query = """
       SELECT TOP @limit c.title, c.content,
              VectorDistance(c.embedding, @queryVector) AS similarityScore
       FROM c
       WHERE VectorDistance(c.embedding, @queryVector) > @threshold
       ORDER BY VectorDistance(c.embedding, @queryVector)
   """
   ```
   - Impact: Essential for vector similarity search

**Vector Search Validation Results**:
```
Found 5 results ordered by similarity:
  1. Embedding Generation Techniques       | Score: 0.0533
  2. Change Feed Processing                | Score: 0.0461
  3. Python SDK for Cosmos DB              | Score: 0.0306
  4. Indexing Policies for Vector Search   | Score: 0.0281
  5. Multi-Region Replication              | Score: 0.0273
```

**Rules Validated**:
- ✅ Rule 10.1 (vector-enable-feature.md): Feature enabled in Azure account
- ✅ Rule 10.2 (vector-embedding-policy.md): 1536 dims, Cosine, float32 configured correctly
- ✅ Rule 10.3 (vector-index-type.md): QuantizedFlat index working, embeddings excluded from default indexing
- ✅ Rule 10.4 (vector-distance-query.md): VectorDistance() queries returning ranked results

**Best Practices Demonstrated**:
- ✅ Rule 1.2 (model-embed-related): Messages embedded in session documents
- ✅ Rule 2.3 (partition-high-cardinality): userId and category partition keys
- ✅ Rule 2.2 (index-exclude-unused): Vector embeddings excluded from default indexing
- ✅ Rule 3.6 (query-use-projections): Embeddings excluded from SELECT when not needed

**Lessons Learned**:

1. **SDK Version Critical**: Python requires azure-cosmos >= 4.7.0 for vector search (tested 4.14.5)
2. **Embedding Normalization**: Mock embeddings must be normalized to unit length for cosine similarity
3. **Windows Compatibility**: Use ASCII-safe characters in console output (avoid Unicode symbols)
4. **Query Patterns**: VectorDistance() must appear in SELECT, WHERE, and ORDER BY clauses
5. **Indexing Performance**: Excluding /embedding/* from default indexing is critical for RU costs
6. **Microsoft Samples**: GitHub samples provide authoritative patterns for repository implementation

**Gap Analysis**: 
- ✅ No gaps found - all 4 vector search rules worked perfectly
- ✅ Python implementation validated .NET rules work cross-language
- ✅ Azure cloud deployment validated (vs emulator in iteration 001)

**FILES CREATED**:
- ✅ `chat_repository.py` - Chat session data access layer (200 lines)
- ✅ `document_repository.py` - Document + vector search operations (220 lines)
- ✅ `create_test_data.py` - Test data generator with mock embeddings
- ✅ `test_vector_search.py` - Vector search validation script
- ✅ Updated `main.py` - Wired repositories to FastAPI endpoints
- ✅ `ITERATION.md` - Complete documentation

**NO SKILL MODIFICATIONS NEEDED** - All existing rules sufficient and accurate.

**Post-Iteration Rule Additions**:

After completing the iteration successfully, user provided GitHub samples showing proper implementation patterns. Based on this, added 2 new rules:

1. **vector-repository-pattern.md** (HIGH impact)
   - Issue: Agent had vector query rule but not complete repository implementation pattern
   - Solution: Created comprehensive rule showing data access layer with upsert, vector search, get/delete methods
   - Examples: Python, .NET, JavaScript, Java repository classes
   - Covers: Clean abstraction, testability, proper error handling, RU logging

2. **vector-normalize-embeddings.md** (MEDIUM impact)
   - Issue: No guidance on embedding normalization for cosine similarity or testing
   - Solution: Created rule explaining L2 normalization, deterministic mock embeddings
   - Examples: All languages showing normalized embedding generation with magnitude verification
   - Covers: Why normalize, formula, production vs testing, common mistakes

**Updated Files**:
- ✅ `rules/vector-repository-pattern.md` - NEW (HIGH impact)
- ✅ `rules/vector-normalize-embeddings.md` - NEW (MEDIUM impact)
- ✅ `AGENTS.md` - Recompiled with 56 total rules (was 54)

**Next Testing Priorities**:
- MEDIUM: Test vector search with Java (validate rules cross-language)
- MEDIUM: Test vector search with JavaScript/Node.js
- LOW: Test DiskANN vector index type (for > 50K vectors scenario)

---

#### 2026-01-27: Enum Serialization Mismatch Bug (ecommerce-order-api)

- **Scenario**: ecommerce-order-api
- **Iteration**: 001-dotnet
- **Issue**: Agent generated code where Cosmos SDK stored enums as integers but queries searched for strings, causing status queries to return empty results
- **Improvement**: ✅ Added new rule `sdk-serialization-enums.md` with guidance on consistent enum serialization
- **Priority**: HIGH
- **Files Modified**: 
  - `rules/sdk-serialization-enums.md` (new)
  - `AGENTS.md` (added section 4.10)

#### 2026-01-27: Missing Pagination Pattern (ecommerce-order-api)

- **Scenario**: ecommerce-order-api
- **Iteration**: 001-dotnet
- **Issue**: List queries returned all results without pagination, which would fail at scale
- **Analysis**: Rule `query-pagination.md` already exists with good content - the agent simply didn't apply it
- **Action**: No rule change needed, but highlights that agents may not always apply existing rules
- **Priority**: HIGH (for observation)
- **Files Modified**: None - rule already exists

#### 2026-01-27: Missing RU Diagnostics Logging (ecommerce-order-api)

- **Scenario**: ecommerce-order-api
- **Iteration**: 001-dotnet
- **Issue**: No logging of CosmosDiagnostics or RU consumption for debugging/monitoring
- **Analysis**: Rule `monitoring-ru-consumption.md` already exists with comprehensive SDK examples - agent didn't apply it
- **Action**: No rule change needed - rule has .NET examples with logging, telemetry, and middleware patterns
- **Priority**: MEDIUM (for observation)
- **Files Modified**: None - rule already exists

---

#### 2026-02-18: Iteration 001 - Multi-Tenant SaaS Scenario (Java / Spring Boot 3)

- **Scenario**: multitenant-saas
- **Iteration**: 001-java
- **Result**: ✅ SUCCESSFUL - All endpoints functional, tenant isolation verified, HPK working correctly
- **Score**: 7/10
- **Key Achievement**: Skills feedback loop validated — HPK multi-tenant design from iteration-001-dotnet replicated correctly. Java-specific `@PostConstruct`/`@Bean` circular dependency anti-pattern identified and fixed. All 13 endpoints tested successfully against emulator.

**New Rules Created** ⭐:

1. **sdk-java-cosmos-config.md** (HIGH)
   - Documents `@PostConstruct` + `@Bean` circular dependency anti-pattern in Spring Boot
   - Correct pattern: dependent `@Bean` methods with parameter injection chain
   - Includes HPK container initialization example
   - Includes `SmartInitializingSingleton` alternative for post-init logic

**Rules Strengthened** 📝:

1. **index-composite.md** — STRENGTHENED (HIGH)
   - Added "Multi-Tenant Composite Index Patterns" section
   - Added composite indexes for type discriminator queries: `(type, status, createdAt)`, `(type, assigneeId, dueDate)`, `(type, priority, createdAt)`
   - Added Java `IndexingPolicy` code example
   - Added explanation of why type discriminators require composite indexes
   - Added rule: "Always define composite indexes when using type discriminators in shared containers"

2. **query-pagination.md** — STRENGTHENED (MEDIUM)
   - Added "Unbounded Query Anti-Pattern" section
   - Added Java anti-pattern example: returning all results without pagination
   - Added rule of thumb: "If a query can return more than 100 items, it must use pagination"
   - Documents cascading failure risk from excessive RU consumption

3. **sdk-etag-concurrency.md** — STRENGTHENED (MEDIUM)
   - Added "Always use" guidance for denormalized data updates
   - Added "Critical: ETags for Denormalized Data Updates" section
   - Added Java anti-pattern: concurrent denormalized count updates without ETag
   - Added Java correct pattern: ETag-protected count update with retry
   - Documents why denormalized fields are highest-risk for lost updates

**Issues Encountered & Resolved**:

1. **Circular Dependency in CosmosConfig** — ❌ RUNTIME ERROR → FIXED
   - Problem: `@PostConstruct` method called `cosmosClient()` @Bean method, causing Spring circular dependency
   - Solution: Removed `@PostConstruct`, moved database/container initialization into dependent `@Bean` methods: `cosmosClient()` → `cosmosDatabase(CosmosClient)` → `cosmosContainer(CosmosDatabase)`
   - Status: ✅ Fixed. Potential new rule candidate.

2. **Emulator SSL Certificate** — ⚠️ EXPECTED FRICTION → RESOLVED
   - Problem: Cosmos DB emulator uses self-signed cert, Java SDK needs it in truststore
   - Solution: Exported cert from Windows cert store, copied JDK cacerts locally, imported cert, ran with `-Djavax.net.ssl.trustStore=cacerts`
   - Status: ✅ Resolved using approach from Rule 4.6 / gaming-leaderboard iteration-002. `sdk-emulator-ssl.md` update recommended in prior iteration still applies.

3. **No Custom Indexing Policy** — ⚠️ DESIGN GAP (NOT FIXED)
   - Problem: Default "index everything" policy used; no composite indexes defined
   - Impact: Excessive write RU consumption; sorted queries inefficient at scale
   - Status: ⚠️ Not fixed. Documented as primary scoring gap (Indexing: 3/10).

**Comparison with Iteration 001 (.NET)**:

| Aspect | Iter-001 (.NET) | Iter-001 (Java) | Delta |
|--------|----------------|----------------|-------|
| 2026-04-18 | iot-device-telemetry | Batch #232 (skills, java) | Aggregated 5 iterations | See batch-results/ |
| 2026-04-01 | gaming-leaderboard | Batch #191 (skills, python) | Aggregated 5 iterations | See batch-results/ |
| HPK design | ✅ /tenantId, /projectId | ✅ /tenantId, /type, /projectId | ✅ Added /type level |
| Build success | ❌ Newtonsoft.Json issue | ❌ CircularDependency | ⟷ Different issues |
| Endpoint testing | ❌ Packaging prevented | ✅ All 13 endpoints tested | ✅ Improved |
| Tenant isolation | ⚠️ Not verified | ✅ Explicitly verified | ✅ Improved |
| Indexing | ❌ Not defined | ❌ Not defined | ⟷ Same gap |
| Schema versioning | ✅ Applied | ✅ Applied | ⟷ Same |
| Packaging | ❌ Wrong structure | TBD | TBD |

**Test Results**:
- ✅ POST /api/tenants — Tenant creation with type discriminator
- ✅ GET /api/tenants/{tenantId} — Tenant retrieval by HPK
- ✅ POST /api/tenants/{tenantId}/users — User creation with tenant isolation
- ✅ GET /api/tenants/{tenantId}/users — User listing (single-partition query)
- ✅ POST /api/tenants/{tenantId}/projects — Project creation with self-referencing projectId
- ✅ GET /api/tenants/{tenantId}/projects/{projectId} — Project with denormalized counts
- ✅ POST /api/tenants/{tenantId}/projects/{projectId}/tasks — Task creation with count update
- ✅ GET /api/tenants/{tenantId}/projects/{projectId}/tasks — Task listing by project
- ✅ PUT /api/tenants/{tenantId}/projects/{projectId}/tasks/{taskId} — Status update with count refresh
- ✅ POST /api/tenants/{tenantId}/projects/{projectId}/tasks/{taskId}/comments — Embedded comment
- ✅ GET /api/tenants/{tenantId}/tasks?assigneeId=X — Cross-project assignee query
- ✅ GET /api/tenants/{tenantId}/tasks?status=open — Status-based query
- ✅ GET /api/tenants/{tenantId}/analytics — Tenant analytics from denormalized counts
- ✅ Tenant isolation: tenant-beta sees 0 items from tenant-acme

**Best Practices Applied Successfully**:
1. ✅ **Hierarchical Partition Keys** — 3-level HPK (/tenantId, /type, /projectId) — Rule 2.3
2. ✅ **Type Discriminators** — Single container, 4 entity types — Rule 1.9
3. ✅ **Denormalized Reads** — Task counts on projects, names on tasks — Rule 1.2
4. ✅ **Embedded Documents** — Comments bounded at 20 per task — Rule 1.3, 1.7
5. ✅ **Schema Versioning** — schemaVersion field on BaseEntity — Rule 1.8
6. ✅ **Singleton CosmosClient** — Spring @Bean singleton — Rule 4.16
7. ✅ **Gateway for Emulator** — Auto-detect based on endpoint — Rule 4.6
8. ✅ **contentResponseOnWriteEnabled** — Java SDK optimization — Rule 4.9
9. ✅ **Parameterized Queries** — SqlParameter on all queries — Rule 3.5
10. ✅ **Projections** — Field selection on list queries — Rule 3.6

**Best Practices NOT Applied**:
- ❌ Custom indexing policy / composite indexes (Rules 5.1, 5.2)
- ❌ Pagination with continuation tokens (Rule 3.4)
- ❌ ETag concurrency on updates (Rule 4.7)
- ❌ Preferred regions / availability strategy (Rules 4.8, 4.12)
- ❌ SDK diagnostics logging (Rule 4.5)
- ❌ Async API (Rule 4.1)

**Lessons Learned**:
1. **Indexing is consistently the weakest area** — Both .NET and Java iterations missed custom indexing, suggesting the skill rules need stronger emphasis
2. **`@PostConstruct` + `@Bean` is a Java Spring trap** — Not Cosmos-specific but commonly hit when initializing Cosmos DB resources
3. **HPK design improved over .NET iteration** — Adding /type as middle level provides better partition isolation for mixed entity queries
4. **Emulator SSL is well-understood now** — Third iteration dealing with emulator SSL, process is documented and repeatable
5. **Denormalized count maintenance works end-to-end** — Pattern of updating parent counts after child CRUD is effective but needs concurrency protection

**FILES MODIFIED**:
- ✅ `skills/cosmosdb-best-practices/rules/sdk-java-cosmos-config.md` — NEW (HIGH)
- ✅ `skills/cosmosdb-best-practices/rules/index-composite.md` — STRENGTHENED (multi-tenant composite index patterns)
- ✅ `skills/cosmosdb-best-practices/rules/query-pagination.md` — STRENGTHENED (unbounded query anti-pattern)
- ✅ `skills/cosmosdb-best-practices/rules/sdk-etag-concurrency.md` — STRENGTHENED (denormalized data guidance)
- ✅ `skills/cosmosdb-best-practices/AGENTS.md` — Recompiled (62 total rules, up from 61)
- ✅ `testing/scenarios/multitenant-saas/iterations/iteration-001-java/ITERATION.md` — NEW

---

#### 2026-04-18: Batch 232 - IoT Device Telemetry (Java / skills loaded)

- **Scenario**: iot-device-telemetry
- **Iteration**: batch-232-java-skills
- **Result**: ⚠️ PARTIAL — high stochastic variance, but 7 consistent failures identified and classified
- **Score**: 2/10 (batch mean)

**Rules Created** 🆕:
1. **query-latest-by-timestamp.md** — Require `ORDER BY <timestamp> DESC` + `TOP 1` for deterministic latest-item queries (HIGH)
2. **query-aggregate-single-pass.md** — Compute min/max/avg in one scoped aggregate query to avoid inconsistent stats (HIGH)

**Rules Updated** 🔧:
1. **model-schema-versioning.md** — Clarified that `schemaVersion` must be written on every persisted document type, not only primary entities (MEDIUM)
2. **query-point-reads.md** — Added explicit parent-existence validation pattern using point reads before writing child/event records (HIGH)

**Consistent Failure Classification (always-fail tests)**:
1. `test_documents_have_schema_version` — 🔧 **Unclear existing rule**
   - Existing rule was present (`model-schema-versioning`) but not explicit enough about applying versioning to all document types.
2. `test_stats_humidity_values_correct` — ❌ **Cosmos DB anti-pattern**
   - Symptom matches client-side/partial aggregation instead of one scoped aggregate query.
3. `test_stats_temperature_min_correct` — ❌ **Cosmos DB anti-pattern**
   - Same root cause class as humidity stats: inconsistent aggregation logic.
4. `test_latest_reading_is_most_recent` — ❌ **Cosmos DB anti-pattern**
   - Missing explicit `ORDER BY timestamp DESC` + `TOP 1` pattern for latest reads.
5. `test_ingest_telemetry_empty_body_returns_4xx` — 📜 **Contract violation**
   - API input validation behavior, not a Cosmos-specific best-practice gap.
6. `test_ingest_telemetry_for_nonexistent_device_returns_4xx` — 🔧 **Unclear existing rule**
   - Existing point-read guidance did not clearly instruct referential existence validation before child writes.
7. `test_ingest_telemetry_missing_device_id_returns_4xx` — 📜 **Contract violation**
   - Required-field validation per API contract, not a Cosmos-specific anti-pattern.

**Batch Notes**:
- Flaky tests were intentionally ignored per `testing-v2/EVALUATE.md`.
- Rule additions are generic and reusable across domains (not scenario-specific framing).

---

## Release History

### v1.0.0 (Initial Release)

- Initial set of 48 rules covering:
  - Data modeling (6 rules)
  - Partition key design (6 rules)
  - Query optimization (6 rules)
  - SDK best practices (9 rules)
  - Indexing strategies (5 rules)
  - Throughput & scaling (5 rules)
  - Global distribution (6 rules)
  - Monitoring & diagnostics (5 rules)

#### 2026-04-30: iteration-001-rust - Ecommerce Order Api (Rust) [skills loaded]

- **Scenario**: ecommerce-order-api
- **Iteration**: iteration-001-rust
- **Skills loaded**: Yes
- **Result**: PARTIAL -- 85/91 tests passed (93.4%)
- **Score**: 8/10

**Results by Category**:
- api_contract: 39 passed, 2 failed, 0 skipped
- build_startup: 2 passed, 0 failed, 0 skipped
- cosmos_infrastructure: 11 passed, 3 failed, 1 skipped
- data_integrity: 5 passed, 0 failed, 0 skipped
- robustness: 30 passed, 0 failed, 0 skipped

**Issues Encountered**:
1. **testing-v2.scenarios.ecommerce-order-api.tests.test_api_contract.TestUpdateOrderStatus::test_update_status_reflects_new_status** -- assert 409 == 200
 +  where 409 = <Response [409]>.status_code
1. **testing-v2.scenarios.ecommerce-order-api.tests.test_api_contract.TestUpdateOrderStatus::test_updated_status_persists_on_get** -- AssertionError: After PATCH, GET should return updated status 'delivered', got 'pending'
assert 'pen
1. **testing-v2.scenarios.ecommerce-order-api.tests.test_cosmos_infrastructure.TestIndexingPolicies::test_has_composite_indexes_for_order_queries** -- AssertionError: No container has composite indexes defined. E-commerce queries like 'orders by statu
1. **testing-v2.scenarios.ecommerce-order-api.tests.test_cosmos_infrastructure.TestDocumentStructure::test_documents_have_type_discriminator** -- Failed: No documents have a type discriminator field. When a container holds multiple entity types (
1. **testing-v2.scenarios.ecommerce-order-api.tests.test_cosmos_infrastructure.TestDocumentStructure::test_documents_have_schema_version** -- Failed: No documents have a schema version field. Include a 'schemaVersion' field in documents so fu

**Test Results**: 85 passed, 6 failed out of 91
