# advanced/

Cost tracking, prompt A/B experiments, evaluation, observability, and self-improvement.

| #   | File               | Shows                                                                    | Offline? |
| --- | ------------------ | ------------------------------------------------------------------------ | -------- |
| 14  | cost-tracking      | Per-task cost tracking and budget enforcement                            | ⚡       |
| 15  | prompt-experiments | A/B experiment framework: define variants, assign users, record outcomes | ✅       |
| 16  | eval-framework     | LLM-as-judge evaluation with structured scoring                          | ⚡       |
| 17  | observability      | Live streaming + JSONL file export + metrics dashboard                   | ⚡       |
| 18  | self-improvement   | Cross-task episodic learning — run 2 leverages run 1 outcomes            | ⚡       |

Run offline only: `bun run ../../index.ts --filter advanced --offline`
