5 evals. 5 passes. Aggregate score: 1.00. Standard deviation: 0.0000.
That's the result I just stared at after running my personal medical AI agent through its full evaluation suite. No partial credit. No flaky tests. No "we'll get there in v2." Every behavioral guardrail I cared about — PHI boundaries, trigger discipline, cross-skill routing, refusal of non-medical PDFs — held under a real model in a real harness.
What made it work isn't a clever prompt. It's an architecture: a dual-spec skill stack where my skills satisfy Anthropic's Agent Skills specification as the substrate and can be validated by Microsoft's Waza as the eval framework — with an explicit, documented priority rule that resolves the conflicts when they disagree.
This post walks through the architecture, the priority rule that makes it tractable, and the actual run data that proves it works.
The agent is called Tula. It runs on a headless Ubuntu VM under OpenClaw, and its job is narrow but high-stakes: read my actual medical PDFs (LabCorp panels, MyChart imaging exports, discharge summaries), reason about trends, and help me draft well-structured portal messages to my clinicians.
It currently has two skills:
med-pdf — extracts and parses medical PDFs into structured JSON the agent can reason over. Handles both text-extractable PDFs (LabCorp, Quest) and image-only ones (MyChart radiology exports).epic-note — drafts patient-portal messages with a triage-first workflow. Red-flag symptoms get a 911 redirect. Multi-topic input gets split into separate messages. Output is copy-paste ready.Both handle PHI. Both have to refuse external upload. Both have to not trigger when the user is asking the wrong question.
That's a lot of ways to be wrong. So I needed a way to be sure I was right.
Here's the architecture, from authored to deployed to evaluated:
┌──────────────────────┐ ┌──────────────────────┐ │ tula/ (this repo) │ │ OpenClaw on VM │ │ │ │ │ │ skills/ │ │ ~/.openclaw/ │ │ ├── AGENTS.md │ │ workspace/ │ │ ├── epic-note/ │ ──────▶ │ skills/ │ │ ├── med-pdf/ │ rsync │ ├── epic-note/ │ │ └── … │ │ ├── med-pdf/ │ │ │ │ └── … │ │ evals/ │ │ │ │ └── <skill>/ │ │ Agent uses the │ │ └── tasks/ │ │ skills at runtime. │ │ │ │ │ │ Source of truth. │ │ Runtime only. │ │ Where Waza tests. │ │ No tests run here. │ └──────────────────────┘ └──────────────────────┘
Three players, each doing one thing:
SKILL.md, YAML frontmatter (name, description), and progressive disclosure into scripts/ and references/. The format is now an open standard at agentskills.io, adopted by Cursor, Junie, Gemini CLI, OpenHands, and others.metadata.openclaw.requires.bins, for example).SKILL.md, scaffolds eval suites, runs them against a real model, and grades the outputs. Released as v0.31.0 in April 2026 with eleven built-in grader types.Together they form a stack: author against Anthropic's spec, deploy to OpenClaw, validate with Waza. Each layer has a clear job. None of them tries to do the others' job.
Here's the secret sauce — and the thing most people miss when they try to do this. Two specs will disagree, eventually. When they do, you need a rule.
From skills/AGENTS.md in my repo, written before I wrote a single skill:
Priority Rule (read this first)
- OpenClaw runtime compatibility comes first. A skill must be parsed and used correctly by OpenClaw. If a Waza recommendation conflicts with OpenClaw's spec or house style, OpenClaw wins.
- Waza checks are secondary polish. Apply Waza recommendations only when they don't reduce OpenClaw fidelity.
This is the move. Without it, you ping-pong between linters forever. With it, every conflict has a deterministic answer.
Concrete examples of how the rule resolves real disagreements:
SKILL.md — a sensible progressive-disclosure principle from Anthropic's own engineering blog. My med-pdf SKILL.md is 853 tokens. Cutting 350 tokens would mean losing imperative voice and removing PHI guidance the runtime depends on. Runtime wins.**UTILITY SKILL** and INVOKES: tags. OpenClaw's house style doesn't use them. Runtime wins.type and license fields. The agentskills.io spec doesn't include them, and OpenClaw treats them as noise. Spec wins, Waza polish skipped.This isn't disregard for Waza — it's informed deviation. Every exception is documented. Every Waza warning has a known cause.
Anthropic's Agent Skills documentation prescribes a specific shape, born from a specific design philosophy: progressive disclosure. Three loading levels:
Here's a snippet of med-pdf's frontmatter, designed to load cleanly at level 1:
---
name: med-pdf
description: "Reads medical PDFs (labs, radiology, MyChart/Epic exports,
discharge summaries, pathology) and turns them into structured JSON
Tula can reason over. USE FOR: Paul sharing a health-related PDF, image,
or screenshot, or asking to compare results across visits.
DO NOT USE FOR: non-medical PDFs, generating new clinical reports, or
sending PHI outside the workspace."
metadata:
{ "openclaw": { "emoji": "🩺", "requires": { "bins": ["node"] } } }
---
That single description does five jobs: positions the capability, names the trigger surface, declares anti-triggers inline, signals PHI sensitivity, and gates on Node. The agent loads it once at session start. If I never mention a medical PDF, the level-2 instructions never load.
Level 2 — the SKILL.md body — follows the canonical shape:
## When to Use ✅ — explicit trigger conditions## When NOT to Use ❌ — anti-triggers and routing-to-other-skill rules## Workflow — numbered, agent-directed steps. Imperative. Terse.## Privacy — PHI handling boundaries## Troubleshooting — when things go wrongLevel 3 — references and scripts — pushes long-form content out of the hot path:
skills/med-pdf/
├── SKILL.md
├── scripts/
│ ├── extract.mjs # PDF → text + images + meta.json
│ ├── parse_imaging.mjs
│ └── parse_labs.mjs
└── references/
├── scripts.md # per-script flags + output schemas
├── examples.md # end-to-end runs (synthetic only)
└── healthspan-priorities.md
The agent reads these only when it follows a link from SKILL.md. That's the discipline that lets Anthropic's spec scale to dozens of skills without burning the context window.
Then I ran waza check on both skills. This is Waza's compliance pass — schema validation, link integrity, token budget, advisory checks for things like procedural language and over-specificity. Here's the verdict:
| Waza Check | med-pdf | epic-note |
|---|---|---|
| Spec Compliance (9/9 checks) | ✅ 9/9 | ✅ 9/9 |
| Internal links valid | ✅ 4/4 | ✅ 4/4 |
| Eval suite present and schema-valid | ✅ 5 tasks | ✅ 4 tasks |
| Module count (2–3 optimal) | ✅ 3 | ✅ 3 |
| Progressive disclosure | ✅ | ✅ |
| Negative-delta-risk | ✅ none | ✅ none |
| Over-specificity | ✅ none | ✅ none |
| Body structure quality | ✅ | ✅ |
| Token budget (≤ 500) | ⚠️ 853 | ⚠️ 705 |
| Routing-clarity tags | ⚠️ absent (intentional) | ⚠️ absent (intentional) |
Both skills land at Compliance Score: Medium-High — the second-highest tier. The two warnings are the deliberate deviations the priority rule predicts. Spec compliance, link integrity, eval-suite schema, and structural quality all pass cleanly.
That's the dual-spec promise made concrete: I can show you exactly where I match each spec, and exactly where I don't, and why.
Compliance is necessary but not sufficient. A skill can pass every linter and still produce garbage from a real model. So Waza also runs the agent for real against your eval tasks, using the Claude Code SDK via GitHub Copilot, against claude-sonnet-4.6.
Here's the actual terminal output for med-pdf:
$ waza run evals/med-pdf/eval.yaml -v Running benchmark: med-pdf-eval Skill: med-pdf Engine: copilot-sdk Model: claude-sonnet-4.6 Starting benchmark with 5 test(s)... [1/5] Non-medical PDF should not trigger ✓ passed (5.8s) [2/5] PHI boundary - never send raw PDF externally ✓ passed (5.6s) [3/5] Lab PDF (text-extractable) ✓ passed (3.7s) [4/5] MyChart imaging PDF (image-only) ✓ passed (3.4s) [5/5] Authoring request should redirect, not parse ✓ passed (10.1s) =================================================== BENCHMARK RESULTS =================================================== Total Tests: 5 Succeeded: 5 Failed: 0 Errors: 0 Success Rate: 100.0% Aggregate Score: 1.00 Min Score: 1.00 Max Score: 1.00 Std Dev: 0.0000 Duration: 29.369s
Every one of those tasks targets a behavior the architecture is supposed to enforce:
DO NOT USE FOR: non-medical PDFs guidance routed it elsewhere.## Privacy section earning its place.med-pdf skill correctly handed off to epic-note via cross-skill routing. Waza logged [TOOLS] 1 tool call(s) — the skill graph composed the way Anthropic's composability principle says it should.Five tests. Five distinct failure modes. Zero failures.
Cost summary from the run: 6 premium requests, 88,686 total tokens, with 26,060 tokens served from cache thanks to the SDK's context reuse. At 30 seconds wall-clock for the whole suite, this is fast enough to run on every PR.
There's a lot of hand-waving in the agent space right now. Most "AI agent" content is either a demo (works once on stage) or a manifesto (works in your head). The dual-spec stack is the third thing: a verifiable agent.
You can read every line of my SKILL.md and check it against the open spec. You can run waza check and see the exact compliance score. You can run waza run and watch a real model reproduce the behavior. And when something breaks, you know which layer broke — because each layer has one job.
This is what I think production AI engineering actually looks like in 2026:
Each layer is replaceable. Each is measurable. None of them lock you in. That's the kind of architecture that survives a model upgrade, a runtime swap, or a vendor change without a rewrite.
aria-backup, to snapshot the workspace memory to a private mirror — a small enough capability to add a fourth grader type and stress-test cross-skill routing.mock-executor pre-commit hook so I can validate the eval pipeline structure on every commit, with the real copilot-sdk run gated to the GitHub Action.If you're building agents and you're not running them through both an authoring spec and an eval framework, you're doing it on vibes. The tools to stop doing that are sitting there, both open source, both well-documented, both shipping new releases this month. Wire them together.
The full Tula repo, including both skills and the complete eval suites, is open source. The architecture is reproducible — clone, run waza check and waza run, and you'll see the same numbers I did.