# MCP Context Gateway — scenario scorecard

> Renders ``benchmarks/results/gateway_latest.json``. Generated by ``scripts/render_gateway_scorecard.py``. **Do not edit by hand** — re-run ``make gateway-scorecard`` after the benchmark.

`benchmark_version = 1.0` · `catalog = examples/architectures/mcp_context_gateway/catalog.yaml` · `scenarios = 5`

## Headline range

Across **5 gateway scenarios** the firewall reduction lands between **0.0%** (small payload — firewall correctly no-ops) and **98.8%** (largest payload). Raw upstream payload sizes span **34–16,507 chars**; injected summaries land at **34–501 chars**.

The shape of the range — including the explicit ``0.0%`` floor — is the load-bearing guarantee that the firewall does not pay any tax on small responses. Anything that fits under the configured threshold (`firewall_threshold=2000` here) flows through the prompt verbatim.

## Per-scenario table

| scenario | raw chars | summary chars | reduction | artifact | answer tokens | answer chars |
|---|---:|---:|---:|:-:|---:|---:|
| `tiny_ack` | 34 | 34 | 0.0% | — | 34 | 218 |
| `small_post` | 77 | 77 | 0.0% | — | 41 | 244 |
| `medium_ticket` | 238 | 238 | 0.0% | — | 85 | 423 |
| `large_log` | 14,206 | 501 | 96.5% | ✓ | 153 | 691 |
| `bigquery_rowset` | 16,507 | 194 | 98.8% | ✓ | 72 | 377 |

## Per-scenario detail

### `tiny_ack`

- **user query**: `Acknowledge the pending PagerDuty incident PD-9931`
- **routing query**: `Acknowledge a PagerDuty incident`
- **chosen tool**: `pagerduty.incidents.ack`
- **catalog size**: 60 tools; **exposed cards**: 5
- **firewall**: 34 chars → 34 chars (0.0%); artifact persisted: no
- **answer phase**: 34 tokens, 218 chars

### `small_post`

- **user query**: `DM alice on Slack to confirm the rollback`
- **routing query**: `Send a direct Slack message to alice`
- **chosen tool**: `slack.dm.send`
- **catalog size**: 60 tools; **exposed cards**: 5
- **firewall**: 77 chars → 77 chars (0.0%); artifact persisted: no
- **answer phase**: 41 tokens, 244 chars

### `medium_ticket`

- **user query**: `Search Linear for the C-12345 plan-change ticket`
- **routing query**: `Search Linear tickets matching customer C-12345 plan change`
- **chosen tool**: `linear.tickets.search`
- **catalog size**: 60 tools; **exposed cards**: 5
- **firewall**: 238 chars → 238 chars (0.0%); artifact persisted: no
- **answer phase**: 85 tokens, 423 chars

### `large_log`

- **user query**: `Pull recent error logs for the api-gateway outage window`
- **routing query**: `Query the events table for api-gateway error logs`
- **chosen tool**: `analytics.events.query`
- **catalog size**: 60 tools; **exposed cards**: 5
- **firewall**: 14,206 chars → 501 chars (96.5%); artifact persisted: yes
- **answer phase**: 153 tokens, 691 chars

### `bigquery_rowset`

- **user query**: `Why did customer C-12345's MRR drop last month?`
- **routing query**: `Execute a BigQuery query to find MRR delta rows for customer C-12345`
- **chosen tool**: `bigquery.run_query`
- **catalog size**: 60 tools; **exposed cards**: 5
- **firewall**: 16,507 chars → 194 chars (98.8%); artifact persisted: yes
- **answer phase**: 72 tokens, 377 chars

## Reading this honestly

These are deterministic single-call scenarios over the same 60-tool catalog. The range is wider than the previous “single-call gateway scenario” anecdote in ``docs/benchmarks.md`` (one number, one payload) — *because* the harness now covers small payloads where the firewall correctly does nothing. Picking the high end (``bigquery_rowset``, 98.8 %) as a headline is dishonest in isolation; this table is the honest framing.

