Mean Reciprocal Rank
Voorhees (1999), TREC-8 QA Report
Open citationThis audit tests framework_aliases.synonyms — the cupertino-specific table that maps colloquial / abbreviated names to canonical framework slugs (nfc → corenfc, wifi → corewlan, bluetooth → corebluetooth, ml → coreml, etc.). The expectation: when a developer types the synonym alone as a query, cupertino should route to the canonical framework root via the synonyms table.
Each card opens its own page. The headline and charts above are all you need at a glance; the cards are for the why and how.
This is the worst-performing class baseline by a wide margin.
Read details →All four wins are explainable by literal-substring or prefix-token matching, not by synonym lookup. The synonyms table may not be contributing to these results at all.
Read details →Every miss is a literal-token match against a deeper path that happens to contain the query word. The canonical CoreX framework is either far down the ranking (rank 2-7) or absent from the top-10 entirely.
Read details →The framework_aliases table exists in the schema and contains 22 rows with a populated synonyms column (verified directly via SELECT identifier, synonyms FROM framework_aliases WHERE synonyms IS NOT NULL).
Read details →Following the feedback_code_changes_as_ideas_for_future rule, three candidate paths, in increasing complexity:
Read details →For the AI agent consumer: if the agent issues a bare NFC query expecting the CoreNFC framework reference, it gets an authentication-services deep page instead.
Read details →22 (synonym, canonical_URI) pairs, all drawn from framework_aliases.synonyms rows in the v1.2.0 candidate DB (excluding data and text as too-generic).
Read details →Five of eight Phase 1.x classes from §1.4 now have documented baselines. Three remain: G (prose), H (symbol-attribute), Phase 1.7 (anti-hallucination agent-end-to-end).
Read details →Every metric and method this audit relies on, with a link to the foundational source. Auto-collected from the audit text.
Voorhees (1999), TREC-8 QA Report
Open citationCormack, Clarke, Büttcher (2009), SIGIR
Open citationManning, Raghavan, Schütze (2008) IIR §8.4
Open citation