Reciprocal Rank Fusion (k=60)
Cormack, Clarke, Büttcher (2009), SIGIR
Open citationThis audit answers a focused question: when a developer queries a concept that exists in both modern Swift form (value type / stdlib) and the original NS-prefixed Objective-C form, does cupertino's ranker put the modern form above the legacy form? This is the most user-visible failure mode for an AI coding agent (Criterion 2: an agent told to use NSURLConnection when URLSession is the right answer in 2026 generates wrong code).
Each card opens its own page. The headline and charts above are all you need at a glance; the cards are for the why and how.
For the other 23 pairs the Obj-C form did not appear in the top-10 at all, which is a stronger statement than "Swift higher in pair": the legacy form is not surfaced at all under a developer's natural query.
Read details →The 30 pairs cover three concept families:
Read details →The 7 pairs where both forms appeared in the top-10. These are the rows where the ranker had to choose between two valid alternatives.
Read details →Cupertino's BM25F weights combined with the RRF source authority produce a strong preference for the modern Swift form. The behaviour is not accidental:
Read details →Deprecation-axis failures are one of the four explicit anti-hallucination concerns in docs/design/cupertino.md §1.1 ("avoid generating code that ... uses deprecated APIs").
Read details →30 (query, swift_URI, objc_URI) triples. For each: run cupertino search "<query>" --limit 10 via the dev binary, find rank of swift_URI and rank of objc_URI in the top-10.
Read details →Two of the eight query classes from §1.4 now have documented baselines on the v1.2.0 candidate DB. Six remain (C acronym, D CamelCase fragment, F cross-source, G prose, H symbol-attribute, and the Phase 1.7 agent-eval).
Read details →Every metric and method this audit relies on, with a link to the foundational source. Auto-collected from the audit text.
Cormack, Clarke, Büttcher (2009), SIGIR
Open citationVoorhees (1999), TREC-8 QA Report
Open citationManning, Raghavan, Schütze (2008) IIR §8.4
Open citation