Back to dashboard
Headline result
30 / 30 (100%)
Method & source Reciprocal Rank Fusion (k=60)Cormack, Clarke, Büttcher (2009), SIGIR

## Read in detail

Each card opens its own page. The headline and charts above are all you need at a glance; the cards are for the why and how.

### Aggregate

For the other 23 pairs the Obj-C form did not appear in the top-10 at all, which is a stronger statement than "Swift higher in pair": the legacy form is not surfaced at all under a developer's natural query.

Read details →

### What the Pairs Look Like

The 30 pairs cover three concept families:

Read details →

### Both-present Subset (the Strict Test)

The 7 pairs where both forms appeared in the top-10. These are the rows where the ranker had to choose between two valid alternatives.

Read details →

### What This Says About the Ranker

Cupertino's BM25F weights combined with the RRF source authority produce a strong preference for the modern Swift form. The behaviour is not accidental:

Read details →

### Implications for Criterion 2 (anti-hallucination)

Deprecation-axis failures are one of the four explicit anti-hallucination concerns in docs/design/cupertino.md §1.1 ("avoid generating code that ... uses deprecated APIs").

Read details →

### What This Baseline Does Not Cover

Read details →

### Method Recap

30 (query, swift_URI, objc_URI) triples. For each: run cupertino search "<query>" --limit 10 via the dev binary, find rank of swift_URI and rank of objc_URI in the top-10.

Read details →

### Combined With the Canonical-lookup Baseline

Two of the eight query classes from §1.4 now have documented baselines on the v1.2.0 candidate DB. Six remain (C acronym, D CamelCase fragment, F cross-source, G prose, H symbol-attribute, and the Phase 1.7 agent-eval).

Read details →

## Sources cited in this measurement

Every metric and method this audit relies on, with a link to the foundational source. Auto-collected from the audit text.

### Reciprocal Rank Fusion (k=60)

Cormack, Clarke, Büttcher (2009), SIGIR

Open citation

### Mean Reciprocal Rank

Voorhees (1999), TREC-8 QA Report

Open citation

### P@k (Precision at k)

Manning, Raghavan, Schütze (2008) IIR §8.4

Open citation