Back to Search-quality version diff: v1.1.0 → v1.2.0 (canonical-lookup-V2 (independent corpus))

    flowchart TD
        Q["30 canonical-lookup query strings (corpus V2)
    zero overlap with the original 50-query corpus"]:::input
        R["per-query regex pattern
    accepts canonical + specific-variant URIs"]:::input
        Q --> H[scripts/eval/search-quality-phase1-extended.py]
        R --> H

        BA["Arm A: brew cupertino 1.1.0
    + ~/.cupertino/search.db (schema 13)"]:::arm
        BB["Arm B: dev cupertino 1.2.0
    + ~/.cupertino-dev/search.db (schema 18)"]:::arm
        H --> BA
        H --> BB

        BA --> SA["per-query rank → RR, P@1, P@5, NDCG@10"]
        BB --> SB["per-query rank → RR, P@1, P@5, NDCG@10"]

        SA --> PAIR["paired buckets"]
        SB --> PAIR
        PAIR --> WX["Wilcoxon signed-rank"]:::stat
        PAIR --> MC["McNemar 2×2"]:::stat
        WX --> MD["audit MD"]:::out
        MC --> MD

        classDef input fill:#0a84ff,stroke:#0040cc,color:#fff
        classDef arm fill:#5856d6,stroke:#3634a3,color:#fff
        classDef stat fill:#ff9500,stroke:#c4730a,color:#fff
        classDef out fill:#34c759,stroke:#1f7a3a,color:#fff