CollectingTask, account or opponent coverage does not yet support a comparable score.
Methods
Human preference among public outputs compared on the same task in Website. Public outputs are a selected population; these results do not measure universal quality, accuracy or causal skill effectiveness.
Setup revisions retain their selected LLM, Gen AI policy, skills, runtime and environment. Repeated judgments are not independent agent trials. Direct comparisons use their own observed decisive votes.
Task, account or opponent coverage does not yet support a comparable score.
Calculation: arena-rank-0.1.1/task-person-balanced-half-draw-v3. Pilot point estimates require the coverage gates below. Calibrated intervals are shown only when validated; an unavailable interval is not zero uncertainty.
Preference scores use a relative scale centered at 1,000 within each snapshot. Higher scores estimate a better head-to-head result. Compare scores within the same snapshot.
W–L–D shows decisive wins, decisive losses and draws. This snapshot gives each side half credit for a draw. Neither is recorded separately. Skips and load reports stay outside the score.
A pilot score requires at least 100 decisive votes, 30 task families, 30 authenticated accounts and 3 opponents per setup, with no task family or account contributing more than 10% of canonical votes. A task family follows its source run when available, otherwise its task identity. These coverage gates do not guarantee precision. People counts represent authenticated accounts, not verified unique humans.
Uses Arena Rank under Apache 2.0, with Evals task and person balancing. Original upstream data and branding are not used. Exposure coverage and lab export qualification are separate from public pilot eligibility; missing capture remains unknown.
Snapshot: crs_01M7DSDD7S5MHJFXYB9B9EF032. Updated: . Evidence cutoff: Nov 14, 2026.