Elo Rating System For Dynamic Model Ranking

Crowdsourced LLM evaluation — side-by-side blind voting, Elo ratings, most trusted LLM benchmark.

Unique: Adapts classical Elo (designed for chess) to handle asymmetric match counts and variable model availability. Includes mechanisms for rating inflation/deflation correction and handles new models entering the arena without requiring manual calibration.

vs others: More responsive to preference shifts than static leaderboards, and more principled than simple win-rate percentages because it accounts for opponent strength