This is an automated archive made by the Lemmit Bot.

The original was posted on /r/singularity by /u/pigeon57434 on 2025-02-26 16:19:40+00:00.

Original Title: LMArena is actually useful now! Introducing Prompt-to-Leaderboard a system that generates a custom leaderboard for any prompt giving infinitely granular control and more accurate rankings from LMArena

they also released a technical paper about it

you can run any prompt you want and it will generate a leaderboard for answering that specific prompt so apparently if you want specifically this prompt answered this is the leaderboard for this prompt and this prompt only

or you can explore their premade leaderboard for many niche categories for example if you want to know what model is the best at a very niche specific type of puzzle here you go

this should make it so you can use LMArena for you specific niche use cases which makes the rankings more accurate because many people complain that models like gpt-4o score so high on the overall category but in here you get more granular results for more granular question sets making the arena actually useful again

they also mention this could be used as a router because if you know the best model for each prompt you can just route to that model and get the best possible answer any model can offer to any question no matter the question the tested this on lmarena under “experimental-router-0112” and got higher performance than any single model by itself