How it works

1. Sign in and open Quick Eval from the home page.
2. Select one or more candidate models and a judge model from the catalog (platform, sponsor, or your saved models).
3. Pick a prompt from the public or private library (or use your own saved prompts).
4. Start the run — models generate answers in parallel (or sequentially for precise timing). Outputs stream in real time.
5. When generation finishes, the judge model scores each output. Review scores, confirm or reject them, and export results if needed.

For larger comparisons, create a batch task with multiple prompts and models. The same judge pipeline runs per prompt; you can track progress and compare aggregates when the task completes.

1. Create a task, select prompts and models, and choose a judge.
2. Run the task — each prompt × model combination is executed and scored.
3. Review per-prompt results, task-level aggregates, and exports.

After models finish generating, their outputs are sent to a judge model you select (separate from the candidates) for automated multi-dimension scoring. You can accept or reject scores; acceptance feeds into leaderboard weighting. Judge scores are informational — always review before production use.

Eligible completed evaluations from public prompts may contribute aggregated model scores to public leaderboards. Your account email and raw prompts are not shown on the board. Use private prompts or non-public tasks if you do not want results to qualify. See our Privacy Policy for details.

If you add custom provider credentials, they are stored encrypted and used only to call the providers you select when you start an evaluation. Keys are never shown to other users. See Privacy Policy §5 for encryption and access controls.

• Global leaderboard algorithm (full documentation)
• Privacy Policy
• Help & FAQ
• Start Eval →