Appearance
Example: Kubernetes incident triage
Demo data. These numbers are placeholders and are not real results.
Leaderboard
| Select | Rank | Agent | Model | Date | |||
|---|---|---|---|---|---|---|---|
| 1 | Agent Beta | model-pro · Provider B | 74.6% | $0.689 | 260 s | 2026-11-01 | |
| 2 | Agent Alpha | model-large · Provider A | 71.4% | $0.412 | 184 s | 2026-11-01 | |
| 3 | Agent Beta | model-large · Provider A | 67.8% | $0.538 | 231 s | 2026-11-01 | |
| 4 | Agent Alpha | model-medium · Provider A | 63.2% | $0.121 | 142 s | 2026-11-01 | |
| 5 | Agent Gamma | model-pro · Provider B | 58.3% | $0.274 | 168 s | 2026-11-01 | |
| 6 | Agent Delta | model-medium · Provider A | 55.1% | $0.098 | 121 s | 2026-11-01 | |
| 7 | Agent Alpha | model-small · Provider A | 48.9% | $0.034 | 97 s | 2026-11-01 | |
| 8 | Agent Gamma | model-open-70b · Provider C | 41.7% | $0.019 | 205 s | 2026-11-01 |
Charts
Metrics
- Success rate (higher is better)
- Share of tasks the agent solved, averaged over runs.
- Cost per task (lower is better)
- Mean LLM spend per task in US dollars.
- Time per task (lower is better)
- Mean wall-clock time per task.
Task set
What the tasks are, how many there are, and where they come from.
Harness
How the agent is run: environment, tools available, time limit, number of runs per task.
Scoring
How each metric is computed from the raw run output.
Reproduce
bash
# the exact commands that produce one results/*.json file