Correct answer known from: Official counts for the options. The larger set reduces sampling uncertainty but does not validate the difficulty labels.
Guessing scores 0.
Results in the task’s own metric
Dresden, Germany · Accuracy ↑ · 60 items · Constant baseline: 33.333%
Recomputed from saved answers. Model labels retain the original version and repeat protocol. No overall model average.
| Model | Raw result | Valid / asked |
|---|---|---|
| MiniMax-M3 (API, temp 1.0, majority of 5 runs) | 48.333% | 60 / 60 |
| DeepSeek-V4-Flash (API, temp 1.0, majority of 3 usable runs) | 40.000% | 60 / 60 |
| GLM-5.2 (API, temp 1, majority of 3) | 40.000% | 60 / 60 |
| Gemma-4-26B (API, temp 1, majority of 5) | 40.000% | 60 / 60 |
| MiniMax-M3 (API, temp 1, majority of 3) | 40.000% | 60 / 60 |
| gpt-oss-120B (API, temp 1, majority of 3) | 38.333% | 60 / 60 |
| Llama-3.3-70B (API, temp 1, majority of 5) | 36.667% | 60 / 60 |
| Qwen3.8-27B (API, temp 1.0, majority of 4 usable runs) | 31.667% | 60 / 60 |
Loading…
Benchmark task
Question-by-question comparison
Show the full prompt sent to each model Hide the full prompt sent to each model
Scenario matrix
Best model per axis
mean judge score across all cells (0–100). Winner per column highlighted.
50
70
90
Click any cell to load its prompt and responses into the browser below.
Prompts & responses
Pick a scenario here, or click a cell in the heat-map above.
Show the prompt sent to each model Hide the prompt sent to each model identical for all models
All cases
Model responses
Each model receives the exact same prompt above.