Correct answer known from: The official count recorded for each of the three options, with dates and measurement methods in the answer key.
Guessing scores 0, because one of the options is always right by chance.
Results in the task’s own metric
Dresden, Germany · Accuracy ↑ · 12 items · Constant baseline: 33.333%
Recomputed from saved answers. Model labels retain the original version and repeat protocol. No overall model average.
| Model | Raw result | Valid / asked |
|---|---|---|
| GPT-5 (no search) | 58.333% | 12 / 12 |
| DeepSeek-V4-Flash (API, temp 1.0, majority of 6 usable runs) | 50.000% | 12 / 12 |
| Gemini (no search) | 50.000% | 12 / 12 |
| GLM-5.3 (API, temp 1.0, majority of 3 usable runs) | 41.667% | 12 / 12 |
| MiniMax-M3 (API, temp 1.0, majority of 10 runs) | 41.667% | 12 / 12 |
| Kimi-K2 (no search) | 33.333% | 12 / 12 |
| Qwen3.8-27B (API, temp 1.0, majority of 9 usable runs) | 33.333% | 12 / 12 |
| Gemma-4-26B (API, temp 1, majority of 5) | 25.000% | 12 / 12 |
| gpt-oss-120B (API, temp 1, majority of 5) | 25.000% | 12 / 12 |
| Llama-3.3-70B (API, temp 1, majority of 5) | 0.000% | 12 / 12 |
Loading…
Benchmark task
Question-by-question comparison
Show the full prompt sent to each model Hide the full prompt sent to each model
Scenario matrix
Best model per axis
mean judge score across all cells (0–100). Winner per column highlighted.
50
70
90
Click any cell to load its prompt and responses into the browser below.
Prompts & responses
Pick a scenario here, or click a cell in the heat-map above.
Show the prompt sent to each model Hide the prompt sent to each model identical for all models
All cases
Model responses
Each model receives the exact same prompt above.