Correct answer known from: The mode each traveller actually chose, from the ModeCanada survey.
The sample's own shares score 0, and they are a strong answer here: car and air take 86 per cent of the choices between them.
Results in the task’s own metric
Montreal–Toronto, Canada · Brier (sum over modes) ↓ · 200 items · Constant baseline: 0.608
Recomputed from saved answers. Model labels retain the original version and repeat protocol. No overall model average.
| Model | Raw result | Valid / asked |
|---|---|---|
| Llama-3.3-70B (API, temp 0) | 0.702 | 200 / 200 |
| gpt-oss-120B (API, temp 0) | 0.709 | 200 / 200 |
| Gemma-4-26B (API, temp 0) | 0.710 | 200 / 200 |
Loading…
Benchmark task
Question-by-question comparison
Show the full prompt sent to each model Hide the full prompt sent to each model
Scenario matrix
Best model per axis
mean judge score across all cells (0–100). Winner per column highlighted.
50
70
90
Click any cell to load its prompt and responses into the browser below.
Prompts & responses
Pick a scenario here, or click a cell in the heat-map above.
Show the prompt sent to each model Hide the prompt sent to each model identical for all models
All cases
Model responses
Each model receives the exact same prompt above.