Given one traveller, what each of air, train, bus and car would cost them in money and time, and their household income, give the probability that they chose each mode.
Sydney–Canberra–Melbourne, Australia · Brier (sum over modes) ↓ · 210 items · Constant baseline: 0.734
Recomputed from saved answers. Model labels retain the original version and repeat protocol. No overall model average.
| Model | Raw result | Valid / asked |
|---|---|---|
| Gemma-4-26B (API, temp 0) | 0.724 | 210 / 210 |
| Llama-3.3-70B (API, temp 0) | 0.757 | 210 / 210 |
| gpt-oss-120B (API, temp 0) | … | 208 / 210 |
Correct answer known from: The mode each of 210 travellers actually chose, from a 1987 intercity survey between Sydney, Canberra and Melbourne.
The historical baseline uses mode shares in this choice-based sample, not population shares. The new probability protocol fits its baseline on development cases only.
Click any cell to load its prompt and responses into the browser below.
Pick a scenario here, or click a cell in the heat-map above.
Each model receives the exact same prompt above.