Measured traffic knowledge in Dresden. Inspect each task, its reference data and the answers behind the scores.
Compare models within a task. Select a model score to inspect the questions and answers.
Given ten named street segments in Dresden, estimate how many vehicles use each one on an average day.
Reference The city's own traffic counts for those segments, taken from the Dresden open data portal.
Scoring fold error · Skill score: 100 exact, 0 baseline, negative below baseline.
For each of twelve questions, choose which of three Dresden street segments carries the most traffic.
Reference The official count recorded for each of the three options, with dates and measurement methods in the answer key.
Scoring MC skill, chance-corrected · Skill score: 100 exact, 0 baseline, negative below baseline.
Choose the busiest of three Dresden street segments in each of sixty questions.
Reference Official counts for the options. The larger set reduces sampling uncertainty but does not validate the difficulty labels.
Scoring MC skill, chance-corrected · Skill score: 100 exact, 0 baseline, negative below baseline.
For fifteen Dresden segments, estimate what share of the traffic is heavy vehicles.
Reference The city's own heavy-vehicle counts for those segments.
Scoring mean abs. error, pp · Skill score: 100 exact, 0 baseline, negative below baseline.
For fifty combinations of city, policy goal and target year, propose a package of transport measures that would reach the goal.
Reference There is no single right answer here. Each proposal is read by an LLM judge and scored on five axes: effectiveness, cost, political acceptance, local specificity and local correctness.
Scoring 5-axis judge · Judge rating, 0–100. Not factual accuracy.
Prompts, answer keys, model responses and scores.