You can't improve what you can't see

Overview
A right or wrong answer tells you a student is stuck. It doesn't tell you why. This study of a Swedish school district's math diagnostic testing shows what gets lost when a test can only see the final answer.

About this research
Conducted by the Magma Math Research Lab in partnership with KTH Royal Institute of Technology, this study examines how Haninge municipality (Sweden) uses a districtwide math diagnostic test for incoming high school students, and where a multiple-choice format runs into its own limits.

What You'll Learn

Overview
A right or wrong answer tells you a student is stuck. It doesn't tell you why. This study of a Swedish school district's math diagnostic testing shows what gets lost when a test can only see the final answer.

About this research
Conducted by the Magma Math Research Lab in partnership with KTH Royal Institute of Technology, this study examines how Haninge municipality (Sweden) uses a districtwide math diagnostic test for incoming high school students, and where a multiple-choice format runs into its own limits.

What You'll Learn

• Why even a statistically well-built multiple-choice test, one where every question reliably separated strong performers from weak ones, still couldn't tell administrators why students were getting questions wrong.
• How four specific test questions were answered incorrectly by more students than answered them correctly, and what those wrong answers revealed about the misconceptions behind them.
• Why the researchers' central recommendation was to capture students' steps, not just their final answers, and what that would have changed about the results.
• Where a real district's teachers and administrators weren't reading the same data the same way, and what that gap cost them.

Rebecca Dou

Title Organization