Have you ever aced a compliance or safety module by ruling out the obviously wrong answers? And if you missed one, retaken it until you got them all right? Now think about how much of it you remembered a week later.
Regardless of what you actually learned, that module shows up as "100%, complete" on a performance dashboard. As someone who cares about employee upskilling and training, it tells them almost nothing about whether their team learned anything.
When a company pays for training, it does not want the knowledge to last a few hours. It wants it to last the whole time that person works there. A multiple-choice quiz can show that someone recognizes the right answer. It cannot show that they can say it out loud, to an upset customer, with little time to think.
Recognizing is not knowing
Picking an answer from a list feels like knowing it. It usually is not.
Jeffrey Karpicke and his colleagues found that most learners study by rereading, and very few test themselves (Karpicke, Butler & Roediger, 2009). Rereading makes the material feel familiar, and learners mistake that familiar feeling for mastery. A quiz that only asks them to spot the answer they have seen before feeds the same illusion.
The people with the least skill are often the most confident. Kruger and Dunning (1999) found that the lowest scorers in their studies overestimated their own performance the most. Researchers still debate how big that effect is, but the lesson for training holds: confidence is not evidence.
The fix is to make people produce the answer instead of picking it. In a landmark study, students who practiced by recalling what they had read remembered much more a week later than students who read it again (Roediger & Karpicke, 2006). The more a learner has to work to get to an answer, the longer it lasts.
Test the real task
Assessment researchers have a name for the alternative: authentic assessment. It means judging people on tasks that look like the real work, not on stand-ins for it (Wiggins, 1990). If the job is a spoken conversation, choosing option B on a screen is the wrong test.
How Mentor126 asks for proof
Mentor126 still uses Quizzes for quick checks. This helps guide the pace and content throughout a course. When a lesson needs proof that learning is taking place, it uses open-ended Assessments and Roleplays instead.
- •Learners write or speak their answer. Nothing to pick from, nothing to eliminate.
- •A rubric written for that lesson sets the score. Each criterion is scored, and the total is added up from those scores.
- •The screen closes around the Assessment. When a learner starts one, the chat and the input bar disappear until they submit.
- •Only a passing grade completes it. The Mentor has no other way to mark it done. A learner cannot talk their way past it, and the Mentor cannot wave them through.
So when a manager sees "complete" in Mentor126, it does not mean they simply clicked through the module, it means the learner produced the answer and met the standard.
The takeaway
If we want teams that can handle high-stakes, unpredictable moments, we have to stop rewarding the illusion of competence. Stop handing people multiple-choice answers on a silver platter and expecting them to handle a live call on Monday morning. In the real world, nobody gives you four options. You have to come up with your own answer.
References
Karpicke, J. D., Butler, A. C., & Roediger, H. L. (2009). Metacognitive strategies in student learning: Do students practise retrieval when they study on their own? Memory, 17(4), 471–479.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121–1134.
Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255.
Wiggins, G. (1990). The case for authentic assessment. Practical Assessment, Research & Evaluation, 2(2).
