Veronica Yan, a PhD student in the Bjork cognitive psychology lab at UCLA, gave a talk tonight at Pearson’s Mastering Leadership Conference about different study practices and student learning. I heard Robert Bjork talk about “desirable difficulties” a couple of years ago at Pearson’s Biology Leadership Conference, and it transformed the way I teach and how I advise students to improve their study practices. The research findings, that spaced study is superior to massed study, that testing (retrieval practice) is superior to repeated study, and that interleaved study is superior to blocked study, are all counter-intuitive, but robust and reproducible in multiple contexts. Robin Heyden summarized Robert Bjork’s talk very nicely in her blog post so I won’t repeat those points here. In any case, the NY Times also ran a story in 2010. Veronica Yan summarized some of the same work, and expanded into new but related territory.
Multiple Choice Questions
After relating the results that showed the remarkable effects of retrieval practice (repeated testing) on long-term memory, Veronica discussed multiple choice tests, a “necessary evil” for those of us who teach large classes. Poorly designed multiple choice questions, with non-competitive distractors (answers that are clearly wrong), encourage students to answer via pattern-recognition, and do not elicit the cognitive benefits of retrieval practice. However, multiple-choice questions with competitive distractors force students to engage in ways that benefit recall of related information. Students given multiple choice questions with competitive lures performed better on a subsequent cued recall test of related information, than students given questions with non-competitive lures. This improved recall of related information occurred even if the students answered the original multiple choice questions incorrectly.
This brings up the fascinating idea that making mistakes can enhance learning, as long as the mistakes are corrected. Veronica Yan described an experiment where some subjects are given 13 seconds to study a word association, such as whale: mammals. Another group of subjects are given 8 seconds to guess at the association of whale: ___?, then shown whale: mammals for 5 seconds. In later tests, the group that initially guessed at alternative associations (such as whale:dolphin) did better at remembering the correct associations that those who had studied just the correct associations.
She then discussed research by Kapur and Bielaczyc (2012) on “productive failure.” Groups of students in 3 Singapore schools worked on complex problems. The “productive failure” groups worked for 6 periods with no instruction or assistance from their teacher, and then received 1 period of instruction. These students got 0%, 7%, and 16% correct solutions to their complex problems. The “traditional instruction” group received 7 periods of directed instruction from their teacher, and achieved 91%, 93%, and 92% correct solutions. But on a post-test, where students were given 3 new well-structured problems, 1 complex problem, and 1 graphic representation problem, the “productive failure” group performed much better!
Veronica Yan compared this to the common Japanese teaching practice, where students work on complex problems and write out their incorrect solutions for all to see, and receive diagnostic correction. Exploring mistakes and receiving corrective feedback appears to be a powerful way of learning.
Attitudes and assumptions that impede learning
Veronica Yan concluded by suggesting that students and teachers need to change attitudes toward mistakes and learning. Massed and blocked study feels more effective, and yields better short-term results. More effective study that is spaced and interleaved feels more difficult, and takes more effort, but yields better long-term results. Indeed, if learning feels easy, then you may not really be learning.
I thought about my own students’ mixed reactions to my efforts to institute “desirable difficulties” in my classes. Some get angry that I’m not teaching, and say so in their course evaluations. Others get discouraged when they feel confused, and give up. I fear that in making mistakes, they say to themselves, “I’m just bad at this” or “this must not be the right subject for me”. I fear even more that making mistakes may reinforce any stereotype threat. I raised these concerns with Veronica after her talk, and she suggested that we need to find ways to help students change their attitudes and be better informed about how they learn best.