“Is It Time To Ban Computers From Classrooms?” asks the title of an NPR blog post by Tania Lombrozo. Her post is a mostly accurate summary of a new working paper by Carter, Greenberg and Walker: “The Impact of Computer Usage on Academic Performance: Evidence from a Randomized Trial at the United States Military Academy” reporting the results of a study conducted in Spring 2015 and Fall 2015 with 726 West Point students taking an introductory Economics course.
The punchline of the paper is that allowing laptop or tablet computers in the classroom lowered final exam scores by 0.18 standard deviation (1.7% – the mean was 72% with a standard deviation of 9%). The students were randomly assigned to sections, and each professor taught both a section that allowed computer use and a section that banned computer use. All sections took the same final exam, that included multiple choice, short answer, and essay questions. The authors could thus account for professor effect, GPA, and student self-selection, making this truly a randomized controlled study.
The authors argue that the size of the negative effect of computer use could be considerably larger in other college or university settings. The class size at West Point is capped at 18 students per section, and professors are expected to interact with all the students in a class session. West Point students may be more motivated because their overall grades and class rank affect their priority for post-graduation postings, and they are subject to discipline for inappropriate classroom behavior. At other colleges or universities, students in larger classes may may have less motivation to stay on task and experience greater temptation to engage in distracting computer use.
The study methodology and data analysis look really solid. These conclusions are in agreement with previous studies that all reported a negative effect of computer use on student performance. But I say these studies are misdirected.
I’ll start with key details of this study. The authors had 2 different “treatments” in the computer use category, in addition to the sections that completely banned computer use. Some sections had no restrictions on use of laptops or tablets. Other section allowed only tablets (iPads), that had to stay flat on the desktop during class. In lieu of measuring actual student computer use, the professors monitored and recorded student computer use during 3 class sessions over the course of the semester. In the sections that allowed unrestricted computer use, 80% of students used a laptop or tablet during at least 1 of the 3 class session. In the sections that restricted use to tablets that had to stay flat on the desk, only 40% of students used a tablet in at least 1 of the 3 class sessions. Despite the disparity in observed computer use in the two treatments, the supposed negative effect of computer use was indistinguishable (0.18 vs 0.17 standard deviations).
Another key detail is that the negative effect was observed only with the multiple-choice and short-answer questions that were machine-graded. The grades on the essay questions, graded by the professors, were the same between banned, restricted, and unrestricted computer use sections. Although the authors discount the essay question grades as being subjective, they do acknowledge that the essay questions addressed higher-order conceptual thinking than the multiple-choice and short answer questions.
If computer usage is detrimental, why doesn’t it scale with the amount of usage, and why does it not affect higher-order conceptual thinking?
Of the course, the biggest problem with this study and others like it, is that the course did not integrate computers into their classroom lessons. The authors themselves admonish in their concluding paragraph:
We want to be clear that we cannot relate our results to a class where the laptop or tablet is used deliberately in classroom instruction, as these exercises may boost a student’s ability to retain the material.
In other words, there was not a functional role for computers in these classrooms. Computers were irrelevant to learning – only 40% of students used a tablet in the tablet-restricted sections.
I argue that this and other such studies are tantamount to studying the effect of any potentially distracting, useless item in the classroom. The effect of pet rocks in the classroom. The effect of goldfish in the classroom. The effect of allowing eating candy in the classroom. There is no potential upside to having computers, or pet rocks, or goldfish, or candy in the classroom, if the lesson does not use any of these things. These studies are asking the wrong or irrelevant questions. What I’d really like to see are studies of how students using computers can enhance mastering of particular learning objectives, compared to other methods.
So by all means, if you are teaching in a way that has no use for computers, ban them. But in this information age, I prefer to challenge my students to explore, analyze, evaluate, create, and communicate with the universe of information and computing tools now available to them.