COVID Learning Loss Comparable to That Inflicted by Hurricane Katrina, Study Finds; Math Drops Outpace Reading

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter

Learning loss during the pandemic may have exceeded the damage inflicted on New Orleans students by Hurricane Katrina, according to a recently released study of standardized test scores. Setbacks in math achievement exceeded those for reading, and achievement gaps between comparatively rich and poor schools expanded dramatically.

As the United States approaches the end of another school year marked irrevocably by COVID, the paper’s findings are merely the latest to show substantial academic reversals resulting from lengthy school closures, uneven execution of virtual learning, and the effects of countless social and economic upheavals over the last two years.

Study co-author Megan Kuhfeld, a senior research analyst at the nonprofit research group NWEA, declared herself “surprised in some ways that the results were a lot bigger” than those following the seminal natural disaster of recent American history. But while the “concentration of disrupted schooling” immediately following Katrina was especially intense, Kuhfeld argued, most affected children were relocated and enrolled in new schools within a matter of months. 

“If you think about how prolonged the disruptions were, it does make sense to me that we saw larger drops than we did during the hurricane,” she said. 

The study, circulated as a working paper by Brown University’s Annenberg Institute for School Reform, examines student performance on NWEA’s Measures of Academic Progress (MAP) test, which assesses growth in both reading and math. The test is typically administered in participating K-12 schools three times during the school year, and results include demographic information detailing student age, gender and race or ethnicity.

All told, the study’s sample comprised roughly 5.4 million students in grades 3–8 across a swath of 12,000 elementary and middle schools; those schools represented between 12 percent and 15 percent of all American schools in each grade. To illustrate the learning trajectory of these students during the pandemic, Kuhfeld and her collaborators tracked the scores of same-age test-takers during the fall of 2019, 2020, and 2021.

As expected, the arrow pointed downward over those three academic years. Average math scores dropped by a range of 3.3 points to 4.5 points (sixth graders and eighth graders, respectively)during the period under examination, while reading scores dropped between 1.5 and three points (eighth graders and third graders, respectively).

To put those figures in perspective, the authors followed up with a March essay released by the Brookings Institution, which compared the negative impact captured on the MAP tests to the positive growth catalyzed by various education policy interventions in separate studies. The pandemic-related hit to math scores among elementary schoolers between 2019 and 2021 was nearly three times as large as the benefit that similar students have received from a specialized course of summer instruction in the subject. Score slippage for middle schoolers was larger in magnitude than the boost that comparable children received from high-dosage tutoring, one of the most effective education interventions that has ever been measured. 

Mike Magee, who recently stepped down as CEO of the education advocacy group Chiefs for Change, called the study “yet another indication that an alarming number of students have fallen far behind during the pandemic.” He called for a range of remedies — including extending the school day, offering higher-quality instructional materials, and deploying more tutors — to arrest the downward momentum in achievement.

“Covid has exacerbated inequities and widened gaps,” Magee wrote in an email. “If schools do not have an uncompromising focus on academics, the lasting consequences will be catastrophic — and Black and brown children, and those who live in poverty, will hurt the most.”

Indeed, the study also detected widening disparities in academic performance between students of different levels of socioeconomic advantage. In just two years, the already sizable gap in reading performance between students enrolled at relatively high-poverty schools (those in which over 75 percent of students are eligible to receive free or reduced-price lunch) and relatively low-poverty schools (those in which fewer than 25 percent are eligible) expanded by 15 percent. The gap in math performance grew by 20 percent.

Longtime education observer Tom Loveless, the former director of Brookings’s Brown Center on Education Policy, was struck by the sheer toll exacted by COVID’s disruption to learning, but also the smaller stories within the learning loss.

“What’s interesting is that the losses from the pandemic seem to be the opposite of the learning gains registered over the past three decades,” Loveless wrote in an email. “Since the early 1990s, greater gains were made by younger kids than older kids, and math gains outpaced gains in reading. Now — at least in early studies of pandemic effects — we are seeing greater losses by younger students compared to older ones and a larger setback in math compared to reading.”

The explanation for the subject-specific differences in MAP scores is still unclear, though Loveless added that math performance may simply be “more sensitive to schooling” — and therefore vulnerable to decline when schools shut down. Evidence for that theory can be found in a curious time lag: Even after the sudden impact of school closures in the spring of 2020, reading scores measured that autumn were still only modestly affected, sinking much further over the course of the 2020-21 school year. But significant declines in math scores were apparent in fall 2020 MAP performance.

Kuhfeld agreed that previous research has shown that math learning “seems to be more sensitive to time in school,” which could account for the steeper backsliding in that subject. But she added that the MAP results can’t capture the lost growth among early readers in the first and second grades, who were just beginning to develop the foundational skills of literacy when COVID hit.

“Those kids who were first-graders when the pandemic started are now in third grade,” Kuhfeld observed. “There’s probably going to be a cumulative effect where kids who missed out on those formative, learning-to-read experiences will probably see even larger negative effects on reading than we’ve observed so far.”

Further research will additionally clarify the extent to which the educational consequences of the pandemic differed among students in different learning environments, she added. One study — circulated last fall by a team of academics including Brown economist Emily Oster — has already found that learning loss was much greater in school districts that kept remote schooling for longer. Kuhfeld argued that there was likely to be “huge variation” based on the time children spent in physical classrooms.

“It was probably worse for certain groups that either didn’t have access to in-person instruction, or maybe had access but chose not to go for health or other reasons,” she said. “The kind of scary part is knowing that this was the average effect; there were certainly a number of students who had larger drops than what we’re reporting.”

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter