Textbook Adoption Was Once About ‘Going to the Right Dinner’ With Publishers. Now EdReports Is Disrupting the $8 Billion Industry by Putting Teachers in Charge

In early 2011, when Maryland and other states were adopting the Common Core State Standards, teachers in the Baltimore City Public Schools were starting to grumble.

“The materials in a lot of districts fell woefully short of the new standards,” said Sonja Santelises, now Baltimore’s superintendent. “[I was] hearing classroom teachers rightly point out that it’s great to have these standards, but it’s actually unfair to expect teachers to teach to a particular standard and not have the instructional materials to support that.”

“I remember the frustration that I experienced,” she said.

More than two years later, Santelises became a founding board member of EdReports, a nonprofit organization that publishes independent, teacher-led reviews of textbooks. Over the past four years, EdReports has published more than 500 reviews of math, English language arts and science curricular materials, injecting a steady stream of analyses into an $8 billion industry that critics say had long operated in an objectivity-free void. Now states and districts are increasingly relying on EdReports’ reviews, shifting pressure onto publishers to increase the alignment of their materials to the Common Core.

In California, home to more than 3 million students and the country’s largest textbook market, EdReports created a state-specific site, CalCurriculum, to offer localized guidance. Nationally, EdReports says that more than 9 million students are using materials purchased at least in part based on their reviews, which are free and publicly available. Two of the larger school districts in the United States — Baltimore and Charlotte-Mecklenburg — now require a review from EdReports or a similar organization before the district purchases curricular materials. Such requirements are bolstered by EdReports’ reviews of more than 90 percent of existing curricular material in English and math. And the organization counts 22 examples of publishers using their evidence to improve materials.

‘Like Noah’s Ark’

In the telling of multiple EdReports officials, the nonprofit’s origin story is akin to an organizational immaculate conception, when a single meeting launched an industry-upending startup. In April 2011, Bill Gates and 23 other leading education thinkers gathered at Sunnylands, the former estate of philanthropists Walter and Leonore Annenberg, in southern California. Sal Khan of Khan Academies was there. Also there was Carol Dweck, a Stanford researcher whose work focuses on growth mindsets — or “the belief that intelligence can be developed.”

“In certain ways, it was like Noah’s Ark,” said Maria Klawe, the president of Harvey Mudd College, who sat on Microsoft’s board and co-organized the retreat with Gates. “There were two researchers, one for Japan, one from Berkeley. Two state superintendents of education. Two governors.”

The group’s intended focus was technology-enabled math education for K-12, but 48 hours later, Klawe said, there was unanimous agreement that “no one knew” what constituted a really good math curriculum in the Common Core era. The group’s takeaway was that “we should have an organization that can review curriculum and whether or not it’s aligned to the Common Core,” she said.

In the absence of such an organization, states and districts had historically made textbook purchases based in part on “going to the right dinner [with publishing companies], and that’s not how curricula should be decided and adopted,” said Baltimore’s Santelises.

The Common Core faced strong political headwinds as EdReports launched, but today more than half of states use all or most of the standards and are on the hunt for aligned resources, which helps ensure the organization’s relevance.

Based on Consumer Reports, EdReports buys sets of new curricular materials and sends them to trained teams of educators around the country, who are chosen from a pool of more than 400 classroom teachers, instructional coaches and administrators from across 47 states. Each team has five reviewers, and over the course of several months, they peruse every page of the materials, pausing to consult by phone each week and share evidence that will ultimately justify a score on EdReports’ rubric. The team then drafts a review and shares it with the respective publishing company, which may respond in a post that EdReports publishes alongside its review.

The initial results struck a nerve. EdReports published its first collection of math material reviews in 2014, which found that 19 of the 20 curricular series did not align with the Common Core for all grade levels. Publishing companies were not pleased.

The Association of American Publishers said the EdReports process resulted in “a very shallow, incomplete review.” The National Council of Teachers of Mathematics accused EdReports of publishing “reviews that fall short of providing useful and accurate information about many critical features of materials reviewed.” And in response to the critical review of its “Investigations in Number, Data, and Space” curriculum for grades 3-5, Pearson said that EdReports took a “very narrow interpretation, and [fell] short of the true intention of the goals of the Common Core State Standards for Mathematics.”

Eric Hirsch, the founding executive director of EdReports, said the backlash was even greater than he anticipated. “Oh my gosh, that was wow,” he said. “It’s like, ‘Now I’m starting to understand why somebody hadn’t taken on this challenge.’”

Many publishers critique the manner in which EdReports conducts its reviews, requiring materials to go through certain “gateways,” or categorical hurdles, before advancing. If a set of materials are found lacking in the alignment phase, for instance, the review would effectively end, and the EdReports team would not further advance the materials to the usability phase.

Hirsch prides EdReports on being flexible, so he and his team engaged the critics and made slight modifications to the review process (though notably without changing the gateways system). EdReports is keen not to present itself as an adversary of publishing companies or an arbiter of which curricula districts should adopt. The focus, Hirsch says, is to increase awareness of how well materials align with the Common Core. By highlighting quality, EdReports hopes states and districts can make informed decisions about which curricular products to use and publishers will be publicly pressured to continuously improve their products.

This is not to suggest, of course, that greater awareness of high-quality curricular materials — or simply having more of them — will unilaterally improve public schools. EdReports’ own analysis shows that “only a small fraction of students (22 percent in math and 15 percent in ELA) are exposed to aligned curriculum at least once a week,” Hirsch wrote in a recent blog post. A March report from Harvard’s Center for Education Policy Research found little “evidence of differences in achievement growth for schools using different elementary math textbooks and curricula.”

Rather, EdReports hopes that its reviews, and the higher-quality published material they inspire, are the beginning of a broader change in how teachers spend their time. In Rhode Island, where the organization is helping 14 districts make curricular adoption decisions, the assistant commissioner for the state education department, Mary Ann Snider, said better resources mean teachers “don’t have to be the author of all of these curricular materials.” State surveys had previously found that teachers spent an average of 8-10 hours a week finding materials for their classes. “It’s changing everyone’s thinking about what does it mean to be a teacher,” she said.

Given how central they are to curricula, teachers sit on EdReports’ board and serve as the majority of its reviewers. Teachers examine materials after hours and receive a stipend for their efforts. The pay is meager — at less than $17.50 an hour, it’s comparable to the salary of an Amazon warehouse employee, a fact Hirsch says is under review — but teachers say the expertise they develop is invaluable.

Sara Schwerdtfeger, a former elementary school teacher, is an associate professor at Emporia State University in central Kansas. She’s served as an EdReports reviewer since 2014, and she believes the experience has helped her and her peers.

“What teachers really think about is what’s going to help other teachers,” Schwerdtfeger said. “That’s sort of why I do it: to get really good instruction materials in teachers’ hands, because they’re going to be able to really use them and make [a] difference in our students.”

Disclosure: The Bill & Melinda Gates Foundation, the Charles and Lynn Schusterman Family Foundation, and the Carnegie Corporation of New York provide financial support to EdReports and to The 74.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter