‘Miles Ahead’ of Other Districts: Using New Dashboard, St. Paul Tracks Learning Recovery Spending — and Adjusts Programs on the Fly

Updated

At the October conference of the Council of Great City Schools, an organization made up of urban districts, leaders of St. Paul Public Schools took a victory lap. Despite wave after wave of national data showing alarming pandemic-related learning losses, St. Paul’s reading scores were inching up.

One reason for the progress, the superintendent and his chief of strategy said, was the decision to use some of the district’s $300 million in federal pandemic recovery funds to provide intensive literacy instruction for struggling readers. That program, launched in fall 2021, was called WINN — What I Need Now.

The district’s Office of Teaching & Learning identified the top reading teachers and assigned them, in pairs, to every elementary school in the district to work intensively with students who were behind. The progress was impressive enough last year that this year, district leaders added two more coaches to the program.

But as promising as WINN appears to be, it’s not the St. Paul strategy that’s drawing national attention. What has researchers and policymakers buzzing are the district’s efforts to track the effectiveness of WINN and dozens of other initiatives to help students make up lost ground — and to use that information to refine their programs as they go.

In the coming days, a working group of staffers from different district departments will officially unveil a public dashboard that shows how the money is being spent in real time and how well each pandemic recovery strategy is — or isn’t — working.

Other examples include $10 million in grants to 37 community partners to provide afterschool programs and mental health services, many tailored to particular ethnic or immigrant communities; $6 million to boost instruction for children who missed special education services during the pandemic; $2 million to pay for 625 bilingual students to take language assessments, enabling them to graduate with a collective $11 million worth of college credits; and $25 million in funds given directly to schools to meet whatever unique needs they have.

With the pandemic working group, a new Innovation Office is responsible for monitoring the impact of more than 50 recovery strategies, collecting information on how well each meets short-, medium- and long-term goals — all ultimately helping close longstanding achievement gaps and increase overall academic performance in the district.

Already, the data is being used to fine-tune programs in progress. Information about job stress among new special education teachers, for instance, has led the district to boost spending on recruitment of classroom aides by $400,000.

St. Paul is one of only a “handful” of districts that are gathering data and using it to make changes in real time, says Marguerite Roza, director of Georgetown University’s Edunomics Lab, who recently called on school systems to quickly commit federal relief funds to address the dismal results of the most recent National Assessment of Educational Progress.

“The fact that they’re doing measurement of their investments and looking at them both internally and publicly is miles ahead of many other districts,” she says. “It’s what it takes to do continuous improvement.”

School Board Chair Jim Vue was delighted to hear that policy watchers are tracking St. Paul’s efforts — but says the district’s academic recovery strategy was hatched organically and early, before the government had even outlined rules for spending the relief funds. As in other high-poverty school districts, St. Paul’s leaders for years have talked about what they might be able to do with more funding. 

“Now, we were being given this money, and it’s a chance to show we can do something great,” says Vue. “We knew who we were serving, and we knew what their struggles were.” 

Some 30% of St. Paul’s 30,000 students are Asian, a fourth are Black, 22% are white and 14% Latino. More than 60% are impoverished and 29% are learning English. Adds Vue: “We knew we had to be specifically targeting different groups in our schools.” Closing and consolidating schools after years of declining enrollment and per-pupil funding has allowed the district to dedicate the lion’s share of its COVID relief money to meet those needs. 

Academic recovery: WINN for the win

All St. Paul students lost academic ground during the pandemic, but longstanding gaps between affluent whites and historically underserved children dramatically widened. In 2019, 40% of district students passed state reading tests — an average that obscures big gaps. While 65% of white children read at grade level, just 16.5% of Black students, 20% of Asians, 22% of Latinos and 10% of Native American pupils met or exceeded standards. 

On the 2022 tests, half of students lost ground or remained below grade level in reading, as did 59% in math. Only 21% in reading and 19% in math made academic progress.

Many districts, faced with staffing shortages and other logistical hurdles, have turned to online programs to provide the high-dosage tutoring — trained adults working in person with two or three students three or more times a week — that experts say is needed to reverse these declines. St. Paul chose instead to reconfigure school staffing and schedules. While school closures typically mean layoffs, St. Paul identified several dozen educators — both in schools slated for closure and in some staying open — with strong reading instruction backgrounds. The district placed 72 of them to work in pairs in elementary and middle schools, and asked six more to coach them. The cost: $11 million a year.

Schools juggled their schedules so the WINN teachers could join every early-grades reading class to work with kids who need intensive help, says Innovation Office Director Leah Corey. Last year, this resulted in 2,300 students — 20% of the district’s kindergarten through third graders — receiving WINN services. 

The program’s structure offers a number of advantages in terms of meeting the district’s long-term needs, she says. At a time when many school systems, including St. Paul, are attempting to align their teaching to the science of reading, the WINN teachers are demonstrating evidence-based literacy instruction in classrooms throughout the city. Their presence also frees up regular classroom teachers to work with other students.

Pupils who struggle get the cutting-edge instruction before they fail, unlike typical reading remediation programs. And with a variety of expertise, the teachers can provide tailored assistance to English learners, students with dyslexia or those receiving special education services.

Data gathered in WINN classrooms is already guiding district decisions. Elementary students’ reading levels are measured using an assessment known as the FAST, which gives information on specific skills each has mastered or needs to learn. These results are helping teachers decide which students need small-group assistance, for example, and will help WINN coaches plan professional development for classroom teachers, with an end goal of boosting the district’s overall reading proficiency.

When district leaders looked at scores from the 2021-22 school year, they saw strong progress among students who participated in WINN. In grades 2 and 3, gains by pupils who received the support dramatically outpaced those of classmates. In third grade, for example, WINN students’ scores increased an average of 60%, compared with 40% among non-participants. 

The data is preliminary, cautions Stacey Gray Akyea, the district’s chief of equity, strategy and innovation. But it’s promising enough that this year, the district added two more staffers to the program.

Goal: Diversity up, vacancies down

Long before teacher shortages dominated headlines, educators of color were in desperately short supply. Nowhere has that been more true than in Minnesota, where, in recent years, the teaching force has consistently been more than 90% white. Despite research showing the positive impact educators of color have on all children, barriers to their hiring and retention persist.

St. Paul used some of its recovery funding to create a 14-member recruitment and retention team that has both boosted the percentage of new hires of color and slashed the number of unfilled teaching jobs dramatically. Thirty percent of teachers hired between January and November 2022, came from underrepresented demographic groups, versus 23% during the same time span in 2021. 

Between Aug. 7 and Nov. 22, the number of open teaching positions fell from 113 to 44. By mid-October, the number of special education vacancies — particularly hard to fill everywhere — had dropped from 42 to 22.

Until recently, would-be teachers trained at Historically Black Colleges and Universities and Tribal Colleges found it all but impossible to secure licenses to work in Minnesota. All the HBCUs and more than 90% of the tribal colleges are located out of state, and Minnesota had accepted very few transplants. A new law has eased the process somewhat, and St. Paul’s recruiters have planned hiring fairs at several HBCUs in the coming months. They have also held local job fairs, making offers on the spot to top prospects. 

With a goal of keeping diverse educators, the team is also interviewing new teachers of color to learn what will keep them in St. Paul schools. Upon hearing that special education teachers are stretched too thin, district leaders earmarked an additional $400,000 to recruit classroom aides to work with children with disabilities.

Nine of the team members — many existing employees doing new jobs — are being paid with recovery funds. In fiscal year 2023, the amount of stimulus funding in question comes to $1.6 million. In 2024, that figure will rise to $3.3 million. 

In addition to the interviews, team members will analyze recruiting and retention statistics by race and will share the information with principals and other supervisors with an eye toward helping them create inclusive cultures, increasing the number of teachers of color who stay and, ultimately, decreasing racial disparities in student achievement.

In the cases of both the intensive literacy program and the educator recruitment and retention teams, Roza sounds both positive and cautionary notes. It’s important to remember, she says, that the country is likely headed into a recession, which means jobs created now may be imperiled when stimulus funds dry up. Tapping existing staff to perform new jobs is a valuable way to avert layoffs and start delivering pandemic recovery services quickly at a time when hiring is hard, she says. 

A trusted — and underused — path to student engagement

In 1989, three University of Minnesota professors launched an experiment to prevent students with behavioral challenges and learning disabilities from dropping out. A school staff member mentors one or more students, using a research-based set of relationship-building skills, and monitors data about their attendance, behavior and academic performance. 

More than 30 years later, Check and Connect is recognized as a leading strategy to promote student engagement, boost graduation rates and improve children’s mental health. Still, schools often struggle to implement the program. Even when they can find grant funding to pay for it, they are loath to add extra duties to teachers’ already full calendars.

For five years, St. Paul has had a state grant to operate Check and Connect with a small group of Black and Native American students who receive special education services. Recent results were so promising that this year the district expanded the program to nine schools, using federal funds budgeted for special education recovery services. 

Pre-pandemic, data on student disengagement showed unmet needs that have likely widened since the first school closures in spring 2020. In 2019, according to state statistics, some 73% of Black St. Paul students attended class 90% of the time. Just 62% of Native American children and 71% of students with disabilities had consistent attendance. By St. Paul’s calculations, in the 2021-22 school year, 75% of Native American students missed 11 or more days of class, as did 67% of Black children. 

But despite unprecedented rates of disengagement among at-risk high school students nationwide, this year, seven of eight seniors enrolled in the St. Paul program graduated on time. District leaders concede it’s a small sample but point out that the results for Check and Connect participants are higher than both the district’s 76% overall graduation rate and the 49% of students with disabilities who graduate on time. 

Because the engagement effort worked so well with a student demographic that’s particularly at risk of dropping out, last fall district leaders trained 40 school support staffers to be mentors and plans to add more in the spring. They also hired a specialist with the expertise to work with Black and Native students in schools that are too small to have a full program, and are piloting Check and Connect in elementary schools. Because mentors are expected to forge long-term, trusting relationships with families under very high stress, a high degree of cultural competence is required. 

To learn whether they are succeeding, the counselors and other staff involved in the mentoring program will survey students about their experiences and look at graduation and college- and career-readiness rates. The end goal is to increase academic achievement among students receiving special education services.

‘Library of lessons’ on Native history, culture

Home to a large number of Native Nations, Minnesota historically has struggled with teaching Indigenous culture and history. Since a mandate to include Native American topics became law in 2007, numerous proposals to update state academic standards have sparked repeated waves of controversy. 

An effort earlier this year to update state math standards to include references to Dakota and Anishinaabe communities ran into stiff opposition both from anti-critical race theory conservatives and Indigenous people. The ongoing revision of the social studies standards has also been a flashpoint. 

Over the next two years, St. Paul Public Schools will invest some $1.4 million in changing its approaches to addressing the needs of its almost 900 Native American students. A full-time staff member is bolstering the district’s resources. In addition to reviewing curricula for places where Indigenous history and current issues should be taught, the district is creating a library of lessons, helping individual schools adopt Native circle-based restorative practices and training all staff on contemporary issues. 

The district also hired a counselor to work with Native American students and recently passed a policy allowing Indigenous smudging rituals — the burning of sage and other herbs to purify people and spaces — in schools. 

Addressing longstanding equity issues is hugely important, says Roza. But districts that do what St. Paul is doing — tracking the outcomes of their recovery efforts on individual dashboards — need to start with an end goal in mind and reverse-engineer the data they track.

“They need to be clear about what they hope to get from it,” she says. “If you think it will boost attendance for Native Americans, then track attendance. If it will boost reading and math, then track test scores.”

Staff responsible for the Native American curriculum effort will monitor how often teachers use the lessons the district develops and how often they include materials about Indigenous culture in their instruction; will survey educators about their confidence regarding Native topics; and, ultimately, will look for increased academic progress among Indigenous students.

Any degree of progress toward helping students rebound from the pandemic is, of course, welcome, says Roza. But shifting to a culture where making changes at any point in the year is an innovation that will serve St. Paul going forward. 

“ ‘We’ll see in next year’s test scores’ — that’s an unacceptable response,” says Roza. “The problem is complicated? Well, let’s roll up our sleeves and solve it.”

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter