‘Don’t Get Gaggled’: Minneapolis School District Spends Big on Student Surveillance Tool, Raising Ire After Terminating Its Police Contract

Minneapolis education leaders have spent hundreds of thousands of dollars this year to surveil children online, even after the district ended its police department contract and launched school safety reforms that officials said would build trust between adults and students.

The district terminated its longstanding relationship with the city’s police department after George Floyd died at the hands of a Minneapolis officer in May. But since the pandemic closed campuses in March and required students to attend online classes from home, the district has shelled out more than $355,000 for a digital surveillance tool called Gaggle, according to contracts obtained by The 74 through a public records request.

Gaggle is currently used in hundreds of districts across the U.S., relying on artificial intelligence and a team of moderators paid as little as $10 an hour to scan billions of student emails, chat messages and files each year in search of references to sex, drugs and violence.

Even while the police-free schools movement has garnered momentum in the wake of Floyd’s death, with districts nationwide reexamining the role of cops on campus, it has not appeared to slow the recent growth of the nearly $3 billion-a-year school security industry. A Gaggle executive said their service is key to student safety, and the company saw a sales surge with more than 100 school districts becoming new customers since schools went virtual in March.

“With school now taking place in our students’ living rooms and bedrooms, safety is more important than ever,” Jeff Patterson, Gaggle’s founder and CEO, said in the media release. “Many educators are concerned that without in-person school, they may not be able to identify students in abusive situations or those suffering from mental illness.”

But there’s little research to back up the company’s claims and critics argue that Gaggle and similar products could be detrimental to child development and amount to pervasive government surveillance. Civil rights groups and racial justice advocates are especially concerned about online surveillance tools during the pandemic as students across the country spend the majority of their academic lives in front of screens.

In Minneapolis, the latest revelation further outraged activists who cheered the district’s decision to terminate the police contract but grew wary after officials sought to substitute campus cops with “public safety support specialists” with law enforcement backgrounds.

“My concern was that they would replace physical policing with technological policing, which appears to be something like Gaggle,” said Marika Pfefferkorn, executive director of the Midwest Center for School Transformation and a proponent of the police-free schools movement. Pfefferkorn pushed the district to split with the cops but said the move was just the first step in curtailing the policing and surveillance of students — particularly those of color. Instead, Gaggle “has the potential to further criminalize students.”

No such thing as confidentiality online

An initial six-month district contract with Gaggle, signed by Chief Operations Officer Karen DeVet just a week after the virus shuttered city schools, totaled $99,603 and was in place through the end of September. A second, three-year contract was signed months after Floyd’s death and went into effect this month at an annual rate of $255,750. School Board Chair Kim Ellison signed the second contract on Sept. 18. District and school board officials didn’t respond to multiple requests for comment.

In the contract, Gaggle notes that it “cannot guarantee security and confidentiality through its services” and “may choose to turn over” student messages to the police. However, the company said it “shall not be responsible for contacting, notifying or alerting” law enforcement and cannot guarantee that “all unsafe communications can or will be detected while monitoring your student communications or website content.”

Through the contracts, Gaggle helps the district monitor student activities on a range of Google services, including email, Docs, a video platform, the chat service Google Hangouts and other Google Classroom tools. Through artificial intelligence, the company scans students’ emails, chat messages and other materials for specific words and phrases that may indicate harm. Moderators evaluate flagged content and notify school officials about references to self-harm, depression, drug use and violent threats. Gaggle’s algorithm scans student content for trigger words including “bomb,” “drunk,” “gun” and “kill me,” according to a 2019 Buzzfeed News investigation. But it also scans for LGBTQ-specific words like “gay” and “lesbian,” which are often flagged as potential bullying.

Such keywords could lead Gaggle to disproportionately subject LGBTQ students to school surveillance, Pfefferkorn said.

“Over and over again, we continue to see with algorithms that bias is often baked in,” she said.

---

“Any time you have a service turned on, you see pornography, you can see drugs and alcohol use or use being talked about. You, of course, have anxiety, depression and suicide being talked about.” —Bill McCullough, Gaggle’s vice president of sales. 

---

In a brief message buried on one Minneapolis high school’s website — with the headline “Don’t Get Gaggled” — district staff noted that distance learning presents new challenges in supporting students’ mental and emotional health needs and offers a reminder that “there is no such thing as confidentiality online.” The webpage links to a video featuring counseling services manager Derek Francis, who notes that the district “will be monitoring chats and postings for inappropriate content and will follow up as is appropriate.”

“Make sure you’re not saying things online that you would never say to someone’s face,” Francis warns students. “We don’t want you to end up regretting something that you post.”

Prior to the pandemic, the Minneapolis district didn’t believe Gaggle’s services were necessary, said Bill McCullough, the company’s vice president of sales. But when the virus closed buildings, “they wanted us to start the service as quickly as possible,” he said. After an initial six-month pilot, the district “realized that this service is extremely valuable and moved to a full contract this fall.”

“Any time you have a service turned on, you see pornography, you can see drugs and alcohol use or use being talked about,” he said. “You, of course, have anxiety, depression and suicide being talked about.”

Ben Feist, the chief programs officer at the American Civil Liberties Union of Minnesota, has urged the state to adopt student data privacy protections for years, due in part to surveillance concerns with companies like Gaggle. In an interview, he said the Minneapolis district’s partnership with Gaggle is “massively intrusive” at a time when students’ use of technology for school has reached “complete saturation.”

---

“As far as I can tell, nobody has really thought this through, at least from any type of privacy lens. It’s hugely troubling.” —Ben Feist, chief program officer for ACLU of Minnesota

---

By terminating the police contract, district leaders have said they’re working to dismantle what they called a “white supremacist culture.” But Feist said that Gaggle could perpetuate racial disparities in student discipline. The Minneapolis district educates about 35,000 students, roughly 65 percent of whom are youth of color.

“There’s every reason to believe that the implementation of this type of surveillance is going to have a disproportionate impact on students of color and bring more people into a surveillance net that could have been avoided,” he said. “As far as I can tell, nobody has really thought this through, at least from any type of privacy lens. It’s hugely troubling.”

Searching for ‘sad kids’

Gaggle and similar student surveillance platforms have long marketed themselves as crucial to preventing school violence. After the 2018 mass school shooting in Parkland, Florida, for example, companies bombarded education leaders with sales pitches touting their wares as the key ingredient to precluding more carnage. In the pandemic era, Gaggle is marketing itself as a tool for mental health intervention.

“People are using our product to identify, largely, who the sad kids are in the school district,” said McCullough, who noted concerns that the pandemic has taken a toll on students’ emotional wellbeing and could lead to a spike in youth suicides. Such a trend has emerged in students’ emails and other digital communications, he said, with an uptick in student comments about depression, suicide and domestic abuse. “But thankfully kids are still talking about it and we’re able to go and identify those kids who are in crisis” to connect them with mental health services. McCullough declined to detail how his service has been used in Minneapolis, citing student privacy concerns.

Last school year, Gaggle monitored more than 4.5 million students’ online activities across the U.S., efforts it claims saved 927 lives, according to a company media release. In total, the company scanned 6.25 billion items within school accounts for content deemed harmful, including 64,000 references to suicide or self harm, 38,000 references of violence toward others and 18,000 instances of nudity or sexual content.

School surveillance doesn’t stop when classes end for the day. Prior to the pandemic, about 40 percent of incidents occurred after school hours, according to company data. But since March, incidents happening after hours increased to 55 percent. While threats of violence decreased by 43 percent after the pandemic closed campuses, the platform observed an uptick in students sending each other nude selfies.

Several years ago, concerning material was most often found in student emails, McCullough said. But now, messages are most often flagged in Google Docs, which students have used as makeshift chat rooms. In this context, students are often “their most authentic self” and typically share documents “with just a few friends,” he said.

Minneapolis and other districts have also paid Gaggle to monitor student communications in the chat tool Google Hangouts, which has taken on a new role in education during the pandemic. Without face-to-face interaction between students, they’re using Hangouts to collaborate on science projects and other assignments, McCullough said.

But critics argue that schools’ use of tools like Gaggle could discourage students from expressing themselves. Elizabeth Laird, the senior fellow of student privacy at the Center for Democracy and Technology, questioned whether such surveillance runs counter to schools’ mission of providing supportive environments where students can speak freely and learn from mistakes.

“When people are surveilled in this way, it really limits that kind of free expression and can have a chilling effect on what they’re comfortable saying and doing,” she said. Laird also raised concerns about the accuracy of algorithms, which often struggle to decipher true threats from slang or humor. Such noisy data could flag some students unnecessarily and miss signs that are genuinely concerning, she said.

School interactions with police are also a concern. In a recent parent survey, the Center for Democracy and Technology found that while most parents support the use of education technology, they’re also concerned about protecting their children’s digital privacy from vulnerabilities like hacks. Of the 1,200 participants who completed the online survey in May and June, 55 percent of parents — and 61 percent of those who are Black — said they’re concerned that student data could be shared with the police.

Though Gaggle rarely contacts the police directly, McCullough said, the district hasn’t said how it’s responding to tips generated from the surveillance service.

The district completed a chaotic candidate search last month and hired 11 “public safety support specialists” to replace the school-based police. The district has refused to disclose the names and qualifications of the 11 people who filled the openings, but documents obtained by The 74 suggest that more than half bring experience in policing, security or corrections — bolstering critics’ fears that the district ended the police contract but created an internal security force. According to an August school board agenda, the specialists training is supposed to encompass “school security 101,” de-escalating conflicts, dismantling the “school-to-prison pipeline” — and Gaggle.

Pfefferkorn, the local activist, blasted Minneapolis schools for a lack of transparency in its student surveillance practices and demanded that officials answer hard questions.

“It’s an opportunity for the district to hold a meeting where they share” how they’re using Gaggle to monitor students, she said. “Although you’re in a contract, contracts have been broken.”

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter