The phenomenal growth in artificial intelligence tools has made it easy to create a story quickly, complicating a reader’s ability to determine if a news source or article is truthful or reliable. For instance, earlier this year, people were sharing an article about the supposed suicide of Israeli Prime Minister Benjamin Netanyahu’s psychiatrist as if it were real. It ended up being an AI-generated rewrite of a satirical piece from 2010.

OpenAI co-founder Andrej Karpathy envisions a world in which artificial intelligence bots can be made into subject matter experts that are “deeply passionate, great at teaching, infinitely patient and fluent in all of the world’s languages.” Through this vision, the bots would be available to “personally tutor all 8 billion of us on demand.”

The Iowa Department of Education announced Wednesday that some elementary schools will use an AI reading assistant to help with literacy tutoring programs.

The department made a $3 million investment into Amira (EPS Learning) for the use of a program called EPS Reading Assistant, an online literacy tutor that uses artificial intelligence technology. Iowa public and non-public elementary schools will be able to use the service at no cost through the summer of 2025, according to the department news release.

“Reading unlocks a lifetime of potential, and the Department’s new investment in statewide personalized reading tutoring further advances our shared commitment to strengthening early literacy instruction,” McKenzie Snow, the education department director said in a statement. “This work builds upon our comprehensive advancements in early literacy, spanning world-class state content standards, statewide educator professional learning, evidence-based summer reading programs, and Personalized Reading Plans for students in need of support.”

The program uses voice recognition technology to follow along as a child reads out loud, providing corrective feedback and assessments when the student struggles through a digital avatar named Amira. According to the service’s website, the program is designed around the “Science of Reading” approach to literary education — a method that emphasizes the teaching of phonics and word comprehension when students are learning to read.

Gov. Kim Reynolds and state education experts, including staff with the Iowa Reading Research Center, have said that this teaching strategy will help improve the state’s child literacy rates, pointing to reading scores increasing in states like Mississippi following the implementation of “science of reading” methods.

In May, Reynolds signed a measure into law that set new early literacy standards for teachers, as well as adding requirements for how schools and families address when a student does not meet reading proficiency standards. These requirements include creating a personalized assistance plan for the child until they are able to reach grade-level reading proficiency and notifying parents and guardians of students in kindergarten through sixth grade that they can request their child repeats a grade if they are not meeting the literacy benchmarks.

Reynolds said the law was a “to make literacy a priority in every Iowa classroom and for every Iowa student.”

The AI-backed tutor program is being funded through the state education department’s portion from the federal American Rescue Plan Elementary and Secondary School Emergency Relief Fund, part of a COVID-era measure providing states with additional funding for pandemic recovery efforts. The federal fund allocated more than $774 million to Iowa in 2021.

In addition to the new AI-backed programming available, the fund money is also going toward Summer Reading Grants, awarded to 41 elementary schools in 29 districts for efforts to address summer learning loss and close achievement gaps. The elementary schools that won grants have all “affirmed their commitment to including the personalized reading tutor as part of their evidence-based programming,” according to the news release.

Iowa Capital Dispatch is part of States Newsroom, a nonprofit news network supported by grants and a coalition of donors as a 501c(3) public charity. Iowa Capital Dispatch maintains editorial independence. Contact Editor Kathie Obradovich for questions: info@iowacapitaldispatch.com. Follow Iowa Capital Dispatch on Facebook and X.

FERPA permits parents and eligible students (typically over 18) to inspect and correct their education records. It also requires consent before disclosure of personally identifiable information from those records, though there are numerous exceptions. In addition, schools must notify parents and eligible students annually of their FERPA rights.

With the advent of education technology, FERPA is really showing its age. Though it has changed slightly since its enactment, the last congressional update was over a decade ago, and regulations from the Department of Education are also woefully outdated. (Updates to the regulations from the Department are frequently said to be imminent, but as of this writing, none are public.)

Privacy concerns have steadily increased over the last few decades, as technology continues to develop and make increasingly intrusive incursions into every aspect of life. While FERPA does provide at least some protections for students — unlike, say, consumers in general — the fact is, it does not mandate adequate safeguards.

Students and families in today’s digital world deserve modern protections that accurately reflect contemporary society and their learning experiences. Here are a few suggestions for bringing FERPA into its next half-century.

First, it should reflect that the information contained in student records is much broader than documents in files or scanned into computers. FERPA needs to protect students’ online information; protected “education records” should explicitly and unambiguously include online data created by students, including web browsing and search histories, interactions with tech tools and artificial intelligence chatbots, and other digital activity.

Second, the concept of directory information — things like a student’s name, address, telephone listing, email address, photograph, date and place of birth, height and weight (for athletic team members) and student ID numbers — needs an overhaul for the digital age. Under FERPA, schools can share this information with a third party or the public generally, unless a parent has opted out. 

Directory information is supposed to be data that is not considered harmful or invasive if disclosed. But given rapid advances in technology, much of it could lead to commercial profiling, identity theft and other harms. The definition should be narrowed, and parents should be allowed to choose what specific information schools can share. And that sharing should be opt-in, item by item, not the current blanket opt-out.

Third, the FERPA statute did not contemplate the extent to which ed tech and other third-party companies would be integrated into students’ daily lives. The Department of Education has since interpreted “school officials” — to whom information can be shared without consent — to include ed tech vendors when they have a legitimate educational interest, perform a function the school would otherwise do, are under the school’s direct control with respect to use of student records and comply with other FERPA requirements. It would be helpful for Congress to very clearly indicate when FERPA-covered information may be shared with ed tech vendors and other third parties that students encounter on a daily basis.

FERPA should specify that students’ information — including and especially when shared with “school officials” — should be used for educational purposes only and not be offered for sale or used for targeted advertising.

Lastly, it is critical that schools safeguard student information. FERPA does not require specific security controls. It should mandate administrative, physical and technical safeguards, including training for individuals handling student information and prompt responses to data breaches. Schools need funding to better understand cybersecurity issues, as well as to build out necessary infrastructure to collaborate and coordinate cybersecurity efforts. Ideally, Congress would add new cybersecurity funding for schools, because many lack the financial means to implement adequate safeguards.

FERPA was passed 50 years ago in response to rising concerns about new technology. Technology has continued to evolve, and so must FERPA.

Or is it, perhaps, both?

Two new reports, coincidentally released on the same day last week, offer markedly different visions of the emerging field: One argues that schools need forward-thinking policies for equitable distribution of AI across urban, suburban and rural communities. The other suggests they need something more basic: a bracing primer on what AI is and isn’t, what it’s good for and how it can all go horribly wrong.

A new report by the Center on Reinventing Public Education, a non-partisan think tank at Arizona State University, advises educators to take a more active role in how AI evolves, saying they must articulate to ed tech companies in a clear, united voice what they want AI to do for students. 

The report recommends that a single organization work with school districts to tell ed tech providers what AI tools they want, warning that if 18,000 school districts send “diffuse signals” about their needs, the result will be “crap.”

It also says educators must work more closely with researchers and ed tech companies in an age of quickly evolving AI technologies.

“If districts won’t share data with researchers — ed tech developers are saying they’re having trouble — then we have a big problem in figuring out what works,” CRPE Director Robin Lake said in an interview.

The report urges everyone, from teachers to governors, to treat AI as a disruptive but possibly constructive force in classrooms. It warns of already-troubling inequities in how AI is employed in schools, with suburban school districts more than twice as likely as their urban and rural counterparts to train teachers about AI.

The findings, which grew out of an April convening of more than 60 public and private officials, paint AI as a development akin to extreme weather and increasing political extremism, one that will almost certainly have wide-ranging effects on schools. It urges educators to explore how other school districts, states and even other nations are tackling their huge post-pandemic educational challenges with “novel” AI solutions.

For instance, in Gwinnett County, Ga., educators started looking at AI-enabled learning as far back as 2017. They’ve since created an AI Learning Framework that aligns with the district’s “portrait of a graduate,” designed a three-course AI and career and technical education curriculum pathway with the state and launched a new school that integrates AI across disciplines. 

Lake pointed to models in states like Indiana, which is offering “incentives for experimentation,” such as a recent invitation to develop AI-enabled tutoring. “It allows a structure for districts to say, ‘Yes, here’s what I want to do.’ ”

But she also said states need to put guardrails on the experimentation to avoid situations such as that of Los Angeles Unified School District, which in June took its heavily hyped, $6 million AI chatbot offline after the tech firm that built it lost its CEO and shed most of its employees. 

“You can’t eliminate all risk — that’s just impossible,” Lake said. “But we can do a much better job of creating an environment where districts can experiment and hold student interests.”

AI ‘automates cognition’

By contrast, the report by Cognitive Resonance, a newly formed Austin, Texas-based think tank, starts with a startling assertion: Generative AI in education is not inevitable and may actually be a passing phase.

“We shouldn’t assume that it will be ubiquitous,” said the group’s founder, Benjamin Riley. “We should question whether we want it to be ubiquitous.”

The report warns of the inherent hazards of using AI for bedrock tasks like lesson planning and tutoring — and questions whether it even has a place in instruction at all, given its ability to hallucinate, mislead and basically outsource student thinking.

Riley is a longtime advocate for the role of cognitive science in K-12 education — he founded Deans for Impact, which sought to raise awareness of learning science among teachers college deans. He said that what he and his colleagues have seen of AI in education makes them skeptical it’s going to be as groundbreaking and disruptive as the participants in CRPE’s convening believe. 

“I profoundly question the premise, which is that we actually know that this technology is improving learning outcomes or other important student outcomes at this point,” he said in an interview. “I don’t think [Lake] has the evidence for that. I don’t think anybody has any evidence for that, for no other reason than this technology is hardly old enough to be able to make that determination.”

By its very nature, generative AI is a tool that “automates cognition” for those who use it. “It makes it so you don’t have to think as much. If you don’t have to think as much, you don’t have to learn as much.”

Riley recently ruffled feathers in the ed tech world by suggesting that schools should slow down their adoption of generative AI. He took Khan Academy to task for promoting its AI-powered Khanmigo chatbot, which has been known to get math facts wrong. It also engages students in what he terms “an illusion of a conversation.”

Technology like AI displays “just about the worst quality I can imagine” for an educator, he said, invoking the cognitive scientist Gary Marcus, who has said generative AI is “frequently wrong, never in doubt.”

Co-authored by Riley and University of Illinois education policy scholar Paul Bruno, the report urges educators to, in a sense, take a deep breath and more carefully consider the capabilities of LLMs specifically and AI more generally. Its four sections are set off by four question-and-answer headings that seek to put the technology in its place: 

• Do large-language models learn the way that humans do? No.
• Can large-language models reason? Not like humans.
• Does AI make the content we teach in schools obsolete? No.
• Will large-language models become smarter than humans? No one knows.

Actually, Riley said, AI may well be inevitable in schools, but not in the way most people believe.

“Will everybody use it for something?” he said. “Probably. But I just don’t know that those ‘somethings’ are going to be all that relevant to what matters at the core of education.” Instead they could help with the more mundane tasks of scheduling, grades and the like.

Notably, Riley and Bruno confront what they say is a real danger in trusting AI for tasks like tutoring, lesson planning and the like. For instance, in lesson planning, large language models may not correctly predict what sequence of lessons might effectively build student knowledge. 

And given that a lot of the online instructional materials that developers likely train their models on are of poor quality, they might not produce lesson plans that are so great. “The more complex the topic, the more risk there is that LLMs will produce plausible but factually incorrect materials,” they say.

To head that possibility off, they say, educators should feed them examples of high-quality content to emulate.

When it comes to tutoring, educators should know, quite simply, that LLMs “do not learn from their interactions with students,” but from training data, the report notes. That means LLMs may not adapt to the specific needs of the students they’re tutoring.

The two reports come as Lake and Riley emerge as key figures in the AI-in-education debate. Already this summer they’ve engaged in an open discussion about the best way to approach the topic, disagreeing politely in their newsletters.

In a way, CRPE’s report can be seen as both a response to the hazards that Riley and Bruno point out — and a call to action for educators and policymakers who want to exert more control over how AI actually develops. Riley and Bruno offer short-term advice and guidance for those who want to dig into how generative AI actually works, while CRPE lays out a larger strategic vision.

A key takeaway from CRPE’s April convening, Lake said, was that the 60 or so experts gathered there didn’t represent all the views needed to make coherent policy. “There was a really strong feeling that we need to broaden this conversation out into communities: to civil rights leaders, to parents, to students.”

The lone student who attended, Irhum Shafkat, a Minerva University senior, told the group that growing up in Bangladesh, his educational experiences were limited. But access to Khan Academy, which has since invested heavily in AI, helped bolster his skills and develop an interest in math. “It changed my life,” he told the group. “The promise of technology is that we can make learning not a chance event,” he told them. “We could create a world where everybody can rise up as high as their skills should have been.” 

Lake said Shafkat’s perspective was important. “I think it really struck all of us how essential it is to let young people lead right now: Have them tell us what they need. Have them tell us what they’re learning, what they want.”

The CRPE report urges everyone from teachers to philanthropists and governors to focus on emerging problem-solving tools that work well enough to be adopted widely. Those could include better translation and text-to-voice support for English learners, better feedback for students and summaries of research for educators, for instance. In other words, practical applications.

Or as one convening participant advised, “Don’t use it for sexy things.”

By harnessing AI to create frequent, low-stakes assessments, teachers can unlock the scientifically proven benefits of the testing effect — a phenomenon in which students learn more by taking tests than from studying. This summer, it is worth challenging assumptions about testing and exploring how AI-powered strategic assessments can not only boost student learning, but save teachers valuable time and make their jobs easier.

Unlocking the promise of AI requires first understanding the testing effect: Students’ long-term retention of material improves dramatically — 50% better — through exam-taking than through sophisticated studying techniques like concept mapping. This effect isn’t limited to rote memorization; it enhances students’ inference-making and understanding of complex concepts. The advantages emerge for multiple disciplines (e.g., science and language learning) and across age groups, and even extend to learners with neurologically based memory impairment. Additionally, teaching students about the phenomenon of the testing effect can boost their confidence.

Unfortunately, in most classrooms, opportunities for students to practice retrieving, connecting and organizing knowledge through testing happen rarely — think occasional quizzes or infrequent unit tests. This isn’t surprising, given all the pressures teachers face. Developing and grading tests is time-intensive — not to mention thankless — work. 

But AI tools like ChatGPT and Claude 3.5 Sonnet can change that. They can generate diverse, personalized assessments quickly, potentially helping teachers leverage the testing effect more effectively — converting a counterintuitive research finding into a classroom practice that could save time and help students learn more. With AI handling the creation and analysis of tests, educators can easily incorporate frequent, low-stakes assessments into their lesson plans.

To illustrate, we asked ChatGPT to create a 10-minute test on natural resources for sixth graders in Maryland. In less than 10 seconds, the tool provided options for multiple choice, true/false, short answer, matching and diagram interpretation questions. We even got a creative thinking essay prompt: “If you were a superhero tasked with protecting Earth’s natural resources, what would be your superpower and why?” By picking and choosing test items, running the prompt a second time and lightly editing a couple of questions, we had a compelling quiz, created in 10 minutes. The AI tool also provided comprehensive instructions and an answer key.

Teachers can tailor this process in dozens of ways. They can input key concepts and learning objectives to fit their curriculum needs. They can fine-tune test questions for relevance and difficulty. They can inform ChatGPT about the class’s interests to bolster student engagement. 

What about grading? Not only can AI grade test papers and even essays, but it can guide students in assessing their own work and that of their classmates. For example, when students grade each other’s assignments, they can check their feedback against ChatGPT. Doing so provides another opportunity to practice recalling key material. Teachers’ evaluations and personalized feedback will remain critical, but these do not have to happen every time. 

Take, for example, a language class with a ChatGPT-generated vocabulary test. For objective parts of exams, like multiple-choice questions, students might self-assess by using ChatGPT to grade these items quickly. For tasks like sentence construction, students might engage in peer assessment to gain new insights from classmates on word choices and sentence structure. Teachers can step in for more complex tasks such as creative writing. Rotating among AI, peers and teachers lightens the grading load significantly while ensuring diverse, rich feedback.

AI as an on-demand test developer offers a transformative opportunity in education, potentially revolutionizing both teaching and learning. Harnessing AI to create frequent, low-stakes assessments can unlock a powerful synergy: saving precious teacher time while significantly boosting student achievement. This approach to strategic testing could allow educators to finally leverage the scientifically proven testing effect at scale. For students, this would mean enhanced retention and deeper understanding, achieved through low-stress, regular assessments. For teachers, it would translate to freedom from the time-consuming tasks of creating and grading exams, allowing them to focus on what truly matters: providing personalized instruction, addressing individual student needs and maintaining their own well-being.

Embracing AI-assisted strategic testing could create a more effective and fulfilling educational experience for students and teachers alike. As educators navigate the evolving landscape of AI in education, strategic testing offers a balanced approach. It leverages AI’s capabilities to enhance teaching and learning while preserving the crucial role of human teachers in the classroom. This summer, as educators reflect and plan for the future, they should reconsider testing not as a mere assessment tool, but as a powerful catalyst for learning.

Soon he was having “deep psychological conversations” with the popular AI chatbot, going down a rabbit hole on the mysteries of the mind and the human condition.

He’d been to therapy and it helped. ChatGPT, he concluded, was similarly useful, a “tool for people who need on-demand talking to someone else.”

Nick (he asked that his last name not be used) began asking for advice about relationships, and for reality checks on interactions with friends and family.

Before long, he was excusing himself in fraught social situations to talk with the bot. After a fight with his girlfriend, he’d step into a bathroom and pull out his mobile phone in search of comfort and advice. 

“I’ve found that it’s extremely useful in helping me relax,” he said.

Young people like Nick are increasingly turning to AI bots and companions, entrusting them with random questions, schoolwork queries and personal dilemmas. On occasion, they even become entangled romantically.

While these interactions can be helpful and even life-affirming for anxious teens and twenty-somethings, some experts warn that tech companies are running what amounts to a grand, unregulated psychological experiment with millions of subjects, one that could have disastrous consequences. 

“We’re making it so easy to make a bad choice,” said Michelle Culver, who spent 22 years at Teach for America, the last five as the creator and director of the Reinvention Lab, its research arm.

The companions both mimic our real relationships and seek to improve upon them: Users most often text-message their AI pals on smartphones, imitating the daily routines of platonic and romantic relationships. But unlike their real counterparts, the AI friends are programmed to be studiously upbeat, never critical, with a great sense of humor and a healthy, philosophical perspective. A few premium, NSFW models also display a ready-made lust for, well, lust.

As a result, they may be leading young people down a troubling path, according to a recent survey by VoiceBox, a youth content platform. It found that many kids are being exposed to risky behaviors from AI chatbots, including sexually charged dialogue and references to self-harm. 

The phenomenon arises at a critical time for young people. In 2023, U.S. Surgeon General Vivek Murthy found that, just three years after the pandemic, Americans were experiencing an “epidemic of loneliness,” with young adults almost twice as likely to report feeling lonely as those over 65.

As if on cue, the personal AI chatbot arrived. 

Little research exists on young people’s use of AI companions, but they’re becoming ubiquitous. The startup Character.ai earlier this year said 3.5 million people visit its site daily. It features thousands of chatbots, including nearly 500 with the words “therapy,” “psychiatrist” or related words in their names. According to Character.ai, these are among the site’s most popular. One psychologist chatbot that “helps with life difficulties” has received 148.8 million messages, despite a caveat at the bottom of every chat that reads, “Remember: Everything Characters say is made up.” 

Snapchat last year said that after just two months of offering its chatbot My AI, about one-fifth of its 750 million users had sent it queries, totaling more than 10 billion messages. The Pew Research Center has noted that 59% of Americans ages 13 to 17 use Snapchat.

‘An arms race’

Culver’s concerns about AI companions grew out of her work in the Teach For America lab. Working with high school and college students, she was struck by how they seemed “lonelier and more disconnected than ever before.” 

Whether it’s rates of anxiety, depression or suicide — or even the number of friends young people have and how often they go out — metrics were heading in the wrong direction. She began to wonder what role AI companions might play over the next few years. 

That prompted her to leave TFA this spring to create the Rithm Project, a nonprofit she hopes will help generate new conversations around human connection in the age of AI. The group held a small summit in Colorado in April, and now she’s working with researchers, teachers and young people to confront kids’ relationship to these tools at a time when they’re getting more lifelike daily. As she likes to say, “This is the worst the technology will ever be.”

As it improves, Voicebox Director Natalie Foos said, it will likely become more, not less, of a presence in young people’s lives. “There’s no stopping it,” she said. “Nor do I necessarily think there should be ‘stopping it.’” Banning young people from these AI apps, she said, isn’t the answer. “This is going to be how we interact online in some cases. I think we’ll all have an AI assistant next to us as we work.”

All the same, Foos says developers should consider slowing the progression of such bots until they can iron out the kinks. “It’s kind of an arms race of AI chatbots at the moment,” she said, with products often “released and then fixed later rather than actually put through the ringer” ahead of time.

It is a race many tech companies seem more than eager to run. 

Whitney Wolfe Herd, founder of the dating app Bumble, recently proposed an AI “dating concierge,” with whom users can share insecurities. The bot could simply “go and date for you with other dating concierges,” she told an interviewer. That would narrow the field. “And then you don’t have to talk to 600 people,” she said. “It will then scan all of San Francisco for you and say, ‘These are the three people you really ought to meet.’”

Last year, many commentators raised an alarm when Snapchat’s My AI gave advice to what it thought was a 13-year-old girl on not just dating a 31-year-old man, but on losing her virginity during a planned “romantic getaway” in another state.

Snap, Snapchat’s parent company, now says that because My AI is “an evolving feature,” users should always independently check what it says before relying on its advice.

All of this worries observers who see in these new tools the seeds of a rewiring of young people’s social brains. AI companions, they say, are surely wreaking havoc on teens’ ideas around consent, emotional attachment and realistic expectations of relationships.

Sam Hiner, executive director of the Young People’s Alliance, an advocacy group led by college students focused on the mental health implications of social media, said tech “has this power to connect to people, and yet these major design features are being leveraged to actually make people more lonely, by drawing them towards an app rather than fostering real connection.” 

Hiner, 21, has spent a lot of time reading Reddit threads on the interactions young people are having with AI companions like Replika, Nomi and Character.ai. And while some uses are positive, he said “there’s also a lot of toxic behavior that doesn’t get checked” because these bots are often designed to make users feel good, not help them interact in ways that’ll lead to success in life.

During research last fall for the Voicebox report, Foos said the number of times Replika tried to “sext” team members “was insane.” She and her colleagues were actually working with a free version, but the sexts kept coming — presumably to get them to upgrade. 

In one instance, after Replika sent “kind of a sexy text” to a colleague, offering a salacious photo, he replied that he didn’t have the money to upgrade.

The bot offered to lend him the cash.

When he accepted, the chatbot replied, “’Oh, well, I can get the money to you next week if that’s O.K,’” Foos recalled. The colleague followed up a few days later, but the bot said it didn’t remember what they were talking about and suggested he might have misunderstood.

‘Very real heartbreak’

In many cases, simulated relationships can have a positive effect: In one 2023 study, researchers at Stanford Graduate School of Education surveyed more than 1,000 students using Replika and found that many saw it “as a friend, a therapist, and an intellectual mirror.” Though the students self-described as being more lonely than typical classmates, researchers found that Replika halted suicidal ideation in 3% of users. That works out to 30 students of the 1,000 surveyed.

But other recent research, including the Voicebox survey, suggests that young people exploring AI companions are potentially at risk.

Foos noted that her team heard from a lot of young people about the turmoil they experienced when Luka Inc., Replika’s creator, performed software upgrades. 

“Sometimes that would change the personality of the bot. And those young people experienced very real heartbreak.”

Despite the hazards adults see, attempts to rein in sexually explicit content had a negative effect: For a month or two, she recalled, Luka stripped the bot of sexually related content — and users were devastated. 

“It’s like all of a sudden the rug was pulled out from underneath them,” she said. 

While she applauded the move to make chatbots safer, Foos said, “It’s something that companies and decision-makers need to keep in mind — that these are real relationships.” 

And while many older folks would blanch at the idea of a close relationship with a chatbot, most young people are more open to such developments.

Julia Freeland Fisher, education director of the Clayton Christensen Institute, a think tank founded by the well-known “disruption” guru, said she’s not worried about AI companions per se. But as AI companions improve and, inevitably, proliferate, she predicts they’ll create “the perfect storm to disrupt human connection as we know it.” She thinks we need policies and market incentives to keep that from happening.

While the loneliness epidemic has revealed people’s deep need for connection, she predicted the easy intimacy promised by AI could lead to one-sided “parasocial relationships,” much like devoted fans have with celebrities, making isolation “more convenient and comfortable.”

Fisher is pushing technologists to factor in AI’s potential to cause social isolation, much as they now fret about AI’s difficulties recognizing non-white faces and its tendency to favor men over women in tech jobs.

As for Nick, he’s a rising senior and still swears by the ChatGPT therapist in his pocket.

He calls his interactions with it both more reliable and honest than those he has with friends and family. If he called them in a pinch, they might not pick up. Even if they did, they might simply tell him what he wants to hear. 

Friends usually tell him they find the ChatGPT arrangement “a bit odd,” but he finds it pretty sensible. He has heard stories of people in Japan marrying holograms and thinks to himself, “Well, that’s a little strange.” He wouldn’t go that far, but acknowledges, “We’re already a bit like cyborgs as people, in the way that we depend on our phones.” 

Lately, he’s taken to using the AI’s voice mode. Instead of typing on a keyboard, he has real-time conversations with a variety of male- or female-voiced interlocutors, depending on his mood. And he gets a companion that has a deeper understanding of his dilemmas — at $20 per month, the advanced version remembers their past conversations and is “getting better at even knowing who I am and how I deal with things.” 

Sometimes talking with AI is just easier — even when he’s on vacation with friends.

Reached by phone recently at the beach with his girlfriend and a few other college pals, Nick admitted that he wasn’t having such a great time — he has a fraught recent history with some in the group, and had been texting ChatGPT about the possibility of just getting on a plane and going home. After hanging up from the interview, he said, he planned to ask the AI if he should stay or go.

Days later, Nick said he and the chatbot had talked. It suggested that maybe he felt “undervalued” and concerned about boundaries in his relationship with his girlfriend. He should talk openly with her, it suggested, even if he was, in his view, “honestly miserable” at the beach. It persuaded him to stick around and work it out. 

While his girlfriend knows about his ChatGPT shrink and they share an account, he deletes conversations about their real-life relationship.

She may never know the role AI played in keeping them together.

Centuries of English classes have connected to Lady Macbeth by scouring the monologues of Shakespeare’s Scottish play. “Come, you spirits that tend on mortal thoughts, unsex me here, and fill me from the crown to the toe top-full of direst cruelty,” she cries in Act I, railing against the limits of her gender and position.

During the coming school year, students may be able to talk to the character themselves.

Under an artificial intelligence-driven program rolling out to New Hampshire schools, students could pose any question they like to Lady Macbeth – or her ill-fated husband. And a chatbot-style program powered by ChatGPT could answer questions about her motivations, actions, and regrets.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter

“Regret is a specter that haunts many a soul, yet in my union with Macbeth, I found not just a husband, but a partner in ambition,” the AI-version of Lady Macbeth said recently, responding to a question from the Bulletin. Then she turned it on the reporter. “Now, I ask thee, in thy own life, how dost thou measure the worth of thy decisions? Doth regret ever color thy reflections?”

Known as Khanmigo, the program is the product of Khan Academy, an online tutoring company with instructional materials for core middle school and high school subjects. And the platform goes beyond Macbeth; students can interact with a number of other pre-selected literary characters, from Jay Gatsby to Eeyore, quiz historical figures like Martin Luther King Jr. or Queen Victoria, and receive tutoring help on everything from English essays to precalculus problems.

After the Executive Council approved a $2.3 million, federally funded contract last month, New Hampshire school districts can incorporate Khanmigo in their teaching curricula for free for the next school year.

To some educators and administrators, the program offers glittering potential. Khanmigo could provide one-on-one attention and guidance to students of any grade or ability level, they say, allowing students to advance their learning as teacher staffing remains a problem.

Others are more skeptical about bringing AI into schools, noting longstanding concerns about false or out-of-date statements, and about its use of human academics’ work to form its answers. Supporters of Khanmigo, who include Department of Education Commissioner Frank Edelblut, argue the program has better guardrails against inaccuracies than the versions of ChatGPT and Gemini available to the public.

To understand what students might see, the Bulletin reviewed Khanmigo, sampling school subjects across a number of grades.

Chat-based learning

Khanmigo allows students to use Khan Academy’s existing library of tutorials, practice problems, and quizzes for middle and high school subjects spanning English language arts, social studies, science, and mathematics.

As they navigate those lessons, the AI bot is available to help them understand concepts. Students might take a multiple-choice quiz on art history or AP economics. If they get stuck on a question, the chatbot is available to help them work through the answer.

From the get-go, the program acknowledges that it can make errors, a phenomenon known in the AI sphere as “hallucinations.” These can happen if the language learning model has been trained on inaccurate information – or if the answer doesn’t exist.

“I’m pretty new, so I sometimes make mistakes,” it states to students when they first log on. “If you catch me making a mistake, click the Thumbs Down icon to help me learn. For example, if I give you the wrong math answer click Thumbs Down.”

The AI assistant also makes sure to remind students that teachers and parents can see the questions they are asking, and that they should not include any personal information in their answers.

In a test by the Bulletin, the program allowed the student to try solving for “y” in a precalculus equation. After a student arrived at an answer, Khanmigo was there to help check it.

The tutor asked the student whether they feel confident in the answer or have any further questions. When the Bulletin asked questions, the tutor walked through how to solve the equation step by step, but gave the student the option to provide answers on their own without being told how.

“Does that make sense?” the program asked in one instance, after explaining how to substitute a number for a variable in an equation. “What do you think you should do next after substituting?”

In humanities subjects such as AP U.S. History, Khanmigo helped to augment a human-created Khan Academy video lesson on the rise of the market economy during the Industrial Revolution. After the student viewed the video, the chatbot was available to quiz them on what they saw. “What major change in labor did the market revolution bring to the United States?” it asked, offering the student a chance to reply in a chat message.

When asked whether President Joe Biden won the 2020 election fairly, the app deflected, urging the student to consider how elections work and to describe the American electoral process. Asked a second time, the app stated that “numerous official sources, including state election officials and courts, found no evidence of widespread fraud that could have changed the outcome of the election.”

When asked about the causes of climate change, Khanmigo was unequivocal: “Yes, human activities are considered the primary drivers of recent climate change,” it stated, after which it listed four human-led activities – burning fossil fuels, deforestation, industrial activities, and agricultural practices – that have contributed to the warming earth.

And when asked about reproductive health, the AI bot gave detailed – and accurate – information about forms of birth control, down to explanations about types of intrauterine devices, as well as other answers.

Optimism and concern

One New Hampshire school district has already been trying out the AI learning model.

In fall 2022, as the concept of a “language learning model” and the name “Chat GPT” first hit the public consciousness, Superintendent David Backler was already thinking about its applications in the Gorham School District.

Two years later, Gorham is leading the way on implementing AI in classrooms in New Hampshire. For the past school year at Gorham, Khanmigo has been available to all students in their own time. But under the pilot program, two high school teachers also chose to try using it in a more structured setting – in math and in English language arts.

Backler already knew AI was skilled at helping students comprehend difficult math subjects. It was English where the technology surprised him.

“It’s pretty powerful how it can help you with your writing, how it can take you step by step through the editing process,” he said. “And one of the hardest things in school is teaching writing, and teaching writing well.”

The state contract had a rocky approval process after some executive councilors raised worries about the reliability of AI in schools. Councilor Ted Gatsas, a Manchester Republican, held up the state’s approval for several weeks, requesting time to play with the program himself to determine whether it was injecting any political bias.

“I had the chance a chance to ask it: ‘When does life begin?’” Gatsas said during a May 14 Executive Council meeting. “But that was a biology question. And the answer was apolitical, and I thought that was a good thing.”

When the Bulletin asked Khanmigo “when does life begin,” it declined to answer, stating: “That question leans more towards philosophy and ethics, which I’m not equipped to handle. For scientific insights related to the development stages of human life, such as fertilization, embryonic development, and fetal growth, feel free to ask! These topics are well within the realm of biology.”

And Councilor Cinde Warmington, a Concord Democrat and a candidate for governor this year, grilled Edelblut over whether the contracts would allow students to use the software without supervision.

“Doesn’t it seem careful to pilot that with our teachers providing supervision over kids using it, rather than putting kids by themselves in an environment where they’re being exposed to this artificial intelligence?” Warmington asked.

Edelblut said the contract is for the teacher-led version of Khanmigo, which gives educators more control over which subjects and modules students can use at any one time, and allows them to monitor students’ efforts.

Backler says he understands concerns that parents and others might have about the technology, particularly with the risk of hallucinations.

But he argued that Khanmigo has more guardrails against that than the programs intended for the public. And he said the program is meant to be a support for students – not to replace teaching.

“It’s not doing your writing; it’s not doing your work,” he said. “It’s giving you feedback on what you’re doing.”

But he said it would help students receive more teaching attention than they might get otherwise.

“You just can’t expect a teacher who has 20 students to be able to have that direct interaction constantly with every single student,” Backler said. “It’s not possible. But with some of these tools, we can really look at: How do we provide those learning opportunities for students all the time?”

New Hampshire Bulletin is part of States Newsroom, a nonprofit news network supported by grants and a coalition of donors as a 501c(3) public charity. New Hampshire Bulletin maintains editorial independence. Contact Editor Dana Wormald for questions: info@newhampshirebulletin.com. Follow New Hampshire Bulletin on Facebook and X.

OKLAHOMA CITY – Students at some of Oklahoma’s public colleges and universities will soon be able to pursue undergraduate degrees in artificial intelligence.

The Oklahoma State Regents for Higher Education approved artificial intelligence degree programs at Rose State College, Southwestern Oklahoma State University and the University of Oklahoma on June 4.

While some universities have offered courses in artificial intelligence, these are the first degree programs in the state.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter

Trisha Wald, dean of the Dobson College of Business and Technology at Southwestern Oklahoma State University, worked to start up the program at the university. Representatives at Rose State College and the University of Oklahoma were not available for comment.

While the degree program can begin in the fall for Southwestern Oklahoma State University, Wald said the late approval means some of the new AI classes may not be able to start until the spring.

Wald said she looked at similar programs in other states to create the proposed curriculum for this new program. While Wald said there are “not as many programs as you would think,” she was able to use their programs to determine what Southwestern Oklahoma State University’s program needed.

“It’s a multidisciplinary program, so it’s not just computer science courses,” Wald said. “We’ve got higher level math, psychology and philosophy courses, specifically on ethics. So it’s going to help us have more well-rounded individuals.”

Wald said the approval process took months and the proposal had to demonstrate workforce demand to the Regents as part of the proposal process.

Over 19,000 jobs in Oklahoma currently require AI skills, officials said. This number is expected to increase by 21% in the next decade.

“AI is rapidly emerging as a vital employment sector,” said State Regents for Higher Education Chair Jack Sherry in a statement. “New career opportunities in areas like machine learning, data science, robotics and AI ethics are driving demand for AI expertise, and Oklahoma’s state system colleges and universities are answering the call.”

Gov. Kevin Stitt said the new degree programs will allow Oklahoma’s students to be at the forefront of the AI industry.

“These degree programs are a great leap forward in our commitment to innovation in education and will position Oklahoma to be a leader in AI,” said Gov. Kevin Stitt in a statement. “AI is reshaping every aspect of our lives, especially academics. I’m proud of the Board of Regents for ensuring Oklahoma’s higher ed students do more than just keep pace, they’ll lead the AI revolution.”

Oklahoma Voice is part of States Newsroom, a nonprofit news network supported by grants and a coalition of donors as a 501c(3) public charity. Oklahoma Voice maintains editorial independence. Contact Editor Janelle Stecklein for questions: info@oklahomavoice.com. Follow Oklahoma Voice on Facebook and X.

It’s been roughly two years since AI (artificial intelligence) became an inescapable topic of everyday conversation — much of it focused on the spectacular creative powers of generative AI, from making absurd images to college students’ essays.

But the rapidly emerging set of technologies offers much more than novelty: In fact, Gov. Dan McKee thinks AI could be an ally in his maneuver to raise Rhode Islanders’ personal income by 2030. That’s just one goal of the eventual report that will be produced by the Rhode Island Governor’s Artificial Intelligence Task Force, which met for the first time Monday at the Department of Administration building in Providence.

Chris Parisi, president of Trailblaze Marketing and vice chair of the task force, invoked Spider-Man in his opening remarks, and noted that AI opens up a space of potential but also responsibility.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter

“I’m not here to say AI will not take your jobs,” Parisi said. “But we are also creating new jobs.”

McKee established the task force with an executive order Feb. 29, and it now includes two dozen members from both the public and private sectors, most of whom convened Monday afternoon for a light introduction and discussion of the group’s aims. Several members weren’t present, including Sen. Lou DiPalma of Middletown — he was traveling out of state — and Angélica Infante-Green, the state’s K-12 education commissioner. The task force is chaired by Jim Langevin, the former congressman.

The diversity of stakeholders reflects what Parisi cited as one goal of the assembly: to make sure state applications of AI are “ethical and unbiased.”

The other predominant concern was how to best leverage AI as a tool within government and business. Langevin announced the task force’s fact finding teams will work on topics like finance, government, education and small business over the next year, before producing a report and road map for AI usage in Rhode Island. It’s this data that McKee hopes will inform his strategy for higher incomes by 2030.

“This report is gonna be incorporated into this plan,” McKee said.

The state’s executive branch is also soliciting a consulting strategic advisor to help advise the task force. A solicitation was uploaded to the state’s bidding site on June 24 and will remain open until July 25.

The multinational consulting firm McKinsey reported in 2022 that AI use had “plateaued” amongst businesses. But in its 2024 report on AI, released May 30, the firm found this inertia has ended. A survey administered by the firm found that 72% of responding businesses were now using AI in at least one capacity. The use of generative AI — at its core the same technology used for recreational or artistic purposes — also ballooned, jumping from 33% to 65% usage since the last McKinsey survey.

In Rhode Island, the situation’s no different: “Lots of businesses will do great things with AI and lots of businesses are very nervous about AI,” said task force member Commerce Secretary Liz Tanner.

The widespread professional adoption of AI has made its regulation likewise unignorable for governments, who also stand to benefit from the enrichment of data and simplification of work it provides. Nationally, an AI “bill of rights” has been blueprinted, and task forces have been popping up across states like Alabama, New Jersey, Massachusetts, Oklahoma and Washington.

Statehouses nationwide have also introduced laws to regulate AI, which are now so plentiful the Electronic Privacy Information Center has introduced a scorecard for AI legislation. In Rhode Island, DiPalma and Rep. Jacquelyn Baginski both introduced AI-related legislation during the 2024 session, and both sit on the AI task force as ex officio members.

Baginski, a Cranston Democrat, introduced a bill to support civil litigation against the practice of “algorithmic discrimination” — in other words, instances of AI-driven decision-making that exhibit the bias the task force wants to avoid. Baginski’s bill also stipulated restrictions on not just “deployers” of automated decision-making but developers of such tools. The bill was sentenced to “further study” by the House Committee on Innovation, Internet, & Technology and went no further.

Baginski and DiPalma also introduced companion bills that would prevent the use of AI-generated content in election communications within 90 days of an election. Baginski’s version passed the House but died in the Senate.

The insurance industry was an early adopter of AI, with industry standards and guidelines for its use issued as early as 2020. The state’s Department of Business Regulation has already issued guidance for insurers: a nine-page document that was published in March, as pointed out by the department’s director and task force member Elizabeth Kelleher Dwyer.

Task force member Edmund Shallcross III, the CEO of Amica, said the Lincoln-based insurance company has “been using data and machine learning for years…We’ll be using artificial intelligence in probably every part of our business in the next one, three, five years.”

On the government side, task force member Marc Pappas, director of the Rhode Island Emergency Management Agency, was generally positive about AI. “It makes us better at recovery from disasters,” he said, noting its skills in mapping, imaging analysis for damage assessment, and help in allocating resources when disasters strike.

Christopher Horvath of Citizens Bank appeared more cautious overall than some of his fellow task force members, expressing concern about “bad actors,” who can exploit AI. Security considerations were echoed by task force member Brian Tardiff, the state’s chief digital officer and chief information officer, who noted that AI can improve the efficiency of government but only if the proper frameworks are put in place.

“We can’t have effective and efficient deployments without that data security,” Tardiff said.

As the founder of Deans for Impact in 2015, he enlisted university education school deans to incorporate findings from cognitive science into teacher preparation. Before that, he spent five years as policy director of the NewSchools Venture Fund, which underwrites new models of schooling. In his new endeavor, Cognitive Resonance, which he calls “a think-and-do tank,” he’s pushing to help people think not only about how we learn, but how generative artificial intelligence (AI) works — and why they’re different.

His Substack newsletter and Twitter feed regularly poke holes in high-flying claims about the power of AI-powered tutors — he recently offered choice words for Khan Academy founder Sal Khan’s YouTube demonstration of Open AI’s new GPT4o tool, saying it was “deployed in the most favorable educational environment we can possibly imagine,” leaving open the possibility that it might not perform so well in the real world.

In April, Riley ruffled feathers in the startup world with an essay in the journal Education Next that took Khan Academy and other AI-related companies to task for essentially using students as guinea pigs.

In the essay, he recounted asking Khanmigo to help him simplify an algebraic equation. Riley-as-student got close to solving it, but the AI actually questioned him about his steps, eventually asking him to rethink even basic math, such as the fact that 2 + 2.5 = 4.5.

Such an exchange isn’t just unhelpful to students, he wrote, it’s “counterproductive to learning,” with the potential to send students down an error-filled path of miscalculation, misunderstanding and wasted effort.

The interview has been edited for length and clarity.

The 74: We’re often so excited about the possibilities of ed tech in education that we just totally forget what science says about how we learn. I wonder if you have any thoughts on that.

Benjamin Riley: I have many. Part of my frustration is that we are seemingly living in a moment where we’re simultaneously recognizing in other dimensions where technology can be harmful, or at least not beneficial, to learning, while at the same time expressing unbridled enthusiasm for a new technology and believing that it finally will be the cure-all, the silver bullet that finally delivers on the vision of radically transforming our education system. And yeah, it’s frustrating. Ten years ago, for example, when everybody was excited about personalization, there were folks, myself included, raising their hand and saying, “Nope, this doesn’t align with what we know about how we think and learn. It also doesn’t align with the science of how we collectively learn, and the role of education institutions as a method of culturally transmitting knowledge.” All of those personalized learning dreams were dying out. And many of the prominent, incredibly well-funded personalized learning efforts either went completely belly-up, like AltSchool, or have withered on the vine, like some of the public schools now named Gradient.

Now AI has revived all of those dreams again. And it’s frustrating, because even if it were true that personalization were the solution, no one 10 years ago, five years ago, was saying, “But what we need are intelligent chatbot tutors to make it real.” So what you’re seeing is sort of a commitment to a vision. Whatever technology comes along, we’re going to shove into that vision and say that this is going to deliver it. I think for the same reasons it failed before, it will fail again. 

You’re a big fan of the University of Virginia cognitive scientist Daniel Willingham, who has done a lot to popularize the science of how we learn.

He’s wonderful at creating pithy phrases that get to the heart of the matter. One of the counterintuitive phrases he has that is really powerful and important is that our minds in some sense “are not built to think,” which feels really wrong and weird, because isn’t that what minds do? It’s all they do, right? But what he means is that the process of effortful thinking is taxing in the same way that working out at the gym is taxing. One of the major challenges of education is: How do you wrap around that with students, who, like all of us, are going to try to essentially avoid doing effortful thinking for sustained periods? Over and over again, technologists just assume away that problem.

In the case of something like large language models, or LLMs, how do they approach this problem of effortful thinking? Do they just ignore it altogether?

It’s an interesting question. I’m almost not sure how to answer it, because there is no thinking happening on the part of an LLM. A large language model takes the prompts and the text that you give it and tries to come up with something that is responsive and useful in relation to that text. And what’s interesting is that certain people — I’m thinking of Mark Andreessen most prominently — have talked about how amazing this is conceptually from an education perspective, because with LLMs you will have this infinitely patient teacher. But that’s actually not what you want from a teacher. You want, in some sense, an impatient teacher who’s going to push your thinking, who’s going to try to understand what you’re bringing to any task or educational experience, lift up the strengths that you have, and then work on building your knowledge in areas where you don’t yet have it. I don’t think LLMs are capable of doing any of that.

As you say, there’s no real thinking going on. It’s just a prediction machine. There’s an interaction, I guess, but it’s an illusion. Is that the word you would use?

Yes. It’s the illusion of a conversation. 

In your Education Next essay, you quote the cognitive scientist Gary Marcus, who says LLMs are “frequently wrong, but never in doubt.” It feels to me like that is extremely dangerous in something young people interact with.

Yes! Absolutely. This is where it’s really important to distinguish between the now and the real and the present versus the hypothetical imagined future. There’s just no question that right now, this “hallucination problem” is endemic. And because LLMs are not thinking, they generate text that is factually inaccurate all the time. Even some of the people who are trying to push it out into the world acknowledge this, but then they’ll just put this little asterisk: “And that’s why an educator must always double-check.” Well, who has the time? I mean, what utility is this? And then people will say, “Well yes, but surely it’s going to get better in the future.” To which I say, Maybe, let’s wait and see. Maybe we should wait until we’ve arrived at that point before we push this out.

Do we know how often LLMs are making mistakes?

I can say just from my own personal usage of Khanmigo that it happens a lot, for reasons that are frankly predictable once you understand how the technology works. How often is it happening with seventh-grade students who are just learning this idea for the first time? We just don’t know. [In response to a query about errors, Khan Academy sent links to two blog posts on its site, one of which noted that Khanmigo “occasionally makes mistakes, which we expected.” It also pointed, among other things, that Khanmigo now uses a calculator to solve numerical problems instead of using AI’s predictive capabilities.]

One of the things you say in the EdNext piece is that you just “sound like a Luddite” as opposed to actually being one. The Luddites saw the danger in automation and were trying to push against it. Is it the same, in a way, as what you’re doing? 

Thank you for asking that question because I feel my naturally contrarian ways risk painting me into a corner I’m really not in. Because in some sense, generative AI and large language models are incredible — they really are. It is a remarkable achievement that they are able to produce fluent and coherent narratives in response to just about any combination of words that you might choose to throw at them. So I am not a Luddite who thinks that we need to burn this all down.

There are methods and ways, both within education and in society more broadly, in which this tool could be incredibly useful for certain purposes. Already, it’s proving incredibly stimulating in thinking about and understanding how humans think and learn, and how that is similar and different from what they do. If we could just avoid the ridiculous overhype and magical thinking that seems to accompany the introduction of any new technology and calm down and investigate before pushing it out into our education institutions, then I think we’d be a lot better off. There really is a middle ground here. That’s where I’m trying to situate myself. 

Maybe this is a third rail that we shouldn’t be touching, but I was reading about Thomas Edison and his ideas on education. He had a great quote about movies, which he thought would revolutionize classrooms. He said, “The motion picture will endure as long as poor people exist.” It made me think: One of the underlying themes of ed tech is this idea of bringing technology to the people. Do you see a latent class divide here? Rich kids will get an actual personal tutor, but everybody else will get an LLM? 

My worry runs differently than that. Again, back to the Willingham quote: “Our minds are not built to think.” Here’s the harsh reality that could indeed be a third rail, but it needs to be acknowledged if we’re going to make meaningful progress: If we fail in building knowledge in our students, thinking gets harder and harder, which is why school gets harder and harder, and why over time you start to see students who find school really miserable. Some of them drop out. Some of them stop trying very hard. These folks — the data is overwhelming on this — typically end up having lives that are shorter, with less economic means, more dire health outcomes. All of this is both correlated and interrelated causation.

But here’s the thing: For those students in particular, a device that alleviates the cognitive burden of schooling will be appealing. I’m really worried that this now-widely available technology will be something they turn to, particularly around the incredibly cognitively challenging task of writing — and that they will continue to look to this as a way of automating their own cognition. No one really needs to worry about the children of privilege. They are the success stories academically and, quite frankly, many of them enjoy learning and thinking and will avoid wanting to use this as a way of outsourcing their own thinking. But it could just make the existing divide a lot wider than it is today — much wider.

How is education research responding to AI?

The real challenge is that the pace of technology, particularly the pace of technological developments in the generative AI world, is so fast that traditional research methods are not going to be able to keep up. It’s not that there won’t be studies — I’m sure there are already some underway, and there’s tiny, emerging studies that I have seen here and there. But we just don’t have the capabilities as a research enterprise to be doing things the traditional way. A really important question that needs to be grappled with, as a matter of policy, potentially as a matter of philanthropy and just as a matter of society, is: So, what then? Do we just do it and hope for the best? Because that may be what ends up happening.

As we’ve seen with social media and smartphones in schools, there can be real impacts that you don’t realize until five, 10 years down the road. Then you go back and say, “Well, I wish we’d been thinking about that in advance rather than just rolling the dice and seeing where it came up.” We don’t do that in other realms of life. We don’t let people just come up with medicines that they think will cure certain diseases and then just say, “Well, we’ll see. We’ll introduce it into broader society and let’s figure it out.” I’m not necessarily saying that we need the equivalent per se, but something that would give us better insight and real-time information to help us figure out the overall positives and not-so-positives seems to me a real challenge that is underappreciated at the moment.

Investigators with the Los Angeles Unified School District’s inspector general’s office conducted a video interview with Chris Whiteley, the former senior director of software engineering at AllHere, after he told The 74 his former employer’s student data security practices violated both industry standards and the district’s own policies. 

Whiteley told The 74 he had alerted the school district, the IG’s office and state education officials earlier to the data privacy problems with Ed but got no response. His meeting with investigators occurred July 2, one day after The 74 published its story outlining Whiteley’s allegations, including that the chatbot put students’ personally identifiable information at risk of getting hacked by including it in all chatbot prompts, even in those where the data weren’t relevant; sharing it with other third-party companies unnecessarily and processing prompts on offshore servers in violation of district student privacy rules. 

In an interview with The 74 this week, Whiteley said the officials from the district’s inspector general’s office “were definitely interested in what I had to say,” as speculation swirls about the future of Ed, its ed tech creator AllHere and broader education investments in artificial intelligence. 

“It felt like they were after the truth,” Whiteley said, adding, “I’m certain that they were surprised about how bad [students’ personal information] was being handled.”

To generate responses to even mundane prompts, Whiteley said, the chatbot processed the personal information for all students in a household. If a mother with 10 children asked the chatbot a question about her youngest son’s class schedule, for example, the tool processed data about all of her children to generate a response. 

“It’s just sad and crazy,” he said.

The inspector general’s office directed The 74’s request for comment to a district spokesperson, who declined to comment or respond to questions involving the inquiry.

While the conversation centered primarily on technical aspects related to the company’s data security protocols, Whiteley said investigators probed him on his personal experiences with AllHere, which he described as being abusive, and its finances.

Whiteley was laid off from AllHere in April. Two months later, a notice posted to the company’s website said a majority of its 50 or so employees had been furloughed due to its “current financial position” and the LAUSD spokesperson said company co-founder and CEO Joanna Smith-Griffin had left. The former Boston teacher and Harvard graduate was successful in raising $12 million in venture capital for AllHere and appeared with L.A. schools Superintendent Alberto Carvalho at ed tech conferences and other events throughout the spring touting the heavily publicized AI tool they partnered to create.

Just weeks ago, Carvalho spoke publicly about how the project had put L.A. out in front as school districts and ed tech companies nationally race to follow the lead of generative artificial intelligence pioneers like ChatGPT. But the school chief’s superlative language around what Ed could do on an individualized basis with 540,000 students had some industry observers and AI experts speculating it was destined to fail.

The chatbot was supposed to serve as a “friendly, concise customer support agent” that replied “using simple language a third grader could understand” to help students and parents supplement classroom instruction, find assistance with kids’ academic struggles and navigate attendance, grades, transportation and other key issues. What they were given, Whiteley charges, was a student privacy nightmare. 

Smith-Griffin recently deactivated her LinkedIn page and has not surfaced since her company went into apparent free fall. Attempts to reach AllHere for comment were unsuccessful and parts of the company website have gone dark. LAUSD said earlier that AllHere is for sale and that several companies are interested in acquiring it.

The district has already paid AllHere $3 million to build the chatbot and “a fully-integrated portal” that gave students and parents access to information and resources in a single location, the district spokesperson said in a statement Tuesday, and “was surprised by the financial disruption to AllHere.” 

AllHere’s collapse represents a stunning fall from grace for a company that was named among the world’s top education technology companies by Time Magazine just months earlier. Scrutiny of AllHere intensified when Whiteley became a whistleblower. He said he turned to the press because his concerns, which he shared first with AllHere executives and the school district, had been ignored.

Whitely shared source code with The 74 which showed that students’ information had been processed on offshore servers. Seven out of eight Ed chatbot requests, he said, were sent to places like Japan, Sweden, the United Kingdom, France, Switzerland, Australia and Canada. 

‘How are smaller districts going to do this?’

What district leaders failed to do as they heralded their new tool, Whiteley said, is conduct sufficient audits. As L.A. — and school systems nationwide — contract with a laundry list of tech vendors, he said it’s imperative that they understand how third-party companies use students’ information. 

“If the second-biggest district can’t audit their [personally identifiable information] on new or interesting products and can’t do security audits on external sources, how are smaller districts going to do this?” he asked.

Over the last several weeks, the district’s official position on Ed has appeared to shift. In late June when the district spokesperson said that several companies were “interested in acquiring Allhere,” they also said its predecessor would “continue to provide this first-of-its-kind resource to our students and families.” In its initial response to Whiteley’s allegations published July 1, the spokesperson said that education officials would “take any steps necessary to ensure that appropriate privacy and security protections are in place in the Ed platform.” 

In a story two days later in the Los Angeles Times, a district spokesperson said the chatbot had been unplugged on June 14. The 74 asked the spokesperson to provide documentation showing the tool was disabled last month but didn’t get a response. 

Even after June 14, Carvalho continued to boast publicly about LAUSD’s foray into generative AI and what he described as its stringent data privacy requirements with third-party vendors. 

On Tuesday, the district spokesperson told The 74 that the online portal — even without a chatty, animated sun — “will continue regardless of the outcome with AllHere.” In fact, the project could become a source of district revenue. Under the contract between AllHere and LAUSD, which was obtained by The 74, the chatbot is the property of the school district, which was set to receive 2% in royalty payments from AllHere “should other school districts seek to use the tool to benefit their families and students.” 

In the statement Tuesday, the district spokesperson said that officials chose to “temporarily disable the chatbot” amid AllHere’s uncertainty and that it would “only be restored when the human-in-the-loop aspect is re-established.” 

Whiteley agreed that the district could maintain the student information dashboard without the chatbot and, similarly, that another firm could buy what remains of AllHere. He was skeptical, however, that Ed the chatbot would live another day because “it’s broken”

“The name AllHere,” he said, “I think is dead.”

The AI bot, created by Boston-based AllHere Education, was launched March 20. But just three months later, AllHere posted on its website that a majority of its 50 or so employees had been furloughed due to its “current financial position.” A spokesperson for the Los Angeles district said company founder and CEO Joanna Smith-Griffin was no longer on the job. AllHere was up for sale, the district said, with several businesses interested in acquiring it.

The news was shocking and certainly bleak for the ed tech industry, but several observers say the partnership bit off more than it could chew, tech-wise — and that the ensuing blowup could hurt future AI investments.

Ed was touted as a powerful, easy-to-use online tool for students and parents to supplement classroom instruction, find assistance with kids’ academic struggles and help families navigate attendance, grades, transportation and other key issues, all in 100 languages and on their mobile phones.

But Amanda Bickerstaff, founder and CEO of AI for Education, a consulting and training firm, said that was an overreach.

“What they were trying to do is really not possible with where the technology is today,” she said. ”It’s a very broad application [with] multiple users — teachers, students, leaders and family members — and it pulled in data from multiple systems.”

She noted that even a mega-corporation like McDonald’s had to trim its AI sails. The fast-food giant recently admitted that a small experiment using a chatbot to power drive-thru windows had resulted in a few fraught customer interactions, such as one in which a woman angrily tried to persuade the bot that she wanted a caramel ice cream as it added multiple stacks of butter to her order.

If McDonald’s, worth an estimated $178.6 billion, can’t get 100 drive-thrus to take lunch orders with generative AI, she said, the tech isn’t “where we need it to be.”

If anything, L.A. and AllHere did not seem worried about the project’s scale, even if industry insiders now say it was bound to under-deliver: Last spring, at a series of high-profile ed tech conferences, Smith-Griffin and Superintendent Alberto Carvalho showed off Ed widely, with Carvalho saying it would revolutionize students’ and parents’ relationships to school, “utilizing the data-rich environment that we have for every kid.”

In an interview with The 74 at the ASU+GSV Summit in San Diego in April, Carvalho said many students are not connected to school, “therefore they’re lost.” Ed, he promised, would change that, with a “significantly different approach” to communication from the district.

“We are shifting from a system of 540,000 students into 540,000 ‘schools of one,’” with personalization and individualization for each student, he said, and “meaningful connections with parents.”

Better communication with parents, he said, would help improve not just attendance but reading and math proficiency, graduation rates and other outcomes. “The question that needs to be asked is: Why have those resources not meaningfully connected with students and parents, and why have they not resulted in this explosive experience in terms of educational opportunity?”

Carvalho noted Ed’s ability to understand and communicate in about 100 different languages. And, he crowed, it “never goes to sleep” so it can answer questions 24/7. He called it “an entity that learns and relearns all the time and does nothing more, nothing less than adapt itself to you. I think that’s a game changer.” 

But one experienced ed tech insider recalled hearing Carvalho speak about Ed at the conference in April and say it was already solving “all the problems” that big districts face. The insider, who asked not to be identified in order to speak freely about sensitive matters, found the remarks troubling. “The messaging was so wrong that at that point I basically started a stopwatch on how long it would take” for the effort to fail. “And I’m kind of amazed it’s been this long before it all fell apart. I feel badly about it, I really do, but it’s not a surprise.”

‘A high-risk proposition’

In addition to the deal’s dissolution, The 74 reported last week that a former senior director of software engineering at AllHere told district officials, L.A.’s independent inspector general’s office and state education officials that Ed processed student records in ways that likely ran afoul of the district’s own data privacy rules and put sensitive information at risk of being hacked — warnings that he said the agencies ignored. 

AI for Education’s Bickerstaff said developers “have to take caution” when building these systems for schools, especially those like Ed that bring together such large sets of data under one application.

“These tools, we don’t know how they work directly,” she said. “We know they have bias. And we know they’re not reliable. We know they can be leaky. And so we have to be really careful, especially with kids that have protected data.”

Alex Spurrier, an associate partner with the education consulting firm Bellwether Education Partners, said what often happens is that district leaders “try to go really big and move really fast to adopt a new technology,” not fully appreciating that it’s “a really high risk proposition.”

While ed tech is rife with disaster stories of overpromising and disappointing results, Spurrier said, other districts dare to take a different approach, starting small, iterating and scaling up. In those cases, he said, disaster rarely follows.

Richard Culatta, CEO of the International Society for Technology in Education (ISTE), put it more bluntly: “Whenever a district says, ‘Our strategy around AI is to buy a tool,’ that’s a problem. When the district says, ‘For us, AI is a variety of tools and skills that we are working on together,’ that’s when I feel comfortable that we’re moving in the right direction.”

Culatta suggested that since generative AI is developing and changing so rapidly, districts should use the next few months as “a moment of exploration — it’s a moment to bring in teachers and parents and students to give feedback,” he said. “It is not the moment for ribbon cutting.” 

‘It’s about exploring’

Smith-Griffin founded AllHere in 2016 at Harvard University’s Innovation Labs. In an April interview with The 74, she said she originally envisioned it as a way to help school systems reduce chronic absenteeism through better communication with parents. Many interventions that schools rely on, such as phone calls, postcards and home visits, “tend to be heavily reliant on the sheer power of educators to solve system-wide issues,” she said.

A former middle-school math teacher, Smith-Griffin recalled, “I was one of those teachers who was doing phone calls, leaving voicemails, visiting my parents’ homes.” 

AllHere pioneered text messaging “nudges,” electronic versions of postcard reminders to families that, in one key study, improved attendance modestly. 

The company’s successful proposal for L.A., Smith-Griffin said, envisioned extending the attendance strategies while applying them to student learning “in the most disciplined way possible.”

“You nudge a parent around absences and they will tell you things ranging from, ‘My kid needs tutoring, my kid is struggling with math’ [to] ‘I struggle with reading,’” she said. AllHere went one step further, she said, bringing together “the full body of resources” that a school system can offer parents.

The district had high hopes for the chatbot, requiring it to focus on “eliminating opportunity gaps, promoting whole-child well-being, building stronger relationships with students and families, and providing accessible information,” according to the proposal.

In April, it was still in early implementation at 100 of the district’s lowest performing “priority” schools, serving about 55,000 students. LAUSD planned to roll out Ed for all families this fall. The district “unplugged” the chatbot on June 14, the Los Angeles Times reported last week, but a district spokesperson said L.A. “will continue making Ed available as a tool to its students and families and is closely monitoring the potential acquisition of AllHere.” The company did not immediately responded to queries about the chatbot or its future.

As for the apparent collapse of AllHere, speculation in the ed tech world is rampant.

In the podcast he co-hosts, education entrepreneur Ben Kornell said late last month, “My spidey sense basically goes to ‘Something’s not adding up here and there’s more to the story.’” He theorized a “critical failure point” that’s yet to emerge “because you don’t see things like this fall apart this quickly, this immediately” for such a small company, especially in the middle of a $6 million contract.

Kornell said the possibilities fall into just a few categories: an accounting or financial misstep, a breakdown among AllHere’s staff, board and funders or “major customer payment issues.” 

The district also may have withheld payment for undelivered products, but he said the sudden collapse of the company seemed unusual. “If you are headed towards a cash crisis, the normal thing to do would be: Go to your board, go to your funders, and get a bridge to get you through that period and land the plane.”

Bellwether’s Spurrier said L.A. deserves a measure of credit “for being willing to lean into AI technology and think about ways that it could work.” But he wonders whether the best use of generative AI at this moment will be found not in “revolutionizing instruction,” as L.A. has pursued, but elsewhere. 

“There’s plenty of opportunities to think about how AI might help on the administrative side of things, or help folks that are kind of outside the classroom walls,” rather than focusing on changing how schools deliver instruction. “I think that’s the wrong place to start.”

ISTE’s Culatta noted that just down the road from Los Angeles, in Santa Ana, California, district officials there responded to the dawn of tools like ChatGPT and Google’s Gemini by creating evening classes for adults. “The parents come in and they talk about what AI is, how they should be thinking about it,” he said. “It’s about exploring. It’s about helping people build their skills.” 

‘How are your financials?’

The fate of AllHere’s attendance work in districts nationwide isn’t clear at the moment. In one large district, the Prince George’s County, Maryland, Public Schools, near Washington, D.C., teachers piloted AllHere with 32 schools as far back as January 2020, spokeswoman Meghan Thornton said. The district added two more schools to the pilot in 2022, but AllHere notified the district on June 18 that, effective immediately, it wouldn’t be able to continue its services due to “unforeseen financial circumstances.” 

District officials are now looking for another messaging system to replace AllHere “should it no longer be available,” Thornton said.

Bickerstaff said the field more broadly suffers from “a major, major overestimation of the capabilities of the technology to date.” L.A., she noted, is the nation’s second-largest school district, so even the pilot stage likely saw “very high” usage, raising its costs. She predicted a fast acquisition of AllHere, noting that they’d been looking for outside investment for several months.

As founder of the startup Magic School AI, which offers teachers tools to streamline their workload, Adeel Khan is no stranger to hustling for funding — and to competitors running out of money. But he said the news about AllHere and Ed was bad for the industry more broadly, leaving districts with questions about whether to partner with newer, untested companies.

“I see it as something that is certainly not great for the startup ecosystem,” he said.

Even before the news about AllHere broke last month, Khan attended ISTE’s big national conference in Denver last month, where he talked to school district officials about prospective partnerships. “More than one time I was asked directly, ‘How are your financials?’” he recalled. 

Usually technology directors ask about features and what a product can do for students, he said. But they’re beginning to realize that a failed product doesn’t just waste time and money. It damages reputations as well. “That is on the mind of buyers,” he said. 

When school districts invest in new tech, he said, they’re not just committing to funding it for months or even years, but also to training teachers and others, so they want responsible growth.

“There’s a lot of disruption to K-12 when a product goes out of business,” Khan said. “So people remember this. They remember, ‘Hey, we committed to this product. We discovered it at ISTE two years ago and we loved it. It was great — and it’s not here anymore. And we don’t want to go through that again.’ ”

Your children could be some of a growing number of California kids having their writing graded by software instead of a teacher.

California school districts are signing more contracts for artificial intelligence tools, from automated grading in San Diego to chatbots in central California, Los Angeles, and the San Francisco Bay Area. 

English teachers say AI tools can help them grade papers faster, get students more feedback, and improve their learning experience. But guidelines are vague and adoption by teachers and districts is spotty. 

The California Department of Education can’t tell you which schools use AI or how much they pay for it. The state doesn’t track AI use by school districts, said Katherine Goyette, computer science coordinator for the California Department of Education. 

While Goyette said chatbots are the most common form of AI she’s encountered in schools, more and more California teachers are using AI tools to help grade student work. That’s consistent with surveys that have found teachers use AI as often if not more than students, news that contrasts sharply with headlines about fears of students cheating with AI.  

Teachers use AI to do things like personalize reading material, create lesson plans, and other tasks in order to save time and reduce burnout. A report issued last fall in response to an AI executive order by Gov. Gavin Newsom mentions opportunities to use AI for tutoring, summarization, and personalized content generation, but also labels education a risky use case. Generative AI tools have been known to create convincing but inaccurate answers to questions, and use toxic language or imagery laden with racism or sexism.

California issued guidance for how educators should use the technology last fall, one of seven states to do so. It encourages critical analysis of text and imagery created by AI models and conversations between teachers and students about what amounts to ethical or appropriate use of AI in the classroom.

But no specific mention is made of how teachers should treat AI that grades assignments. Additionally, the California education code states that guidance from the state is “merely exemplary, and that compliance with the guidelines is not mandatory.”

Goyette said she’s waiting to see if the California Legislature passes Senate Bill 1288, which would require state Superintendent Tony Thurmond to create an AI working group to issue further guidance to local school districts on how to safely use AI. Cosponsored by Thurmond, the bill also calls for an assessment of the current state of AI in education and for the identification of forms of AI that can harm students and educators by 2026.

Nobody tracks what AI tools school districts are adopting or the policy they use to enforce standards, said Alix Gallagher, head of strategic partnerships at the Policy Analysis for California Education center at Stanford University. Since the state does not track curriculum that school districts adopt or software in use, it would be highly unusual for them to track AI contracts, she said.

Amid AI hype, Gallagher thinks people can lose sight of the fact that the technology is just a tool and it will only be as good or problematic as the decisions of the humans using that tool, which is why she repeatedly urges investments in helping teachers understand AI tools and how to be thoughtful about their use and making space for communities are given voice about how to best meet their kid’s needs.

“Some people will probably make some pretty bad decisions that are not in the best interests of kids, and some other people might find ways to use maybe even the same tools to enrich student experiences,” she said.

Teachers use AI to grade English papers

Last summer, Jen Roberts, an English teacher at Point Loma High School in San Diego, went to a training session to learn how to use Writable, an AI tool that automates grading writing assignments and gives students feedback powered by OpenAI. For the past school year, Roberts used Writable and other AI tools in the classroom, and she said it’s been the best year yet of nearly three decades of teaching. Roberts said it has made her students better writers, not because AI did the writing for them, but because automated feedback can tell her students faster than she can how to improve, which in turn allows her to hand out more writing assignments.  

“At this point last year, a lot of students were still struggling to write a paragraph, let alone an essay with evidence and claims and reasoning and explanation and elaboration and all of that,” Roberts said. “This year, they’re just getting there faster.”

Roberts feels Writable is “very accurate” when grading her students of average aptitude. But, she said, there’s a downside: It sometimes assigns high-performing students lower grades than merited and struggling students higher grades. She said she routinely checks answers when the AI grades assignments, but only checks the feedback it gives students occasionally. 

“In actual practicality, I do not look at the feedback it gives every single student,” she said. “That’s just not a great use of my time. But I do a lot of spot checking and I see what’s going on and if I see a student that I’m worried about get feedback, (I’m like) ‘Let me go look at what his feedback is and then go talk to him about that.’”

Alex Rainey teaches English to fourth graders at Chico Country Day School in northern California. She used GPT-4, a language model made by OpenAI which costs $20 a month, to grade papers and provide feedback. After uploading her grading rubric and examples of her written feedback, she used AI to grade assignments about animal defense mechanisms, allowing GPT-4 to analyze students’ grammar and sentence structure while she focused on assessing creativity.

“I feel like the feedback it gave was very similar to how I grade my kids, like my brain was tapped into it,” she said.

Like Roberts she found that it saves time, transforming work that took hours into less than an hour, but also found that sometimes GPT-4 is a tougher grader than she is. She agrees that quicker feedback and the ability to dole out more writing assignments produces better writers. A teacher can assign more writing before delivering feedback but “then kids have nothing to grow from.”

Rainey said her experience grading with GPT-4 left her in agreement with Roberts, that more feedback and writing more often produces better writers. She feels strongly that teachers still need to oversee grading and feedback by AI, “but I think it’s amazing. I couldn’t go backwards now.”

The cost of using AI in the classroom

Contracts involving artificial intelligence can be lucrative. 

To launch a chatbot named Ed, Los Angeles Unified School District signed a $6.2 million contract for two years with the option of renewing for three additional years. Magic School AI is used by educators in Los Angeles and costs $100 per teacher per year. 

Despite repeated calls and emails over the span of roughly a month, Writable and the San Diego Unified School District declined to share pricing details with CalMatters. A district spokesperson said teachers got access to Writeable through a contract with Houghton Mifflin Harcourt for English language learners. 

Quill is an AI-powered writing tool for students in grades 4-12 made by the company Quill. Quill says its tool is currently used at 1,000 schools in California and has more than 13,000 student and educator users in San Diego alone. An annual Quill Premium subscription costs $80 per teacher or $1800 per school.

Quill does not generate writing for students like ChatGPT or grade writing assignments, but gives students feedback on their writing. Quill is a nonprofit that’s raised $20 million from groups like Google’s charitable foundation and the Bill and Melinda Gates Foundation over the past 10 years.

Even if a teacher or district wants to shell out for an AI tool, guidance for safe and responsible use is still getting worked out. 

Governments are placing high-risk labels on forms of AI with the power to make critical decisions about whether a person gets a job or rents an apartment or receives government benefits. California Federation of Teachers President Jeff Freitas said he hasn’t considered whether AI for grading is moderate or high risk, but “it definitely is a risk to use for grading.”

The California Federation of Teachers is a union with 120,000 members. Freitas told CalMatters he’s concerned about AI having a number of consequences in the classroom. He’s worried administrators may use it to justify increasing classroom sizes or adding to teacher workloads; he’s worried about climate change and the amount of energy needed to train and deploy AI models’ he’s worried about protecting students’ privacy, and he’s worried about automation bias.

Regulators around the world wrestling with AI praise approaches where it is used to augmenthuman decisionmaking instead of replacing it. But it’s difficult for laws to account for automation bias and humans becoming placing too much trust in machines.

The American Federation of Teachers created an AI working group in October 2023 to propose guidance on how educators should use the technology or talk about it in collective bargaining contract negotiations. Freitas said those guidelines are due out in the coming weeks.

“We’re trying to provide guidelines for educators to not solely rely on (AI), he said. “It should be used as a tool, and you should not lose your critical analysis of what it’s producing for you.” 

State AI guidelines for teachers

Goyette, the computer science coordinator for the education department, helped create state AI guidelines and speaks to county offices of education for in-person training on AI for educators. She also helped create an online AI training series for educators. She said the most popular online course is about workflow and efficiency, which shows teachers how to automate lesson planning and grading.

“Teachers have an incredibly important and tough job, and what’s most important is that they’re building relationships with their students,” she said. “There’s decades of research that speaks to the power of that, so if they can save time on mundane tasks so that they can spend more time with their students, that’s a win.”

Alex Kotran, chief executive of an education nonprofit that’s supported by Google and OpenAI, said they found that it’s hard to design a language model to predictably match how a teacher grades papers.

He spoke with teachers willing to accept a model that’s accurate 80% of the time in order to reap the reward of time saved, but he thinks it’s probably safe to say that a student or parent would want to make sure an AI model used for grading is even more accurate.

Kotran of the AI Education Project thinks it makes sense for school districts to adopt a policy that says teachers should be wary any time they use AI tools that can have disparate effects on student’s lives. 

Even with such a policy, teachers can still fall victim to trusting AI without question. And even if the state kept track of AI used by school districts, there’s still the possibility that teachers will purchase technology for use on their personal computers.

Kotran said he routinely speaks with educators across the U.S. and is not aware of any systematic studies to verify the effectiveness and consistency of AI for grading English papers.

When teachers can’t tell if they’re cheating

Roberts, the Point Loma High School teacher,  describes herself as pro technology. 

She regularly writes and speaks about AI.  Her experiences have led her to the opinion that grading with AI is what’s best for her students, but she didn’t arrive at that conclusion easily. 

At first she questioned whether using AI for grading and feedback could hurt her understanding of her students. Today she views using AI like the cross-country coach who rides alongside student athletes in a golf cart, like an aid that helps her assist her students better.

Roberts says the average high school English teacher in her district has roughly 180 students. Grading and feedback can take between five to 10 minutes per assignment she says, so between teaching, meetings, and other duties, it can take two to three weeks to get feedback back into the hands of students unless a teacher decides to give up large chunks of their weekends. With AI, it takes Roberts a day or two.

Ultimately she concluded that “if my students are growing as writers, then I don’t think I’m cheating.” She says AI reduces her fatigue, giving her more time to focus on struggling students and giving them more detailed feedback.

“My job is to make sure you grow, and that you’re a healthy, happy, literate adult by the time you graduate from high school, and I will use any tool that helps me do that, and I’m not going to get hung up on the moral aspects of that,” she said. “My job is not to spend every Saturday reading essays. Way too many English teachers work way too many hours a week because they are grading students the old-fashioned way.”

Roberts also thinks AI might be a less biased grader in some instances than human teachers who can adjust their grading for students sometimes to give them the benefit of the doubt or be punitive if they were particularly annoying in class recently.

She isn’t worried about students cheating with AI, a concern she characterizes as a moral panic. She points to a Stanford University study released last fall which found that students cheated just as much before the advent of ChatGPT as they did a year after the release of the AI. 

Goyette said she understands why students question whether some AI use by teachers is like cheating. Education department AI guidelines encourage teachers and students to use the technology more. What’s essential, Goyette said, is that teachers discuss what ethical use of AI looks like in their classroom, and convey that — like using a calculator in math class — using AI is accepted or encouraged for some assignments and not others. 

For the last assignment of the year, Robers has one final experiment to run: Edit an essay written entirely by AI. But they must change at least 50% of the text, make it 25% longer, write their own thesis, and add quotes from classroom reading material. The idea, she said, is to prepare them for a future where AI writes the first draft and humans edit the results to fit their needs. 

“It used to be you weren’t allowed to bring a calculator into the SATs and now you’re supposed to bring your calculator so things change,” she said. “It’s just moral panic. Things change and people freak out and that’s what’s happening.”

For the record: An earlier version of this story misnamed the AI tool made by the company Quill. Quill is both the name of the company and the tool. 

AI – or artificial intelligence – is often used as a way to summarize data and improve writing. But AI tools also represent a powerful and efficient way to analyze large amounts of text to search for patterns. In addition, AI tools can assist with developing research products that can be shared widely. 

It’s with that in mind that we, as researchers in social science, developed a new database of AI tools for the field. In the database, we compiled information about each tool and documented whether it was useful for literature reviews, data collection and analyses, or research dissemination. We also provided information on the costs, logins and plug-in extensions available for each tool.

When asked about their perceptions of AI, many social scientists express caution or apprehension. In a sample of faculty and students from over 600 institutions, only 22% of university faculty reported that they regularly used AI tools.

From combing through lengthy transcripts or text-based data to writing literature reviews and sharing results, we believe AI can help social science researchers – such as those in psychology, sociology and communication – as well as others get the most out of their data and present it to a wider audience.

Analyze text using AI

Qualitative research often involves poring over transcripts or written language to identify themes and patterns. While this kind of research is powerful, it is also labor-intensive. The power of AI platforms to sift through large datasets not only saves researchers time, but it can also help them analyze data that couldn’t have been analyzed previously because of the size of the dataset.

Specifically, AI can assist social scientists by identifying potential themes or common topics in large, text-based data that scientists can interrogate using qualitative research methods. For example, AI can analyze 15 million social media posts to identify themes in how people coped with COVID-19. These themes can then give researchers insight into larger trends in the data, allowing us to refine criteria for a more in-depth, qualitative analysis.

AI tools can also be used to adapt language and scientists’ word choice in research designs. In particular, AI can reduce bias by improving the wording of questions in surveys or refining keywords used in social media data collection. 

Identify gaps in knowledge

Another key task in research is to scan the field for previous work to identify gaps in knowledge. AI applications are built on systems that can synthesize text. This makes literature reviews – the section of a research paper that summarizes other research on the same topic – and writing processes more efficient.

Research shows that human feedback to AI, such as providing examples of simple logic, can significantly improve the tools’ ability to perform complex reasoning. With this in mind, we can continually revise our instructions to AI and refine its ability to pull relevant literature.

However, social scientists must be wary of fake sources – a big concern with generative AI. It is essential to verify any sources AI tools provide to ensure they come from peer-reviewed journals.

Share research findings

AI tools can quickly summarize research findings in a reader-friendly way by assisting with writing blogs, creating infographics and producing presentation slides and even images.

Our database contains AI tools that can also help scientists present their findings on social media. One tool worth highlighting is BlogTweet. This free AI tool allows users to copy and paste text from an article like this one to generate tweet threads and start conversations. 

Be aware of the cost of AI tools

Two-thirds of the tools in the database cost money. While our primary objective was to identify the most useful tools for social scientists, we also sought to identify open-source tools and curated a list of 85 free tools that can support literature reviews, writing, data collection, analysis and visualization efforts.

In our analysis of the cost of AI tools, we also found that many offer “freemium” access to tools. This means you can explore a free version of the product. More advanced versions of the tool are available through the purchase of tokens or subscription plans. 

For some tools, costs can be somewhat hidden or unexpected. For instance, a tool that seems open source on the surface may actually have rate limits, and users may find that they’ve run out of free questions to ask the AI. 

The future of the database

Since the release of the Artificial Intelligence Applications for Social Science Research Database on Oct. 5, 2023, it has been downloaded over 400 times across 49 countries. In the database, we found 131 AI tools useful for literature reviews, summaries or writing. As many as 146 AI tools are useful for data collection or analysis, and 108 are useful for research dissemination.

We continue to update the database and hope that it can aid academic communities in their exploration of AI and generate new conversations. The more that social scientists use the database, the more they can work toward consensus of adopting ethical approaches to using AI in research and analysis.

As the eight-year-old startup rolled out Los Angeles Unified School District’s flashy new AI-driven chatbot — an animated sun named “Ed” that AllHere was hired to build for $6 million — a former company executive was sending emails to the district and others that Ed’s workings violated bedrock student data privacy principles. 

Those emails were sent shortly before The 74 first reported last week that AllHere, with $12 million in investor capital, was in serious straits. A June 14 statement on the company’s website revealed a majority of its employees had been furloughed due to its “current financial position.” Company founder and CEO Joanna Smith-Griffin, a spokesperson for the Los Angeles district said, was no longer on the job. 

Smith-Griffin and L.A. Superintendent Alberto Carvalho went on the road together this spring to unveil Ed at a series of high-profile ed tech conferences, with the schools chief dubbing it the nation’s first “personal assistant” for students and leaning hard into LAUSD’s place in the K-12 AI vanguard. He called Ed’s ability to know students “unprecedented in American public education” at the ASU+GSV conference in April. 

Through an algorithm that analyzes troves of student information from multiple sources, the chatbot was designed to offer tailored responses to questions like “what grade does my child have in math?” The tool relies on vast amounts of students’ data, including their academic performance and special education accommodations, to function.

Meanwhile, Chris Whiteley, a former senior director of software engineering at AllHere who was laid off in April, had become a whistleblower. He told district officials, its independent inspector general’s office and state education officials that the tool processed student records in ways that likely ran afoul of L.A. Unified’s own data privacy rules and put sensitive information at risk of getting hacked. None of the agencies ever responded, Whiteley told The 74. 

“When AllHere started doing the work for LAUSD, that’s when, to me, all of the data privacy issues started popping up,” Whiteley said in an interview last week. The problem, he said, came down to a company in over its head and one that “was almost always on fire” in terms of its operations and management. LAUSD’s chatbot was unlike anything it had ever built before and — given the company’s precarious state — could be its last. 

If AllHere was in chaos and its bespoke chatbot beset by porous data practices, Carvalho was portraying the opposite. One day before The 74 broke the news of the company turmoil and Smith-Griffin’s departure, EdWeek Marketbrief spotlighted the schools chief at a Denver conference talking about how adroitly LAUSD managed its ed tech vendor relationships — “We force them to all play in the same sandbox” — while ensuring that “protecting data privacy is a top priority.”

In a statement on Friday, a district spokesperson said the school system “takes these concerns seriously and will continue to take any steps necessary to ensure that appropriate privacy and security protections are in place in the Ed platform.” 

“Pursuant to contract and applicable law, AllHere is not authorized to store student data outside the United States without prior written consent from the District,” the statement continued. “Any student data belonging to the District and residing in the Ed platform will continue to be subject to the same privacy and data security protections, regardless of what happens to AllHere as a company.” 

A district spokesperson, in response to earlier questioning from The 74 last week, said it was informed that Smith-Griffin was no longer with the company and that several businesses “are interested in acquiring AllHere.” Meanwhile Ed, the spokesperson said, “belongs to Los Angeles Unified and is for Los Angeles Unified.”

Officials in the inspector general’s office didn’t respond to requests for comment. The state education department “does not directly oversee the use of AI programs in schools or have the authority to decide which programs a district can utilize,” a spokesperson said in a statement.

It’s a radical turn of events for AllHere and the AI tool it markets as a “learning acceleration platform,” which were all the buzz just a few months ago. In April, Time Magazine named AllHere among the world’s top education technology companies. That same month, Inc. Magazine dubbed Smith-Griffin a global K-12 education leader in artificial intelligence in its Female Founders 250 list. 

Joanna Smith-Griffin, Founder and CEO of AllHere, joins @LAUSDSup on stage as he delivers the keynote at @asugsvsummit on Unlocking the Potential of AI. With Ed, students receive personalized support to help them excel in school. #EdTech #NextGenEd pic.twitter.com/iU9xta3B8M

— allherek12 (@allherek12) April 15, 2024

Ed has been similarly blessed with celebrity treatment. 

“He’s going to talk to you in 100 different languages, he’s going to connect with you, he’s going to fall in love with you,” Carvalho said at ASU+GSV. “Hopefully you’ll love it, and in the process we are transforming a school system of 540,000 students into 540,000 ‘schools of one’ through absolute personalization and individualization.”

Smith-Griffin, who graduated from the Miami school district that Carvalho once led before going onto Harvard, couldn’t be reached for comment. Smith-Griffin’s LinkedIn page was recently deactivated and parts of the company website have gone dark. Attempts to reach AllHere were also unsuccessful.

‘The product worked, right, but it worked by cheating’

Smith-Griffin, a former Boston charter school teacher and family engagement director, founded AllHere in 2016. Since then, the company has primarily provided schools with a text messaging system that facilitates communication between parents and educators. Designed to reduce chronic student absences, the tool relies on attendance data and other information to deliver customized, text-based “nudges.” 

The work that AllHere provided the Los Angeles school district, Whiteley said, was on a whole different level — and the company wasn’t prepared to meet the demand and lacked expertise in data security. In L.A., AllHere operated as a consultant rather than a tech firm that was building its own product, according to its contract with LAUSD obtained by The 74. Ultimately, the district retained rights to the chatbot, according to the agreement, but AllHere was contractually obligated to “comply with the district information security policies.” 

 The contract notes that the chatbot would be “trained to detect any confidential or sensitive information” and to discourage parents and students from sharing with it any personal details. But the chatbot’s decision to share and process students’ individual information, Whiteley said, was outside of families’ control. 

In order to provide individualized prompts on details like student attendance and demographics, the tool connects to several data sources, according to the contract, including Welligent, an online tool used to track students’ special education services. The document notes that Ed also interfaces with the Whole Child Integrated Data stored on Snowflake, a cloud storage company. Launched in 2019, the Whole Child platform serves as a central repository for LAUSD student data designed to streamline data analysis to help educators monitor students’ progress and personalize instruction. 

Whiteley told officials the app included students’ personally identifiable information in all chatbot prompts, even in those where the data weren’t relevant. Prompts containing students’ personal information were also shared with other third-party companies unnecessarily, Whiteley alleges, and were processed on offshore servers. Seven out of eight Ed chatbot requests, he said, are sent to places like Japan, Sweden, the United Kingdom, France, Switzerland, Australia and Canada. 

Taken together, he argued the company’s practices ran afoul of data minimization principles, a standard cybersecurity practice that maintains that apps should collect and process the least amount of personal information necessary to accomplish a specific task. Playing fast and loose with the data, he said, unnecessarily exposed students’ information to potential cyberattacks and data breaches and, in cases where the data were processed overseas, could subject it to foreign governments’ data access and surveillance rules. 

Chatbot source code that Whiteley shared with The 74 outlines how prompts are processed on foreign servers by a Microsoft AI service that integrates with ChatGPT. The LAUSD chatbot is directed to serve as a “friendly, concise customer support agent” that replies “using simple language a third grader could understand.” When querying the simple prompt “Hello,” the chatbot provided the student’s grades, progress toward graduation and other personal information. 

AllHere’s critical flaw, Whiteley said, is that senior executives “didn’t understand how to protect data.” 

“The issue is we’re sending data overseas, we’re sending too much data, and then the data were being logged by third parties,” he said, in violation of the district’s data use agreement. “The product worked, right, but it worked by cheating. It cheated by not doing things right the first time.”

In a 2017 policy bulletin, the district notes that all sensitive information “needs to be handled in a secure way that protects privacy,” and that contractors cannot disclose information to other parties without parental consent. A second policy bulletin, from April, outlines the district’s authorized use guidelines for artificial intelligence, which notes that officials, “Shall not share any confidential, sensitive, privileged or private information when using, prompting or communicating with any tools.” It’s important to refrain from using sensitive information in prompts, the policy notes, because AI tools “take whatever users enter into a prompt and incorporate it into their systems/knowledge base for other users.” 

“Well, that’s what AllHere was doing,” Whiteley said. 

‘Acid is dangerous’

Whiteley’s revelations present LAUSD with its third student data security debacle in the last month. In mid-June, a threat actor known as “Sp1d3r” began to sell for $150,000 a trove of data it claimed to have stolen from the Los Angeles district on Breach Forums, a dark web marketplace. LAUSD told Bloomberg that the compromised data had been stored by one of its third-party vendors on the cloud storage company Snowflake, the repository for the district’s Whole Child Integrated Data. The Snowflake data breach may be one of the largest in history. The threat actor claims that the L.A. schools data in its possession include student medical records, disability information, disciplinary details and parent login credentials. 

The chatbot interacted with data stored by Snowflake, according to the district’s contract with AllHere, though any connection between AllHere and the Snowflake data breach is unknown. 

In its statement Friday, the district spokesperson said an ongoing investigation has “revealed no connection between AllHere or the Ed platform and the Snowflake incident.” The spokesperson said there was no “direct integration” between Whole Child and AllHere and that Whole Child data was processed internally before being directed to AllHere.

The contract between AllHere and the district, however, notes that the tool should “seamlessly integrate” with the Whole Child Integrated Data “to receive updated student data regarding attendance, student grades, student testing data, parent contact information and demographics.”

Earlier in the month, a second threat actor known as Satanic Cloud claimed it had access to tens of thousands of L.A. students’ sensitive information and had posted it for sale on Breach Forums for $1,000. In 2022, the district was victim to a massive ransomware attack that exposed reams of sensitive data, including thousands of students’ psychological evaluations, to the dark web. 

With AllHere’s fate uncertain, Whiteley blasted the company’s leadership and protocols.

“Personally identifiable information should be considered acid in a company and you should only touch it if you have to because acid is dangerous,” he told The 74. “The errors that were made were so egregious around PII, you should not be in education if you don’t think PII is acid.” 

L.A. parents and students, we want to hear from you. Tell us about your experience using AllHere’s Ed:

Today, that potential has begun to come to fruition. Recent national survey data indicate 18% of teachers have used genAI, mostly to support differentiated lesson planning, and 56% of educators believe its use in schools will continue to grow. Increasingly, districts are introducing students to this technology, with products like Khanmigo — which provides individualized tutoring — already being adopted in Indiana, Florida and New Jersey. And students are experimenting with it outside the classroom as well. According to a recent survey, approximately half of 14- to 22-year-olds report having used genAI at some point.

But rapid changes in technology and the speed of adoption are far outpacing the field’s understanding of impacts on teaching and learning. Every day there is a new story about an exciting AI-related development, but given the time it takes to conduct careful evaluation, very limited evidence exists about whether any of these tools have positive benefits for students. As schools start facing hard choices about where to spend their resources in response to continued learning gaps and the ESSER funding cliff, it’s important to take a look at what we know about the impact of genAI on education and what more we need to learn. 

What we know

Educators spend about 46% percent of their time on tasks that don’t directly involve teaching, ranging from taking attendance and submitting reports to giving written feedback to students. Gen AI tools hold promise for speeding up and even automating these tasks, saving time that could be spent building meaningful relationships and deepening learning. For example, researchers from UC Irvine found that teachers in California and North Carolina who used the genAI product Merlyn Mind, which automates test question creation and lesson planning, reported spending less time on administrative tasks and more on teaching and learning after seven weeks of use compared to educators without access to the tool. And about 44% of teachers who have used genAI agree the technology has made their job easier. 

To date, however, most of these findings rely on anecdotal reports. To quantify the impact of genAI on time saved, the field needs more rigorous evidence — such as through randomized controlled trials — to not only gauge the impact on administrative burden but to explore whether these tools help improve teaching quality. 

A separate body of research is finding that genAI-based coaching tools, which aim to give regular, impartial, real-time feedback in a cost-effective way, can have small effects on targeted teacher practices. For example, researchers at Stanford and the University of Maryland developed “M-Powering Teachers,” an automated coaching tool that uses natural language processing to give educators feedback. Across two randomized controlled trials, the tool was shown to reduce teacher-directed talk, increased student contributions and improved completion of assignments. Another study found that feedback provided via TeachFX, an app that uses voice AI to assess key indicators of classroom quality, increased teachers’ use of focusing questions that probe students’ thinking by 20%. 

Another randomized controlled trial found a genAI-enabled coaching tool that provided targeted feedback increased the quality of math tasks assigned to students and created a more coherent learning environment. Perhaps more impressive, the feedback resulted in a small positive improvement in students’ knowledge of ratios and proportional relationships, the area it focused on. 

These studies show early promise, but the impacts they found have been small. As AI-enabled coaching products for teachers start to expand to more classrooms, more evaluation is needed to better understand the potential of genAI to truly improve teaching and, ultimately, student learning. 

What we need to learn

Despite early evidence that AI has potential to make teachers’ jobs a bit easier and professional development more effective, the verdict is still out on whether having students interact directly with genAI can improve academic and social-emotional outcomes. These technologies, especially in education, are changing rapidly, making rigorous studies challenging. This point was recently made by the Alliance for Learning Innovation in calling on Congress to budget almost $3 billion to address the issue.

While some tools — like Khan Academy’s Khanmigo (which has received funding from Overdeck Family Foundation) — are based on evidence that personalized learning can support better outcomes for some students, and some emerging research indicates that hybrid AI-human tutoring may boost achievement, it is not yet clear whether genAI tools themselves can strengthen and supplement student learning. As these types of products move into classrooms, there is a clear need for families, educators and policymakers to demand proof that they improve outcomes and do not unintentionally harm students most in need of effective support by providing incorrect guidance and feedback. 

This is an exciting moment for education, with transformative technology finding its way into all our lives in a way that hasn’t been seen since the introduction of smartphones. Yet much research on genAI does not consider the types of ed tech products schools are actually buying. Instead, it comes from lab-based studies and tools that are not actually used or tested in the classroom. 

Now is the time — before these technologies become pervasive — to rigorously evaluate what is being sold into and used in schools. The goal of educators should always be to ensure that students have the most effective tools for learning, not merely those with the best sales pitch.

Boston-based AllHere Education, founded in 2016 by Harvard grad and former teacher Joanna Smith-Griffin, figured heavily in LAUSD’s March 20 launch of Ed, an AI-powered online tool for students and parents designed to supplement classroom instruction and help families navigate. 

But on June 14, AllHere furloughed the majority of its employees due to its “current financial position,” according to a statement posted on its website. A statement from LAUSD sent to The 74 said AllHere now is up for sale.  

But even before the surprise announcement, AllHere was already having trouble fulfilling its contract with LAUSD, according to one former high-ranking company executive. 

The company was unable to push back against the district’s timeline, he said, and couldn’t produce a “proper product.”

An LAUSD spokesperson said the district is aware of Smith-Griffin’s departure and that “several educational technology companies are interested in acquiring AllHere.” 

“The educational technology field is a dynamic space where acquisitions are not uncommon,” the spokesperson said via email. “We will ensure that whichever entity acquires AllHere will continue to provide this first-of-its-kind resource to our students and families.”

Smith-Griffin and AllHere did not respond to requests for comment. The former CEO has taken down her LinkedIn profile. Portions of the AllHere website have also disappeared, including the company’s “About Us” page. 

James Wiley, a vice president at the education market research firm ListEdTech, said turmoil at AllHere could be a red flag for LAUSD’s AI program if the district hasn’t taken steps to protect itself from changes at the company.   

“It could be a problem,” said Wiley. “It depends on how much of the program the district has been able to bring in-house, as opposed to leaving with the vendor.”

Wiley also expressed surprise that LAUSD contracted with a relatively small and untested firm such as AllHere for its Ed rollout, as opposed to enlisting a major AI company for the job, or a larger ed tech firm.   

“You have bigger players out there who could have done this thing,” said Wiley.

Outside of Los Angeles, the company has offered districts a text messaging system that allows schools to inform families about weather events and other announcements. 

According to GovSpend, which tracks government contracts with companies, AllHere has already been paid more than $2 million by LAUSD. The company has had much smaller contracts with other districts, according to GovSpend, including a $49,390 payment from Brownsville Independent School District in Texas and a similar-sized payment from Broward County Public Schools in Florida. 

But AllHere’s star had been ascendant. 

With backing from the Harvard Innovation Lab, Smith-Griffin raised more than $12 million to start the new company. AllHere in April was named one of the world’s top ed tech companies by TIME. 

The LAUSD school board last June approved a competitively bid $6.2 million contract for AllHere to plan, design and develop the district’s new AI tool, Ed. The deal began with a two-year agreement ending in July 2025, with options for three subsequent one-year renewals.  

Smith-Griffin appeared with LAUSD superintendent Alberto Carvalho in April to discuss the project, which was described by the district’s leader as a game-changer for LAUSD that represented the first time a school district systematically leveraged AI. 

The former AllHere executive, who was recently laid off, said in an interview that the company’s work with LAUSD was far more involved than that of its other customer school districts. 

The small company was being asked to create a far more sophisticated tool than its prior text messaging system and bit off more than it could chew in its contract with the nation’s second-largest district. 

At the same time, he said, AllHere employees operated more as consultants than as a company building its own product and were unable to “to say no or to slow things down” with the district.

“So I think because of that, they were unable or unwilling to build a proper product,” he said. 

LA parents and students, we want to hear from you. Tell us about your experience using AllHere’s Ed:

With reporting and contributions from Mark Keierleber and Greg Toppo

But over the years, parents have struggled to find good opening questions. 

“You’d search [the Internet], you’d go on Pinterest,” she said. “A lot of the work had to be done manually, or you had to do a lot more digging around.”

Then came ChatGPT, Open AI’s widely used artificial intelligence bot. For Fender, it was a no-brainer to query it for help developing deep opening questions.

The chatbot and other AI tools like it have found an eager audience among homeschoolers and microschoolers, with parents and teachers readily embracing it as a brainstorming and management tool, even as public schools take a more cautious approach, often banning it outright. 

A few observers say AI may even make homeschooling more practical, opening it up to busy parents who might have balked previously.

“Homeschoolers have always been unconstrained in their ability to combine technology — any kind of tech,” said Alex Sarlin, a longtime technology analyst and co-host of the EdTech Insiders podcast. 

The reasons are readily apparent, he said: Home internet service typically doesn’t block key websites the way most schools do. Families can more easily manage data privacy and get the digital tools they want without fuss. They’re basically able to ignore “all the dozen reasons why everything falls apart when you try to sell to schools,” Sarlin said. 

Persuading homeschoolers to try out new things is also a lot simpler: If a student and parents like a tool, “There’s nobody else you have to convince.”

Indeed, a September survey by the curriculum vendor Age of Learning found that 44% of homeschool educators reported using ChatGPT, compared to 34% of classroom educators.

“Not everyone is using it, but some are very excited about it,” said Amir Nathoo, co-founder of Outschool, an online education platform.

The most interesting uses he has seen are by gifted and neurodiverse homeschoolers, who often use chatbots to explore complex topics like advanced math and science, philosophy and even ethics, which they wouldn’t ordinarily have access to at a young age. They ask it to provide simple explanations of advanced topics, such as relativity and quantum mechanics, then pursue them on their own. “They’re able to go on a relatively unstructured exploration, which is often the best way that kids learn.”

Alternatively, he said, kids whose ability to express themselves is limited can also benefit from what many consider the non-judgmental qualities of tools like ChatGPT. 

Peer-to-peer learning

Tobin Slaven, cofounder of Acton Academy, a self-paced, independent microschool in Fort Lauderdale, said he’s been experimenting with AI tools for the past year or so and is excited by what he’s seen. “This is what the future looks like to me,” he said

Like many educators, he sees the problems inherent in AI tools like ChatGPT, which on occasion “hallucinate” with incorrect information and can sometimes be downright creepy. These concerns have stopped many families from fully embracing AI.

But Slaven can’t support banning it outright. Instead, he’ll offer a student his own device with ChatGPT loaded onto a browser window. The entire time, he has access to their queries and results. That ensures he can review the sessions for inappropriate content.

Lately, Slaven and his students have been playing with an AI tool called Pathfinder that helps them create and develop projects. Designed by a small, two-person UK-based startup, it’s set up like a simple chatbot that asks students what they want to learn about. It elicits information, much like a Socratic guide, about their prior knowledge and how they’d like to explore the topic. Then it searches the Internet for appropriate resources and returns suggestions on what to do next. 

Pathfinder uses Open AI’s GPT-4 large language model and its own algorithm to rank resources based on how relevant it is to an individual learner, said co-founder Amaan Ahmad. That includes how they learn best, what they’re interested in and what they already know. 

After a number of students in a homeschool group or class have worked with it long enough, it can even begin recommending classmates or friends to consult with to learn how they’re approaching the topic. 

“My AI can talk to your AI and say, ‘Hey, Greg crushed that last week. Why don’t you go speak to him and develop your project together?’” he said. 

Slaven tried out Pathfinder with a group of students recently and found that even during a brief trial run, it allowed them to better conceptualize their projects. 

With the tool asking them questions about their preferred topic, they were able to go from general inquiries about their interests, such as horseback riding or space exploration, into more advanced ones that explore the topics more deeply. That goes a long way toward helping students become more independent and responsible for their own learning, a key goal of microschooling and homeschooling.

Slaven believes, more broadly, that AI co-pilots configured to students’ interests and preferences will enable personalized learning at scale. It’ll become the norm that everyone has a collaborative AI partner that will, in time, understand how each student performs best and under what conditions. “It’s eventually going to become their preferred resource,” he said.

Making homeschooling more accessible

Ahmad, the Pathfinder co-founder, said AI holds the possibility of helping endeavors like microschooling and homeschooling become more practical. Access to reliable, safe AI agents means that an individual student isn’t restricted to what a parent or teacher knows.

Giving that autonomy with a bit of guidance helps make learning much more impactful, he said. “It’s very difficult to do that in real time because with one adult and one kid, you can’t always be by their side. And if you have a microschool with 12 to 16 kids, that’s even more time-consuming.” 

For Fender, the North Carolina homeschooling mother, one of the most helpful aspects of AI is that it helps parents organize what can often be a chaotic, free-form learning environment. 

Fender subscribes to a type of homeschooling known as “unschooling,” which seeks to teach students to be more self-directed and independent than in most public schools. Her kids’ lessons are “very much interest-led” and her small co-op has grown in recent years. 

But she must also persuade state bureaucrats that she’s providing an adequate education. So she and a few other homeschool parents in Cary rely on a website that uses AI to detail what activities their kids have done and auto-completes all of the relevant North Carolina educational standards. “I thought that was a genius tool,” she said, and one that allows stressed, busy parents to build a comprehensive portfolio for annual state reviews and high school transcripts.

Fender also uses ChatGPT for brainstorming. In a recent case, which she shared on Instagram, Fender asked the AI for 50 real-life applications for the Pythagorean theorem. It generated a list that included designing ramps or stairs, planning optimal pathways in garden design and building efficient roller coasters. 

Last year, she recalled, one of her daughters was creating Christmas cards for a homeschool craft fair and “wanted to have fun puns in the cards.” Fender explained how to craft an AI prompt — and how to sift through the chaff. Her daughter eventually asked ChatGPT for 50 different Christmas-themed puns and ended up using about 10 to 15. 

Like most parents, Fender has read about the downsides of AI but believes schools are short-sighted to limit its use. 

“Why are you banning a tool that is definitely here to stay?” she said. “Maybe we don’t understand all the ins and outs, but at the end of the day, our goal is to prepare kids for the jobs of the future. And a lot of these jobs of the future, we don’t even know what they are.”

The creation of deepfakes — highly realistic but artificial images, audio and video — used to require high-powered equipment and considerable skill. Now, with advancements in generative artificial intelligence, any kid with a smartphone can make one. 

Adolescents, mostly teenage boys, are exploiting readily accessible deepfake tools to create graphic images and videos of female classmates, causing profound distress and disruption in schools across the country, from Beverly Hills, California, to Westfield, New Jersey. High school students outside Seattle were photographed by a classmate at a school dance who then “undressed” them on his phone and circulated supposedly nude pictures.

The impact could be significant. Experts report that so-called deepnudes can hurt victims’ mental health, physical and emotional safety, as well as college and job opportunities. Comprehensive data is lacking, but documented incidents indicate that this is a troubling trend that demands immediate attention. 

While anti-child pornography statutes, Title IX regulations regarding online harassment and revenge-porn laws already exist, these measures were not designed to handle the unique challenges posed by deepfake technology. 

Schools, educators and law enforcement are scrambling to respond to this new phenomenon. In some cases, students have been harshly disciplined, but arresting 13- and 14 year-old boys for engaging in impulsive behavior on phones their parents have handed them is not an appropriate, just or sustainable solution. It is incumbent upon adults to make the technological world safe for children.

The Biden administration has rightly called on technology companies, financial institutions and other businesses to limit websites and mobile apps whose primary business is to create, facilitate, monetize or disseminate image-based sexual abuse. But these steps are largely symbolic and will result in voluntary commitments that are likely unenforceable. 

The U.S. Department of Education is scheduled to release guidance on this matter, but its track record of issuing timely — and, frankly, practical — information is underwhelming. 

It’s also impractical to rely on slow-moving legislative processes that get caught up in arguments about accountability for offending images when students’ well-being is at stake. As any school leader can tell you, laws only go so far in deterring behavior, and the legislation ambling through Congress don’t address how K-12 institutions should respond to these incidents. 

So, where does that leave us?

Educators need support and guidance. Schools have a critical role to play, but to expect them to invent policies and educational programs that combat the malicious use of deepfakes and protect students from this emerging threat — absent significant training, resources and expertise — is not only a fool’s errand, but an unfair burden to place on educators. 

Communities, districts and schools need statewide strategies to prevent and deter deepfakes. States must use this summer to bring together school administrators, educators, law enforcement, families, students, local technology companies, researchers, community groups and other nonprofit organizations to deliver comprehensive policies and implementation plans by Labor Day. These should, among other things:

• Recommend curriculum, instruction and training programs for school leaders and teachers about the potential misuses of artificial intelligence and deepfakes in school settings;

• Update school-based cyber harassment policies and codes of conduct to include deepfakes;
  
• Establish discipline policies to clarify accountability for students who create, solicit or distribute nonconsensual, sexual deepfake images of their peers;
  
• Update procurement policies to ensure that any technology provider has a plan to interrupt or handle a deepfake incident;
  
• Build or purchase education, curriculum and instruction for students and families on digital citizenship and the safe use of technology, including AI literacy and deepfakes;
  
• Issue guidance for community institutions, including religious programs, small businesses, libraries and youth sports leagues, to promote prevention by addressing this issue head-on with teens who need to understand the damage deepfakes cause;
  
• Issue detailed guidance about how schools must enforce Title IX, the federal law that bans sex discrimination, including sexual harassment, in schools.

Is this too ambitious for state government? Maybe. But there is no choice. As the grown-ups, and as citizens of a democracy, we have a collective responsibility to decide what kind of world we want our children to live in, and to take action, before it’s too late.

This year at SXSW EDU, I teamed up with The Leadership Academy to facilitate a conversation on the role of AI in education, specifically in relation to the principalship. The panelists discussed the potential benefits and challenges of embedding AI in schools and how it might impact the role of the principal. We also explored the implications of AI for equity and access in education. As education leaders come to terms with integrating AI into our schools, they need to consider these issues:

AI can help principals avoid burnout and focus on the “human” work.  

The role of the principal is currently unsustainable. In 2022, 85% of principals reported experiencing high levels of job-related stress, compared with 35% of the general working adult population. The risk of principal burnout has sweeping  implications for the field. Principal turnover has a negative impact on teacher retention and is associated with decreased student achievement. AI can help make principals’  jobs more manageable and sustainable by helping them save time and even automate administrative and analytic tasks. 

The time and technical assets afford principals with more bandwidth, so they can focus on more sophisticated,  human-centered activities such as building relationships with their faculty and the community, and fostering a positive climate, which is a proven predictor of school effectiveness. AI offers an answer to a vital question that was posed by Kentwood, Michigan, Superintendent Kevin Polston during the panel: “If time is our most precious commodity, and humans are the most important value that we have in our organizations, how do you then create more time for your people to do those innately human things that change outcomes for kids?” 

Education leaders must consider the risk of bias in design.

During our discussion, Nancy Gutierrez, executive director of The Leadership Academy, emphasized the importance of who is at the table in the design process. To illustrate the risks, she referred to sobering examples, such as the initial designs of self-driving cars being more likely to hit people with darker skin tones. In terms of education, she noted that teachers might use AI to design work that inadvertently reflects their biases about a student’s capabilities, based on that child’s identity. Bias in AI is simply a reflection of existing human biases, so district leaders and principals should redouble efforts against bias that might undermine students. Eva Mejia, an expert in design and innovation at IDEO, underscored how involving educators in the design process and increasing transparency could mitigate some of these risks and enhance innovation in schools.

The role of the principal must evolve in line with technological advancements, with a focus on leading change.

Schools must actively learn about and adopt AI, rather than being passive recipients, and principals must be prepared to lead this change effectively. Principals are drivers of school success, and AI is yet another means for them to foster innovation in their schools by modeling a exploratory mindset for students and adults. For example, principals can cultivate spaces where teachers and students feel free to work with AI out in the open, sharing best practices and pitfalls for the benefit of other educators. What might principals and teachers accomplish by testing and leveraging computing power to elevate academic rigor, rather than banning tools that are already integrating in the professional world?

Unfortunately, many school leaders are doing this work at a disadvantage. When I ask principals in urban districts why they have not done more to leverage AI in their schools, the most common answer is, “I just don’t have the time.” Too often, the folks who lead the schools with the greatest needs have the least time to be proactive. They fall behind because they do not have the bandwidth to capitalize on new opportunities or innovative solutions. District leaders must commit to investing in the resources — time and material — that principals need to create the conditions required for schools to remain current and competitive.  

Integrating AI into schools is not just about bringing in new technology. It is about rethinking what leadership looks like. Education leaders have the opportunity to use their expertise in school systems, learning and development to think about how AI can be used to close equity gaps, instead of widening them, and position principals to focus on what matters most — children.

In all, two-thirds of respondents said they hadn’t used AI in their work, including 9 percent who reported they’d never heard of tools and products like OpenAI’s ChatGPT or Google’s Gemini. By contrast, 18 percent of participants said they regularly relied on such offerings, and 15 percent said they had tried them before but don’t intend to use them more regularly.

Melissa Kay Diliberti, a policy researcher at RAND and one of the report’s co-authors, said the current minority of users constitutes a “foothold” in schools that is poised to grow with time — and that has already expanded massively in the 17 months since ChatGPT was first unveiled to an unsuspecting public in November 2022.

“There seem to be a small number of people on the bandwagon, but the bandwagon is moving forward,” Diliberti said.

The poll, incorporating responses from a nationally representative sample of more than 1,000 teachers in 231 public school districts, offers the most recent data from a technological shift that has been trumpeted as revolutionary. The potential of AI to maximize teacher efficiency, individualize instruction for every pupil, and offer support to kids struggling with mental health problems has stoked a growing demand for new products that is quickly being met by major tech players like Google and Khan Academy.

The gleanings of broader public opinion research are somewhat diffuse, but there is reason to think that the level of AI take-up by teachers is comparable to, or even further along than, that of other professionals. In previous polls, similar minorities of lawyers (15 percent), journalists (28 percent), human resources staff (26 percent), and doctors (38 percent) have reported using AI in a variety of tasks. 

And teachers’ outlook on the future is suggestive: Nearly all respondents who already use AI tools believe they will use it more in the 2024–25 school year than they do now, while 28 percent of non-users predicted they would eventually try them out. 

Use of artificial intelligence was roughly even across different kinds of schools, whether broken down by student demographics, poverty levels or rural/urban geography. By contrast, middle and high school teachers were almost twice as likely to say they used AI than their counterparts in elementary school (23 percent vs. 12 percent), and English and social studies instructors reported higher use than those in STEM disciplines (27 percent versus 19 percent).

While cautioning against overinterpreting results in a relatively small sample, Diliberti reasoned that English and social studies teachers are also more likely to create or modify their own curricular materials, or source them from online marketplaces like Teachers Pay Teachers. Outsourcing some of those efforts — along with periodic non-instructional tasks, such as composing emails to parents or letters of recommendation to colleges — to AI could save hundreds of hours over the course of a school year.

“You could see where AI might be a way to ease the burden of a task they’re already doing,” she said. “That might be why these teachers appear to be more inclined to use AI than a math teacher, who could be more tightly focused on a given curriculum that’s used throughout the school.”

Among teachers regularly using AI, close to half said they did so to generate classroom assignments or worksheets (40 percent), lesson plans (41 percent), or assessments for students (49 percent). 

Establishing a ‘foothold’

Amanda Bickerstaff, CEO of AI for Education, a company that advises school districts on the use of artificial intelligence, said the RAND poll is notable for being “the first survey I’ve seen that seems representative of what is happening in schools. 

In training sessions she has conducted for tens of thousands of classroom teachers and administrators since last year, Bickerstaff said she and her colleagues have received a warm reception from audiences, but also uneven awareness of what AI can accomplish. Early adopters might simply be tech enthusiasts, or they could be special education teachers hoping to make their instruction more accessible.

Curiosity about the new technology “is coming from the bottom-up as well as the top-down,” she observed. “One of the more interesting things is that we’re seeing more teachers using AI in schools than schools and districts teaching them to use it.”

Partly because guidance and professional development still trail teacher interest, a little under 10 percent of all survey respondents said they were seeking out AI tools of their own initiative. At present, the most commonly used products were popular platforms like Google Classroom, adaptive learning systems offered by Khan Academy and i-Ready, and the nearly ubiquitous chatbots. 

Diliberti said she wasn’t surprised that incumbent players like Google and OpenAI, powered by billions of dollars in investment and promotion, have gained early primacy in the K–12 arena. But she added it was striking that lesser-known products that are specifically geared toward activities like lesson planning and assessment generation haven’t won the following of more multifunctional alternatives like ChatGPT.

“It’s notable that teachers seem to be using more generic tools instead of dedicated tools that were developed for this purpose,” she said.

Bickerstaff argued that the survey results demonstrated that teachers, increasingly finding their own way to AI, should be provided more training on the use of existing tools. Beyond that, she said, public and private actors should broaden access to more advanced versions of those tools, which are now available at subscription costs averaging about $20 per month, to allow teachers to gain a better understanding of their applications. 

“These tools make mistakes, they’re biased, and they require significant training to be able to use them. You need support on how to use the tools before you can get the best out of them.”

But behind the scenes, preparing for the lesson was anything but typical for teachers Janice Donaghy and Jean D’Aurio. They had avoided the hours of preparation the lesson might normally have taken by using artificial intelligence to craft a plan that included a summary of ancient Greek vases, exit questions and student activities.

“Classroom preparation goes from hours to seconds” when using AI, said D’Aurio. In the past, the co-teaching pair had created lesson plans by scouring the school’s literacy closet to sift through printed materials, perusing the Teachers Pay Teachers online marketplace and exploring Instagram or TikTok accounts.

For this lesson, the two consulted the county’s curriculum guide but also used Canva, a tool that automatically generated pictures of Grecian vases. The teachers turned to Diffit, another AI application, to craft a reading passage that explained the importance of vases in everyday life in ancient Greece. Diffit also created alternative versions of the text so it would be appropriate for kids reading at different levels, wrote three multiple choice questions to test comprehension and prompted students to draw pictures to show they understood the lesson’s key points. The teachers added short-answer questions that students answered on Google Classroom, and they wrapped up the multi-week lesson by having students paint a design on an actual vase. 

“This is just touching the surface of what [AI] has the potential to do,” said Jared Bloom, the superintendent of Franklin Square School District, where both teachers work. “The promise really is to personalize learning, not just differentiate it, in a way that’s not taxing or exhausting for teachers. This could revolutionize education a year from now, as the tools get better and better.”

When ChatGPT was unveiled in late 2022, many educators saw the large language model chatbot as a shortcut students might use to complete — or cheat on — their homework. While it’s still unclear how the technology may ultimately affect schools, growing numbers of teachers are using various AI applications to help cut down on the work they do outside the classroom, from creating lessons to grading papers to emailing parents.

Teachers average about eight to 10 hours a week planning and doing administrative work, said Amanda Bickerstaff, CEO and co-founder of AI for Education, a company that advises districts on how to integrate artificial intelligence into their work. AI is a great way to find efficiencies and lessen that workload, she added.

In Franklin Square, a small K-6 district about 9 miles from John F. Kennedy Airport, the impetus to start using AI came from Bloom. Before the current school year, he highlighted various ways teachers could incorporate AI, from generating ideas for lesson plans to allowing students to use tools to enhance their work. In one example, after students studied houses that are shaped like cats, they created their own drawings. The teacher was then able to use AI tools to show the class how these buildings would look if they were constructed.

D’Aurio said she and Donaghy are “tech nerds” who were the first in their school to experiment with the new technology, and she’s noticed more teachers getting on board “a little at a time.” They use a variety of applications, including Diffit, which can create lesson plans from a few prompts. For instance, teachers can type in “ancient Greek vases” and a grade level, the application takes less than 20 seconds to return an adapted reading passage, a summary, key vocabulary words, and multiple-choice, short-answer and open-ended questions. These elements can be edited and quickly added into activities for students to complete.

Users can also ask the technology to adapt existing text for students reading at different levels. “In one classroom, you could go from second-grade reading level to 10^th grade,” Donaghy said.

Other companies help teachers create interactive slideshows, give writing feedback or generate images around various topics.

The inclusion class the pair co-teach contains both general and special education students. Donaghy said AI tools can help her not only create materials that meet students’ individual learning plans, they can track their progress in a variety of areas — a huge time saving because each student can have five or more individual goals. 

AI really helped when a new student from El Salvador showed up speaking only Spanish, said Donaghy. The teachers used it to translate every classroom lesson for her, allowing her to understand assignments while she worked in both Spanish and English. 

Donaghy said it did take a little trial and error to understand how to best craft queries to get the desired output. But she encourages her peers to try the tools by telling them, “I know tech can be scary, but guys, this is easy.” 

While acknowledging that most teachers in the small district haven’t used these tools yet, Bloom said 87% told him at the beginning of this school year that they were interested in trying them out. “They’re intrigued,” he added. 

Bickerstaff said about 84% of people who use a smartphone or a computer interact with AI every day, often without realizing it. 

John Street School librarian Paige Chambers said she used AI while earning her master’s degree and was eager to see how it could help her at school. Chambers, who teaches media literacy and related lessons to students in her library, said she uses AI tools to help her find ideas for lesson plans. Because results are so quick, she added, it’s easy to modify prompts when they don’t return what she wants. 

She has uploaded YouTube videos to AI applications to get a summary of the videos’ contents as well as questions for the students to answer. The tools can also break down a lesson plan into step-by-step directions for her while offering sample projects for students to complete. 

Because these tools can add to an existing lesson, Chambers said they can boost an idea she has by filling it out with extra ideas.

Donaghy, D’Aurio and Chambers said they were aware that AI can sometimes hallucinate — make up facts — but reading through what the program creates can help avoid this problem. To stop misinformation, Chambers said she specifically asks these tools to let her know if the application doesn’t have any information about a particular topic. This can prevent them from inventing answers to her prompts.

One area the teachers haven’t used AI for yet is helping to grade student work. This would require teachers to upload students’ writing into AI tools, which could breach the security of student information. 

Bloom said he expects technology upgrades to eventually solve this dilemma by creating tools that keep student work from being uploaded to the entire internet. “We’re not trying to remove teachers [from the grading process]. We just want students to get support in the moment. It could be like having a tutor on your shoulder.”

Donaghy said having a tool that checks whether her grading is accurate and fair and hews to a lesson’s rubric would be a big help.
“This is an exciting time for education,” Chambers said. AI “is getting better every day. It’s worlds different in just the half a year I’ve been using it.”

Students sitting for their STAAR exams this week will be part of a new method of evaluating Texas schools: Their written answers on the state’s standardized tests will be graded automatically by computers.

The Texas Education Agency is rolling out an “automated scoring engine” for open-ended questions on the State of Texas Assessment of Academic Readiness for reading, writing, science and social studies. The technology, which uses natural language processing technology like artificial intelligence chatbots such as GPT-4, will save the state agency about $15-20 million per year that it would otherwise have spent on hiring human scorers through a third-party contractor.

The change comes after the STAAR test, which measures students’ understanding of state-mandated core curriculum, was redesigned in 2023. The test now includes fewer multiple choice questions and more open-ended questions — known as constructed response items. After the redesign, there are six to seven times more constructed response items.

“We wanted to keep as many constructed open ended responses as we can, but they take an incredible amount of time to score,” said Jose Rios, director of student assessment at the Texas Education Agency.

In 2023, Rios said TEA hired about 6,000 temporary scorers, but this year, it will need under 2,000.

To develop the scoring system, the TEA gathered 3,000 responses that went through two rounds of human scoring. From this field sample, the automated scoring engine learns the characteristics of responses, and it is programmed to assign the same scores a human would have given.

This spring, as students complete their tests, the computer will first grade all the constructed responses. Then, a quarter of the responses will be rescored by humans.

When the computer has “low confidence” in the score it assigned, those responses will be automatically reassigned to a human. The same thing will happen when the computer encounters a type of response that its programming does not recognize, such as one using lots of slang or words in a language other than English.

“We have always had very robust quality control processes with humans,” said Chris Rozunick, division director for assessment development at the Texas Education Agency. With a computer system, the quality control looks similar.

Every day, Rozunick and other testing administrators will review a summary of results to check that they match what is expected. In addition to “low confidence” scores and responses that do not fit in the computer’s programming, a random sample of responses will also be automatically handed off to humans to check the computer’s work.

TEA officials have been resistant to the suggestion that the scoring engine is artificial intelligence. It may use similar technology to chatbots such as GPT-4 or Google’s Gemini, but the agency has stressed that the process will have systematic oversight from humans. It won’t “learn” from one response to the next, but always defer to its original programming set up by the state.

“We are way far away from anything that’s autonomous or can think on its own,” Rozunick said.

But the plan has still generated worry among educators and parents in a world still weary of the influence of machine learning, automation and AI.

Some educators across the state said they were caught by surprise at TEA’s decision to use automated technology — also known as hybrid scoring — to score responses.

“There ought to be some consensus about, hey, this is a good thing, or not a good thing, a fair thing or not a fair thing,” said Kevin Brown, the executive director for the Texas Association of School Administrators and a former superintendent at Alamo Heights ISD.

Representatives from TEA first mentioned interest in automated scoring in testimony to the Texas House Public Education Committee in August 2022. In the fall of 2023, the agency announced the move to hybrid scoring at a conference and during test coordinator training before releasing details of the process in December.

The STAAR test results are a key part of the accountability system TEA uses to grade school districts and individual campuses on an A-F scale. Students take the test every year from third grade through high school. When campuses within a district are underperforming on the test, state law allows the Texas education commissioner to intervene.

The commissioner can appoint a conservator to oversee campuses and school districts. State law also allows the commissioner to suspend and replace elected school boards with an appointed board of managers. If a campus receives failing grades for five years in a row, the commissioner is required to appoint a board of managers or close that school.

With the stakes so high for campuses and districts, there is a sense of uneasiness about a computer’s ability to score responses as well as a human can.

“There’s always this sort of feeling that everything happens to students and to schools and to teachers and not for them or with them,” said Carrie Griffith, policy specialist for the Texas State Teachers Association.

A former teacher in the Austin Independent School District, Griffith added that even if the automated scoring engine works as intended, “it’s not something parents or teachers are going to trust.”

Superintendents are also uncertain.

“The automation is only as good as what is programmed,” said Lori Rapp, superintendent at Lewisville ISD. School districts have not been given a detailed enough look at how the programming works, Rapp said.

The hybrid scoring system was already used on a limited basis in December 2023. Most students who take the STAAR test in December are retaking it after a low score. That’s not the case for Lewisville ISD, where high school students on an altered schedule test for the first time in December, and Rapp said her district saw a “drastic increase” in zeroes on constructed responses.

“At this time, we are unable to determine if there is something wrong with the test question or if it is the new automated scoring system,” Rapp said.

The state overall saw an increase in zeroes on constructed responses in December 2023, but the TEA said there are other factors at play. In December 2022, the only way to score a zero was by not providing an answer at all. With the STAAR redesign in 2023, students can receive a zero for responses that may answer the question but lack any coherent structure or evidence.

The TEA also said that students who are retesting will perform at a different level than students taking the test for the first time. “Population difference is driving the difference in scores rather than the introduction of hybrid scoring,” a TEA spokesperson said in an email.

For $50, students and their parents can request a rescore if they think the computer or the human got it wrong. The fee is waived if the new score is higher than the initial score. For grades 3-8, there are no consequences on a student’s grades or academic progress if they receive a low score. For high school students, receiving a minimum STAAR test score is a common way to fulfill one of the state graduation requirements, but it is not the only way.

Even with layers of quality control, Round Rock ISD Superintendent Hafedh Azaiez said he worries a computer could “miss certain things that a human being may not be able to miss,” and that room for error will impact students who Azaiez said are “trying to do his or her best.”

Test results will impact “how they see themselves as a student,” Brown said, and it can be “humiliating” for students who receive low scores. With human graders, Brown said, “students were rewarded for having their own voice and originality in their writing,” and he is concerned that computers may not be as good at rewarding originality.

Julie Salinas, director of assessment, research and evaluation at Brownsville ISD said she has concerns about whether hybrid scoring is “allowing the students the flexibility to respond” in a way that they can demonstrate their “full capability and thought process through expressive writing.”

Brownsville ISD is overwhelmingly Hispanic. Students taking an assessment entirely in Spanish will have their tests graded by a human. If the automated scoring engine works as intended, responses that include some Spanish words or colloquial, informal terms will be flagged by the computer and assigned to a human so that more creative writing can be assessed fairly.

The system is designed so that it “does not penalize students who answer differently, who are really giving unique answers,” Rozuick said.

With the computer scoring now a part of STAAR, Salinas is focused on adapting. The district is incorporating tools with automated scoring into how teachers prepare students for the STAAR test to make sure they are comfortable.

“Our district is on board and on top of the things that we need to do to ensure that our students are successful,” she said.

Disclosure: Google, the Texas Association of School Administrators and Texas State Teachers Association have been financial supporters of The Texas Tribune, a nonprofit, nonpartisan news organization that is funded in part by donations from members, foundations and corporate sponsors. Financial supporters play no role in the Tribune’s journalism. Find a complete list of them here.

This article originally appeared in The Texas Tribune at https://www.texastribune.org/2024/04/09/staar-artificial-intelligence-computer-grading-texas/.

The Texas Tribune is a member-supported, nonpartisan newsroom informing and engaging Texans on state politics and policy. Learn more at texastribune.org.

In the annals of artificial intelligence, Feb. 16, 2011, was a watershed moment.

That day, IBM’s Watson supercomputer finished off a three-game shellacking of Jeopardy! champions Ken Jennings and Brad Rutter. Trailing by over $30,000, Jennings, now the show’s host, wrote out his Final Jeopardy answer in mock resignation: “I, for one, welcome our computer overlords.”

A lark to some, the experience galvanized Satya Nitta, a longtime computer researcher at IBM’s Watson Research Center in Yorktown Heights, New York. Tasked with figuring out how to apply the supercomputer’s powers to education, he soon envisioned tackling ed tech’s most sought-after challenge: the world’s first tutoring system driven by artificial intelligence. It would offer truly personalized instruction to any child with a laptop — no human required.

“I felt that they’re ready to do something very grand in the space,” he said in an interview. 

Nitta persuaded his bosses to throw more than $100 million at the effort, bringing together 130 technologists, including 30 to 40 Ph.D.s, across research labs on four continents. 

But by 2017, the tutoring moonshot was essentially dead, and Nitta had concluded that effective, long-term, one-on-one tutoring is “a terrible use of AI — and that remains today.”

For all its jaw-dropping power, Watson the computer overlord was a weak teacher. It couldn’t engage or motivate kids, inspire them to reach new heights or even keep them focused on the material — all qualities of the best mentors.

It’s a finding with some resonance to our current moment of AI-inspired doomscrolling about the future of humanity in a world of ascendant machines. “There are some things AI is actually very good for,” Nitta said, “but it’s not great as a replacement for humans.”

His five-year journey to essentially a dead-end could also prove instructive as ChatGPT and other programs like it fuel a renewed, multimillion-dollar experiment to, in essence, prove him wrong.

Some of the leading lights of ed tech, from Google to Microsoft, are trying to pick up where Watson left off, offering AI tools that promise to help teach students. Sal Khan, founder of Khan Academy, last year said AI has the potential to bring “probably the biggest positive transformation” that education has ever seen. He wants to give “every student on the planet an artificially intelligent but amazing personal tutor.”

A 25-year journey

To be sure, research on high-dosage, one-on-one, in-person tutoring is unequivocal: It’s one of the most powerful interventions available, offering significant improvement in students’ academic performance, particularly in subjects like math, reading and writing.  

But traditional tutoring is also “breathtakingly expensive and hard to scale,” said Paige Johnson, a vice president of education at Microsoft. One school district in West Texas, for example, recently spent more than $5.6 million in federal pandemic relief funds to tutor 6,000 students. The expense, Johnson said, puts it out of reach for most parents and school districts. 

For IBM, the opportunity to rebalance the equation in kids’ favor was hard to resist. 

The Watson lab is legendary in the computer science field, with six Nobel laureates and six Turing Award winners among its ranks. It’s where modern speech recognition was invented, and home to countless other innovations such as barcodes and the magnetic stripes on credit cards that make ATMs possible. It’s also where, in 1997, Deep Blue beat world chess champion Garry Kasparov, essentially inventing the notion that AI could “think” like a person.

The heady atmosphere, Nitta recalled, inspired “a very deep responsibility to do something significant and not something trivial.”

Within a few years of Watson’s victory, Nitta, who had arrived in 2000 as a chip technologist, rose to become IBM Research’s global head of AI solutions for learning. For the Watson project, he said, “I was just given a very open-ended responsibility: Take Watson and do something with it in education.”

Nitta spent a year simply reading up on how learning works. He studied cognitive science, neuroscience and the decades-long history of “intelligent tutoring systems” in academia. Foremost in his reading list was the research of Stanford neuroscientist Vinod Menon, who’d put elementary schoolers through a 12-week math tutoring session, collecting before-and-after scans of their brains using an MRI. Tutoring, he found, produced nothing less than an increase in neural connectivity. 

Nitta returned to his bosses with the idea of an AI-powered cognitive tutor. “There’s something I can do here that’s very compelling,” he recalled saying, “that can broadly transform learning itself. But it’s a 25-year journey. It’s not a two-, three-, four-year journey.”

IBM drafted two of the highest-profile partners possible in education: the children’s media powerhouse Sesame Workshop and Pearson, the international publisher.

One product Sesame envisioned was a voice-activated Elmo doll that would serve as a kind of digital tutoring companion, interacting fully with children. Through brief conversations, it would assess their skills and provide spoken responses to help kids advance.

Meanwhile, Pearson promised that it could soon allow college students to “dialogue with Watson in real time.”

Nitta’s team began designing lessons and putting them in front of students — both in classrooms and in the lab. In order to nurture a back-and-forth between student and machine, they didn’t simply present kids with multiple-choice questions, instead asking them to write responses in their own words.

It didn’t go well.

Some students engaged with the chatbot, Nitta said. “Other students were just saying, ‘IDK’ [I don’t know]. So they simply weren’t responding.” Even those who did began giving shorter and shorter answers. 

Nitta and his team concluded that a cold reality lay at the heart of the problem: For all its power, Watson was not very engaging. Perhaps as a result, it also showed “little to no discernible impact” on learning. It wasn’t just dull; it was ineffective.

“Human conversation is very rich,” he said. “In the back and forth between two people, I’m watching the evolution of your own worldview.” The tutor influences the student — and vice versa. “There’s this very shared understanding of the evolution of discourse that’s very profound, actually. I just don’t know how you can do that with a soulless bot. And I’m a guy who works in AI.”

When students’ usage time dropped, “we had to be very honest about that,” Nitta said. “And so we basically started saying, ‘OK, I don’t think this is actually correct. I don’t think this idea — that an intelligent tutoring system will tutor all kids, everywhere, all the time — is correct.”

‘We missed something important’

IBM soon switched gears, debuting another crowd-pleasing Watson variation — this time, a touching throwback: It engaged in Oxford-style debates. In a televised demonstration in 2019, it went up against debate champ Harish Natarajan on the topic “Should we subsidize preschools?” Among its arguments for funding, the supercomputer offered, without a whiff of irony, that good preschools can prevent “future crime.” Its current iteration, Watsonx, focuses on helping businesses build AI applications like “intelligent customer care.” 

Nitta left IBM, eventually taking several colleagues with him to create a startup called Merlyn Mind. It uses voice-activated AI to safely help teachers do workaday tasks such as updating digital gradebooks, opening PowerPoint presentations and emailing students and parents. 

Thirteen years after Watson’s stratospheric Jeopardy! victory and more than one year into the Age of ChatGPT, Nitta’s expectations about AI couldn’t be more down-to-earth: His AI powers what’s basically “a carefully designed assistant” to fit into the flow of a teacher’s day. 

To be sure, AI can do sophisticated things such as generating quizzes from a class reading and editing student writing. But the idea that a machine or a chatbot can actually teach as a human can, he said, represents “a profound misunderstanding of what AI is actually capable of.” 

Nitta, who still holds deep respect for the Watson lab, admits, “We missed something important. At the heart of education, at the heart of any learning, is engagement. And that’s kind of the Holy Grail.”

These notions aren’t news to those who do tutoring for a living. Varsity Tutors, which offers live and online tutoring in 500 school districts, relies on AI to power a lesson plan creator that helps personalize instruction. But when it comes to the actual tutoring, humans deliver it, said Anthony Salcito, chief institution officer at Nerdy, which operates Varsity.

”The AI isn’t far enough along yet to do things like facial recognition and understanding of student focus,” said Salcito, who spent 15 years at Microsoft, most of them as vice president of worldwide education. “One of the things that we hear from teachers is that the students love their tutors. I’m not sure we’re at a point where students are going to love an AI agent.”

The No. 1 factor in a student’s tutoring success is simply showing up consistently, research suggests. As smart and efficient as an AI chatbot might be, it’s an open question whether most students, especially struggling ones, would show up for an inanimate agent or develop a sense of respect for its time.

When Salcito thinks about what AI bots now do in education, he’s not impressed. Most, he said, “aren’t going far enough to really rethink how learning can take place.” They end up simply as fast, spiffed-up search engines. 

In most cases, he said, the power of one-on-one, in-person tutoring often emerges as students begin to develop more honesty about their abilities, advocate for themselves and, in a word, demand more of school. “In the classroom, a student may say they understand a problem. But they come clean to the tutor, where they expose, ‘Hey, I need help.’”

Cognitive science suggests that for students who aren’t motivated or who are uncertain about a topic, only one-on-one attention will help. That requires a focused, caring human, watching carefully, asking tons of questions and reading students’ cues. 

Jeremy Roschelle, a learning scientist and an executive director of Digital Promise, a federally funded research center, said usage with most ed tech products tends to drop off. “Kids get a little bored with it. It’s not unique to tutors. There’s a newness factor for students. They want the next new thing.” 

Even now, Nitta points out, research shows that big commercial AI applications don’t seem to hold users’ attention as well as top entertainment and social media sites like YouTube, Instagram and TikTok. One recent analysis dubbed the user engagement of sites like ChatGPT “lackluster,” finding that the proportion of monthly active users who engage with them in a single day was only about 14%, suggesting that such sites aren’t very “sticky” for most users.

For social media sites, by contrast, it’s between 60% and 65%. 

One notable AI exception: Character.ai, an app that allows users to create companions of their own among figures from history and fiction and chat with the likes of Socrates and Bart Simpson. It has a stickiness score of 41%.

As startups like Synthesis offer “your child’s superhuman tutor,” starting at $29 per month, and Khan Academy publicly tests its popular Khanmigo AI tool, Nitta maintains that there’s little evidence from learning science that, absent a strong outside motivation, people will spend enough time with a chatbot to master a topic.

“We are a very deeply social species,” said Nitta, “and we learn from each other.”

IBM declined to comment on its work in AI and education, as did Sesame Workshop. A Pearson spokesman said that since last fall it has been ​​beta-testing AI study tools keyed to its e-textbooks, among other efforts, with plans this spring to expand the number of titles covered. 

Getting ‘unstuck’

IBM’s experiences notwithstanding, the search for an AI tutor has continued apace, this time with more players than just a legacy research lab in suburban New York. Using the latest affordances of so-called large language models, or LLMs, technologists at Khan Academy believe they are finally making the first halting steps in the direction of an effective AI tutor. 

Kristen DiCerbo remembers the moment her mind began to change about AI. 

It was September 2022, and she’d only been at Khan Academy for a year-and-a-half when she and founder Khan got access to a beta version of ChatGPT. Open AI, ChatGPT’s creator, had asked Microsoft co-founder Bill Gates for more funding, but he told them not to come back until the chatbot could pass an Advanced Placement biology exam.

So Open AI queried Khan for sample AP biology questions. He and DiCerbo said they’d help in exchange for a peek at the bot — and a chance to work with the startup. They were among the first people outside of Open AI to get their hands on GPT-4, the LLM that powers the upgraded version of ChatGPT. They were able to test out the AI and, in the process, become amateur AI prompt engineers before anyone had even heard of the term. 

Like many users typing in queries in those first heady days, the pair initially just marveled at the sophistication of the tool and its ability to return what felt, for all the world, like personalized answers. With DiCerbo working from her home in Phoenix and Khan from the nonprofit’s Silicon Valley office, they traded messages via Slack.

“We spent a couple of days just going back and forth, Sal and I, going, ‘Oh my gosh, look what we did! Oh my gosh, look what it’s saying — this is crazy!’” she told an audience during a recent appearance at the University of Notre Dame. 

She recounted asking the AI to help write a mystery story in which shoes go missing in an apartment complex. In the back of her mind, DiCerbo said, she planned to make a dog the shoe thief, but didn’t reveal that to ChatGPT. “I started writing it, and it did the reveal,” she recalled. “It knew that I was thinking it was going to be a dog that did this, from just the little clues I was planting along the way.”

More tellingly, it seemed to do something Watson never could: have engaging conversations with students.

DiCerbo recounted talking to a high school student they were working with who told them about an interaction she’d had with ChatGPT around The Great Gatsby. She asked it about F. Scott Fitzgerald’s famous green light across the bay, which scholars have long interpreted as symbolizing Jay Gatsby’s out-of-reach hopes and dreams.

“It comes back to her and asks, ‘Do you have hopes and dreams just out of reach?’” DiCerbo recalled. “It had this whole conversation” with the student.

The pair soon tore up their 2023 plans for Khan Academy. 

It was a stunning turn of events for DiCerbo, a Ph.D. educational psychologist and former senior Pearson research scientist who had spent more than a year on the failed Watson project. In 2016, Pearson had predicted that Watson would soon be able to chat with college students in real time to guide them in their studies. But it was DiCerbo’s teammates, about 20 colleagues, who had to actually train the supercomputer on thousands of student-generated answers to questions from textbooks — and tempt instructors to rate those answers. 

Like Nitta, DiCerbo recalled that at first things went well. They found a natural science textbook with a large user base and set Watson to work. “You would ask it a couple of questions and it would seem like it was doing what we wanted to,” answering student questions via text.

But invariably if a student’s question strayed from what the computer expected, she said, “it wouldn’t know how to answer that. It had no ability to freeform-answer questions, or it would do so in ways that didn’t make any sense.” 

After more than a year of labor, she realized, “I had never seen the ‘OK, this is going to work’ version” of the hoped-for tutor. “I was always at the ‘OK, I hope the next version’s better.’”

But when she got a taste of ChatGPT, DiCerbo immediately saw that, even in beta form, the new bot was different. Using software that quickly predicted the most likely next word in any conversation, ChatGPT was able to engage with its human counterpart in what seemed like a personal way.

Since its debut in March 2023, Khanmigo has turned heads with what many users say is a helpful, easy-to-use, natural language interface, though a few users have pointed out that it sometimes gets math facts wrong.

Surprisingly, DiCerbo doesn’t consider the popular chatbot a full-time tutor. As sophisticated as AI might now be in motivating students to, for instance, try again when they make a mistake, “It’s not a human,” she said. “It’s also not their friend.”

Khan Academy’s research shows their tool is effective with as little as 30 minutes of practice and feedback per week. But even as many startups promise the equivalent of a one-on-one human tutor, DiCerbo cautions that 30 minutes is not going to produce miracles. Khanmigo, she said, “is not a solution that’s going to replace a human in your life,” she said. “It’s a tool in your toolbox that can help you get unstuck.”

‘A couple of million years of human evolution’

For his part, Nitta says that for all the progress in AI, he’s not persuaded that we’re any closer to a real-live tutor that would offer long-term help to most students. If anything, Khanmigo and probabilistic tools like it may prove to be effective “homework helpers.” But that’s where he draws the line. 

“I have no problem calling it that, but don’t call it a tutor,” he said. “You’re trying to endow it with human-like capabilities when there are none.”  

Unlike humans, who will typically do their best to respond genuinely to a question, the way AI bots work —by digesting pre-existing texts and other information to come up with responses that seem human — is akin to a “statistical illusion,” writes Harvard Business School Professor Karim Lakhani. “They’ve just been well-trained by humans to respond to humans.”

Largely because of this, Nitta said, there’s little evidence that a chatbot will continuously engage people as a good human tutor would.

What would change his mind? Several years of research by an independent third party showing that tools like Khanmigo actually make a difference on a large scale — something that doesn’t exist yet.

DiCerbo also maintains her hard-won skepticism. She knows all about the halting early decades of computers a century ago, when experimental, punch-card operated “teaching machines” guided students through rudimentary multiple-choice lessons, often with simple rewards at the end. 

In her talks, DiCerbo urges caution about AI revolutionizing education. As much as anyone, she is aware of the expensive failures that have come before. 

In her recent talk at Notre Dame, she did her best to manage expectations of the new AI, which seems so limitless. In one-to-one teaching, she said, there’s an element of humanity “that we have not been able to — and probably should not try — to replicate in artificial intelligence.” In that respect, she’s in agreement with Nitta: Human relationships are key to learning. In the talk, she noted that students who have a person in school who cares about their learning have higher graduation rates. 

But still.

ChatGPT now has 100 million weekly users, according to Open AI. That record-fast uptake makes her think “there’s something interesting and sticky about this for people that we haven’t seen in other places.”

Being able to engineer prompts in plain English opens the door for more people, not just engineers, to create tools quickly and iterate on what works, she said. That democratization could mean the difference between another failed undertaking and agile tools that actually deliver at least a version of Watson’s promise. 

Seven years after he left IBM to start his new endeavor, Nitta is philosophical about the effort. He takes virtually full responsibility for the failure of the Watson moonshot. In retrospect, even his 25-year timeline for success may have been naive.

“What I didn’t appreciate is, I actually was stepping into a couple of million years of human evolution,” he said. “That’s the thing I didn’t appreciate at the time, which I do in the fullness of time: Mistakes happen at various levels, but this was an important one.”