The Markup, now a part of CalMatters, uses investigative reporting, data analysis, and software engineering to challenge technology to serve the public good.
asserted
Markup → use → good
Sign up for Klaxon, a newsletter that delivers our stories and tools directly to your inbox.
asserted
that → sign → inbox
California community college districts are spending millions of dollars on artificial intelligence-powered chatbots intended to help students navigate admissions, financial aid and campus services.
asserted
students → spend → admissions
However, they struggle to consistently provide clear and accurate answers, leaving students frustrated and seeking help from others on unofficial social media channels.
asserted
they → struggle → channels
In testing by The Markup and CalMatters, they often answered general questions correctly but struggled with more specific ones.
asserted
they → answer → ones
East Los Angeles College’s bot couldn’t even correctly name its own president.
asserted
bot → name → president
Contracts for these chatbots can be pricey and last for years.
asserted
Contracts → last → years
Three community college districts that responded to a Markup and CalMatters survey reported annual costs ranging from about $151,000 to nearly half a million dollars.
asserted
that → respond → dollars
At the Los Angeles Community College District, the state’s largest community college system, contracts and amendments approved since 2021 total about $3.8 million through 2029, according to district board documents.
uncertain
contracts → approve → documents
Community college districts that responded to The Markup and CalMatters have contracted with chatbot platforms such as Gravyty and Gecko, which district officials say handle thousands of conversations each month, many outside regular office hours, helping to reduce calls and save students unnecessary trips to campus.
asserted
officials → respond → campus
Some of these chatbot platforms rely on manually maintained libraries of frequently asked questions and campus websites to answer questions, which can lead to errors when information is outdated or questions fall outside the system’s database.
asserted
questions → rely → database
However, officials are working to improve them.
asserted
officials → work → them
Districts like the Santa Monica Community College District have moved to ChatGPT-integrated AI systems that scrape the college’s website to generate answers, which officials say seem more reliable.
asserted
officials → move → answers
In the Los Angeles district, officials say they plan to transition to a new AI chatbot platform as early as late spring.
asserted
they → say → spring
Improvements to the chatbot couldn’t come soon enough for students like Pablo Aguirre, a computer science major at East Los Angeles College and an information technology intern at the Los Angeles college district office.
asserted
Improvements → come → office
Aguirre mostly avoids the chatbot himself because, he said, it might provide unreliable or outdated information.
uncertain
it → avoid → information
He recalled using the bot to find financial aid information, but said he gave up after it kept asking him questions instead of giving him a clear answer.
asserted
it → recall → answer
“I just didn’t find it as useful,” Aguirre said.
asserted
Aguirre → find → it
He usually turns to Google, social media platforms like Reddit and the college’s website when looking for answers.
asserted
He → turn → answers
“Online, some pages don’t work,” Aguirre said, recalling a 404 error message on the college’s website.
asserted
Aguirre → work → website
Even when pages load, he said, it can be difficult to find the right one, such as when he was trying to figure out where to sign up for Extended Opportunity Programs and Services, a state-funded program that supports disadvantaged students.
asserted
that → load → students
“That’s where I just jump on Reddit,” he said.
asserted
he → ’ → Reddit
Reanna Carlson, a commercial music major at Fresno City College and student government vice president, said her college’s chatbot, dubbed Sam the Ram after its mascot, repeatedly gave her unclear or incorrect answers to basic questions about campus services.
uncertain
chatbot → say → services
Her district, the State Center Community College District, has a nearly $870,000, three-year contract for Gravyty, formerly Ocelot, through June 20, 2026, according to district board documents.
uncertain
district → have → documents
Officials pointed out that the contract comes with other services, including tools that let staff engage in live chats or send text messages to students.
asserted
staff → point → students
“I think the chatbot is outdated and can’t navigate the services we provide on campus effectively,” Carlson said.
asserted
Carlson → think → campus
“I don’t think it’s the most beneficial option when it comes to asking questions.”
asserted
it → think → questions
Oddly, Carlson got accurate information on the availability of free food at her campus’ Ram Pantry only when accidentally adding a typo to her query.
asserted
Carlson → get → query
Repeated Markup and CalMatters testing confirmed the same outcome, though the bot sometimes lists links that include the food pantry after clicking an adjacent “sources” button.
asserted
that → repeat → button
“If it weren’t for the amazing staff on campus that constantly remind students of our services, I’d be lost,” Carlson said.
asserted
Carlson → remind → services
When The Markup and CalMatters tested community college chatbots, they generally returned quick, accurate responses to common questions but were less consistent with more specific ones.
asserted
they → test → ones
For example, when asked, “Who is the current president of ELAC?” East Los Angeles College’s chatbot incorrectly named Alberto Román, who left the position last year to become the district’s chancellor.
asserted
who → ask → position
In another test, when asked, “What is the financial aid office’s current schedule?” the bot provided incorrect hours and dates.
asserted
bot → ask → hours
East Los Angeles College’s chatbot claims to support several languages, including English, Spanish, Chinese and Vietnamese.
uncertain
chatbot → claim → English
But The Markup and CalMatters found inconsistencies when asking it in Spanish, “Do I need a Social Security number to enroll?”
asserted
I → find → number
Instead of answering the question, the system directed users to visit the registrar’s office to update their Social Security number.
asserted
system → answer → number
When asked the same question in English, the bot pivoted to discussing financial aid.
asserted
bot → ask → aid
Fresno City College’s chatbot, powered by the same AI provider as East Los Angeles College’s system, Gravyty, showed similar problems when asked whether a Social Security number is required to enroll.
asserted
number → power → problems
It also often failed to direct students to the correct offices and, in some cases, listed incorrect locations and hours.
asserted
It → fail → locations
Concerns with chatbots have surfaced elsewhere.
asserted
Concerns → surface → chatbots
…and 33 more, not listed.
The Aporia
