“Chatbots are obsolete” – Markup

The Markup, now part of CalMatters, uses investigative reporting, data analysis, and software engineering to challenge technology in the public interest. register to Klaxona newsletter that delivers stories and tools straight to your inbox.

California community college districts are spending millions of dollars on artificial intelligence-powered chatbots aimed at helping students navigate admissions, financial aid, and campus services.

However, they struggle to consistently provide clear and accurate answers, leading students to become frustrated and seek help from others on unofficial social media channels.

In tests by The Markup and CalMatters, I answered general questions correctly, but often struggled with more specific questions. East Los Angeles College’s bot couldn’t even name the president correctly.

Contracts for these chatbots are expensive and can last for years. The three community college districts that responded to the Markup and CalMatters survey reported annual costs ranging from about $151,000 to nearly $500,000. In the Los Angeles Community College District, the state’s largest community college system, contracts and amendments approved after 2021 total about $3.8 million through 2029, according to district board documents.

Community college districts in response to The Markup and CalMatters have contracted with chatbot platforms such as Gravyty and Gecko, which district officials say handle thousands of conversations each month, many of them outside of normal business hours, helping to reduce phone calls and unnecessary trips for students to campus.

Some of these chatbot platforms rely on manually maintained FAQ libraries or campus websites to answer questions, which can result in errors if the information is outdated or the question is not in the system’s database.

However, authorities are working on improvements. School districts like the Santa Monica Community College District are moving to ChatGPT-integrated AI systems that scrape college websites and generate answers, which officials say are likely more reliable. In the Los Angeles area, officials say they plan to migrate to a new AI chatbot platform as early as late spring.

Improving the chatbot didn’t come quickly for students like Pablo Aguirre, a computer science major at East Los Angeles College and an information technology intern at the Los Angeles College District office.

Aguirre himself says he mostly avoids chatbots because they can provide unreliable or outdated information. He recalled using the bot to find financial aid information, but said he gave up because it kept asking questions without giving clear answers.

“It didn’t really help us,” Aguirre said. When he looks for answers, he typically turns to Google, social media platforms like Reddit, and university websites.

“Some pages don’t work online,” Aguirre said, recalling a 404 error message on the university’s website. Even when a page loads, it can be difficult to find the right page, he said. For example, when trying to figure out where to enroll in Expanded Opportunity Programs and Services, a state-funded program that helps disadvantaged students. “That’s why I jumped on Reddit,” he said.

Aguirre’s experience is not unique. Liana Carlson, a commercial music major and student government vice president at Fresno City College, said the university’s chatbot, called “Rum Sam,” after the university’s mascot, repeatedly gave unclear or inaccurate answers to basic questions about campus services. Her district, the State Center Community College District, has signed Gravity, formerly of Ocelot, to a three-year contract worth about $870,000 through June 20, 2026, according to district board documents. Officials noted that the deal also comes with other services, including tools that allow staff to live chat and send text messages to students.

“I think chatbots are outdated and cannot effectively navigate the services we offer on campus,” Carlson said. “I don’t think that’s the most beneficial option if you ask the question.”

Oddly, Carlson only received accurate information about the availability of free food at the campus Rum Pantry when she accidentally added a typo to her query. I repeatedly tested Markup and CalMatters and saw the same results, except that the bot would sometimes list links containing pantries after clicking the adjacent “source” button.

“I would have been lost if it wasn’t for the wonderful staff on campus who constantly remind students of our services,” Carlson said.

When The Markup and CalMatters tested a community college chatbot, it provided quick and accurate answers to general questions, but less consistently when asked more specific questions.

For example, when asked “Who is the current president of ELAC?” East Los Angeles College’s chatbot incorrectly named Alberto Roman, who resigned last year to become the school’s president. In another test, when asked, “What is the financial aid office’s current schedule?” The bot provided the wrong time and date.

East Los Angeles College’s chatbot claims to support multiple languages, including English, Spanish, Chinese, and Vietnamese. But when The Markup and CalMatters asked in Spanish, “Do I need a social security number to register?”, they found a discrepancy. Instead of answering questions, the system directed users to go to the registrar’s office to update their Social Security numbers. When I asked the same question in English, the bot moved on to discussing financial aid.

Fresno City College’s chatbot, which is powered by Gravyty, the same AI provider as East Los Angeles College’s system, showed a similar issue when asked whether a social security number was required for registration. Additionally, students were often not directed to the correct office, and in some cases, the wrong location or business hours were listed.

Concerns about chatbots have surfaced elsewhere as well. In New York City, Mayor Zoran Mamdani terminated the city-run AI chatbot in February after reports from The Markup and THE CITY found that it provided guidance that could lead to illegal activities.

Santa Monica College’s chatbot, powered by Gecko, was able to successfully answer most questions. The Uni-College School District uses chatbots integrated with ChatGPT to scan university websites, which are regularly updated and monitored by staff. The district has been contracting with Gecko since 2019 and renewed its annual contract for the tool late last year for $57,000, according to district board documents. Bots initially presented a major problem. When asked about mental health counseling, Bott did not mention the campus Wellness and Wellbeing Center. Now it is.

District officials say problems with chatbots stem not from the technology itself, but from how the tools are configured and the information they provide.

The Los Angeles area initially adopted chatbots through Ocelot, which later merged with Gravyty. The same chatbot platform is also used on the California Student Aid Commission website.

Betsy Regalado, one of the district’s vice chancellors, said the current system relies on a manually maintained library of frequently asked questions that staff at each of the district’s nine colleges help review at least once or twice a year to ensure maintenance and accuracy. He added that the chatbot is primarily aimed at the public rather than registered students, and that students can access more detailed personal information through the campus portal.

“The current chatbots that we use use a library of questions, and if that library doesn’t have that question, poor people won’t get an answer, and they won’t get an accurate answer,” Regalado said.

The district plans to migrate all nine colleges to Gravyty’s platform as early as late spring at no additional cost under its current contract through 2029, she said. The new system uses AI to scrape university and external websites to generate responses.

“We’re ready for the modernization[of chatbots]and the change to generative AI. That’s the new world out there,” Regalado said.

Santa Monica College’s chatbot similarly initially relied on a manually loaded library of frequently asked questions and answers before transitioning to a fully AI system. said Esau Tovar, director of enrollment services at the university. He said in an email that the bot was “not designed to address every aspect of student life,” but was designed to answer common questions from students.

Tovar said because the bot pulls responses from university websites, accuracy depends on how up-to-date and complete that information is. As a result, bots provide “better answers with fewer errors” rather than “better answers that may have more errors” as universities prioritize keeping their websites up to date.

Acknowledging the limitations, community college districts justify the costs by pointing to high student use, which would be significantly higher if call center staff were on duty 24 hours a day.

Regalado said universities in the Los Angeles area average 5,000 to 7,000 interactions a month. Other school districts have reported similar monthly usage, with the State Center Community College district, which includes campuses in Fresno and neighboring counties, reporting 5,000 interactions and Santa Monica College reporting 4,000 conversations. Regalado said the district will continue to support the chatbot as long as it continues to be used frequently.

Tovar said chatbots are useful for international students abroad because they provide 24-hour support regardless of time zone or location. He said having staff available to answer the tens of thousands of questions a chatbot receives 24 hours a day would significantly increase costs.

“All technology has a cost, and you can’t help every student if you can only reach them through traditional methods,” Tovar said.

But high usage and expanded access don’t always translate to trust, especially when students need accurate answers on sensitive topics.

Brian Hartanto, a civil engineering major at the University of Santa Monica from Indonesia, said the university’s new chatbot system is smoother and could be a useful starting point, especially for students who are used to communicating in languages ​​other than English. But as an international student, he worries that following inaccurate guidance could jeopardize his visa eligibility.

“Maintaining your status as an international student at this time is a very sensitive issue,” Hartand said. “I still rely on human or email communication.”

Martin Romero is a contributor to the College Journalism Network, a collaboration between CalMatters and student journalists from across California. CalMatters’ higher education reach is supported by a grant from the College Futures Foundation.