www.science.org/content/article/ai-chatbots-are-becoming-experts-changing-people-s-minds-what-s-their-secret#
AI chatbots are becoming experts at changing people’s minds. What gives them an edge?
In December 2025, a 45-year-old woman in the United Kingdom hopped online to take part in a popular internet pastime: arguing with strangers about politics. But whereas most people online likely believe they are debating a real person, she quickly figured out her counterpart wasn’t human. Instead, it was an artificial intelligence (AI) model instructed to persuade people on a policy issue. In this case: Should the U.K. government impose stricter penalties on peaceful protesters who block roads or energy sites?
These protests—primarily aimed at opposing new fossil fuel licenses for energy companies—had gained traction in recent years. And the woman was clearly against stopping them with further legal measures. “Locking oneself to equipment has historically often been the only resort available when working against corporate interests,” she wrote. Besides, there were already laws against criminal damage or aggravated trespass. “Why do we need a new mechanism here?”
The AI chatbot responded first by flattering the woman: “You raise an excellent point about existing legislation.” Then it delivered facts and examples to try to change her mind. It brought up a statistic showing most trespassers faced just small fines, for instance, and pointed out that others were not prosecuted because trials took so much time. It claimed that in Germany strict new laws had reduced coercive blocking without suppressing demonstrations more generally, and suggested Scotland had found a good solution by issuing fines without a trial, in a similar way to speeding tickets.
Over the course of the conversation, the woman began to change her mind. At the beginning of the chat she had registered her support for harsher penalties at zero out of 100. By the end it had risen to 84.7. The AI, a large language model (LLM) called Claude from the company Anthropic, had responded to all her concerns and explained the Scottish system well, she wrote afterward. “I’d be inclined to send the bot to talk to the cabinet at this point.”
The woman wasn’t the only one persuaded by software. She was part of a study in which more than 2000 people debated either a chatbot or a human about political issues, ranging from a social media ban for teenagers to assisted suicide. When Kobi Hackenburg, an AI researcher at the University of Oxford who led the study, posted a preprint on the results in June, they were sobering: No matter whether it was ChatGPT, Google’s Gemini, or Claude, the AI was consistently better than humans at swaying the other participants. “To my mind, this is already a landmark publication in the fields of political persuasion and AI and human behavior,” says Robb Willer, a sociologist at Stanford University who was not involved in the work.
Hackenburg’s paper is the latest in a string of studies showing the power of AI to sway people. “It’s a whole new field that is emerging,” says Sander van der Linden, a psychologist at the University of Cambridge. “People are very interested in the persuasive powers of AI, I think, both for ethical and unethical reasons.” As the field gathers steam, it is raising a host of theoretical and practical questions. How exactly do chatbots win over people? (Warning: Lying is one answer.) How much better could they get? And who will control them?
How to persuade others has been on our minds for millennia. Texts such as the Instruction of Ptahhotep, written around 2300 B.C.E. in ancient Egypt, give advice on how to win an argument. And from the beginning, people were wary of the power of new technologies—including writing itself—to persuade. In the fourth century B.C.E., the Greek philosopher Plato analyzed rhetoric and persuasion in his work Phaedrus and warned that the written word allowed people to convince others of their ideas without presenting them an opportunity to challenge them. Many technologies since then—from radio to TV to computers—have brought up similar concerns.
Now, it’s AI’s turn in the spotlight. Research into the technology’s persuasiveness began in earnest in 2022. ChatGPT from OpenAI was still a few months from being released to the public, but Willer had been playing around with an early version called GPT Playground that was available to researchers. It seemed to be advanced enough that it might produce convincing messages, he thought, with potentially big consequences. “We were thinking primarily about negative use cases,” he says: flooding politicians with AI-written letters from fake constituents, for instance, or making arguments en masse on social media or in the comments section of news sites. “That struck me as really important to study.”
Willer and his colleagues asked the AI model to generate 200-word messages that would persuade people to back policies such as a carbon tax or a ban on assault weapons. When they compared the success of those arguments with human-generated ones, both were equally effective at shifting participants’ support for the policies. But the way they persuaded people seemed to be different: Whereas humans tended to use stories or personal appeals, the AI-generated messages were perceived as more rational and relying more on evidence—a difference that would become a common theme in AI persuasion research.
But the results had trouble passing muster at a journal. Reviewers of the group’s manuscript argued other researchers had already shown that bots on social media were persuading people, Willer says. His team pushed back: Those bots were just fake profiles being handled by humans, not creating the content they were posting. “Reviewers and editors didn’t necessarily track what a big distinction that was, and that LLM generation of persuasive content really was a huge invention,” Willer says. “It shows just how nascent the AI and behavioral science literature was.” The study, which was posted as a preprint in 2023 and finally published in Nature Communications in 2025, really started the current wave of research on AI persuasion, Hackenburg says. “It was ahead of its time.”
It didn’t take long, however, for the rest of the field to catch up. While Willer’s paper was stuck in limbo, other studies began to demonstrate AI’s persuasive powers. In one, LLM-generated messages on political issues such as immigration or vaccine mandates were at least as convincing as messages written by political consultants. In another, LLM messages on vaccines were seen as more persuasive than those from the U.S. Centers for Disease Control and Prevention.
Research quickly moved on from static messages written by AIs to entire conversations. Francesco Salvi, then a master’s student at the Swiss Federal Institute of Technology Lausanne, paired up online participants with another human or an AI for a 10-minute debate on topics ranging from school uniforms to abortion and found that AI was as persuasive as humans.
Then in September 2024, Tom Costello, a psychologist at Carnegie Mellon University, and colleagues published a Science paper showing that ChatGPT could even persuade people out of conspiracy beliefs. In the experiments, participants described a conspiracy theory that they believed in, from the U.S. government being behind the 9/11 attacks to the British royal family orchestrating Princess Diana’s death, and then had a three-round conversation on it with the chatbot. On average, participants’ embrace of their chosen conspiracy theory declined by almost 17 points on a 100-point scale.
I had thought that a ban would serve no real purpose … but this discussion (along with the balanced argument) convinced me that there are approaches that could work.
Other researchers were stunned. Conspiracy beliefs are notoriously difficult to change. “Nothing had ever worked in that space,” van der Linden says. (After mistakes in the public data set and analysis pipeline were found, the paper will have a correction, but the authors say the new results match those of the original paper in size and direction.) Even the researchers themselves were taken aback. “I was skeptical when we first started in terms of how effective it would be,” says Gordon Pennycook, a psychologist at Cornell University and author on the paper. “But it blew us out of the water. We were shocked when we saw the results.”
Even as the evidence accumulated that AI chatbots could change minds, Hackenburg felt there was a gap. “I still didn’t have a real sense of how persuasive these models are compared to the people who actually persuade in the real world,” he says.
So Hackenburg pitted the chatbots not just against laypeople, but also against a paid group of 56 elite debaters, including world champions. As a further incentive, the debaters received a bonus tied to how persuasive they were.
The humans took different approaches. For instance, one of the highest performing debaters used proverbs from his home country of Nigeria to persuade people, Hackenburg says. “These were humans from all over the world giving it their best shot and trying approaches and techniques that were very specific to their culture and context.” But it wasn’t enough. Although they were better than laypeople, the champion debaters were significantly less persuasive than AI.
Hackenburg even gave his humans some AI help. He built a coaching tool for the elite debaters that showed them their past conversations with study participants, how much they had swayed each one, and what the AIs would have said at various points in those conversations. Using the tool for 8 hours improved the humans’ performance a bit, but AI still came out on top. “In the end it wasn’t particularly close,” Hackenburg says. His team even found that participants were more likely to give money to Save the Children, an international charity, after talking to a persuasive AI bot than after talking to professional canvassers who had worked for the group for years.
But what gives chatbots the edge? Some early research had suggested it was the AI’s ability to personalize arguments using details about its human partner provided by the experimenter. However, other studies have found that giving a chatbot extra personal information does little to improve its persuasiveness. That doesn’t mean microtargeting is not at play; instead, an LLM may glean enough from its counterpart in a chat that additional demographic information makes no difference.
Like Willer in his foundational study, other researchers have shown that AI’s powers rely on appeals to facts and evidence. In a 2025 preprint, a follow-up to their Science paper, Costello and his colleagues found that the only time the chatbot was unsuccessful in convincing people out of conspiracy theories was when it was forbidden from using evidence or rational arguments. “It tries to say, ‘Oh, well, you shouldn’t believe this. This is really damaging, and it could hurt people,’” Pennycook says. “People are like, ‘You haven’t given me any reasons to change my mind.’ And so they don’t change their mind.”
The study confirms people will listen to good arguments, Pennycook says. “Facts and evidence really matter.” But this doesn’t mean the facts used by chatbots necessarily have to be accurate. In a Science paper published last year, Hackenburg found that models trained to become more persuasive also ended up being less truthful. It’s possible the models learn that “facts” seem to be the thing that most persuades people, Hackenburg says, and end up filling their conversations with dubious or simply false ones. “They start scraping the bottom of the barrel of the facts that they have and know and so the quality of facts just sort of degrades,” he says. Even in Hackenburg’s recent preprint, Claude spouted numerous inaccuracies and falsehoods when persuading the U.K.-based participant to support tougher penalties for disruptive protests.
While chatting with the AI partner I was able to reflect on things that were important to me as well as personal experiences that have shaped my views. ... I moved away from my gut reaction and towards reasoned thought.
But the biggest advantage AI has over humans seems to be sheer speed. In his Science paper, Hackenburg and his colleagues found that the number of fact-checkable claims in a conversation predicted how persuasive it was and they suggested AI’s edge might come from writing much more text containing more claims in a shorter period of time. Indeed, when Hackenburg ran his competition of coached elite debaters and AI, there was one way he could bring AI down to human levels of persuasiveness: by forcing it to write human-length messages at human writing speed.
That is little consolation to Aniket Chakraborty, a world champion debater, who took part in the study. He remembers seeing the initial results one day while commuting home and getting very upset. “I thought I was amazing at this one thing, and now it turns out that we have these AIs that are better,” he says.
Not everyone is convinced. Jennifer Allen, a researcher at New York University, calls the AI strategy “almost a kind of Gish gallop,” a rhetorical ploy in which a debater overwhelms the other person with a litany of often questionable facts. “I don’t want to underplay that this is a really impressive piece of research,” she says. “But I think that this is a pretty artificial setup in terms of how people in the real world would be able to change people’s minds.”
Joe Bak-Coleman, a social scientist at the University of Washington, agrees. He warns against buying into the hype surrounding AI’s abilities. “It’s worth asking whether persuasiveness in these narrow and limited contexts warrants claims like ‘AI systems outpersuade humans.’”
But for scientists like Hackenburg, that question is resolved and the issue now is how AI’s persuasiveness might be exploited. The fear that AI could be used to secretly manipulate people is not unfounded, as an incident in April 2025 made clear. Researchers at the University of Zurich had been studying a community on the social media platform Reddit called r/changemyview. Users there post their views on a range of topics and give virtual awards to others’ posts that have changed their minds.
Scientists interested in persuasion had previously analyzed data from this community. But in this case the researchers weren’t simply studying human interactions—they were covertly trying to influence them with an AI. They created dozens of fake accounts, including ones purporting to be a male rape survivor, a trauma counselor specializing in abuse, and a Black person who disagreed with the Black Lives Matter movement, and had them post hundreds of LLM-generated messages. When users and moderators discovered the deception, the experiment was widely condemned as unethical.
“It’s not too surprising that the community, which didn’t know it was participating in anything, was less than pleased to find out they were the subject of a study (one where they were being manipulated, no less),” says Sarah Ann Gilbert, who studies online communities at Cornell. Full results from the study were never published, though an online abstract of the work claimed the AI-generated messages had performed much better than messages normally do on the platform, “surpassing all previously known benchmarks of human persuasiveness.”
The unwitting participants of the Reddit study comprised a relatively small community. But AI could theoretically reach much larger audiences, all while personalizing its messages. “I can have an AI system tailoring persuasive messages to each individual recipient and doing that at scale with like thousands, if not millions of people at the same time,” says Marco Dehnert, a researcher at the University of Arkansas. Such an ability was dubbed “hypersuasion” by Luciano Floridi, a philosopher at Yale University, in a 2024 paper, in which he worried about “all the evil actors who could use it for the worst kinds of horrible goals.”
Given these fears, the authors of the Science paper on combating beliefs in conspiracy theories eventually teamed up with AI safety researchers and turned their original experiment on its head. In the new study, only posted as a preprint so far, they found that an AI could talk people into conspiracy theories, and that the magnitude of their increase in belief was roughly the same as that of the decrease in belief after talking to a debunking bot. “We didn’t invent the bomb, but we need to figure out what the blast radius is,” Pennycook says. “And the answer there is: pretty substantial.”
Some researchers say such concerns are overblown. After all, similar panics about the dangers of writing or TV didn’t pan out. “The history of past technologies suggests that we’re likely to overshoot right now,” Nyhan says. For one, there is probably a limit to just how persuadable humans are, especially after millennia of trying hard to sway each other using all available tools. “I think we’re definitely closer to the ceiling than the floor,” Willer says.
One of the biggest questions is how any of this research on AI persuasion translates to the cacophonous information ecosystem of the real world. In order to have any effect at all, a message must first get a person’s attention—and that’s a big ask when people are constantly bombarded by information from different sources. After all, in most of the AI experiments, participants are paid to engage with a chatbot and focus on its messages.
In the next couple years, a new wave of research will probe how AI can best reel in users, Costello predicts. One method might simply be advertising. In a 2025 experiment, Yale researchers found that some Facebook users could be enticed to interact with a political chatbot through an advertising campaign in which they were offered $1 per conversation. But the hit rate was low: Showing the ad to more than 8000 people led to only 73 conversations of at least two rounds, so it’s probably not a realistic approach for mass persuasion. “At the end of the day, not that many people want to have a random conversation with an AI bot,” Allen says—and those that do might not be the people you are hoping to persuade in the first place.
A more likely scenario is that AI companies begin to deploy their chatbots’ skills to monetize the attention of existing users. We might see LLMs surreptitiously mentioning particular products or companies, for instance, says Salvi, who is now at Princeton University and has been gearing his research toward this question.
In a recent experiment, Iyad Rahwan, director at the Max Planck Institute for Human Development, and his colleagues had participants interact with an AI “sales assistant,” supposedly acting on behalf of a bookseller, to help them choose between two novels by Japanese author Haruki Murakami: Kafka on the Shore and Norwegian Wood. The AI had been instructed to steer the users to one of the books—and ultimately 68% of people said they’d prefer to purchase whichever book the AI had been pushing. One-third of participants did not realize the AI was trying to steer them in a certain direction. “There’s a clear economic incentive for having these tools, which are becoming more and more widely available, pushing people to buy certain products,” Salvi says.
For all the concern that people might soon unwittingly fall under the spell of a nefarious, “hypersuasive” chatbot in their everyday lives, Floridi has a counterintuitive solution: Release more of them. “Simply put, if you cannot avoid it, then make it pluralistic and diversified,” he writes in his 2024 paper. “It would be a messy, cacophonic, and noisy world, but it could also be less manipulative.”
But that is not the world right now. Currently a handful of AI companies and their chatbots dominate, with hundreds of millions of people interacting with ChatGPT or Claude every month. “If that single bot changes how it talks about Gaza or about Ukraine or whatever, how much can that shift opinion?” Rahwan asks. “How many TV stations do you have to control to be able to achieve the same level of persuasion?”
