The creators of artificial intelligence (AI) chatbot Claude claim to have caught Chinese government-sponsored hackers using the tool to carry out automated cyberattacks against around 30 global organizations.
Anthropic said hackers tricked a chatbot into performing automated tasks under the guise of conducting cybersecurity research.
Company stated in blog post it was “the first recorded cyber espionage campaign orchestrated by an AI.”
But skeptics question the accuracy of this claim and its motives.
Anthropic said it discovered the hacking attempts in mid-September.
Posing as legitimate cybersecurity workers, the hackers gave the chatbot small, automated tasks that added up to a “highly sophisticated espionage campaign.”
Researchers from Anthropic said they had “high confidence” that the people who carried out the attacks were a “Chinese state-sponsored group.”
They said the people selected targets included large technology companies, financial institutions, chemical manufacturing companies and government agencies, but the company did not elaborate.
The hackers then created an unspecified program, using Claude's coding assistance, to “autonomously compromise a selected target with minimal human intervention.”
Anthropic claims that the chatbot was able to successfully hack various unnamed organizations, extract sensitive data and sort through it for valuable information.
The company said it has since banned the hackers from using the chatbot and notified affected companies and law enforcement.
But Martin Zugek of cyber firm Bitdefender said the cybersecurity world had mixed feelings about the news.
“The Anthropic report makes bold, speculative claims but does not provide verifiable intelligence evidence about the threats,” he said.
“While the report highlights a growing issue of concern, it is important for us to gain as much information as possible about how these attacks occur so that we can assess and determine the true danger of AI attacks.”
Anthropic's announcement is perhaps the most high-profile example of companies alleging that attackers are using artificial intelligence tools to perform automated hacks.
It's a danger that many have worried about, but other artificial intelligence companies have also claimed that nation-state hackers have used their products.
In February 2024, OpenAI published a blog post in collaboration with cyber experts from Microsoft that said the company had harmed five government actors, including some from China.
“These participants typically sought to use OpenAI services to query open source information, translate, find coding errors, and perform basic coding tasks,” the firm said at the time.
Anthropic did not say how it came to the conclusion that the hackers in this latest campaign were linked to the Chinese government.
The Chinese Embassy in the United States told reporters it was not involved in the incident.
This comes as some cybersecurity companies have come under fire for overhyping cases of AI being used by hackers.
Critics say the technology is still too cumbersome to be used in automated cyberattacks.
In November, Google cyber experts published a research paper which highlighted growing concerns about hackers using AI to create entirely new forms of malware.
But the paper concluded that these tools were not that successful and were only in the testing phase.
The cybersecurity industry, like the artificial intelligence business, is keen to claim that hackers are using the technology to attack companies to boost interest in their own products.
In its blog, Anthropic argues that the answer to stopping AI attackers is to use AI defenders.
“The very same capabilities that allow Claude to be used in these attacks also make him critical to cyber defense,” the company said.
And Anthropic admitted that its chatbot made mistakes. For example, he made up fake usernames and login passwords and claimed to have extracted classified information that was actually publicly available.
“This remains a barrier to fully autonomous cyberattacks,” Anthropic said.
