BEIJING — The United States and China are both gripped by safety fears as they reckon with their rapid advances in AI, but intensifying competition and deep mutual distrust could derail the kind of cooperation experts say is vital to head off the strategically vital technology’s greatest dangers.
Artificial intelligence is increasingly at the center of the rivalry between Washington and Beijing, and it is expected to be a major topic of discussion at a summit in Washington next week between President Donald Trump and Chinese leader Xi Jinping.
The summit comes as AI leaders and researchers in the U.S. warn that companies should “slow the pace” of AI development, citing recent incidents involving autonomous agents. Trump has dismissed such concerns as a “hoax” while agreeing with executives such as Anthropic CEO Dario Amodei that the U.S. must maintain AI dominance over China through export controls and other measures.
FOR SUBSCRIBERS
How exactly could AI cause widespread danger?
China, which tightly regulates AI and largely avoids the existential worries the technology provokes in the U.S., has interpreted calls to slow its development as targeted squarely at its own progress. U.S. companies say the Chinese AI sector is being driven in part by large-scale “illicit distillation” of their most advanced models.
Chinese officials, who say distillation is a conventional practice that U.S. companies also use on Chinese models, in turn accuse America’s AI executives of trying to stave off competition.
Senior Col. Zhu Qichao, a noted voice on Chinese military AI and national security, called for “cooperation” with the U.S. while warning against “vicious competition.”
“Because with the development of emerging technologies, it definitely comes with many possibilities,” said Zhu, the director of the Institute for Defense Technology and Strategic Studies at the People’s Liberation Army National University of Defense Technology.
“On one hand, it benefits humanity, but on the other hand, there are also considerations regarding risks,” he told NBC News on Tuesday at the Xiangshan Forum in Beijing.
AI and the worries around it were a major focus at the annual defense forum, where rooms were packed with scholars, military officials and commentators from the U.S. and other countries, many of which are being pressured to pick sides in the U.S.-China AI race.
Though Trump and Xi agreed at their May summit in Beijing to hold high-level AI talks, no such talks have been publicly confirmed ahead of Xi’s visit next week.
Wu Xinbo, dean of the Institute of International Studies at Fudan University in Shanghai, said China was wary that the talks could be used to further slow its progress.
“On the other hand, I think both sides need to sit down to talk about the risks posed by AI development,” he said at the forum, adding that “it will take a long way before the two sides can really forge some important consensus on this issue.”
For U.S.-China cooperation to work, the U.S. needs to be open to both sides benefiting, said Francesca Giovannini, executive director of the Harvard Belfer Center’s Initiative on Managing the Atom.
“The United States operates on a very specific type of theory. And the theory is, whoever wins the AI race wins everything,” she said in an interview at the forum. “So if this is your zero-sum game idea, you won’t be conceding anything.”
China is unlikely to agree to a slowdown at the behest of U.S. AI companies, said George Chen, a partner and chair of digital practice at the Asia Group, an American strategic advisory firm based in Washington.
“I don’t think what happens in the U.S. really matters to China,” he said Tuesday in a phone interview from Hong Kong. “They want to do AI at their own pace, and to a very large degree a lot of people in the industry, in the government, they have a very proud sense that this is finally the chance for China.”
China, which sees its “open-weight” AI models as a tool for diplomacy, has promoted the technology as a global “public good” that should be managed by the international community. At a summit in India on Sunday, Xi offered the 11-member BRICS group of nations help developing open-source AI and urged them to join China’s global AI governance initiative.
But China is also keenly aware of the risks it faces from its own AI success, with Xi telling a Shanghai conference in July that authorities must “ensure that AI is always under human control.”
The industry must make mandatory filings and submit security assessments to officials, with many of the companies having to submit technical filings to an algorithm registry. The government has also imposed rules requiring watermarking and safety requirements.
Foremost among the risks are potential threats to China’s ruling Communist Party, which worries the technology could be weaponized against it and has imposed ideological controls to ensure products fall in line with socialist values, for example.
“Hostile forces and individuals with ulterior motives misuse generative AI technologies — such as deepfake audio-video, AI-generated graphics and text, and intelligent bot farms — to manufacture political rumors, spread harmful information, and incite confrontation at low cost and on a massive scale,” Chen Yixin, the Chinese minister of state security, said in an article Sunday in the state-backed China Cybersecurity Magazine that called for greater party control and government oversight of AI.
Chen said AI could enable misinformation campaigns and cyberattacks on China’s critical infrastructure and, in an apparent reference to the U.S., warned that countries with “advantages” in AI could restrict other nations’ access to their technologies.
He also warned of espionage and data leaks, days after Anthropic released a report that found Chinese AI labs had secretly routed requests to its Claude models, some of which contained sensitive information from individuals, major companies and state-affiliated actors.
Like the U.S., China increasingly views AI as a national security issue, down to the individual researchers working on it. On Tuesday, China tightened travel curbs on its citizens, barring some whose work has implications for national technology security from leaving the country.
The new rules add to an extensive AI regulatory regime that includes restrictions on AI companions over mental health concerns, mandated labeling of AI-generated content and a requirement that trained models must use data conforming to Chinese values.
Experts say it could take a major crisis for the U.S. and China to finally work together on AI, much like the 1962 Cuban missile crisis pushed the U.S. and the then-Soviet Union to start negotiating on nuclear weapons.
AI has already had profound military implications, including for targeting, and could be integrated into systems supporting the nuclear chain of command, said David Finkelstein, vice president of the Center for Naval Analyses.
“This is scary stuff, and I think that this is an area where the two sides ought to be sitting down and talking to each other,” he said at the forum.
U.S.-China competition is inevitable, but there is still room for the two sides to negotiate because their respective positions “are not necessarily very much entrenched,” said David Santoro, president and chief executive of Pacific Forum International, a Honolulu-based think tank.
“There are still ways that we can better understand each other and hopefully come up with mechanisms so we can better control this emerging technology,” he said in an interview at the forum.
Santoro said he was “not very optimistic,” however, about the prospect for a major agreement in the near term.
“I don’t know that we will be able to get to a position where we agree on something substantive,” he said. “But at least the first step has got to be talking to each other.”
Eric Baculinao and Erin Tan reported from Beijing and Mithil Aggarwal from Hong Kong.
