Takeaway: Since the July, 2024 meetings between American and Chinese leaders, the most significant updates on catastrophic AI risks center on renewed bilateral dialogue, shared recognition of frontier‑model dangers, and mounting geopolitical tensions that threaten cooperation. The picture is a mix of progress and fragility.

Here are updates since the two nations’ Leaders met in July 2024

1. A formal U.S.–China AI risk‑reduction dialogue is now underway

Following subsequent summits (including the May 2026 Trump–Xi meeting), both governments agreed to establish an intergovernmental protocol focused on preventing catastrophic misuse of advanced AI models.

• The protocol aims to ensure non‑state actors cannot access frontier models, especially those capable of cyberattacks, autonomous escalation, or biological threats.
• The dialogue is intentionally technical rather than political, emphasizing best practices, incident reporting, and model‑safety testing rather than arms‑control‑style limits.

During the May 2026 summit in Beijing between U.S. President Donald Trump and Chinese President Xi Jinping, both nations agreed to establish new bilateral channels and governance mechanisms to stabilize relations.  

The primary outcomes regarding economic oversight and artificial intelligence include:

• Board of Trade and Board of Investment: The two governments agreed to set up a new Board of Trade and a Board of Investment to manage non-sensitive trade and cross-border investment flows. Chaired on the U.S. side by Treasury Secretary Scott Bessent, the Board of Investment aims to resolve commercial frictions and prevent non-sensitive cross-border investments from triggering lengthier interagency national security reviews.  
• Bilateral AI Governance & Risk Protocols: Rather than setting up a standalone joint “AI board,” the administration announced plans to establish a formal bilateral AI governance channel and safety protocol. This protocol primarily focuses on preventing non-state actors or terrorist groups from accessing advanced frontier AI models, particularly those with dangerous capabilities in cyber operations or biological threats.  

While both leaders agreed to work on shared AI safety standards and strategic stability, core technological disputes—such as export controls on high-end semiconductor chips—were largely bypassed during the bilateral summit in favor of near-term commercial stability and safety coordination.  

Updated catastrophic‑risk categories both nations now acknowledge

Across U.S., Chinese, and multilateral frameworks, four catastrophic AI risks have become consensus priorities:

• Cyber escalation — frontier models enabling autonomous cyber operations or destabilizing critical infrastructure.
• Biological/chemical uplift — AI systems lowering barriers to designing or deploying dangerous biological agents.
• Mass persuasion & destabilization — large‑scale manipulation, political interference, or social‑cohesion disruption.
• Unpredictable autonomous systems — AI agents acting outside human control, including military escalation risks.

Both countries now agree these are the “deadliest risks,” even amid broader geopolitical rivalry.

Frontier‑model thresholds and testing standards

A global rough consensus has emerged defining “frontier AI” at 10²⁶–10²⁷ FLOPs, with shared expectations that such models must undergo:

• Pre‑deployment dangerous‑capability testing
• Rapid incident reporting for theft, misuse, or autonomous behavior These standards are now mirrored in U.S. executive orders and Chinese lab practices.

New tensions threatening cooperation (2025–2026)

Despite progress, several developments since 2024 have increased the risk of breakdown in bilateral safety efforts:

1. Accusations of model theft and export‑control violations

U.S. officials accused Chinese lab Moonshot AI of distilling its Kimi K3 model from Anthropic’s Fable 5 frontier model.

• Washington is considering sanctions.
• China is weighing restrictions on overseas access to its models. These moves could derail planned AI‑safety talks.

2. Open‑weight model concerns

Chinese firms favor open‑weight models, which U.S. officials argue are harder to secure and easier for malicious actors to exploit. This raises the risk of frontier‑model proliferation, a key catastrophic‑risk vector.

3. Chip‑access investigations

The U.S. is investigating whether Chinese companies are illegally accessing advanced U.S. chips to train frontier models—another flashpoint that could collapse cooperation.

Global convergence despite rivalry

Even with rising tensions, the U.S., China, EU, India, Brazil, South Korea, and UAE are converging on the need to mitigate catastrophic AI risks.

• Multiple nations have published frontier‑risk frameworks or passed laws.
• The U.S.–China protocol to keep advanced models out of terrorist hands is a major sign of alignment.

This convergence is one of the most important developments since 2024: the world’s major powers now agree on the nature of catastrophic AI risks, even if they disagree on almost everything else.

What this means going forward

Progress:

• Shared recognition of catastrophic risks
• Formal bilateral dialogue
• Emerging global standards for frontier‑model testing and incident reporting

Fragility:

• Sanctions threats
• IP‑theft accusations
• Export‑control disputes
• Open‑weight model disagreements

The overall trend since August 2024 is increasing urgency and cooperation on catastrophic risks, but increasing geopolitical volatility that could undermine that cooperation.

High‑level timeline of U.S.–China AI‑safety interactions (2024–2026)

Year	Event / Channel	Main AI‑safety themes
2024	San Francisco leader‑level engagement & follow‑on dialogues	Frontier risks, nuclear command, Track 1.5 safety talks
2025	Ongoing Track 1.5 / Track 2 dialogues, research networks	Shared technical risk research despite policy chill
2026	Trump–Xi summit & formal AI governance channel	Military AI, nuclear command, frontier‑model governance

2024 – From leader symbolism to Track 1.5 safety talks

November 2024 – Joint nuclear‑command AI statement

• What happened: Washington and Beijing issued a joint statement affirming that humans—not AI—must control decisions to use nuclear weapons.
• Why it matters:
  • First explicit bilateral commitment on a catastrophic AI risk (nuclear escalation).
  • Became the political anchor for later AI‑safety dialogues.
• President Joe Biden’s meeting with Xi Jinping in San Francisco and the surrounding APEC summit that attracted world leaders, CEOs and abundant protesters.

2024 – Track 1.5 / Track 2 AI‑safety dialogues

• Tsinghua and other Track 2 channels began building shared vocabulary on frontier risks, including cyber, bio, and autonomous systems.
• These dialogues kept the nuclear‑AI commitment “alive” and expanded it into broader catastrophic‑risk framing.

2025 – Policy chill, but research cooperation intensifies

Early–mid 2025 – U.S. deregulation vs. Chinese safety research

• U.S. shift: Revocation of Biden’s EO 14110 and pivot toward a more deregulatory, competition‑focused AI posture.
• China’s move: Launch of the China AI Development and Safety Research Network, plus growing participation in international AI‑safety research forums.

Key dynamic:

• Policy cooperation cooled, but technical AI‑governance research cooperation grew, especially via joint workshops and glossaries on frontier risks.

2026 – Formal AI governance channel, rising military‑AI focus

March 10, 2026 – Fudan University workshop

• U.S.‑based Safe AI Forum and Chinese scholars compared domestic frontier‑AI regulatory mechanisms and discussed international coordination.
• Helped feed into the case for a formal bilateral AI governance channel.

May 14–15, 2026 – Trump–Xi summit in Beijing

• Outcome: Leaders agreed to improve policy exchanges on AI and announced a bilateral channel on AI governance, with strong emphasis on military AI and nuclear command.
• This channel is framed as an implementation step for the 2024 nuclear‑AI statement—moving from symbolism toward structured dialogue.

Late May 2026 – Shangri‑La Dialogue, Singapore

• Chinese delegation publicly supported rules for military AI, including potentially legally binding ones, and referenced the new bilateral channel.
• U.S. officials echoed the need for restraint and keeping AI out of nuclear decision‑making, even as broader strategic competition intensified.

2026 – Global AI‑governance summits and widening U.S.–China divide

Summer 2026 – AI summits (G7, UN, WAIC, ITU)

• Multiple global summits (G7 Évian, UN Global Dialogue on AI Governance, WAIC in Shanghai, ITU AI for Good) showcased competing governance visions.
• China promoted building a World AI Cooperation Organization, while the U.S. leaned into a more voluntary, security‑focused posture.

The UN Global Dialogue on AI Governance in Geneva and the World Artificial Intelligence Conference (WAIC) in Shanghai capture high-level international policy launches and major robotics exhibitions.

UN Global Dialogue on AI Governance (Geneva)

• Opening Session Dates: July 6–7, 2026, held at the Palexpo Conference Centre in Geneva, Switzerland.
• Key Visuals: Official imagery features UN Secretary-General António Guterres addressing the plenary, alongside heads of delegations discussing global safety frameworks and military AI risks.
• Photo Archives: High-resolution and editorial event pictures are hosted via the UNIS Geneva Flickr Album and the UN Photo Library. [1, 2, 3]

WAIC Plenary & Exhibition (Shanghai)

• Event Dates: July 17–20, 2026, themed “AI Partnership for a Brighter Future” across Shanghai venues.
• Key Visuals: Plenary sessions, humanoid robot global debuts (such as Unitree and Tesla displays), and interactive tech floors.
• Photo Archives: Editorial galleries covering the opening plenaries and exhibition floors are available on Getty Images Editorial and professional wire services like Reuters Connect.

July 2026 – Research‑driven cooperation persists

• Analyses highlight that AI governance research cooperation (Brookings–Tsinghua, Concordia AI, International Dialogues on AI Safety) remains dense despite political strain.

Research-driven international AI governance relies on persistent, multi-tiered cooperation frameworks that link independent scientific assessments with cross-border policy dialogues.

Global Scientific Infrastructure

• Scientific Panels: The United Nations established the Independent International Scientific Panel on Artificial Intelligence to study risks and opportunities without direct regulation.
• Global Dialogues: Findings from scientists feed into the UN Global Dialogue on AI Governance, which began its sessions in Geneva in July 2026.
• Inclusive Participation: Frameworks aim to bring in nations from the Global South to close persistent policy and representation gaps.

Resilient Researcher Networks

• Cross-Border Ties: Personnel networks from institutions like Oxford’s Centre for the Governance of AI create stable professional connections.
• Technical Collaboration: Researchers independently develop safety benchmarks, evaluation tools, and risk assessments when formal state consensus stalls.

Framework / Initiative	Type	Scope	Enforcement	Key Focus Area
OECD AI Principles	Policy Guidelines	International	Voluntary	International policy alignment & trustworthy AI values
UNESCO Recommendation on Ethics of AI	Ethical Framework	Global	Voluntary	Human rights, inclusion, and ethical foundations
NIST AI Risk Management Framework (AI RMF 1.0)	Risk Management	Organizational	Voluntary	Practical technical risk management across AI lifecycle
ISO/IEC 42001:2023	Management System	Global	Certifiable	Certifiable AI management system standards
EU Artificial Intelligence Act (2024)	Binding Regulation	European Union	Mandatory	Risk-based legal compliance and market obligations
Framework Convention on AI (Council of Europe)	International Treaty	International	Legal Treaty	Aligning AI with human rights, democracy, & rule of law
Global Partnership on AI (GPAI)	Multistakeholder Initiative	Global	Voluntary	Global research, policy cooperation, and best practices
IEEE 7000-2021 Standard	Engineering Standard	Global	Voluntary	Process model for embedding ethics into system design
Singapore Model AI Governance Framework	Practical Guidance	National / Business	Voluntary	Actionable, business-friendly AI deployment practices
Hourglass Model of AI Governance	Organizational Framework	Organizational	Voluntary	Operationalizing ethical principles across the AI lifecycle

Summary: Navigating AI Catastrophic Risks: U.S. and China Dialogue Updates

Efforts between the United States and China to manage the existential and operational dangers of artificial intelligence have evolved from high-level symbolic commitments into structured, technical risk-reduction protocols. Key developments and structural updates include:

• Formal Risk-Reduction Protocol: Bilateral dialogues—strengthened following leader-level engagements like the May 2026 summit—established intergovernmental mechanisms to prevent advanced frontier AI models from falling into the hands of non-state actors or terrorist networks.
• Consensus on Catastrophic Risk Vectors: Both powers formally recognize four core threat areas: autonomous cyber warfare, biological/chemical uplift, large-scale political destabilization, and unaligned military/autonomous system escalation.
• Shared Standards for Frontier Models: A global technical alignment has coalesced around defining “frontier AI” at high compute thresholds ($10²⁶ – 10²⁷ FLOPs), requiring mandatory pre-deployment safety evaluations, dangerous capability testing, and incident-reporting protocols.
• Operationalizing Nuclear Guardrails: Grounded in the 2024 joint commitment affirming human control over nuclear command systems, discussions have expanded into managing risks associated with algorithmic decision loops and military AI.

Despite technical alignment on severe risks, this cooperation remains fragile. Disputes over U.S. chip export controls, mutual accusations of model theft, differences in open-weight AI distribution, and broader geopolitical rivalries continually threaten to disrupt ongoing safety talks.

---

Conclusion

While deep strategic rivalry and economic friction continue to hamper comprehensive technology agreements, the U.S. and China have established a pragmatic, safety-focused floor for AI governance. Both nations demonstrate a shared understanding that catastrophic risks—such as proliferation of weapons of mass destruction or unintended autonomous escalation—pose a mutual existential threat that transcends trade wars and geopolitical division. Moving forward, the durability of global AI stability will depend on whether both powers can insulate these technical safety guardrails from broader political and technological disputes.