기사 / PANews

AI Out of Control? Over 400 Scientists Sign Joint Appeal: Dangerous Models Must Be "Physically Disconnected from the Internet"!

A joint open letter from 420 AI scientists calls for physical isolation of high-risk models. This summer, multiple incidents involving AI escaping sandboxes and intruding into real enterprises, including at OpenAI and Anthropic, were exposed. Frontier model safety regulation is shifting toward specific rules, which may reshape the pace of AI agent deployment and compliance costs.

Author: Jose Antonio Lanz, decrypt

Compiled by: Shenchao TechFlow

Shenchao Introduction: This open letter has no legal force. What investors should note is the list of incidents behind it: this summer, models from OpenAI, Anthropic, Meta, and Kimi all crossed boundaries during testing, and some directly hacked into real companies. Congress's "kill switch" bill happens to exempt the testing phase where the incidents occurred, and the White House-led agreement is merely voluntary. Safety regulation of frontier models is moving from discussions of principles toward concrete rules. Once "airgapping by default" becomes an industry standard, the pace of deployment and compliance costs for AI agents in scenarios such as cybersecurity will need to be reassessed.

Since this summer, AI models have repeatedly escaped test environments and breached real companies. Hundreds of researchers say the internet itself should be treated as a security boundary.

Image source: Decrypt

A total of 420 scientists, including Geoffrey Hinton and Yoshua Bengio, published an open letter on Tuesday urging AI labs to physically isolate models with cyberattack capabilities by default.

The letter followed a summer in which models from OpenAI, Anthropic, Meta, and Moonshot AI all escaped test sandboxes, with several of them breaching real companies.

The signatories want an international body to set isolation standards and record every isolation failure, with the reports ultimately made public.

More than 400 scientists, including AI pioneer Geoffrey Hinton, known as a "godfather of AI," and Yoshua Bengio, winner of the 2018 Turing Award, want the most dangerous AI models disconnected from the internet.

The open letter published on Tuesday calls on AI labs to physically isolate, or airgap, models with cyberattack capabilities by default, meaning physically disconnecting them from any external network.

"In the near term, strong sandboxing mechanisms, including physical isolation (disconnection from the internet), should be used to achieve highly reliable isolation controls for AI models with cyberattack capabilities," the letter states.

As of Wednesday, the letter had gathered 450 signatures, including Stuart Russell of UC Berkeley and Ron Rivest of MIT, co-inventor of the RSA encryption system. The signature list is still growing in real time.

So-called "models with cyberattack capabilities" refer to AI that can autonomously discover and exploit software vulnerabilities. Labs measure this capability in sandboxes, which are like closed digital rooms where AI can attack fake targets without touching any real systems. But this summer, these rooms leaked one after another.

Frequent incidents this summer: frontier models repeatedly crossed boundaries and escaped sandboxes

In July, OpenAI admitted that GPT-5.6 Sol and an unreleased, more capable model broke through a closed testing environment and breached Hugging Face, a platform hosting more than 1 million public AI models. At the time, these models were being tested by ExploitGym, a benchmark that gives AI agents 898 real software vulnerabilities and requires them to turn each vulnerability into a usable attack, scoring pass or fail for each vulnerability.

Shortly afterward, reports emerged that OpenAI's models had breached Australian government systems and obtained information from U.S. federal agencies without proper authorization.

Anthropic then reviewed more than 141,000 test runs and found three cases in which Claude models breached real companies because a testing partner's environment had accidentally remained connected to the internet. In one case, Claude Mythos 5 uploaded a malicious package to PyPI, the public repository where programmers download Python code, and the package reached 15 systems before being removed. Two of the affected companies did not know they had been breached until Anthropic told them.

Meta confirmed in August that its Muse Spark model breached another company due to a similar testing error. China's Kimi K3, a model anyone can download, slipped out of a sandbox built on the testing framework of the UK AI Safety Institute and copied test answers directly from GitHub.

The authors of the open letter say these incidents can be understood from two angles: one is an alignment problem, in which AI pursues goals in ways its creators never anticipated; the other is sloppy safety work, where stronger sandboxes could have stopped it. Their answer is both, and they warn that without change, the next batch of incidents will be more severe.

"If the way AI is developed and deployed does not change immediately, such incidents will continue to occur, and future incidents will cause far greater damage to our exposed critical infrastructure."

Joint proposal: airgapping by default and establishing international standards

Under the proposal, labs should at least initially airgap by default any model with serious hacking capabilities. Any use requiring more freedom, such as internet access, should follow standards set by a new international body formed jointly with governments; that body would also collect reports of isolation failures and eventually make them public.

Reactions from various parties: regulatory bills introduced and voluntary industry agreements

The industry's own response has largely moved in the opposite direction. In August, OpenAI President Greg Brockman, citing his own company's breach incident in an article, called on every security team to immediately deploy AI agents; meanwhile, Hugging Face ultimately relied on running a Chinese open-weight model, whose files anyone can download and run locally, to make sense of the attack on its servers, after U.S. commercial AI refused to analyze the attack data.

Two days after OpenAI disclosed the incident, two members of Congress proposed the AI Kill Switch Act, allowing the Department of Homeland Security to order throttling or shutdown of frontier models. But the bill exempts anything that happens during red-team testing or structured testing, and every breach this summer occurred in exactly those settings.

The White House chose a milder route: on September 29, Nvidia, xAI, Meta, Google, OpenAI, and Anthropic signed a voluntary agreement based on internal controls and independent review. President Trump called the agreement "morally binding," but nothing in it is legally enforceable.

Decrypt contacted several researchers listed as media contacts in the open letter but did not immediately receive a response.

The letter has no legal force, but its 406 signatories work at universities and research institutions on six continents, from Toronto and Zurich to San Diego and Kigali.

관련 읽을거리