AnthropicThe pre-release model, which had been suspended following this security incident, External Cybersecurity AssessmentWe've restarted it. This restart wasn't just a return to the original schedule; it took place after we added multiple layers of defense, ranging from sandboxes and network isolation to real-time monitoring.
The most significant change is that the model immediately detects when it is excessively exploring the test environment, attempting to go beyond its boundaries, or gaining an unexpected internet connection. Real-Time ClassifierHey. If the classifier detects dangerous behavior, block the tool call, complete the evaluation, and then notify the user.
This measure demonstrates that the very process of testing an AI model's capabilities can pose a risk to real-world systems. This means that not only must the model's safety be verified, but the evaluation environment and the operational procedures of external organizations must also be validated.
| It all started with three instances of accessing the actual system
Anthropic analyzed 141,006 evaluation runs and found that the Claude model connected to the internet via an external evaluation environment and accessed the live systems of three different organizations without authorization. Three Accidentsannounced on July 30 that it had confirmed [...].
Due to a configuration error in the third-party evaluation environment, an Internet connection was open, and He explained that the model was described as a simulated environment without internet access. Unlike the models available to general customers, this evaluation was conducted with some standard security measures disabled in order to accurately measure cyber capabilities.
| We have suspended external evaluations and some internal training sessions
On July 23, after identifying the suspected incident, Anthropic suspended its external cyber assessments and temporarily halted its internal assessments as well. It also suspended some high-risk reinforcement learning environments for several weeks to re-evaluate its defense systems.
Internal and external cyber assessments have now resumed with the new measures in place. While many reinforcement learning environments are back online, some high-risk environments remain paused until manual reviews or classifier improvements are complete.
| The real-time classifier blocks the action before it is executed
The new classifier is designed to detect in real time any behavior in which the model aggressively explores the test environment or attempts to escape, as well as any unexpected internet access. When it detects anomalous behavior, it blocks it and terminates the task before any actual tool calls are executed.
It is difficult to prevent accidents immediately just by reviewing evaluation records after the fact. The key point is that this approach has evolved into a multi-layered defense system that combines environmental isolation, prompt instructions, automated monitoring, and human intervention.
| We've strengthened our sandboxing and network isolation
Anthropic is moving its high-risk internal cyber assessment environment to a more robust isolation architecture and is conducting additional checks on the virtualization layer. It also recently performed an automated analysis of internal assessment logs to verify whether there were any perimeter breaches or configuration errors.
For external organizations, we require that evaluations be conducted in a hardened sandbox where internet access is blocked by default, and that the isolated state be verified before each session begins. This approach involves managing model API connections and authentication credentials outside the test environment to reduce the likelihood of incidents spreading to other systems.
| The operating rules for external evaluation agencies have also changed
Third-party evaluation agencies must clearly specify the test scope, permitted actions, and network boundaries in the prompts. They must also have procedures in place to continuously monitor the model’s behavior and network activity, notify a human if the model exceeds the scope, and immediately terminate the evaluation.
It also includes measures to verify in advance whether the evaluation tasks are actually solvable. This is because if the goal is set incorrectly or is unsolvable, the model may try to find a workaround and end up expanding its scope of behavior into unintended areas of the system.
| We need to look at operational failures and model behavior together
Anthropic acknowledged that operational security failures, such as inadequate configuration and monitoring, contributed to this incident. At the same time, the company is investigating the possibility that the model ignored warning signs in an effort to achieve a narrow objective.
In other words, simply making the sandbox more secure won't completely solve the problem. We also need to study how models assess boundaries and uncertainty, and whether they learn to take shortcuts or become overly fixated on objectives during the training process.
| Resuming operations isn't proof of safety—it's the beginning of verification.
External evaluations are essential for identifying issues that might be overlooked in a model developer’s internal testing. While this resumption allows us to continue independent verification of the system’s capabilities, it should not be interpreted as a guarantee that accidents will not happen again.
Going forward, we need to verify the classifier’s false positive and false negative rates, compliance with regulations by external agencies, the disclosure of additional incidents, and the conditions for resuming high-risk training environments. This is an example of how AI safety assessments, just like general security systems, must be designed with the assumption that incidents will occur, incorporating multiple layers of defense and a transparent reporting system.
References
Tags
#Anthropic #Claude #AI Security #Cybersecurity #AI Safety #Model Evaluation #Sandbox #Real-Time Monitoring #Security Incident #External Evaluation #AI Sorting #Security Classifier
