Anthropic CEO Dario Amodei on the pace at which the capabilities of state-of-the-art AI models are improving A three-step framework for adjusting safety measures to a level that can be effectively implementedI proposed this. It's not an argument for stopping model training, but rather an approach to ensure we have enough time for alignment, security, and external validation.
The proposal consists of three stages: ongoing external oversight, common standards among companies in democratic nations, and international coordination. Among these, Anthropic has pledged to be the first to implement the first stage, which involves granting external evaluators ongoing access comparable to that of internal employees.
Anthropic cannot carry out the remaining two stages on its own. This is because competitors, governments, and other countries must be involved, so This should be viewed as a policy proposal announced by the CEO, rather than an agreed-upon rule or international agreement.
| Why Did He Call for a Slowdown Now?
Amodei recently noted that as AI directly contributes to the development of the next generation of AI, the pace of its advancement is accelerating. The concern is that as models become more deeply involved in coding, research, and the automation of experiments, safety research may not be able to keep up with these advancements.
He also cited recent cases in which several AI agents accessed systems outside the evaluation environment or exhibited unintended behavior. He explained that since issues have been reported even in Anthropic’s own models, this cannot be viewed as the failure of just one specific competitor.
He argued that the time gained by slowing down should be used to improve operational stability, conduct alignment research, enhance interpretability to understand the inner workings of the model, and perform more rigorous testing and evaluation. Gaining time is not an end in itself, but rather That means the goal is to tailor the safety system to the appropriate level of capability.
| Step 1 involves having a permanent external evaluator.
The first step involves permanently stationing an independent third-party evaluation team within the AI company. Rather than a one-time audit that only tests the final model, this structure continuously verifies that the training pipeline, risk assessment procedures, and incident response are operating as promised.
Anthropic stated that it plans to provide evaluators with office space, access privileges, and company equipment, and grant them tools and permissions generally similar to those of the internal risk assessment team. Restrictions may be imposed to protect legal and contractual matters, as well as the personal information of customers and partners.
External evaluators should be able to disclose the level of risk, incidents, the company's safety practices, and the access privileges they were actually granted. Anthropic may narrowly redact security, legal, and commercially sensitive information, but I explained that I wouldn't edit the conclusion just because it was unfavorable to the company.
| Phase 2: Common Standards Within Democratic Nations
The second step is a plan for cutting-edge AI companies in democratic nations to jointly establish common safety standards and rules to manage the unlimited pace of technological advancement. The logic is that if only one company were to slow down, it could put itself at a competitive disadvantage, so multiple companies must apply the same standards.
Amodei states that once a model reaches a certain risk threshold, measures such as alignment assessments, interpretability analyses, and learning environment audits, such as A checkpoint system in which users proceed to the next stage after meeting safety requirementsI used that as an example. It's a proposal to evaluate both the actual model's capabilities and its safety status, rather than simply limiting the computational resources used for training.
To ensure that inter-company consultations are not mistaken for collusion, the government may need to mediate the discussions or establish an exception under competition law that is limited to safety cooperation. No specific legislation or common standards have been established yet.
| Step 3 involves international coordination, but verification is a major challenge.
The final step is a plan for the United States and other democratic nations to coordinate AI risk management measures even with authoritarian countries, including China. It was suggested that cooperation of varying intensity—such as banning dangerous uses, conducting common pre-release testing, and imposing limits on the pace at which AI accelerates its own development—could be considered in stages.
While Amodei believes international cooperation is necessary, he acknowledged that it is difficult to verify whether the other side is honoring its commitments. If only one side delays development, the military and economic balance could shift. They assessed that a comprehensive ceasefire agreement without verification mechanisms is unlikely to be feasible.
He argued that, to maintain the technological edge of the United States and its allies, controls must also be implemented on the export of cutting-edge AI chips, circumventing supply chains, model distillation, and the theft of model weights. This section strongly reflects not only universal safety principles but also a U.S.-centric geopolitical perspective.
| OpenAI and xAI also agreed with the intent.
According to Reuters, OpenAI's Sam Altman and xAI's Elon Musk have publicly stated that they agree with Amodei's rationale for implementing speed limits. Altman praised the concept of independent evaluators with employee-level access as a good idea and announced that OpenAI would take similar measures.
However, it has not yet been disclosed which specific evaluation agencies each company will select or to what extent they will allow internal access. A brief consent message Nor does it signify a formal joint agreement on Anthropic's entire three-phase plan.
For external evaluations to be effective, a balance must be struck between the evaluator’s independence and expertise, the right to disclose findings, and the protection of trade secrets. Even if the company bears the cost of the evaluation, What matters is whether the structure makes it impossible to prevent unfavorable outcomes.
| The key question is whether this will evolve from a promise into a system.
In this announcement, The most concrete commitment is Anthropic's pledge to bring in external evaluators on an ongoing basis. Common standards among companies and international coordination are still in the proposal stage, and the participating entities, timeline, and implementation methods have not yet been determined.
The phrase "speed control" also It's not the same as discontinuing the model. Anthropic's position is to continue learning and advancing technology while allocating sufficient time to safety measures and third-party verification in line with improvements in capabilities.
From now on We need to verify the evaluation team's selection process, their actual access privileges, the scope of the public report, and whether other AI companies are participating. Only when these elements are fully defined can we conclude that this proposal has evolved from a voluntary corporate declaration into a verifiable industry standard.
Tags
#Anthropic #DarioAmodei #AI Safety #AI Speed Control #External Evaluation #AI Sorting #FrontierAI #AI Regulation #Model Evaluation #AI Governance #OpenAI #xAI #Artificial Intelligence Policy
