Anthropic and Accenture have agreed to invest at least $1 billion each over the next five years to build the capacity for independent evaluation of frontier AI models. The total investment will be at least $2 billion, and Faculty—a specialized AI division acquired by Accenture—will lead the collaboration.
The key to this collaboration is the "embedded evaluation" approach, in which external evaluators are granted access rights similar to those of employees within the AI company. The plan is not just to review the results once the model is complete, but to continuously monitor the training and deployment processes, as well as safety decision-making and internal operations.
| Bringing External Evaluators into the Development Process
Traditional independent safety assessments often involved testing a completed model or a limited interface over a certain period of time. In such cases, it can be difficult for the assessor to fully understand what decisions were made during model development or how known vulnerabilities were addressed.
The embedded evaluator is granted access comparable to that of employees within an AI company, allowing it to observe the model training process and deployment decisions. The goal is to communicate directly with developers to verify that the company is adhering to the safety measures it promised and to identify risks that internal staff may have overlooked.
| Sorting Evaluation
The faculty plans to conduct red team testing, alignment assessments, and safety mechanism verification on the Anthropic model. Red team testing refers to a process of examining, from an attacker’s perspective, the possibility that the system could fail or be exploited in ways not anticipated by the system.
Accenture's experience in supporting AI adoption by businesses and governments will also be factored into the evaluation. Understanding the real-world environment in which an organization uses the model can help identify operational risks that aren't apparent from laboratory performance alone.
| At least $2 billion
The two companies plan to invest at least $1 billion each over the next five years to build capacity. The key difference is that, rather than a one-time model review contract, significant funding has been allocated to establish an evaluation team, methodology, and long-term operational framework.
Anthropic stated that, since it currently has no joint funding or government funding system in place, it will cover the costs of Faculty evaluations directly. At the same time, it is discussing pilot collaborations with nonprofit evaluation organizations such as METR, utilizing its own funds.
| Independence
While evaluators can make more accurate judgments if they have deep access to internal information, the conflict of interest—where the company being evaluated pays the costs—remains. Actual independence may vary depending on whether unfavorable results can be disclosed and who determines the scope of the report.
Anthropic also acknowledged that there are still no common standards regarding what information to provide and how to report findings. We are now at the stage where we need to define specific measures regarding access rights, confidentiality, disclosure obligations, incident reporting, and safeguards for evaluators.
| Methods for Disclosing Evaluation Results
Anthropic stated that the embedded evaluator can verify the company's safety commitments, report incidents, and explain the risks and benefits to the public. However, there are no established rules yet regarding the extent to which this information should be disclosed.
It must be possible to verify the reliability of the evaluation externally while protecting sensitive model information and security vulnerabilities. Standardized summary reports, disclosure standards for significant events, and common measurement methods among evaluation agencies must be implemented so that investment can translate into public trust.
| This collaboration is not limited to just one company
This partnership is non-exclusive. Anthropic is collaborating with multiple evaluation agencies, and Accenture can play a similar role to other AI developers. This means there’s a possibility that a single evaluation agency could assess multiple companies and compare common risks and best practices.
Conversely, if evaluation capabilities are concentrated in a small number of large consulting firms, a new dependency structure could emerge. That’s why we need a multi-layered ecosystem involving nonprofit organizations, academia, and government evaluation agencies.
| Scaling Up AI Safety Assessments
The confirmed details include a plan for Anthropic and Accenture to each invest at least $1 billion over five years, with Faculty leading an internal, independent evaluation. The scope of the evaluation, the extent of disclosure, and the performance metrics are still being finalized.
This announcement demonstrates that AI safety assessment is evolving from a one-time review into an industrial function that requires a dedicated organization and a substantial budget. Going forward, success will depend not on the amount of investment, but on whether evaluators actually have the authority to independently disclose significant findings and verify corrective actions.
Tags
#Anthropic #Accenture #Faculty #AI Safety #AI Evaluation #Red Team #AI Sorting #FrontierAI #Independent Evaluation #AI Governance #Industrial Policy #METR #Claude
