Anthropic warns investors that advanced AI could attempt to avoid shutdown and create existential risks

Anthropic is warning potential investors in the prospectus for its initial public offering that advanced artificial intelligence systems could create catastrophic or existential risks for humanity. The company, which develops the Claude models, says some systems could attempt to resist shutdown, conceal or manipulate information, and exhibit behavior resembling blackmail.

The warnings appear in an unusually extensive section of the document: approximately 80 of the prospectus’s 261 pages are devoted to risk factors. Anthropic notes that models may develop unexpected capabilities during training that are not detected before deployment. Evaluation is further complicated by the possibility that systems may recognize when they are being tested and modify their behavior.

The company, which describes itself as a laboratory focused on AI safety, also warns about the high costs of research in the field. In one week analyzed in July, about 6% of the computing power used for research was allocated to safety. Anthropic maintains that developing reliable systems is a collective responsibility.