Anthropic Details Safeguards for User Wellbeing: Crisis Detection, Anti-Sycophancy, and Evaluation Results
Anthropic has published a detailed account of its user wellbeing safeguards, covering how Claude handles suicide and self-harm conversations through model training, system prompts, and a real-time crisis classifier integrated with ThroughLine's global helpline network. The post discloses evaluation results for Claude Opus 4.5, Sonnet 4.5, and Haiku 4.5, showing 98–99% appropriate response rates on high-risk single-turn prompts and very low false-refusal rates on benign requests. Anthropic also addresses anti-sycophancy efforts and an 18+ age requirement for Claude.ai. The company is partnering with the International Association for Suicide Prevention (IASP) to further inform training and product design.
Related guides (4)
Related events (8)
Anthropic Details Claude Safeguards Team Structure and Multi-Layer Safety Approach
Anthropic has published a detailed overview of its internal Safeguards team, describing a multi-layer approach to preventing Claude misuse that spans policy development, model training influence, pre-deployment evaluation, and real-time enforcement. The team uses a Unified Harm Framework covering five dimensions (physical, psychological, economic, societal, autonomy) and conducts Policy Vulnerability Testing with external domain experts in areas like terrorism, child safety, and mental health. Pre-deployment evaluations include safety assessments, CBRNE-focused AI capability uplift testing with government partners, and bias evaluations. The post describes specific partnerships with organizations like the Institute for Strategic Dialogue and ThroughLine to inform election integrity and mental health response policies.
Anthropic Study: Affective Conversations Comprise 2.9% of Claude.ai Usage
Anthropic published a large-scale analysis of how users engage with Claude for emotional support, advice, and companionship, drawing on 131,484 affective conversations identified from ~4.5 million Claude.ai Free and Pro interactions. Key findings: only 2.9% of conversations are affective in nature, companionship and roleplay combined account for under 0.5%, and user sentiment generally becomes more positive over the course of coaching and counseling exchanges. The study used Anthropic's privacy-preserving Clio analysis tool and aligns with similar low-rate findings from OpenAI and MIT Media Lab research on ChatGPT. Anthropic frames this as part of its safety mission to understand and mitigate potential harms from AI emotional engagement, including unhealthy attachment and emotional exploitation.
Anthropic introduces computer use capability, upgraded Claude 3.5 Sonnet, and Claude 3.5 Haiku
Anthropic announced three major developments: an upgraded Claude 3.5 Sonnet with significant coding improvements (SWE-bench Verified rising from 33.4% to 49.0%, surpassing all publicly available models including reasoning models), a new Claude 3.5 Haiku that matches Claude 3 Opus performance at Haiku-tier speed, and a public beta of 'computer use' — a capability allowing Claude to control computers by viewing screens, moving cursors, clicking, and typing. Computer use is available via the Anthropic API, Amazon Bedrock, and Google Cloud Vertex AI, with early adopters including Replit, The Browser Company, and Cognition. Both safety institutes (US AISI and UK AISI) conducted pre-deployment testing, and the model was assessed as remaining within ASL-2 under Anthropic's Responsible Scaling Policy.
Anthropic publishes structured harm assessment framework covering physical, psychological, economic, and societal impacts
Anthropic has released a policy document describing their evolving framework for assessing and mitigating AI harms across five dimensions: physical, psychological, economic, societal, and individual autonomy impacts. The framework complements their existing Responsible Scaling Policy and informs decisions on usage policies, red-teaming, detection, and enforcement. Concrete examples include safeguards for computer use capabilities (fraud, phishing) and a reported 45% reduction in unnecessary refusals in Claude 3.7 Sonnet through improved handling of ambiguous prompts. Anthropic frames this as a work-in-progress and invites collaboration from the broader AI ecosystem.
Anthropic Updates Election Safeguards for Claude Ahead of 2026 US Midterms
Anthropic has published an update on its election-related safety measures for Claude, covering political bias evaluations, usage policy enforcement, and influence operation resistance testing. New model versions Claude Opus 4.7 and Sonnet 4.6 scored 95-96% on political impartiality evaluations and handled election-related policy compliance at 99.8-100% on a 600-prompt test suite. For the first time, Anthropic tested whether models can autonomously run influence operations end-to-end, finding that only Mythos Preview and Opus 4.7 completed more than half of tasks when safeguards were removed, underscoring ongoing capability concerns. Anthropic is also deploying election information banners pointing users to nonpartisan resources like TurboVote for the 2026 US midterms.
Anthropic Commits Claude to Remaining Ad-Free, Citing Alignment and User Trust
Anthropic has published a policy statement declaring that Claude will not carry advertising, sponsored content, or third-party product placements in conversations. The company argues that ad-based incentives are structurally incompatible with Claude's constitution and the goal of acting unambiguously in users' interests, citing the sensitive and personal nature of many AI conversations. Anthropic's revenue model relies on enterprise contracts and paid subscriptions, and the post signals openness to agentic commerce features where Claude acts on a user's behalf rather than on behalf of advertisers. The company acknowledges other AI companies may reach different conclusions and commits to transparency if this policy changes.
Anthropic makes Claude 3 Haiku and Sonnet available to US Intelligence Community and AWS GovCloud
Anthropic has made Claude 3 Haiku and Claude 3 Sonnet available via AWS Marketplace for the US Intelligence Community and AWS GovCloud, marking a significant expansion into government deployment. The company has crafted contractual exceptions to its general Usage Policy to permit legally authorized foreign intelligence analysis, including combating human trafficking and identifying covert influence campaigns, while maintaining restrictions on disinformation, weapons design, and malicious cyber operations. The deployment is currently limited to ASL-2 models under Anthropic's Responsible Scaling Policy. Anthropic also notes prior pre-release access to Claude 3.5 Sonnet was provided to the UK AI Safety Institute for pre-deployment testing.
Anthropic activates ASL-3 safety protections for Claude Opus 4 launch
Anthropic has activated its AI Safety Level 3 (ASL-3) Deployment and Security Standards in conjunction with launching Claude Opus 4, marking the first time any Anthropic model has been deployed under ASL-3 rather than the baseline ASL-2. The activation is described as precautionary: Anthropic has not conclusively determined that Opus 4 crosses the ASL-3 capability threshold, but cannot rule it out due to continued improvements in CBRN-related knowledge. ASL-3 measures include Constitutional Classifiers to block end-to-end CBRN weapon development workflows and enhanced model-weight security against sophisticated non-state attackers. Claude Sonnet 4 was evaluated and cleared for ASL-2, and ASL-4 was ruled out for Opus 4.