6arXiv cs.CL (Computation and Language)·
AISPA framework audits system prompts across 88 commercial AI products, finds widespread problematic instructions
Researchers introduce AISPA (Artificial Intelligence System Prompt Assurance), a framework for auditing system prompts in commercial AI applications across eight user-centric dimensions. An audit of 3,249 instructions from 88 commercial products finds that while 98.9% contain at least one protective instruction, roughly 40% also contain at least one instruction that works against user interests. System prompt design varies dramatically across organizations, and only 24% of products cover all eight AISPA dimensions. The findings argue for greater transparency, standardization, and independent oversight of system prompts.
Related guides (3)
Related events (8)
Understanding prompt injections: a frontier security challenge
OpenAI has published a blog post addressing prompt injection attacks as a key security challenge for AI systems. The post covers how these attacks work and outlines OpenAI's multi-pronged approach including research, model training improvements, and safeguard development. This signals OpenAI's formal positioning on agentic security threats as their models are increasingly deployed in tool-using and autonomous contexts.
Simon Willison reports on prompt injection and adversarial attacks after 2,000 people tried to hack his AI assistant
Simon Willison documents the results of a public experiment in which approximately 2,000 people attempted to compromise or manipulate his personal AI assistant. The post covers the attack patterns observed, what succeeded or failed, and lessons learned about prompt injection and adversarial robustness in deployed AI systems. This is a practical, first-hand account of real-world AI security challenges from a respected practitioner.
Anthropic publishes structured harm assessment framework covering physical, psychological, economic, and societal impacts
Anthropic has released a policy document describing their evolving framework for assessing and mitigating AI harms across five dimensions: physical, psychological, economic, societal, and individual autonomy impacts. The framework complements their existing Responsible Scaling Policy and informs decisions on usage policies, red-teaming, detection, and enforcement. Concrete examples include safeguards for computer use capabilities (fraud, phishing) and a reported 45% reduction in unnecessary refusals in Claude 3.7 Sonnet through improved handling of ambiguous prompts. Anthropic frames this as a work-in-progress and invites collaboration from the broader AI ecosystem.
Eticas AI Risk Taxonomy v2.0.0: Open infrastructure for operationalizing AI audits
Eticas presents a structured AI auditing framework that bridges risk cataloging to executable audit methodology, demonstrated end-to-end on PII leakage testing against GPT-4-0314. The taxonomy organizes 76 active subcategories across 10 categories with mappings to 18 external frameworks, and is published under CC BY 4.0 with SKOS/JSON-LD distributions. The key contribution is an operationalization layer that converts named risks into measurable, severity-graded findings — addressing a gap the authors identify across at least 74 existing AI risk taxonomies. The PII leakage demonstration shows disclosure rates ranging from 0% to 84% under adversarial conditioning, graded as SYSTEMIC severity.
Dario Amodei's AI Safety Summit remarks detail Anthropic's Responsible Scaling Policy and ASL framework
Dario Amodei delivered prepared remarks at the UK AI Safety Summit (November 2023) explaining Anthropic's Responsible Scaling Policy (RSP), which was the first such policy published by a major AI lab. The RSP introduces AI Safety Levels (ASL-1 through ASL-4), modeled on biosafety level frameworks, with capability thresholds triggering mandatory safeguards before further training or deployment. Key implementation lessons include deep executive involvement, integrating RSP requirements into product roadmaps, and formal accountability through Anthropic's board and Long Term Benefit Trust. The remarks outline specific ASL-3 requirements around CBRN misuse prevention and security, and preview ASL-4 criteria involving near-human autonomy or becoming a primary source of global security threats.
Anthropic proposes federal AI transparency framework with mandatory Secure Development Frameworks and system cards
Anthropic published a policy proposal calling for a targeted AI transparency framework applicable at federal, state, or international levels, targeting only the largest frontier AI developers (suggested thresholds: ~$100M annual revenue or ~$1B R&D/capex). The framework would require covered developers to publicly disclose a Secure Development Framework covering CBRN and misalignment risks, publish system cards at deployment, self-certify compliance, and face legal liability for false statements. The proposal is explicitly lightweight and flexible, designed to avoid prescriptive standards while creating accountability mechanisms and whistleblower protections during the period before comprehensive safety standards are established.
Designing AI agents to resist prompt injection
OpenAI published a blog post describing how ChatGPT's agent workflows are designed to resist prompt injection and social engineering attacks. The approach focuses on constraining risky actions and protecting sensitive data within agentic pipelines. This represents OpenAI's public articulation of defensive design principles for deployed AI agents.
Anthropic Endorses California SB 53 AI Safety Disclosure Bill
Anthropic has announced its endorsement of California Senate Bill 53, which would require large frontier AI developers to publish safety frameworks, release transparency reports before deploying powerful models, report critical safety incidents within 15 days, and provide whistleblower protections. The bill, authored by Senator Scott Wiener and informed by the Joint California Policy Working Group, takes a disclosure-based approach rather than prescriptive technical mandates, drawing lessons from the failed SB 1047. Anthropic frames the bill as formalizing practices already followed by major labs including Google DeepMind, OpenAI, and Microsoft, while creating a level playing field that prevents competitive pressure from eroding voluntary safety programs. Anthropic notes the bill's compute-based threshold (10^26 FLOPS) is an acceptable starting point but calls for future refinement as AI capabilities advance.