6arXiv cs.AI (Artificial Intelligence)·
Five-layer framework for hidden safety-critical failures in deployed AI systems
A preprint from arXiv argues that AI safety discourse over-indexes on visible, model-centric failures while neglecting quieter systemic risks in deployed socio-technical systems. The authors propose a five-layer diagnostic framework covering epistemic, control, temporal, organizational, and ecosystem integrity. The paper identifies under-recognized risk patterns including uncertainty laundering, prompt injection, memory poisoning, evaluation deception, and model collapse, and calls for a shift from model-centric evaluation toward socio-technical reliability. It concludes with design, governance, and research agenda recommendations.
Related guides (3)
Related events (8)
Concrete Problems in AI Safety
OpenAI, Google Brain, Berkeley, and Stanford researchers co-authored 'Concrete Problems in AI Safety,' a foundational paper exploring research challenges in ensuring modern ML systems operate as intended. The paper identifies and frames specific technical safety problems for the field. Published in June 2016, it became a landmark reference for AI safety research agendas.
OpenAI shares safety and alignment lessons from long-horizon model deployments
OpenAI published a blog post documenting lessons learned from deploying long-running AI models, covering new safety risks, observed failure modes, and iterative safeguard improvements. The post addresses the distinct alignment challenges that emerge when models operate over extended time horizons rather than single-turn interactions. As frontier labs push toward agentic and long-horizon systems, this represents a primary-source safety disclosure from a major lab.
Methodology for auditable trustworthiness levels in AI lifecycle governance
A new arXiv preprint proposes a two-component methodology for tracking AI trustworthiness across a system's lifecycle: a formal framework that learns interpretable trustworthiness levels from measurable dimensions using decision trees, and a governance procedure for design-time labeling, post-deployment monitoring, reassessment, and reporting. The approach introduces diagnostics such as boundary margins and profile drift to detect when a deployed system's trustworthiness has changed materially. The work targets the gap between high-level AI governance frameworks and the concrete, auditable documentation needed for regulatory conformity.
Methodology for harmonizing AI safety capability thresholds across frontier labs
A new arXiv preprint proposes a methodology for deriving harmonized capability thresholds across frontier AI companies, addressing the current inconsistency in published thresholds that makes third-party verification and cross-company comparison difficult. The authors cover three risk domains: cyber misuse, biological misuse, and automated AI R&D, using expected harm modeling for the first two and observed AI progress rates for the third. The work explicitly flags a potential race-to-the-bottom dynamic in safety standards when thresholds are not harmonized, and identifies empirical gaps in existing approaches.
Unfireable Safety Kernel: Formal execution-time alignment layer for escapable AI agents
A new arXiv preprint introduces the concept of 'escapable AI systems' — agents with sufficient reach into their own runtime to subvert in-process safety controls — and proposes a four-property architectural framework for external enforcement. The authors present the Unfireable Safety Kernel, a Rust reference implementation with machine-checked fail-closed invariants via SMT (Z3) and bounded model checking (Kani), evaluated against a self-improving world model adversary across 7,240 authorization attempts with zero successful bypasses. The work positions this 'execution-time alignment' layer as a complement to training-time approaches like RLHF and Constitutional AI, arguing that any control inside the agent's address space is fundamentally reachable by adversarial inputs.
Our approach to AI safety
OpenAI published a high-level overview of its approach to AI safety, framing safe development and deployment as central to its mission. The post appears to be a brief, top-level statement rather than a detailed technical or policy document. It signals OpenAI's public positioning on safety at a time of growing regulatory and public scrutiny.
ResearchArena: Framework for evaluating sabotage and AI control in automated AI R&D
ResearchArena is a new evaluation framework for assessing AI control mechanisms in automated AI R&D pipelines, covering four long-horizon tasks including safety post-training, capabilities post-training, CUDA-kernel optimization, and inference-server optimization. The framework tests frontier agents on both sabotage (covert harmful modifications to submitted artifacts) and monitoring (detecting such sabotage), finding that sabotage hidden in training data is caught fewer than half the time. Monitors that can execute and probe artifacts perform better but still miss embedded sabotage through surface-level inspection, false explanations, or wrong test probes. The work is released as a modular open framework, directly relevant to the emerging challenge of safely deploying AI agents that automate AI research.
Cyber Lack of Security and AI Governance
Zvi Mowshowitz's commentary addresses the intersection of AI capabilities and cybersecurity, framing recent developments around GPT-5.5 and a 'Mythos Moment' as catalysts for both internet security patching efforts and emerging AI regulatory frameworks. The piece situates cybersecurity as the underreported background story of current AI progress. It appears to analyze governance and safety implications of frontier model releases in the context of cyber vulnerabilities.