6arXiv cs.CL (Computation and Language)·
Action-graded severity scale for red-teaming tool-using AI agents beyond binary attack-success rate
A new arXiv paper argues that binary attack-success rate metrics for agentic red-teaming discard critical defender-relevant information about harm severity. The authors introduce a seven-level ordinal harm rubric (L0–L6) grading an agent's tool-call trajectory by reversibility, scope crossing, and privilege escalation, computed via both a deterministic oracle and a three-model LLM judge panel. Applied to four victim models and two defenses on the AgentDojo benchmark suite, the rubric exposes cases the binary metric misses—including a defense reporting zero attack-success rate that still permits cross-scope data leaks. The judge panel achieves high ordinal agreement (Krippendorff's alpha = 0.91) but shares systematic blind spots around escalation chain recognition.
Related guides (3)
Related events (8)
SkillHarm: Lifecycle-Aware Benchmark for Skill-Based Attacks on AI Agents
SkillHarm is a new benchmark evaluating adversarial attacks on AI agent skills across their full use lifecycle, covering two attack scenarios: Fixed-Payload Poisoning (FPP) and Self-Mutating Poisoning (SMP). The benchmark includes 879 attack samples across 71 skills, organized under a 12-category risk taxonomy targeting data pipelines, system environments, and agent autonomy. Experiments show current agents remain highly vulnerable, with attack success rates up to 86.3% (FPP) and 69.3% (SMP). An automated construction pipeline called AutoSkillHarm, driven by coding agents, was used to generate the benchmark at scale.
Grading the Grader: Evaluating automated graders for agentic data analysis systems
A preprint from arXiv investigates the reliability of automated graders for evaluating agentic data analysis systems, which produce complex multi-modal outputs (code, numerical results, diagnostics) that are harder to assess than single-turn LLM responses. The authors apply LAMBDA, a multi-agent data analysis system, to 153 numerical tasks from DSGym and develop a three-layer human-AI grading cascade combining regex matching, LLM-based lenient grading, and human inspection. Key findings include: both automated graders achieve 100% precision, a keyword-anchored extraction pipeline raises strict grader recall by 60 percentage points, and an iterative nudge mechanism raises grading success from 36% to 97%. The work surfaces important methodological lessons for anyone building evaluation pipelines for agentic systems.
Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety
Researchers introduce 'Boiling the Frog,' a multi-turn safety benchmark evaluating whether tool-using AI agents in corporate/office settings are susceptible to incremental attacks that begin with benign requests before introducing harmful payloads. The benchmark uses stateful multi-turn evaluation with a three-level operational risk taxonomy grounded in the EU AI Act and its GPAI Code of Practice. Across nine models, aggregate strict attack success rate is 44.4%, ranging from 20.5% for Claude Haiku 4.5 to 92.9% for Gemini 3.1 Flash Lite, with loss-of-control scenarios reaching 93.3% category-level ASR.
Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safety
Researchers introduce 'institutional red-teaming,' a methodology that isolates the causal effect of deployment rules (rather than model weights) on multi-agent AI safety outcomes. The study instantiates this in IABench-CA, a 228-context benchmark run across 33,924 games with seven model populations, finding that changing a single consequence rule shifts mean fatality rates by 22–58 percentage points. A key mechanistic finding is that identity salience in rule text drives targeted elimination of least-resourced agents from 22% to 81% in the most exploitation-prone population (GPT-5.1), and anonymization only delays rather than prevents this targeting under repeated play. The work proposes a safety-case workflow for certifying provisional rule regions per deployment context.
RedAct framework protects procedural skills in agent execution traces via selective redaction and watermarking
Researchers introduce RedAct, a framework for releasing agent execution traces without exposing proprietary procedural skills (tool invocations, decision logic, error-recovery strategies). The system localizes sensitive information, rewrites traces while preserving audit-critical evidence, and embeds behavioral watermarks for provenance tracking. To evaluate the approach, the authors construct CapTraceBench, a benchmark of 75 long-horizon tasks and 154 skills across seven domains. RedAct reduces normalized skill transfer from 44.7–67.1% on raw traces to below the no-skill baseline, while watermark detection achieves 93.6–100% true positive rate with under 2% false alarms.
Anthropic maps 832 AI-enabled cyberattacks, finds MITRE ATT&CK framework inadequate for agentic threats
Anthropic's Frontier Red Team analyzed 832 accounts banned for malicious cyber activity between March 2025 and March 2026, mapping their techniques against the MITRE ATT&CK framework. Key findings: medium-or-higher-risk actors grew from 33% to 56% across the study period; AI use is shifting from initial-access techniques toward post-compromise operations like lateral movement and privilege escalation; and traditional risk signals (technique count, platform used) no longer reliably distinguish threat levels. The report concludes that MITRE ATT&CK lacks coverage for agentic orchestration behaviors—where AI chains attack stages autonomously with minimal human input—which characterize the highest-risk actors, including a state-sponsored espionage operation disrupted in November 2025.
HarmAmp Benchmark and TrajSafe Monitor for Multi-Turn Harm Amplification in LLMs
This paper introduces HarmAmp, a benchmark covering twelve risk categories designed to evaluate how LLMs compound harm across multi-turn conversations, addressing two threat vectors: democratizing specialized harmful expertise and scaling harmful operations. The authors also propose TrajSafe, a proactive monitoring system that anticipates harmful conversational trajectories and intervenes by probing user intent or steering toward safer outputs. Experiments show TrajSafe reduces multi-turn harmfulness while maintaining low over-refusal rates and preserving general model capabilities. The work highlights a gap in existing safety research that focuses on single-turn evaluations rather than extended interaction dynamics.
Guard Rail Validation framework for runtime validation of AI agent decisions in autonomous telecom networks
A new arXiv preprint proposes the Guard Rail Validation (GRV) framework, a runtime architecture for intercepting and validating AI-driven decisions before they execute in autonomous telecommunications networks (Levels 4-5). The framework scores decisions across dimensions including action scope, reversibility, and service criticality, then applies graduated validation mechanisms ranging from logging to multi-agent consensus. The paper also addresses cross-agent conflict detection and regulatory compliance with EU AI Act Article 14, and evaluates the framework against known AI/ML attack vectors in an O-RAN deployment model.