Entity · paper

Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls

paperactiveprovisionalbeyond-aggregate-risk-role-stratified-conformal-risk-control-for-llm-tool-calls-7b5925a2·1 events·first seen 2d ago

Aliases: Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls

Co-occurring entities

AgentDojo InjecAgent Conformal Risk Control

More like this (12)

Conformal Risk Control Invariant Risk Minimization RAS: Measuring LLM Safety Through Refusal Alignment On-Policy Distillation for LLM Safety: A Routing Approach to Template-Robust Realignment AI Risk Management Framework OpenSkillRisk: Benchmarking Agent Safety When Using Real-World Risky Third-Party Skills Collective Risk Game Subjective Risk Decomposition: A New View for Uncertainty Quantification Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark TrustX Agent Risk Classification Framework OpenSkillRisk Conformal Decision Theory

Recent events (1)

6arXiv · cs.CL·2d ago·source ↗

Role-Stratified Conformal Risk Control for LLM Tool Call Safety

A new arXiv preprint introduces role-stratified conformal risk control, a calibration method that sets separate statistical risk thresholds for individual semantic argument roles within LLM tool calls (e.g., email body vs. recipient vs. credential). The approach addresses a gap in existing aggregate-level risk certification, where failures in rare high-risk fields can be masked by benign arguments. Evaluated across AgentDojo and InjecAgent benchmarks with six language models, the method achieves consistent per-role budget compliance under model transfer, attack transfer, detector noise, and adaptive attacks. The authors argue that structured tool calls should be certified at the semantic-role level rather than the whole-action level.

Evaluation and Benchmarking AI Safety Research AgentDojo Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls InjecAgent +2 more