Entity · paper

Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation

paperactiveprovisional

rethinking-penetration-testing-for-ai-enabled-systems-from-resource-compromise-to-behavioral-objective-violation-1f0b5c3c

·1 events·first seen 3d ago

Aliases: Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation

Co-occurring entities

prompt injection indirect prompt injection

More like this (12)

Policy Vulnerability Testing Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection Adversarial Pragmatics for AI Safety Evaluation: A Benchmark for Instruction Conflict, Embedded Commands, and Policy Ambiguity Inoculation Adapters: Improved Selective Generalization of Capabilities with Fewer Surprising Backdoors Adversarial Attacks on Neural Network Policies Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents MITRE ATT&CK Input-Aware Dynamic Backdoor Attack Against Quantum Neural Networks Concrete Problems in AI Safety AI Cybersecurity Threat Evaluation Framework Criticality-Based Guard Rail Validation for AI Agent Decisions in Autonomous Telecom Networks

Recent events (1)

6arXiv · cs.AI·3d ago·source ↗

Framework for penetration testing AI-enabled systems via behavioral objective violation

A new arXiv preprint proposes reframing penetration testing for AI-enabled systems beyond traditional infrastructure compromise to include adversarial behavioral influence. The authors define 'AI-enabled penetration' as inducing AI-governed behavior that violates operational objectives, covering attack surfaces such as prompt injection, data poisoning, retrieval poisoning, tool misuse, and agentic misalignment. A structured testing workflow and a running example using an AI security operations center assistant illustrate the framework. The work addresses a gap in security evaluation methodology as agentic AI systems become more widely deployed.

AI Safety Research Agent and Tool Ecosystem prompt injection Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation indirect prompt injection