6arXiv cs.AI (Artificial Intelligence)·
VEXAIoT: Multi-agent LLM framework achieves 95% success rate in autonomous IoT vulnerability exploitation
Researchers introduce VEXAIoT, an autonomous multi-agent framework using LLM-based reasoning to discover and exploit vulnerabilities in IoT environments. The system pairs a vulnerability detection agent with an attack execution agent, evaluated across 260 attack executions in IoTGoat and Metasploitable2 environments covering ten OWASP IoT vulnerability categories. It achieves a 95% overall success rate with average execution times under two minutes, demonstrating that LLM agents can automate offensive IoT security workflows at scale in controlled settings.
Related guides (2)
Related events (8)
Practitioner spends $1,500 testing LLM offensive security capabilities against a purpose-built vulnerable app
A developer built a deliberately vulnerable application and ran LLMs against it as automated penetration testers, spending $1,500 on API costs across the experiment. The post evaluates how well current LLMs can identify and exploit real vulnerabilities in a controlled setting. Results provide practical signal on the current state of LLM-assisted offensive security, a capability area with both red-team and safety implications.
VulnClaw: AI agent + MCP toolchain for automated penetration testing workflows
VulnClaw is an open-source Python project that orchestrates an AI agent pipeline using MCP tooling and LLMs to automate the full penetration testing lifecycle: information gathering, vulnerability discovery, exploitation, and report generation via natural language input. The project has accumulated 1,049 GitHub stars with 105 added today, indicating notable community traction. It represents a concrete application of the MCP protocol to security automation.
Google Study Shows LLM-Generated Malware Is Getting Harder to Track and Stop
A Google security report catalogs emerging LLM-enabled cyberattack techniques including morphing malware with mutation engines, logical-flaw discovery in code, and AI-directed obfuscation networks. The report was prompted in part by a real incident where hackers used an LLM to find a zero-day in a widely used web administration tool. Separately, the UK AI Security Institute found that Claude Mythos Preview and GPT-5.5 can reliably execute attacks expected to take humans 3 hours, up from earlier 1-hour benchmarks, with performance scaling further when token limits are relaxed. The findings suggest an accelerating gap between LLM offensive capability and conventional defensive tooling.
Introducing EVMbench: AI Agent Benchmark for Smart Contract Vulnerabilities
OpenAI and Paradigm have jointly introduced EVMbench, a benchmark designed to evaluate AI agents on their ability to detect, patch, and exploit high-severity vulnerabilities in Ethereum Virtual Machine (EVM) smart contracts. The benchmark targets a specialized security domain requiring both code understanding and adversarial reasoning. This represents a new evaluation surface for frontier AI agents in the context of blockchain security.
LLMs susceptible to cognitive heuristics in code vulnerability detection, enabling black-box attacks
A new arXiv paper presents the first systematic study of cognitive heuristics — halo effect, framing effect, and anchoring — in LLM-based code vulnerability detection. Evaluating eight LLMs across three programming languages, the authors find all models susceptible, with framing causing the largest average shift (33.2%), followed by anchoring (23.5%) and halo (18.4%). Critically, the paper demonstrates a proof-of-concept black-box attack that suppresses up to 97% of previously detected vulnerabilities by manipulating surrounding context without changing the code itself. The findings suggest cognitive susceptibility is a systematic and exploitable property of LLM security tooling.
OptiAgent: Multi-agent framework for end-to-end operations research optimization modeling
OptiAgent is a multi-agent LLM framework that converts natural language descriptions of Operations Research problems into solver-ready mathematical formulations and executable code. The architecture uses dedicated agents for extracting decision variables and constraints, with a multi-loop validation system featuring four specialized feedback mechanisms targeting distinct failure modes. The system claims state-of-the-art performance on 3 of 4 benchmarks spanning LP, MILP, and Nonlinear Programming tasks, while also improving transparency through auditable agent reasoning.
Agentic CLEAR: Automating Multi-Level Evaluation of LLM Agents
Agentic CLEAR is an automatic evaluation framework for LLM-based agentic systems that analyzes behavior at three granularity levels: system, trace, and node. Unlike existing tools that rely on static error taxonomies or focus only on observability, it dynamically generates textual insights and integrates above the observability layer with an accessible UI. Experiments across four benchmarks and seven agentic settings demonstrate strong alignment with human-annotated errors and predictive accuracy for task success rates.
Inside VAKRA: Reasoning, Tool Use, and Failure Modes of Agents
IBM Research presents an analysis of VAKRA, a benchmark designed to evaluate agentic AI systems on reasoning and tool use capabilities. The post examines how agents fail across different task categories, surfacing systematic failure modes in multi-step reasoning and tool invocation. The analysis provides diagnostic insights into where current agent architectures break down under realistic task conditions.