6arXiv cs.CL (Computation and Language)·
SkillFuzz: Execution-free fuzzing for implicit intent discovery in LLM agent skill marketplaces
SkillFuzz is a new testing framework that treats skill composition in LLM-based agent marketplaces as a fuzzing problem, using contract-guided Monte Carlo Tree Search to discover 'implicit intents' — unintended behaviors that emerge only when multiple individually-benign skills are co-activated. The approach is execution-free at audit time, relying on structured skill contracts and a skill-free planning baseline as a differential oracle. Across benchmark workloads, it discovers over 1,000 distinct implicit intents within a fixed query budget and confirms more than 80% of highest-risk flagged compositions during execution-time validation.
Related guides (2)
Related events (8)
SkillGenBench: Benchmarking Skill Generation Pipelines for LLM Agents
SkillGenBench is a new benchmark designed to evaluate the ability of LLM agents to generate correct, reusable, and executable skills from raw repositories and documents, rather than merely using pre-provided skills. It covers two generation regimes (task-conditioned and task-agnostic) and two procedural sources (repository-grounded and document-grounded), with standardized execution-based evaluation protocols. Experiments across multiple skill-generation methods reveal substantial performance variation and distinct failure modes depending on source type. The benchmark aims to establish skill generation as an independent research problem within agent systems.
SkillWeaver: Compositional Skill Routing for LLM Agents via Decompose-Retrieve-Compose
Researchers introduce SkillWeaver, a framework for compositional skill routing in LLM agents that decomposes complex queries into atomic sub-tasks, retrieves matching skills from a large library, and composes an executable DAG plan. The paper formalizes the Compositional Skill Routing problem and introduces CompSkillBench, a benchmark of 300 compositional queries over 2,209 real MCP server skills across 24 categories. A key finding is that task decomposition quality is the primary bottleneck, with standard LLM decomposition reaching only 34.2% category recall; the proposed Iterative Skill-Aware Decomposition (SAD) method improves decomposition accuracy from 51.0% to 67.7% in a single iteration. The framework also reduces context window consumption by over 99% compared to naive skill-stuffing approaches.
SkillHarm: Lifecycle-Aware Benchmark for Skill-Based Attacks on AI Agents
SkillHarm is a new benchmark evaluating adversarial attacks on AI agent skills across their full use lifecycle, covering two attack scenarios: Fixed-Payload Poisoning (FPP) and Self-Mutating Poisoning (SMP). The benchmark includes 879 attack samples across 71 skills, organized under a 12-category risk taxonomy targeting data pipelines, system environments, and agent autonomy. Experiments show current agents remain highly vulnerable, with attack success rates up to 86.3% (FPP) and 69.3% (SMP). An automated construction pipeline called AutoSkillHarm, driven by coding agents, was used to generate the benchmark at scale.
SkillComposer: Structured skill composition for LLM agents via constrained autoregressive decoding
A new arXiv preprint introduces SkillComposer, a method that frames skill selection for LLM agents as a structured prediction problem — jointly deciding which skills to activate, how many, and in what order via a constrained autoregressive decoder over skill identifiers. The approach addresses a bottleneck in growing skill libraries where existing retrieval and full-context methods fail to capture the joint nature of skill composition. Evaluated on SkillsBench across two production-grade coding agents (GPT-5.2-Codex and Gemini-3-Pro-Preview), SkillComposer raises pass rates by +23.1 and +18.2 percentage points over no-skill baselines, matching gold-skill retrieval upper bounds at lower prompt-token cost.
SkillOpt: Systematic Text-Space Optimizer for Self-Evolving Agent Skills
SkillOpt introduces a principled optimization framework for agent skills, treating the skill document as an external trainable state analogous to model weights. A separate optimizer model converts scored rollouts into bounded edits (add/delete/replace) on a skill document, accepting only edits that improve held-out validation scores. Evaluated across six benchmarks, seven target models, and three execution harnesses (direct chat, Codex, Claude Code), SkillOpt achieves best or tied performance on all 52 evaluated cells, lifting GPT-5.5 no-skill accuracy by up to +24.8 points inside the Codex agentic loop. Optimized skill artifacts also transfer across model scales and execution environments without further optimization.
SkillCenter: 216K-skill open library for grounded autonomous AI agents
SkillCenter is a new open skill library for autonomous AI agents containing 216,938 structured skills across 24 domain bundles, claimed as the largest by total count. The library combines 114,565 source-grounded skills derived from peer-reviewed journals and ArXiv via an LLM-based quality gate (SkillGate), plus 102,373 community skills from GitHub and the ClawHub marketplace. Each retained skill claim is traceable to an exact source quotation, and the library ships as offline-searchable SQLite FTS5 bundles. The work addresses a gap in agent operational knowledge — making outputs not just executable but correct, secure, and maintainable.
Biased LLM judges silently disable skill retirement in self-evolving agents
A new arXiv paper analyzes how biased LLM judges used as reward signals in self-evolving agent systems can silently disable the 'skill retirement' mechanism that prevents skill libraries from degrading below a no-skill baseline. The authors show that false-pass bias—where failures are incorrectly scored as passes—disables contribution-based retirement past a sharp threshold that cannot be overcome with more data, while leaving aggregate performance metrics unchanged, making the failure invisible. The paper proposes a defect-injection audit to determine pre-deployment whether a judge falls above or below the critical threshold.
Microsoft releases SkillOpt: text-space optimizer for training reusable NL skills in frozen LLM agents
Microsoft has open-sourced SkillOpt, a Python framework that optimizes natural-language skills for frozen LLM agents without modifying model weights. The system uses trajectory-driven edits and validation-gated updates to produce deployable best_skill.md artifacts. The approach is relevant to agent systems that need reusable, inspectable skill libraries without fine-tuning. The repository has accumulated over 11,600 stars with strong recent momentum (+261 today).