paper

Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection

paperactiveprovisionalwords-speak-louder-than-code-investigating-cognitive-heuristics-in-llm-based-code-vulnerability-detection-f1caa3cc·1 events·first seen 15h ago

Aliases: Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection

Co-occurring entities

framing effect halo effect anchoring effect

More like this (12)

Code Is More Than Text: Uncertainty Estimation for Code Generation Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection Do Coding Agents Deceive Us? Detecting and Preventing Cheating via Capped Evaluation with Randomized Tests Can LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QA Exploring Extrinsic and Intrinsic Properties for Effective Reasoning with Code Interpreter Multi-Agentic System Leveraging Open-Source LLMs to Mitigate Disinformation Threats Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction AIR: Adaptive Interleaved Reasoning with Code in MLLMs LLM-based code change labeling pipeline The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs code synthesis LLMs Policy Vulnerability Testing

Recent events (1)

6arXiv · cs.AI·15h ago·source ↗

LLMs susceptible to cognitive heuristics in code vulnerability detection, enabling black-box attacks

A new arXiv paper presents the first systematic study of cognitive heuristics — halo effect, framing effect, and anchoring — in LLM-based code vulnerability detection. Evaluating eight LLMs across three programming languages, the authors find all models susceptible, with framing causing the largest average shift (33.2%), followed by anchoring (23.5%) and halo (18.4%). Critically, the paper demonstrates a proof-of-concept black-box attack that suppresses up to 97% of previously detected vulnerabilities by manipulating surrounding context without changing the code itself. The findings suggest cognitive susceptibility is a systematic and exploitable property of LLM security tooling.

Evaluation and Benchmarking AI Safety Research framing effect halo effect anchoring effect +1 more