5arXiv cs.AI (Artificial Intelligence)·
AirflowAttack: First adversarial attack on infrared remote-sensing VLMs using thermal-airflow perturbations
Researchers introduce AirflowAttack, the first adversarial attack targeting vision-language models deployed on infrared remote-sensing imagery, using physically plausible thermal-airflow turbulence as the perturbation prior. A single input-agnostic perturbation optimized on one surrogate CLIP model achieves 48.5% mean zero-shot attack success rate across five CLIP backbones, outperforming four IR-specific physical baselines. Applied to six state-of-the-art VLMs, the attack reduces scene-classification accuracy by up to 38.2% relative while paradoxically increasing model confidence, causing confabulation of thermal artifacts as genuine evidence. The work also releases a benchmark spanning eleven models and four tasks, exposing systematic vulnerabilities in security-critical IR VLM deployments.
Related guides (3)
Related events (8)
Acoustic adversarial attacks on computer vision using audible frequencies disrupt YOLO11 object detection
Researchers demonstrate physical acoustic attacks on AI-based computer vision systems using audible frequencies (<20 kHz), extending prior ultrasonic work to longer effective ranges. By resonating a commercial camera, they induce motion artifacts that cause YOLO11 to misclassify, miss targets, or hallucinate objects. The paper characterizes which image and object features increase vulnerability, offering a foundation for future mitigation strategies. The attack vector is physically realizable against deployed systems including autonomous vehicles and security cameras.
Training-free mechanistic defense against typographic attacks on CLIP-based vision encoders
Researchers propose a training-free method to defend CLIP-based vision encoders against typographic attacks, where irrelevant text embedded in images biases visual representations toward lexical rather than semantic meaning. The approach uses sampling-based mechanistic interpretability to identify specific Vision Transformer attention heads responsible for encoding lexical information, then applies targeted circuit-level interventions to suppress this behavior. Without any retraining, the method outperforms both supervised and training-free baselines on object classification and improves Visual Question Answering accuracy under typographic attack conditions on RIO-Bench across several state-of-the-art LVLMs.
LLMs fail to reliably self-report adversarial prefill attacks, study finds
A new arXiv paper evaluates whether LLMs can recognize that their own prior responses were elicited by adversarial prefill attacks, testing ten open-weight models (3B–70B) across four safety benchmarks. Models claim intent on prefilled responses only 27.3% of the time on average, and introspective signal is largely mediated by refusal-related reasoning. Three LoRA fine-tuning methods (SFT, GRPO, DPO) improve the intention-probe gap but counterintuitively raise attack success rates on most models, suggesting partial and fragile mitigation. The findings raise concerns about the reliability of LLM self-reports in safety-critical contexts.
RING attack exploits differential privacy to amplify backdoor success in federated learning
A new arXiv paper challenges the assumption that differential privacy (DP) inherently protects federated learning (FL) against backdoor attacks, demonstrating that DP's noise mechanism actually masks the statistical signatures that defenses rely on to detect malicious updates. The authors propose RING, an attack that exploits this masking effect by having compromised clients collaboratively craft adversarial perturbations that reconstruct a strong backdoor signal at aggregation time. Evaluated across four datasets against six state-of-the-art defenses, RING achieves a 90.3% average attack success rate under moderate privacy budgets, up to 26x better than baselines. Proposed countermeasures incur significant utility trade-offs, exposing a fundamental security gap in DP-FL deployments.
Attacking Machine Learning with Adversarial Examples
This 2017 OpenAI blog post introduces adversarial examples — inputs intentionally crafted to cause machine learning models to make mistakes, analogized to optical illusions for machines. It surveys how adversarial examples manifest across different input modalities and discusses the fundamental difficulties in defending against them. The post is an early foundational explainer on adversarial robustness from OpenAI.
AMIA: Attention-based membership inference attacks on tabular foundation models with k-anonymity defense
Researchers demonstrate that tabular foundation models using in-context learning are vulnerable to membership inference attacks (MIAs) via attention mechanism leakage, even when pre-trained on synthetic data. They introduce AMIA, a shadow-model-free attack exploiting transformer attention concentration patterns, achieving a 7.7% average gain over confidence-based attacks. A k-anonymity-inspired inference-time defense reduces membership leakage by 50% against AMIA and 25% against confidence-based attacks with only 3.9% performance degradation. The paper also shows fine-tuning amplifies memorization risk through confidence shifts.
Adversarial Attacks on Neural Network Policies
OpenAI published research examining adversarial attacks on neural network-based reinforcement learning policies. The work investigates how small, carefully crafted perturbations to observations can cause trained RL agents to fail catastrophically. This represents an early investigation into the robustness and safety of learned policies under adversarial conditions.
Building an Early Warning System for LLM-Aided Biological Threat Creation
OpenAI published a blueprint for evaluating whether LLMs can meaningfully assist in biological threat creation. In a controlled study with biology experts and students, GPT-4 was found to provide at most mild uplift in biological threat creation accuracy. The results are inconclusive but are framed as a starting point for ongoing safety research and community deliberation on biosecurity risks from AI.