Entity · paper

Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradation

paperactiveprovisional

bigger-or-cheaper-scale-and-quantization-effects-on-uncertainty-signals-in-vision-language-models-under-image-degradation-3b35eb0f

·1 events·first seen 2d ago

Aliases: Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradation

Co-occurring entities

Qwen2.5-VL

More like this (12)

Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets quantization-induced degradation Scaling Laws for Neural Language Models How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations Evolution of Accuracy and Visual-Cognitive Errors in a Decade of Vision-Language AI Models Sound Probabilistic Safety Bounds for Large Language Models Test-Time Training for Modality Order Consistency in Vision-Language Models From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model Language Modeling Loss Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy Data Selection Through Iterative Self-Filtering for Vision-Language Settings

Recent events (1)

5arXiv · cs.CL·2d ago·source ↗

Scale vs. quantization tradeoffs for uncertainty signals in vision-language models under image degradation

A new arXiv paper evaluates how model scale and 4-bit quantization affect two confidence signals — verbalized confidence and mean token probability — in the Qwen2-VL family across 5,700 predictions under six photographic degradation types. Key findings: scaling from 2B to 7B sharply improves internal uncertainty (AUROC 0.80→0.98) while verbalized confidence remains weak; 4-bit quantization costs little in accuracy (-1.6 pts) but degrades the internal confidence signal (AUROC 0.95→0.80) and collapses verbalized-confidence parse rate from 99% to 64%. The practical recommendation is to prefer a larger quantized model over a smaller full-precision one within a fixed memory budget, and to use error-detection AUROC rather than calibration error as the primary metric.

Evaluation and Benchmarking Inference Economics Qwen2.5-VL Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradation +1 more