benchmark

OCR-Robust

benchmarkactiveprovisionalocr-robust-3c1c66cc·1 events·first seen 6h ago

Aliases: OCR-Robust

Co-occurring entities

How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations

More like this (12)

PP-OCRv6 Robust Classification How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations GLM-OCR olmOCR EDGAR-OCR Azure OCR CoRP OlmOCRBench adversarial robustness ROCm RandOpt

Recent events (1)

4arXiv · cs.CL·6h ago·source ↗

OCR-Robust benchmark evaluates VLM robustness to visual perturbations on OCR-reasoning tasks

Researchers introduce OCR-Robust, a benchmark of 812 samples designed to evaluate how vision-language models handle OCR-reasoning tasks under controlled visual degradation. The benchmark covers documents, scene text, charts, geometry, and tables, applying 5 perturbation types at 3 severity levels each, and evaluates 18 models using metrics including Relative Corruption Retention and a composite Corruption Robustness Index. Key findings show that higher clean accuracy does not guarantee robustness, and that chart and table inputs are substantially more fragile under perturbation than document-like inputs.

Evaluation and Benchmarking Multimodal Progress How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations OCR-Robust