ERUnderstand dataset

ERUnderstand benchmark evaluates VLM understanding of Entity-Relationship diagrams

Researchers introduce ERUnderstand, the first large-scale benchmark for evaluating Vision-Language Models on structured ER diagram understanding, comprising 2,960 diagrams across diverse notations and complexity levels. Evaluation of state-of-the-art VLMs reveals strong performance on common elements (F1 > 0.74) but sharp degradation on weak entities (0.28 F1), multivalued attributes (0.14 F1), and N-ary relationships (0.07 F1). Reasoning-augmented models improve overall performance by 15-25% but remain sensitive to linguistic priors and diagram complexity. The benchmark, dataset, and evaluation toolkit are publicly released.