pluralis-v0-1-4afec358·1 events·first seen Aliases: Pluralis v0.1
Researchers introduce Pluralis v0.1, a 6,448-prompt multimodal benchmark spanning six Asia-Pacific countries and eight languages, designed to evaluate Vision-Language Models on culturally localized safety hazards rather than Western-adapted defaults. The benchmark introduces a novel evaluation paradigm where text and image are individually innocuous but synergistically trigger cultural or legal violations. The authors also present Judge-Pluralis, an LLM-as-a-Judge ensemble for scoring, and document recurring locale-specific failure modes in current VLMs including image misidentification and missed item-context-locale interactions.