beyond-a-single-judge-simulating-social-persona-panels-for-generative-ui-evaluation-8ab14df3·1 events·first seen Aliases: Beyond a Single Judge: Simulating Social Persona Panels for Generative UI Evaluation
Researchers introduce ESPP (Evidence-Grounded, Social-Weighted Persona Panel), a three-stage evaluation method for Generative UI that simulates psychologically diverse personas, has them exchange opinions under a bounded-confidence mechanism, and aggregates judgments via Delphi-inspired social weighting. ESPP raises Pearson correlation with human judgment from 0.716 to 0.922 compared to a naive single-pass LLM judge, with prompt-ensemble baselines recovering only about a third of the gap. The method also reveals that user subgroups agree on overall model rankings but diverge on specific rating dimensions — a structural disagreement a single homogeneous judge would erase.