EvalCards

EvalCards: A unified reporting layer for AI evaluation results with interpretive signals

Researchers introduce EvalCards, an operational schema and tooling layer that composes benchmark metadata, evaluation run data, and model metadata into a unified, interpretable record for AI evaluation reporting. The system derives a reporting schema from 52 papers and 10 stakeholder interviews, implements four interpretive signals (reproducibility, documentation completeness, provenance/risk, score comparability), and deploys a monitoring tool across 5,816 models, 635 benchmarks, and 101,843 results. The work targets the widespread inconsistency in how evaluation results are reported across leaderboards, model cards, and company blogs, making cross-source comparison unreliable. It addresses a structural gap in the evaluation ecosystem by providing extraction infrastructure, not just a proposal.