technique
synthetic data evaluation
techniqueactive
synthetic-data-evaluation-6da364e7·1 events·first seen 25d agoAliases: synthetic data evaluation
Co-occurring entities
More like this (12)
Synthetic Data GeneratorValid Inference with Synthetic Data via Task Exchangeabilitythird-party AI evaluationsEvaluation Cards: An Interpretive Layer for AI Evaluation ReportingPhantoms and Disclosures: a Causal Framework for Auditing Synthetic DataCodex HumanEvalAI-Assisted Systematization for Evaluating GenAI SystemsT-EvalInstructS2S-Evalcounterfactual data augmentationProvenance-Grounded Gating and Adaptive Recovery in Synthetic Post-Training Data CurationText Analytics Evaluation Framework
Recent events (1)
SynAE: Framework for Evaluating Synthetic Data Quality in Tool-Calling Agent Benchmarks
SynAE is a proposed evaluation framework for measuring how well synthetic datasets replicate and augment real data trajectories for multi-turn, tool-calling agent testing. It assesses validity, fidelity, and diversity across four metric categories: task instructions, tool calls, final outputs, and downstream evaluation. The paper demonstrates that no single metric suffices to characterize synthetic data quality, motivating multi-axis evaluation. A demo and code are publicly available.