Entity · technique

disagreement-focused sampling

techniqueactivedisagreement-focused-sampling-2d1ade5c·1 events·first seen May 26, 2026

Aliases: disagreement-focused sampling

Co-occurring entities

human-LLM collaborative annotation speaker-attribute classification WhoSaidIt

More like this (12)

Gibbs Sampling Divergent Questioning Top-p sampling annotator disagreement importance sampling Min-p sampling adversarial pragmatics finite-sample posterior sampling framework Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research discrepancy metric paired-scenario forced-choice probe On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversity

Recent events (1)

4arXiv · cs.CL·May 26, 2026·source ↗

WhoSaidIt: Human-LLM Collaborative Annotation for Multilingual Speaker-Attribute Classification

This paper proposes a human-LLM collaborative re-annotation framework for stabilizing noisy multilingual speaker-attribute labels under resource constraints. LLMs surface recurring annotation rationales through iterative expert interaction, combined with disagreement-focused sampling for targeted re-annotation. The resulting WhoSaidIt dataset covers nine speaker-attribute labels across multiple languages. Benchmarking of recent LLMs reveals substantial cross-lingual annotation divergence and highlights both capabilities and limitations of LLMs in this classification task.

Evaluation and Benchmarking Agent and Tool Ecosystem human-LLM collaborative annotation speaker-attribute classification WhoSaidIt +1 more