Entity · technique

human-LLM collaborative annotation

techniqueactivehuman-llm-collaborative-annotation-083c54f1·1 events·first seen May 26, 2026

Aliases: human-LLM collaborative annotation

Co-occurring entities

speaker-attribute classification WhoSaidIt disagreement-focused sampling

More like this (12)

LLM-augmented clinical NLP pipeline Inter-Annotator Agreement LLM-based code change labeling pipeline Validity of LLMs as data annotators: AMALIA on authority Audio-LLM StreamingLLM From Self to Other: Evaluating Demographic Perspective-Taking in LLM Hate Speech Annotation SpeechLLM LLM-based content classification annotator disagreement Dep-LLM A Human-in-the-Loop Corpus for LLM-Based Simplification of Scientific Summaries

Recent events (1)

4arXiv · cs.CL·May 26, 2026·source ↗

WhoSaidIt: Human-LLM Collaborative Annotation for Multilingual Speaker-Attribute Classification

This paper proposes a human-LLM collaborative re-annotation framework for stabilizing noisy multilingual speaker-attribute labels under resource constraints. LLMs surface recurring annotation rationales through iterative expert interaction, combined with disagreement-focused sampling for targeted re-annotation. The resulting WhoSaidIt dataset covers nine speaker-attribute labels across multiple languages. Benchmarking of recent LLMs reveals substantial cross-lingual annotation divergence and highlights both capabilities and limitations of LLMs in this classification task.

Evaluation and Benchmarking Agent and Tool Ecosystem human-LLM collaborative annotation speaker-attribute classification WhoSaidIt +1 more