3arXiv cs.CL (Computation and Language)·
TikStance: Multimodal hierarchical dataset for stance detection in TikTok political conversations
Researchers introduce TikStance, a dataset of 161 TikTok videos and 13,876 comments annotated for stance detection across three 2024 U.S. election figures (Trump, Biden, Harris). The dataset preserves audiovisual content, hierarchical comment trees, and multi-target stance labels (Favor/Against/None) with high inter-annotator agreement (Krippendorff's α ~0.72–0.74). It is designed to support multimodal stance detection, political communication analysis, and context-aware NLP research.
Related guides (1)
Related events (8)
Annotated dataset for enthymeme detection in political tweets with disagreement-aware training
Researchers present a dataset of 1,482 politically controversial tweets annotated by five annotators for enthymemes — arguments with unstated premises or conclusions — designed to study label variation rather than eliminate it. Annotation guidelines are grounded in Walton's argumentation schemes, and the paper includes a complexity analysis of cognitive load in the task. Preliminary experiments show that models trained on annotator disagreement outperform those trained on hard majority-vote labels, suggesting value in preserving annotation disagreement for subjective NLP tasks.
Stance Detection in Prediction Market Commentary via Counterfactual Augmentation and Market Context
This paper introduces the first stance detection system applied to prediction market commentary (Polymarket), addressing extreme class imbalance (8.7% anti-market comments) through LLM-driven counterfactual augmentation using the Anthropic API. RoBERTa-base is fine-tuned across a 4×3 ablation covering input configurations and augmentation doses. Key findings: market context is the dominant factor (raising 3-class Anti recall from 0.10 to 0.45), 50% synthetic augmentation is optimal, and full augmentation (100%) consistently degrades performance. Attention-based interpretability supports all three findings mechanistically.
IMPACTeen: Annotated dataset for social influence detection in adolescent communication contexts
IMPACTeen is a new Polish/English bilingual dataset of 1,021 social influence scenarios targeting adolescent communication contexts, with 5,100 annotation records from five distinct annotator perspectives (teenagers, parents, psychologists, communication experts, teachers). The dataset covers influence techniques, intentions, consequences, and resistance, and was constructed via constrained LLM generation followed by human editing. It is intended to support research on social influence detection, annotator disagreement modeling, cross-lingual NLP, and LLM training and evaluation.
CATCH-ME dataset: multilingual multi-turn counterspeech against hate speech and misinformation for RAG systems
Researchers introduce CATCH-ME, a large-scale expert-curated multilingual dataset of multi-turn dialogues addressing the intersection of hate speech and misinformation across five languages and seven marginalized groups. The dataset is anchored in verified external knowledge (fact-checking articles and NGO reports) with document- and chunk-level span annotations, making it directly usable for RAG-based counterspeech systems. It addresses a gap in existing resources, which are limited to single-turn English dialogues, and is intended to improve the factual grounding and persuasiveness of LLM-generated counterspeech.
Counterfactual context revision framework for auditing LLM-based stance simulation in online discussions
Researchers introduce a counterfactual context revision framework to audit how LLMs simulate individual users' stances in online discussions. By applying controlled text-only and multimodal (meme-based) revisions to conversational contexts, they measure how readily simulated stances shift in response to semantically independent changes. Results show effective and robust stance transitions across both revision types and polarization-preference mechanisms, raising concerns about whether LLM simulations reflect genuine user-specific beliefs or are highly context-sensitive artifacts. The work contributes an evaluation framework and highlights risks of using LLMs to model online opinion dynamics.
FacialTalker: Facial-expression-aware conversational speech synthesis with DualDPO and VSDD-1K dataset
Researchers introduce FacialTalker, a conversational speech synthesis framework built on a large language model backbone that incorporates facial expression signals via AUTokenizer, a visual tokenizer trained on facial Action Units. A dual direct preference optimization (DualDPO) strategy jointly constrains visual and speech token sequences to improve multimodal alignment. The authors also release VSDD-1K, a 1,033-hour multimodal dialogue dataset of synchronized speaker video and speech collected from real-world internet conversations. Experiments show FacialTalker outperforms baselines on expression perception and speech naturalness.
ToxiREX: Multilingual contextual dataset for implicit toxicity detection with structured reasoning schema
Researchers introduce ToxiREX, a multilingual Reddit-based dataset for detecting implicit and context-dependent toxicity across six languages (English, Arabic, Turkish, Spanish, German, Dutch), anchored to real-world events like the 2023 Turkey earthquakes and the Russian invasion of Ukraine. The dataset includes 125K LLM-annotated training comments and ~3K human-annotated test comments, structured using a toxic reasoning schema that captures implicit toxicity and maps to existing taxonomies. Baseline results from prompted and fine-tuned language models show above-random but substantially suboptimal performance, indicating the task remains challenging. ToxiREX is claimed as the first dataset combining multilingual coverage, conversational context, and implicit toxicity with schema-based structured annotations.
StakeBench: A Market-Commitment-Grounded Benchmark for Financial Language Understanding
StakeBench is a new evaluation framework linking 560,876 comments from 2,261 resolved prediction markets (Polymarket and Manifold) to verified trading positions, actions, and market-odds records, replacing human annotation with observable market behavior as supervision. Four diagnostic tasks test commitment detection, side identification, action anticipation, and collective odds projection, evaluated across 15 LLMs. Results reveal structural failures: models partially recover position-side signals (Directed Accuracy 0.506–0.599) but collapse on action anticipation and fail to beat naive baselines on odds projection. Notably, model scale shows no correlation with performance, and finance-domain fine-tuning does not improve revealed-side identification.