3arXiv cs.CL (Computation and Language)·
CAVE-ABSA: Constraint-aware counterfactual editing framework for aspect-based sentiment analysis
Researchers introduce CAVE-ABSA, a framework for generating and validating aspect-level counterfactuals in Aspect-Based Sentiment Analysis (ABSA). The system localizes opinion spans, performs controlled rewriting, and filters candidates using semantic similarity, AMR-guided structural preservation, fluency, and contradiction detection. The framework targets a known weakness in counterfactual evaluation — existing methods flip sentence-level labels but may violate aspect-level constraints — and is designed to support both robustness evaluation and data augmentation for ABSA models.
Related guides (1)
Related events (8)
SetFitABSA: Few-Shot Aspect Based Sentiment Analysis using SetFit
Hugging Face introduces SetFitABSA, an extension of the SetFit few-shot learning framework for Aspect-Based Sentiment Analysis (ABSA). The approach enables fine-grained sentiment classification at the aspect level with minimal labeled data. This builds on SetFit's contrastive sentence-transformer training paradigm, adapting it to the structured ABSA task of identifying sentiment toward specific aspects within text.
SalAngaBhava: New Sinhala aspect-based sentiment analysis dataset for low-resource NLP
Researchers introduce SalAngaBhava, a manually annotated dataset of Sinhala product reviews labeled with aspect terms and associated sentiments (positive, negative, neutral) for aspect-based sentiment analysis (ABSA). The dataset addresses a gap in low-resource language NLP, specifically for Sinhala, an Indo-Aryan language primarily spoken in Sri Lanka. It is intended to serve as a benchmark for ABSA research and broader Sinhala NLP tasks.
The Abstraction Gap in Vision-Language Causal Reasoning
Researchers introduce a dual-probe methodology and the CAGE benchmark (49,500 questions across 5,500 images) to distinguish linguistic plausibility from faithful causal reasoning in vision-language models. An Abstraction Gap (AG) metric quantifies the normalized performance difference between text-only and chain-of-reasoning probes. Evaluating eight VLMs, seven exhibit AG exceeding 0.50—generating fluent causal text but failing structured causal chain tasks—while one model achieves near-zero AG, suggesting architectural and pretraining choices are decisive. Fine-tuning on 45,000 chain-annotated examples fails to close the gap, pointing to a fundamental capability distinction.
ACL-Verbatim: Hallucination-Free Extractive QA System for Research Papers
The paper introduces ACL-Verbatim, an extractive question answering system built on VerbatimRAG that maps user queries directly to verbatim text spans in ACL Anthology papers, eliminating hallucination by design. The authors contribute a new ground-truth benchmark dataset created via human NLP-researcher annotation over synthetic queries generated using a ScIRGen-based pipeline. A 150M-parameter ModernBERT token classifier trained on silver supervision achieves the best word-level F1 of 53.6, outperforming the strongest LLM-based extractor at 48.7. The work demonstrates that smaller extractive models can outperform large generative LLMs on precision-critical retrieval tasks.
Stance Detection in Prediction Market Commentary via Counterfactual Augmentation and Market Context
This paper introduces the first stance detection system applied to prediction market commentary (Polymarket), addressing extreme class imbalance (8.7% anti-market comments) through LLM-driven counterfactual augmentation using the Anthropic API. RoBERTa-base is fine-tuned across a 4×3 ablation covering input configurations and augmentation doses. Key findings: market context is the dominant factor (raising 3-class Anti recall from 0.10 to 0.45), 50% synthetic augmentation is optimal, and full augmentation (100%) consistently degrades performance. Attention-based interpretability supports all three findings mechanistically.
Automated Benchmark Auditing for AI Agents and Large Language Models (ABA)
The paper introduces Auto Benchmark Audit (ABA), an agentic framework that systematically audits AI benchmark tasks for issues such as ambiguous specifications, environment conflicts, and incorrect ground truths. Applied to 168 benchmarks across nine domains including NeurIPS publications, ABA identifies critical issues in over 25.7% of evaluated tasks. The authors demonstrate that filtering out flawed tasks materially shifts model rankings and improves average performance on SWE-bench Verified and Terminal-Bench 2 by 9.9% and 9.6% respectively, indicating that current benchmark scores are significantly distorted by task quality problems. The agentic tool and annotations are released publicly.
Counterfactual context revision framework for auditing LLM-based stance simulation in online discussions
Researchers introduce a counterfactual context revision framework to audit how LLMs simulate individual users' stances in online discussions. By applying controlled text-only and multimodal (meme-based) revisions to conversational contexts, they measure how readily simulated stances shift in response to semantically independent changes. Results show effective and robust stance transitions across both revision types and polarization-preference mechanisms, raising concerns about whether LLM simulations reflect genuine user-specific beliefs or are highly context-sensitive artifacts. The work contributes an evaluation framework and highlights risks of using LLMs to model online opinion dynamics.
FACTOR: Risk-aware adaptive verification for factual long-form LLM generation
Researchers propose FACTOR (FACTuality-Oriented Risk-aware Verification), an inference-time framework that adapts verification effort based on claim-level hallucination risk rather than applying uniform verification to all claims. The system combines uncertainty estimation, adaptive language inference verification, and candidate re-ranking to focus resources on high-risk claims. Evaluated on the FactScore benchmark, FACTOR improves factuality while simultaneously reducing verification cost, with model-agnostic performance reported across ablation studies.