4arXiv cs.CL (Computation and Language)·
Structured LLM extraction of financial news outperforms sentiment-only approaches for stock prediction
Researchers propose a framework using LLaMA-3.1-70B to extract six semantic dimensions from financial news (event type, impact scope, temporal horizon, semantic confidence, etc.) beyond simple sentiment polarity. Experiments on 41,618 news-stock pairs from the FNSPID dataset show that combining LLM-extracted structured features with FinBERT sentiment achieves F1=0.600, significantly outperforming either alone, with a 53.5% systematic disagreement rate indicating the two signal sources are largely orthogonal. The work argues that compressing financial news to a single sentiment score incurs substantial information loss and that multi-dimensional NLP extraction is systematically exploitable for prediction tasks.
Related guides (2)
Related events (8)
Text Analytics Evaluation Framework: Benchmarking LLMs on Social Media NLP Tasks
Researchers introduce a 470-question evaluation framework to assess LLM performance on aggregated social media text, applied to Twitter datasets across sentiment analysis, hate speech detection, and emotion recognition. Results show performance degrades substantially as input scale exceeds 500 instances, particularly for open-weights models on numerical tasks. Multi-label and target-dependent scenarios also show notable performance drops, and task complexity progressively erodes accuracy from basic semantic identification to comparison and counting operations. The findings point to architectural bottlenecks in current LLMs for rigorous quantitative analysis over large text collections.
Two-stage LLM system extracts fine-grained events from 292K SEC 8-K filings with grounded citations
Researchers present a two-stage LLM pipeline that classifies SEC Form 8-K filings against a 119-type event taxonomy, anchoring every label to a verbatim quote via fuzzy n-gram validation and re-scoring each citation for quality. Applied to 292,984 filings from 2022–2026, the system produces 601,088 grounded event tags with precision rising from 12% to 96% as quality scores increase. The authors release the tagged dataset and validate economic signal via an event study on abnormal returns, confirming the taxonomy captures economically distinct events that the SEC's coarse item codes conflate.
Zero-shot LLMs fail to beat baselines on stock prediction; explainability signals retain practical value
A new arXiv preprint evaluates zero-shot NLP pipelines for predicting short-term stock movements from financial news, finding that across multiple models and prediction horizons, zero-shot approaches consistently fail to outperform simple baselines, with especially weak performance on negative price movements. The authors introduce a multi-layered explainability framework linking predictions to token-, article-, and aggregate-level evidence, finding that explainability signals can reliably distinguish trustworthy from unreliable predictions even when accuracy is low. The work argues for a shift toward decision-support systems emphasizing transparency and uncertainty awareness rather than raw predictive accuracy.
CM-LRS: A capital markets reliability benchmark for LLM workflow outputs
Researchers introduce CM-LRS (Capital Markets LLM Reliability Score), a seven-dimension evaluation framework assessing LLM outputs at the workflow level rather than the question-answer layer, targeting regulated capital-markets use cases such as DCM/ECM term extraction, M&A comparables, and issuer profiling. The benchmark is demonstrated on five workflows using public SEC EDGAR and UK takeover filings, scoring four models across four LLM judges. Key findings: frontier closed-source models cluster tightly (Sonnet 4.6 = 4.31, Opus 4.7 = 4.30, GPT-5.5 = 4.09) while Llama 3.3 70B lags at 3.15, with the gap concentrated in retrieval and synthesis tasks rather than extraction. The work advances domain-specific evaluation methodology for high-stakes financial workflows where regulatory defensibility matters.
news-crawler-LM: Small long-context model for structured news extraction from HTML
Researchers introduce news-crawler-LM, a small long-context language model fine-tuned on human-validated extractions from the Fundus news-crawling library to convert raw HTML into plaintext and structured JSON. The model outperforms baselines on HTML-to-Markdown and HTML-to-JSON tasks by +4.8 BLEU/+6.1 METEOR and +2.2 BLEU/+4.1 METEOR respectively, though it only marginally beats rule-based parsers on HTML-to-plaintext for unseen publishers. All models and artifacts are released publicly, targeting the practical gap between expensive LLMs and brittle rule-based crawlers.
Pair-level judgement outperforms dialogue-level generation for LLM emotion-cause extraction
A new arXiv paper investigates how task formulation affects LLM performance on emotion-cause pair extraction in conversation (ECPEC), finding that pair-level judgement consistently outperforms dialogue-level generation across 18 controlled comparisons. The authors show LLMs can recognize 92.7–98.1% of emotion-cause relations when queried explicitly but struggle to discover complete pair sets autonomously. They introduce an auxiliary retriever targeting ambiguous boundary cases, achieving F1 improvements of 0.50–1.46 points across three datasets at only 1.49x inference cost. The findings have broader implications for how task decomposition and candidate scoping affect LLM utilization in structured extraction tasks.
StakeBench: A Market-Commitment-Grounded Benchmark for Financial Language Understanding
StakeBench is a new evaluation framework linking 560,876 comments from 2,261 resolved prediction markets (Polymarket and Manifold) to verified trading positions, actions, and market-odds records, replacing human annotation with observable market behavior as supervision. Four diagnostic tasks test commitment detection, side identification, action anticipation, and collective odds projection, evaluated across 15 LLMs. Results reveal structural failures: models partially recover position-side signals (Directed Accuracy 0.506–0.599) but collapse on action anticipation and fail to beat naive baselines on odds projection. Notably, model scale shows no correlation with performance, and finance-domain fine-tuning does not improve revealed-side identification.
Investing in Performance: Fine-tune small models with LLM insights — a CFM case study
This Hugging Face blog post presents a case study from CFM (Capital Fund Management) on using large language model outputs to guide fine-tuning of smaller, more efficient models for financial applications. The approach leverages LLM-generated signals or labels to train compact models that can be deployed at lower cost and latency. The case study illustrates an enterprise pattern of distilling LLM capabilities into task-specific smaller models for production use.