4arXiv cs.CL (Computation and Language)·
Two-stage LLM system extracts fine-grained events from 292K SEC 8-K filings with grounded citations
Researchers present a two-stage LLM pipeline that classifies SEC Form 8-K filings against a 119-type event taxonomy, anchoring every label to a verbatim quote via fuzzy n-gram validation and re-scoring each citation for quality. Applied to 292,984 filings from 2022–2026, the system produces 601,088 grounded event tags with precision rising from 12% to 96% as quality scores increase. The authors release the tagged dataset and validate economic signal via an event study on abnormal returns, confirming the taxonomy captures economically distinct events that the SEC's coarse item codes conflate.
Related guides (2)
Related events (8)
Stanford EDGAR Filings Dataset: 152B-token open corpus of SEC filings for LLM pretraining
Stanford researchers introduce the Stanford EDGAR Filings Dataset (SEFD), an open reconstruction of SEC filings into layout-faithful MultiMarkdown, releasing a 152B-token initial snapshot with a larger 550B-token archive described. The dataset targets the growing scarcity of high-quality long-context pretraining data, with less than 0.1% overlap with Common Crawl-derived corpora. Two derived benchmarks are also introduced: EDGAR-Forecast for filing-grounded numerical forecasting and EDGAR-OCR for complex financial table transcription. The work addresses a real gap in open long-context training data outside narrow domains like code.
Structure-Aware Code Change Labeling with LLMs via Two-Stage Taxonomy Pipeline
This paper presents a systematic study of using LLMs for taxonomy-based labeling of code diff hunks, going beyond summarization to assign structured labels capturing semantic attributes like renames, moves, and logic modifications. The authors introduce a two-stage pipeline combining diff-hunk labeling with structural refinement, using few-shot prompting to remain language-agnostic. Evaluated across four LLMs on a curated benchmark of natural and synthetic patches, the best configuration achieves 84% recall and 81% precision. Results suggest LLM-based structured labeling can complement static analysis tools in code review workflows.
Benchmarking LLM judges for citation verification in deep-research systems
A new arXiv paper evaluates 8 LLM judges from 3 model families on citation quality assessment for deep-research systems, testing across 1,248 rubric decisions with human-reviewed gold labels. The study finds that cheaper models remain competitive with frontier models — GPT-5-mini achieves the strongest source-relevance F1 at 0.908 — but judges differ substantially in directional bias (pass-rate drift, false positive/negative rates) even when scalar F1 scores are similar. The key finding is that scalar F1 obscures biases that would be directly reinforced in an RL training loop, making judge calibration a prerequisite before using citation rubrics as reward signals.
German Central Bank case study applies LLMs to securities prospectus eligibility verification
Researchers present the first case study applying LLMs to the Deutsche Bundesbank's securities collateral eligibility verification process, replacing traditional NER-based pipelines with a generative information extraction approach. The system decomposes the task into extraction, normalization, and interpretation stages, handling OCR noise and bilingual (German-English) content in lengthy financial prospectuses. Results show up to 91% precision in document-level eligibility decisions with a conservative false-acceptance profile. The paper also introduces an LLM-as-a-judge evaluation methodology for semantic assessment of extraction quality.
LLMs automate reproducibility assessments in social and behavioral sciences, outperforming human reanalysts
A preprint from arXiv demonstrates that an LLM pipeline can automate reproducibility assessments of published social and behavioral science studies, recovering original effect sizes in 41% of cases (vs. 34% for human reanalysts) and reaching the same qualitative conclusion in 96% of cases (vs. 74% for humans). The study evaluated 76 published studies with predefined claims. The results suggest LLMs could serve as a scalable tool for systematic auditing of empirical research, addressing the resource-intensive nature of traditional reproducibility efforts.
STAGE pipeline generates source-grounded training data for text-to-JSON extraction
Researchers introduce STAGE (Spreadsheet-grounded Text-to-JSON Artifact GEneration), a data generation pipeline that uses LLMs to synthesize training data for structured extraction from long unstructured documents, validating outputs against underlying spreadsheets. Evaluated on STAGE-Eval, an 851-example benchmark, the pipeline substantially improves Qwen3-4B performance, raising exact match from 31.37% to 74.27% and value accuracy from 45.46% to 90.69%. The work targets a practical bottleneck in enterprise document processing: reliably converting financial filings and clinical records into machine-readable JSON.
Text Analytics Evaluation Framework: Benchmarking LLMs on Social Media NLP Tasks
Researchers introduce a 470-question evaluation framework to assess LLM performance on aggregated social media text, applied to Twitter datasets across sentiment analysis, hate speech detection, and emotion recognition. Results show performance degrades substantially as input scale exceeds 500 instances, particularly for open-weights models on numerical tasks. Multi-label and target-dependent scenarios also show notable performance drops, and task complexity progressively erodes accuracy from basic semantic identification to comparison and counting operations. The findings point to architectural bottlenecks in current LLMs for rigorous quantitative analysis over large text collections.
Multi-agent LLM pipeline auto-generates 14,073 chemistry reaction rules from patent corpus
Researchers present a fully automated pipeline using a multi-agent LLM framework to classify chemical reactions and generate verifiable reaction rules across 665,901 US patent reactions. The system expands a standard taxonomy from 68 to 14,073 classes without human curation, using a verification loop that tests each generated rule against the corpus. A lightweight fingerprint classifier trained on the output achieves 97.7% accuracy on unseen reactions, matching leading proprietary classifiers while extending to novel chemistries. The work demonstrates a general approach for converting generative LLMs into self-expanding symbolic rule systems.