3arXiv cs.CL (Computation and Language)·
news-crawler-LM: Small long-context model for structured news extraction from HTML
Researchers introduce news-crawler-LM, a small long-context language model fine-tuned on human-validated extractions from the Fundus news-crawling library to convert raw HTML into plaintext and structured JSON. The model outperforms baselines on HTML-to-Markdown and HTML-to-JSON tasks by +4.8 BLEU/+6.1 METEOR and +2.2 BLEU/+4.1 METEOR respectively, though it only marginally beats rule-based parsers on HTML-to-plaintext for unseen publishers. All models and artifacts are released publicly, targeting the practical gap between expensive LLMs and brittle rule-based crawlers.
Related guides (1)
Related events (8)
Structured LLM extraction of financial news outperforms sentiment-only approaches for stock prediction
Researchers propose a framework using LLaMA-3.1-70B to extract six semantic dimensions from financial news (event type, impact scope, temporal horizon, semantic confidence, etc.) beyond simple sentiment polarity. Experiments on 41,618 news-stock pairs from the FNSPID dataset show that combining LLM-extracted structured features with FinBERT sentiment achieves F1=0.600, significantly outperforming either alone, with a 53.5% systematic disagreement rate indicating the two signal sources are largely orthogonal. The work argues that compressing financial news to a single sentiment score incurs substantial information loss and that multi-dimensional NLP extraction is systematically exploitable for prediction tasks.
Co-LMLM: Continuous-query limited memory language models outperform vanilla LLMs on factual tasks at small scale
Researchers introduce CO-LMLM, a limited memory language model that externalizes factual knowledge to a knowledge base during pretraining and retrieves it at inference via continuous vector queries paired with human-readable text values. The approach removes prior restrictions to relational knowledge bases and Wikipedia-only data by introducing an annotation pipeline for arbitrary text. At 360M parameters, CO-LMLM achieves lower perplexity than models trained on 40x more data and SimpleQA factual performance comparable to GPT-4o mini and above Claude Sonnet 4.5, suggesting significant efficiency gains for factual grounding.
If you're an LLM, please read this — Anna's Archive on llms.txt
Anna's Archive published a blog post addressing LLMs directly, engaging with the emerging llms.txt convention for providing machine-readable site context to language models. The post garnered significant HN engagement (677 points, 386 comments), suggesting it touches on substantive questions about how LLMs interact with web content and what site operators can or should communicate to them. The llms.txt standard is a nascent protocol for structuring web content to be more useful to AI crawlers and inference-time retrieval.
Introducing HELMET: Holistically Evaluating Long-context Language Models
HELMET is a new benchmark designed to holistically evaluate long-context language models across diverse real-world tasks rather than synthetic needle-in-a-haystack tests. The benchmark covers multiple task categories including retrieval, reasoning, summarization, and code, aiming to provide more reliable and comprehensive assessment of long-context capabilities. It is introduced via the Hugging Face blog, suggesting an open release with associated tooling for the community.
Attention Expansion mechanism improves keyphrase extraction from long documents without full-context LLMs
Researchers propose an 'attention expansion' mechanism that augments pre-trained language model token representations with information from out-of-context chunks using static word embeddings, enabling more effective keyphrase extraction from long documents. The approach avoids the computational cost of full-document attention or LLM-based inference while expanding the effective contextual scope of PLM-based models. Evaluated across five PLM backbones and five benchmark corpora, the method consistently improves F1 scores over state-of-the-art baselines in both scientific and news domains.
AutoJourn: LLM pipeline for multi-perspective news generation with bias detection and neutralisation
AutoJourn is a demonstration system that uses LLMs to generate balanced news summaries from social media discussions, incorporating perspective extraction, multi-viewpoint summarisation, and sentence-level bias detection and neutralisation. The pipeline combines prompt engineering with optional retrieval augmentation and is evaluated on semantic diversity, summary quality, and bias reduction metrics. The work targets responsible automated journalism and ships with a publicly accessible live demo.
Recursive Language Models Offer Path To Dramatically Expand Beyond the Context Window
MIT researchers Alex L. Zhang, Tim Kraska, and Omar Khattab propose Recursive Language Models (RLMs), a framework that offloads long-context processing to an external Python REPL environment, allowing models to programmatically fetch and manage text chunks via code generation. The root model spawns submodel instances to handle subtasks, aggregating their outputs recursively. Evaluated on benchmarks requiring reasoning over documents up to 11 million tokens, RLMs substantially outperform both base models and competing agentic strategies such as retrieval and summarization agents. For example, RLM-GPT-5 achieved 91.3% on BrowseComp+ versus GPT-5's inability to produce an answer, and ~50% accuracy on OOLONG-PAIRS at 1 million tokens versus near-zero for baseline approaches.
Latent Context Language Models (LCLMs) achieve competitive encoder-decoder KV cache compression at scale
Researchers introduce Latent Context Language Models (LCLMs), a family of encoder-decoder compressors that map long token sequences to shorter latent embeddings consumed by a decoder, targeting the KV cache memory bottleneck in long-context inference. The authors conduct architecture search and continually pre-train 0.6B-encoder/4B-decoder models on over 350B tokens at compression ratios of 1:4, 1:8, and 1:16. LCLMs improve the Pareto frontier across general-task performance, compression speed, and peak memory, and are demonstrated as efficient backbones for long-horizon agents that can skim compressed context and expand relevant segments on demand. The work closes a previously noted gap between encoder-decoder approaches and KV cache compression methods on the accuracy-efficiency frontier.