4arXiv cs.CL (Computation and Language)·
Training-free graph-based framework for reading order inference in complex document layouts
A new arXiv preprint presents a training-free, graph-based method for inferring reading order in complex historical document layouts, including the challenging Glossa Ordinaria manuscript format where text and commentary are spatially interleaved. The approach scores edges in a directed candidate-transition graph using lightweight language model signals (causal LM likelihood and BERT NSP) and recovers global reading order via a degree-constrained directed path cover with a max-regret inference rule. On wrap-around Glossa layouts the method achieves 95% edge accuracy versus 50% for XY-cut, and 88% versus 75% for XY-cut and 25% for LayoutReader on OmniDocBench multi-column pages. The work is relevant to document digitization pipelines and OCR post-processing for historical archives.
Related guides (1)
Related events (8)
Visual pretraining on document images outperforms text-only pretraining for language intelligence
A new arXiv preprint argues that training language models directly on visual representations of documents (figures, equations, page layouts) consistently outperforms text-only pretraining on the same underlying corpora. The authors conduct a systematic study of unsupervised visual pretraining paradigms across multiple backbones and benchmarks, framing visual pretraining as a scalable alternative to the dominant text-extraction pipeline. The result challenges a foundational assumption in LLM pretraining and has implications for how future foundation models are trained on visually rich sources like PDFs and web pages.
Trajectory Analysis of Masked Diffusion LMs for Graph-to-Text Generation with Lambda-Scaled Structural Decoding
This paper presents the first systematic study of masked diffusion language models (MDLMs) for graph-to-text generation, analyzing the order in which tokens are unmasked during iterative decoding. The authors find MDLMs naturally unmask entities first, then relational/function words, then structural tokens—a pattern disrupted by supervised fine-tuning, which prematurely anchors structural tokens and causes hallucination or omission. They propose lambda-scaled structural decoding, a training-free inference-time fix that recovers +9.4 BLEU-4, and introduce Graph-LLaDA, which integrates a Graph Transformer encoder into LLaDA's decoding process. Cross-dataset evaluation on the LAGRANGE benchmark shows prior baselines overfit to dataset-specific patterns while MDLM-based approaches generalize better.
Multi-expert routing system achieves near-oracle OCR accuracy on low-resource Manchu historical scripts
A new arXiv preprint presents a multi-expert OCR system for historical Manchu manuscripts, routing pages by visual style (regular script, running script, chancery hand) using a lightweight image classifier. The router achieves 99.3% page-level domain accuracy and matches a domain-label oracle, with character error rates of 0.30%, 1.57%, and 4.83% across three script styles. The approach reuses fine-tuning checkpoints as domain specialists, avoiding the need to train dedicated experts for each domain. The work addresses low-resource OCR for endangered historical scripts with a reproducible evaluation protocol.
DocTrace: Structure-Aware On-Demand Hypergraph Memory for Long-Document QA
Researchers introduce DocTrace, a multi-agent RAG framework for long-document question answering that uses query-triggered knowledge organization rather than costly query-agnostic preprocessing. The system combines a lightweight document structural tree index, on-demand hypergraph working memory, and a graph-structured experience memory that stores successful reasoning plans for reuse. Evaluated on four long-document QA datasets, DocTrace outperforms the strongest baseline (ComoRAG) by up to 8.85% F1 and 4.40% EM while reducing computational cost by 53.32%.
GTCL: Latent trajectory discrimination framework for AI-generated text detection
A new arXiv preprint proposes Geometric Trajectory and Contrastive Learning (GTCL), a framework that reframes AI-generated text detection as a problem of distinguishing latent generation trajectories rather than treating documents as static objects. The method segments documents into ordered local units, encodes them in embedding space, and applies contrastive learning to capture geometric regularities of autoregressive generation. GTCL is evaluated on three benchmarks and consistently outperforms detection baselines, suggesting that modeling sequential dynamics provides more robust discriminative signals than aggregate statistics or global embeddings.
Training-free mechanistic defense against typographic attacks on CLIP-based vision encoders
Researchers propose a training-free method to defend CLIP-based vision encoders against typographic attacks, where irrelevant text embedded in images biases visual representations toward lexical rather than semantic meaning. The approach uses sampling-based mechanistic interpretability to identify specific Vision Transformer attention heads responsible for encoding lexical information, then applies targeted circuit-level interventions to suppress this behavior. Without any retraining, the method outperforms both supervised and training-free baselines on object classification and improves Visual Question Answering accuracy under typographic attack conditions on RIO-Bench across several state-of-the-art LVLMs.
Reasoning graphs improve robustness of LLM authorship attribution against obfuscation
A new arXiv paper proposes using graph neural networks over argument-mining-derived reasoning graphs to attribute authorship of LLM-generated text, moving beyond surface-level linguistic features. The approach outperforms a Longformer baseline by up to 27 percentage points under paraphrasing and backtranslation obfuscation attacks, and by 19 points when generalizing to unseen model versions. The work addresses a practical gap in AI-generated text detection: robustness to adversarial rewriting and model version drift.
Systematic study of tree traversal methods in Transformer Grammars reveals trade-offs between composition and lookahead
A new arXiv preprint evaluates Depth-First, Breadth-First, and a novel hybrid Production-Rule Traversal strategy for linearizing syntactic trees in Transformer Grammars. The authors test these methods across language modeling, syntactic generalization, and summarization tasks with varying tree configurations and masking strategies. The study reveals inherent trade-offs between nested composition and global lookahead, offering design recommendations for task-aware Transformer Grammars.