4arXiv cs.CL (Computation and Language)·
VEXMLM: Vocabulary-extended XLM-R for Ge'ez-script African languages (Amharic, Tigrinya)
Researchers introduce VEXMLM, a vocabulary-extended variant of XLM-R targeting Amharic and Tigrinya (Ge'ez-script languages) and evaluated across 19 low-resource African languages. The approach adds 30,000 Ge'ez-script subwords to XLM-R's vocabulary with embedding initialization via subword averaging, followed by two-stage training (continued MLM then task fine-tuning). VEXMLM substantially outperforms XLM-R and Glot500 on QA (87.0 EM vs. 66.0 EM), sentiment analysis, and NER, with OOV-token entity accuracy rising from 81.4% to 94.3% across 11 languages. The work addresses a well-known gap in multilingual NLP: Latin-script-centric tokenizers that fragment non-Latin scripts and inflate OOV rates.
Related guides (1)
Related events (8)
ROMEVA: Geometry-preserving vocabulary expansion for Roman Urdu language models
Researchers propose ROMEVA, a method combining sub-word-average initialization and PCA-guided anchor loss to stabilize embeddings when expanding mBERT's vocabulary for Roman Urdu, a morphologically inconsistent low-resource language with high sub-word fragmentation. The method is evaluated on a 36,130-comment corpus with 500 new tokens added to mBERT. A notable finding is that while ROMEVA best preserves the pretrained embedding geometry, naive fine-tuning outperforms it on downstream sentiment classification, revealing a disconnect between embedding stability and task performance.
DeltaMerge-LowRes: Composing Language and Task Weight Deltas for Low-Resource NLP Adaptation
A new arXiv preprint introduces DeltaMerge-LowRes, a method for adapting multilingual encoders to new languages and tasks in low-resource settings by training language and task deltas separately and composing them in weight space at inference. The paper proposes four composition rules including a novel cross-axis TIES variant that adapts the TIES-Merging algorithm to language/task axes rather than task/task axes. Evaluated across four task families and four African languages (158 cells), cross-axis TIES improves summarization by +4 to +7 chrF and QA F1 by +2.32, while sparsity-aware merging reduces calibration error by 36%. The work advances model merging techniques for low-resource multilingual NLP.
LuxEmo: 21-hour expressive speech corpus for Luxembourgish TTS with emotion categories
Researchers introduce LuxEmo, a 21-hour conversational speech corpus for Luxembourgish derived from RTL youth broadcasts, annotated with four emotion categories via a semi-automatic curation pipeline combining VAD, denoising, language ID, and human validation. The paper benchmarks five expressive TTS systems across cross-lingual transfer, multilingual support, and prosody transfer approaches. The work addresses the underrepresentation of low-resource languages in speech technology research.
Test-time scaling for small VLMs on multilingual visual MCQ: conditions matter more than methods
A new arXiv paper examines whether test-time scaling (TTS) transfers to small open vision-language models using EXAMS-V, a multilingual visual multiple-choice benchmark. The study compares self-consistency, describe-then-reason with PRM-guided beam search, and post-hoc selectors across Qwen2.5-VL-7B-Instruct and Qwen3.5-4B. Key findings: prompt parseability and decoding budget (token limit) dominate gains, while elaborate search/verification methods like PRM-guided beam search underperform plain majority vote at 8x the cost. The best configuration achieves 84.1% on ImageCLEF 2026 test split, ranking first on the Visual MCQ leaderboard.
First Komi-Yazva–Russian parallel corpus and LLM translation evaluation protocol for endangered low-resource language
Researchers introduce the first Komi-Yazva–Russian parallel corpus of 457 aligned sentence pairs from 74 narrative texts, paired with a rigorous evaluation protocol for studying LLM translation under extreme data scarcity. The protocol includes story-level cross-validation, deterministic retrieval-based few-shot prompting, and both reference-based and judge-based metrics to ensure leakage-aware, reproducible evaluation. Results show LLMs produce non-trivial translations but performance varies strongly by model family; retrieval-based few-shot prompting consistently outperforms zero-shot, though gains plateau quickly. The work frames the corpus as both a dataset contribution and a reproducible testbed for endangered-language machine translation research.
LLM-Based Grammar Adaptation for Metamodel-Grammar Co-Evolution in Model-Driven Engineering
This paper proposes using LLMs to automate grammar adaptation when metamodels evolve in model-driven engineering, replacing tedious manual work and outperforming rule-based methods. Evaluated on six real-world Xtext DSLs using Claude Sonnet 4.5, ChatGPT 5.1, and Gemini 3, all three LLMs achieved 100% adaptation consistency on test DSLs versus 62-84% for rule-based approaches. A longitudinal study on QVTo showed LLMs successfully reused learned adaptations across all evolution steps without manual editing. However, on large-scale grammars (EAST-ADL, 297 rules), LLM adaptation consistency dropped well below 90%, revealing a scalability limitation.
In-place tokenizer expansion recipe enables multilingual efficiency gains in pre-trained LLMs
Researchers present a method for expanding a pre-trained LLM's tokenizer without retraining from scratch, using continued BPE merges on a multilingual corpus and a two-stage adaptation procedure (embedding-only training followed by full continued pre-training). Applied to LFM2-8B-A1B (a Mixture-of-Experts model) to produce LFM2.5-8B-A1B with a 128K tokenizer, the approach achieves 2.4–2.6× token compression for Hindi and Vietnamese (up to 4× for Thai), translating to an estimated 2.2–3.7× per-character decode speedup on reference devices. The technique is particularly relevant for on-device models where embedding and LM-head matrices constitute a material fraction of decode bandwidth, making vocabulary size a real efficiency constraint.
Cross-lingual relation extraction for Romanian: QLoRA fine-tuning narrows gap but small encoders remain competitive
Researchers evaluate cross-lingual relation extraction for Romanian by translating the SemEval-2010 Task 8 benchmark and testing Gemma 4 31B under zero-shot, few-shot, and QLoRA fine-tuned settings against encoder baselines (XLM-RoBERTa, Romanian BERT, RoBERTa-large). QLoRA fine-tuning improves macro F1 by over 22 percentage points and reduces the English-Romanian cross-lingual gap from 3.3 to 1.4 pp, but encoder baselines of 125M–560M parameters come within 1–4 pp of the fine-tuned 31B model. The study concludes that large LLMs offer limited advantage over compact encoders for single-task relation extraction in compute-constrained deployment scenarios, and releases the translated dataset, code, and models.