cktn-d86a57d1·1 events·first seen Aliases: CKTN
Researchers introduce CKTN, the first NLP corpus and benchmark for three underrepresented Vietnamese ethnic minority languages (Cham, Khmer, Tay-Nung), comprising 44,367 documents and 24M subword tokens across pretraining, classification, and retrieval tasks. The paper demonstrates that standard multilingual encoders severely fragment these languages and that common adaptation metrics (language-modeling loss, lexical-overlap retrieval) can be misleading proxies for semantic generalization. The authors propose a script-aware adaptation recipe combining vocabulary augmentation with calibrated replaced-token pretraining, achieving substantially reduced fragmentation and best-in-class classification performance. The work highlights systematic gaps in multilingual NLP coverage and evaluation methodology for low-resource, script-diverse languages.