ties-merging-ad6836a1·1 events·first seen Aliases: TIES-Merging
A new arXiv preprint introduces DeltaMerge-LowRes, a method for adapting multilingual encoders to new languages and tasks in low-resource settings by training language and task deltas separately and composing them in weight space at inference. The paper proposes four composition rules including a novel cross-axis TIES variant that adapts the TIES-Merging algorithm to language/task axes rather than task/task axes. Evaluated across four task families and four African languages (158 cells), cross-axis TIES improves summarization by +4 to +7 chrF and QA F1 by +2.32, while sparsity-aware merging reduces calibration error by 36%. The work advances model merging techniques for low-resource multilingual NLP.