κ-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating

κ-LoRA uses condition numbers to selectively update LoRA matrices, cutting fine-tuning cost

A new arXiv preprint introduces κ-LoRA, a method that uses matrix condition numbers to identify which LoRA weight matrices are most worth updating during fine-tuning. By restricting updates to the top 50% of matrices ranked by condition number, κ-LoRA halves the trainable parameter count, reduces fine-tuning time by 16.2% on average, and cuts memory cost by 4.5% while matching standard LoRA accuracy. The core finding — that matrices with larger condition numbers drive most adaptation gains — is presented as a novel theoretical contribution motivating the selective approach. The method targets resource-constrained settings including edge and on-device fine-tuning.