Hadamard modulation

SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning

SMoA is a new parameter-efficient fine-tuning method that addresses LoRA's trade-off between rank size and parameter budget. It partitions model layers into spectral blocks and applies Hadamard-modulated low-rank branches to each diagonal block, enabling broader coverage of pretrained spectral directions without proportionally increasing trainable parameters. Theoretical analysis and empirical results on multiple tasks show SMoA outperforms LoRA and competitive LoRA-style baselines in lower-budget settings.