Bridging the gap between low-rank and orthogonal adaptation via householder reflection adaptation

S Yuan, H Liu, H Xu - Advances in Neural Information …, 2025 - proceedings.neurips.cc
While following different technical routes, both low-rank and orthogonal adaptation
techniques can efficiently adapt large-scale pre-training models in specific tasks or domains …

Efficient Learning With Sine-Activated Low-rank Matrices

Y Ji, H Saratchandran, C Gordon, Z Zhang… - arxiv preprint arxiv …, 2024 - arxiv.org
Low-rank decomposition has emerged as a vital tool for enhancing parameter efficiency in
neural network architectures, gaining traction across diverse applications in machine …