Heterogeneous lora for federated fine-tuning of on-device foundation models

YJ Cho, L Liu, Z Xu, A Fahrezi, G Joshi - arxiv preprint arxiv:2401.06432, 2024 - arxiv.org
Foundation models (FMs) adapt well to specific domains or tasks with fine-tuning, and
federated learning (FL) enables the potential for privacy-preserving fine-tuning of the FMs …

Towards super compressed neural networks for object identification: Quantized low-rank tensor decomposition with self-attention

B Liu, D Wang, Q Lv, Z Han, Y Tang - Electronics, 2024 - mdpi.com
Deep convolutional neural networks have a large number of parameters and require a
significant number of floating-point operations during computation, which limits their …

Efficient compression of overparameterized deep models through low-dimensional learning dynamics

SM Kwon, Z Zhang, D Song, L Balzano… - arxiv preprint arxiv …, 2023 - arxiv.org
Overparameterized models have proven to be powerful tools for solving various machine
learning tasks. However, overparameterization often leads to a substantial increase in …