Alphapruning: Using heavy-tailed self regularization theory for improved layer-wise pruning of large language models

H Lu, Y Zhou, S Liu, Z Wang… - Advances in Neural …, 2025 - proceedings.neurips.cc
Recent work on pruning large language models (LLMs) has shown that one can eliminate a
large number of parameters without compromising performance, making pruning a …

Model balancing helps low-data training and fine-tuning

Z Liu, Y Hu, T Pang, Y Zhou, P Ren, Y Yang - arxiv preprint arxiv …, 2024 - arxiv.org
Recent advances in foundation models have emphasized the need to align pre-trained
models with specialized domains using small, curated datasets. Studies on these foundation …

Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning

P Cong, W Liu, W Yu, H Zhao, T Yang - arxiv preprint arxiv:2502.03884, 2025 - arxiv.org
Large language models (LLMs) have demonstrated remarkable success across various
tasks, accompanied by a continuous increase in their parameter size. Parameter-efficient …