A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations

H Cheng, M Zhang, JQ Shi - IEEE Transactions on Pattern …, 2024 - ieeexplore.ieee.org
Modern deep neural networks, particularly recent large language models, come with
massive model sizes that require significant computational and storage resources. To …

An overview of deep learning methods for multimodal medical data mining

F Behrad, MS Abadeh - Expert Systems with Applications, 2022 - Elsevier
Deep learning methods have achieved significant results in various fields. Due to the
success of these methods, many researchers have used deep learning algorithms in …

Eagles: Efficient accelerated 3d gaussians with lightweight encodings

S Girish, K Gupta, A Shrivastava - European Conference on Computer …, 2024 - Springer
Abstract Recently, 3D Gaussian splatting (3D-GS) has gained popularity in novel-view
scene synthesis. It addresses the challenges of lengthy training times and slow rendering …

Pruning neural networks without any data by iteratively conserving synaptic flow

H Tanaka, D Kunin, DL Yamins… - Advances in neural …, 2020 - proceedings.neurips.cc
Pruning the parameters of deep neural networks has generated intense interest due to
potential savings in time, memory and energy both during training and at test time. Recent …

Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment

L Xu, H **e, SZJ Qin, X Tao, FL Wang - arxiv preprint arxiv:2312.12148, 2023 - arxiv.org
With the continuous growth in the number of parameters of transformer-based pretrained
language models (PLMs), particularly the emergence of large language models (LLMs) with …

Sparse training via boosting pruning plasticity with neuroregeneration

S Liu, T Chen, X Chen, Z Atashgahi… - Advances in …, 2021 - proceedings.neurips.cc
Works on lottery ticket hypothesis (LTH) and single-shot network pruning (SNIP) have raised
a lot of attention currently on post-training pruning (iterative magnitude pruning), and before …

Do we actually need dense over-parameterization? in-time over-parameterization in sparse training

S Liu, L Yin, DC Mocanu… - … on Machine Learning, 2021 - proceedings.mlr.press
In this paper, we introduce a new perspective on training deep neural networks capable of
state-of-the-art performance without the need for the expensive over-parameterization by …

Can subnetwork structure be the key to out-of-distribution generalization?

D Zhang, K Ahuja, Y Xu, Y Wang… - … on Machine Learning, 2021 - proceedings.mlr.press
Can models with particular structure avoid being biased towards spurious correlation in out-
of-distribution (OOD) generalization? Peters et al.(2016) provides a positive answer for …

Towards provably efficient quantum algorithms for large-scale machine-learning models

J Liu, M Liu, JP Liu, Z Ye, Y Wang, Y Alexeev… - Nature …, 2024 - nature.com
Large machine learning models are revolutionary technologies of artificial intelligence
whose bottlenecks include huge computational expenses, power, and time used both in the …

Model sparsity can simplify machine unlearning

J Liu, P Ram, Y Yao, G Liu, Y Liu… - Advances in Neural …, 2024 - proceedings.neurips.cc
In response to recent data regulation requirements, machine unlearning (MU) has emerged
as a critical process to remove the influence of specific examples from a given model …