A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations

H Cheng, M Zhang, JQ Shi - IEEE Transactions on Pattern …, 2024 - ieeexplore.ieee.org
Modern deep neural networks, particularly recent large language models, come with
massive model sizes that require significant computational and storage resources. To …

1xn pattern for pruning convolutional neural networks

M Lin, Y Zhang, Y Li, B Chen, F Chao… - … on Pattern Analysis …, 2022 - ieeexplore.ieee.org
Though network pruning receives popularity in reducing the complexity of convolutional
neural networks (CNNs), it remains an open issue to concurrently maintain model accuracy …

Trainability preserving neural pruning

H Wang, Y Fu - arxiv preprint arxiv:2207.12534, 2022 - arxiv.org
Many recent works have shown trainability plays a central role in neural network pruning--
unattended broken trainability can lead to severe under-performance and unintentionally …

A survey of model compression strategies for object detection

Z Lyu, T Yu, F Pan, Y Zhang, J Luo, D Zhang… - Multimedia tools and …, 2024 - Springer
Deep neural networks (DNNs) have achieved great success in many object detection tasks.
However, such DNNS-based large object detection models are generally computationally …

Sparse double descent: Where network pruning aggravates overfitting

Z He, Z **e, Q Zhu, Z Qin - International Conference on …, 2022 - proceedings.mlr.press
People usually believe that network pruning not only reduces the computational cost of deep
networks, but also prevents overfitting by decreasing model capacity. However, our work …

Why is the state of neural network pruning so confusing? on the fairness, comparison setup, and trainability in network pruning

H Wang, C Qin, Y Bai, Y Fu - arxiv preprint arxiv:2301.05219, 2023 - arxiv.org
The state of neural network pruning has been noticed to be unclear and even confusing for a
while, largely due to" a lack of standardized benchmarks and metrics"[3]. To standardize …

Prune and tune ensembles: low-cost ensemble learning with sparse independent subnetworks

T Whitaker, D Whitley - Proceedings of the AAAI Conference on Artificial …, 2022 - ojs.aaai.org
Ensemble Learning is an effective method for improving generalization in machine learning.
However, as state-of-the-art neural networks grow larger, the computational cost associated …

Channel pruning method for signal modulation recognition deep learning models

Z Chen, Z Wang, X Gao, J Zhou, D Xu… - IEEE Transactions …, 2023 - ieeexplore.ieee.org
Automatic modulation recognition (AMR) plays an important role in communication system.
With the expansion of data volume and the development of computing power, deep learning …

Validating the lottery ticket hypothesis with inertial manifold theory

Z Zhang, J **, Z Zhang, Y Zhou… - Advances in neural …, 2021 - proceedings.neurips.cc
Despite achieving remarkable efficiency, traditional network pruning techniques often follow
manually-crafted heuristics to generate pruned sparse networks. Such heuristic pruning …

Dimensionality reduced training by pruning and freezing parts of a deep neural network: a survey

P Wimmer, J Mehnert, AP Condurache - Artificial Intelligence Review, 2023 - Springer
State-of-the-art deep learning models have a parameter count that reaches into the billions.
Training, storing and transferring such models is energy and time consuming, thus costly. A …