Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks

T Hoefler, D Alistarh, T Ben-Nun, N Dryden… - Journal of Machine …, 2021 - jmlr.org
The growing energy and performance costs of deep learning have driven the community to
reduce the size of neural networks by selectively pruning components. Similarly to their …

The combinatorial brain surgeon: Pruning weights that cancel one another in neural networks

X Yu, T Serra, S Ramalingam… - … Conference on Machine …, 2022 - proceedings.mlr.press
Neural networks tend to achieve better accuracy with training if they are larger {—} even if
the resulting models are overparameterized. Nevertheless, carefully removing such excess …

Filter pruning by switching to neighboring CNNs with good attributes

Y He, P Liu, L Zhu, Y Yang - IEEE Transactions on Neural …, 2022 - ieeexplore.ieee.org
Filter pruning is effective to reduce the computational costs of neural networks. Existing
methods show that updating the previous pruned filter would enable large model capacity …

Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments

A Boggust, V Sivaraman, Y Assogba… - … on Visualization and …, 2024 - ieeexplore.ieee.org
To deploy machine learning models on-device, practitioners use compression algorithms to
shrink and speed up models while maintaining their high-quality output. A critical aspect of …

FCHP: Exploring the discriminative feature and feature correlation of feature maps for hierarchical DNN pruning and compression

H Zhang, L Liu, H Zhou, L Si, H Sun… - IEEE Transactions on …, 2022 - ieeexplore.ieee.org
Pruning can remove the redundant parameters and structures of Deep Neural Networks
(DNNs) to reduce inference time and memory overhead. As one of the important …

Scaling up exact neural network compression by ReLU stability

T Serra, X Yu, A Kumar… - Advances in neural …, 2021 - proceedings.neurips.cc
We can compress a rectifier network while exactly preserving its underlying functionality with
respect to a given input domain if some of its neurons are stable. However, current …

Adaptive Renewable Energy Forecasting Utilizing a Data-Driven PCA–Transformer Architecture

F Saeed, S Aldera - IEEE Access, 2024 - ieeexplore.ieee.org
The incorporation of renewable energy sources into the power grid has necessitated the
development of sophisticated forecasting models that can effectively handle the inherent …

Toward compact deep neural networks via energy-aware pruning

SK Yeom, KH Shim, JH Hwang - arxiv preprint arxiv:2103.10858, 2021 - arxiv.org
Despite the remarkable performance, modern deep neural networks are inevitably
accompanied by a significant amount of computational cost for learning and deployment …

Structured LISTA for multidimensional harmonic retrieval

R Fu, Y Liu, T Huang, YC Eldar - IEEE transactions on signal …, 2021 - ieeexplore.ieee.org
Learned iterative shrinkage thresholding algorithm (LISTA), which adopts deep learning
techniques to optimize algorithm parameters from labeled training data, can be successfully …

Layer-wise data-free cnn compression

M Horton, Y **, A Farhadi… - 2022 26th International …, 2022 - ieeexplore.ieee.org
We present a computationally efficient method for compressing a trained neural network
without using real data. We break the problem of data-free network compression into …