Model compression for deep neural networks: A survey

Z Li, H Li, L Meng - Computers, 2023 - mdpi.com
Currently, with the rapid development of deep learning, deep neural networks (DNNs) have
been widely applied in various computer vision tasks. However, in the pursuit of …

Model compression and hardware acceleration for neural networks: A comprehensive survey

L Deng, G Li, S Han, L Shi, Y **e - Proceedings of the IEEE, 2020 - ieeexplore.ieee.org
Domain-specific hardware is becoming a promising topic in the backdrop of improvement
slow down for general-purpose processors due to the foreseeable end of Moore's Law …

GhostNetv2: Enhance cheap operation with long-range attention

Y Tang, K Han, J Guo, C Xu, C Xu… - Advances in Neural …, 2022 - proceedings.neurips.cc
Light-weight convolutional neural networks (CNNs) are specially designed for applications
on mobile devices with faster inference speed. The convolutional operation can only capture …

A comprehensive survey on model compression and acceleration

T Choudhary, V Mishra, A Goswami… - Artificial Intelligence …, 2020 - Springer
In recent years, machine learning (ML) and deep learning (DL) have shown remarkable
improvement in computer vision, natural language processing, stock prediction, forecasting …

[PDF][PDF] Improved knowledge distillation via teacher assistant

SI Mirzadeh, M Farajtabar, A Li, N Levine… - Proceedings of the AAAI …, 2020 - aaai.org
Despite the fact that deep neural networks are powerful models and achieve appealing
results on many tasks, they are too large to be deployed on edge devices like smartphones …

Filter pruning via geometric median for deep convolutional neural networks acceleration

Y He, P Liu, Z Wang, Z Hu… - Proceedings of the IEEE …, 2019 - openaccess.thecvf.com
Previous works utilized" smaller-norm-less-important" criterion to prune filters with smaller
norm values in a convolutional neural network. In this paper, we analyze this norm-based …

A comprehensive survey on model quantization for deep neural networks in image classification

B Rokh, A Azarpeyvand, A Khanteymoori - ACM Transactions on …, 2023 - dl.acm.org
Recent advancements in machine learning achieved by Deep Neural Networks (DNNs)
have been significant. While demonstrating high accuracy, DNNs are associated with a …

Soft filter pruning for accelerating deep convolutional neural networks

Y He, G Kang, X Dong, Y Fu, Y Yang - arxiv preprint arxiv:1808.06866, 2018 - arxiv.org
This paper proposed a Soft Filter Pruning (SFP) method to accelerate the inference
procedure of deep Convolutional Neural Networks (CNNs). Specifically, the proposed SFP …

Distilling object detectors with fine-grained feature imitation

T Wang, L Yuan, X Zhang… - Proceedings of the IEEE …, 2019 - openaccess.thecvf.com
State-of-the-art CNN based recognition models are often computationally prohibitive to
deploy on low-end devices. A promising high level approach tackling this limitation is …

Model compression and acceleration for deep neural networks: The principles, progress, and challenges

Y Cheng, D Wang, P Zhou… - IEEE Signal Processing …, 2018 - ieeexplore.ieee.org
In recent years, deep neural networks (DNNs) have received increased attention, have been
applied to different applications, and achieved dramatic accuracy improvements in many …