Normalization techniques in training dnns: Methodology, analysis and application

L Huang, J Qin, Y Zhou, F Zhu, L Liu… - IEEE transactions on …, 2023 - ieeexplore.ieee.org
Normalization techniques are essential for accelerating the training and improving the
generalization of deep neural networks (DNNs), and have successfully been used in various …

A comprehensive survey on model compression and acceleration

T Choudhary, V Mishra, A Goswami… - Artificial Intelligence …, 2020 - Springer
In recent years, machine learning (ML) and deep learning (DL) have shown remarkable
improvement in computer vision, natural language processing, stock prediction, forecasting …

Snapfusion: Text-to-image diffusion model on mobile devices within two seconds

Y Li, H Wang, Q **, J Hu… - Advances in …, 2023 - proceedings.neurips.cc
Text-to-image diffusion models can create stunning images from natural language
descriptions that rival the work of professional artists and photographers. However, these …

Pruning and quantization for deep neural network acceleration: A survey

T Liang, J Glossner, L Wang, S Shi, X Zhang - Neurocomputing, 2021 - Elsevier
Deep neural networks have been applied in many applications exhibiting extraordinary
abilities in the field of computer vision. However, complex network architectures challenge …

Autoformer: Searching transformers for visual recognition

M Chen, H Peng, J Fu, H Ling - Proceedings of the IEEE …, 2021 - openaccess.thecvf.com
Recently, pure transformer-based models have shown great potentials for vision tasks such
as image classification and detection. However, the design of transformer networks is …

Ghostnet: More features from cheap operations

K Han, Y Wang, Q Tian, J Guo… - Proceedings of the …, 2020 - openaccess.thecvf.com
Deploying convolutional neural networks (CNNs) on embedded devices is difficult due to the
limited memory and computation resources. The redundancy in feature maps is an important …

Detectors: Detecting objects with recursive feature pyramid and switchable atrous convolution

S Qiao, LC Chen, A Yuille - … of the IEEE/CVF conference on …, 2021 - openaccess.thecvf.com
Many modern object detectors demonstrate outstanding performances by using the
mechanism of looking and thinking twice. In this paper, we explore this mechanism in the …

Dynamic convolution: Attention over convolution kernels

Y Chen, X Dai, M Liu, D Chen… - Proceedings of the …, 2020 - openaccess.thecvf.com
Light-weight convolutional neural networks (CNNs) suffer performance degradation as their
low computational budgets constrain both the depth (number of convolution layers) and the …

Revisiting random channel pruning for neural network compression

Y Li, K Adamczewski, W Li, S Gu… - Proceedings of the …, 2022 - openaccess.thecvf.com
Channel (or 3D filter) pruning serves as an effective way to accelerate the inference of
neural networks. There has been a flurry of algorithms that try to solve this practical problem …

Be your own teacher: Improve the performance of convolutional neural networks via self distillation

L Zhang, J Song, A Gao, J Chen… - Proceedings of the …, 2019 - openaccess.thecvf.com
Convolutional neural networks have been widely deployed in various application scenarios.
In order to extend the applications' boundaries to some accuracy-crucial domains …