A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations

H Cheng, M Zhang, JQ Shi - IEEE Transactions on Pattern …, 2024‏ - ieeexplore.ieee.org
Modern deep neural networks, particularly recent large language models, come with
massive model sizes that require significant computational and storage resources. To …

[HTML][HTML] AutoML: A systematic review on automated machine learning with neural architecture search

I Salehin, MS Islam, P Saha, SM Noman, A Tuni… - Journal of Information …, 2024‏ - Elsevier
Abstract AutoML (Automated Machine Learning) is an emerging field that aims to automate
the process of building machine learning models. AutoML emerged to increase productivity …

A comprehensive survey of neural architecture search: Challenges and solutions

P Ren, Y **ao, X Chang, PY Huang, Z Li… - ACM Computing …, 2021‏ - dl.acm.org
Deep learning has made substantial breakthroughs in many fields due to its powerful
automatic representation capabilities. It has been proven that neural architecture design is …

[HTML][HTML] Automated machine learning: Review of the state-of-the-art and opportunities for healthcare

J Waring, C Lindvall, R Umeton - Artificial intelligence in medicine, 2020‏ - Elsevier
Objective This work aims to provide a review of the existing literature in the field of
automated machine learning (AutoML) to help healthcare professionals better utilize …

AutoML: A survey of the state-of-the-art

X He, K Zhao, X Chu - Knowledge-based systems, 2021‏ - Elsevier
Deep learning (DL) techniques have obtained remarkable achievements on various tasks,
such as image recognition, object detection, and language modeling. However, building a …

Model compression and hardware acceleration for neural networks: A comprehensive survey

L Deng, G Li, S Han, L Shi, Y **e - Proceedings of the IEEE, 2020‏ - ieeexplore.ieee.org
Domain-specific hardware is becoming a promising topic in the backdrop of improvement
slow down for general-purpose processors due to the foreseeable end of Moore's Law …

Full stack optimization of transformer inference: a survey

S Kim, C Hooper, T Wattanawong, M Kang… - arxiv preprint arxiv …, 2023‏ - arxiv.org
Recent advances in state-of-the-art DNN architecture design have been moving toward
Transformer models. These models achieve superior accuracy across a wide range of …

Pc-darts: Partial channel connections for memory-efficient architecture search

Y Xu, L **e, X Zhang, X Chen, GJ Qi, Q Tian… - arxiv preprint arxiv …, 2019‏ - arxiv.org
Differentiable architecture search (DARTS) provided a fast solution in finding effective
network architectures, but suffered from large memory and computing overheads in jointly …

Single path one-shot neural architecture search with uniform sampling

Z Guo, X Zhang, H Mu, W Heng, Z Liu, Y Wei… - Computer vision–ECCV …, 2020‏ - Springer
We revisit the one-shot Neural Architecture Search (NAS) paradigm and analyze its
advantages over existing NAS approaches. Existing one-shot method, however, is hard to …

Progressive differentiable architecture search: Bridging the depth gap between search and evaluation

X Chen, L **e, J Wu, Q Tian - Proceedings of the IEEE/CVF …, 2019‏ - openaccess.thecvf.com
Recently, differentiable search methods have made major progress in reducing the
computational costs of neural architecture search. However, these approaches often report …