Neural architecture search: Insights from 1000 papers

C White, M Safari, R Sukthanker, B Ru, T Elsken… - arxiv preprint arxiv …, 2023 - arxiv.org
In the past decade, advances in deep learning have resulted in breakthroughs in a variety of
areas, including computer vision, natural language understanding, speech recognition, and …

Stronger NAS with weaker predictors

J Wu, X Dai, D Chen, Y Chen, M Liu… - Advances in …, 2021 - proceedings.neurips.cc
Abstract Neural Architecture Search (NAS) often trains and evaluates a large number of
architectures. Recent predictor-based NAS approaches attempt to alleviate such heavy …

Pinat: a permutation invariance augmented transformer for nas predictor

S Lu, Y Hu, P Wang, Y Han, J Tan, J Li… - Proceedings of the …, 2023 - ojs.aaai.org
Time-consuming performance evaluation is the bottleneck of traditional Neural Architecture
Search (NAS) methods. Predictor-based NAS can speed up performance evaluation by …

Nas-bench-x11 and the power of learning curves

S Yan, C White, Y Savani… - Advances in Neural …, 2021 - proceedings.neurips.cc
While early research in neural architecture search (NAS) required extreme computational
resources, the recent releases of tabular and surrogate benchmarks have greatly increased …

Exploring the loss landscape in neural architecture search

C White, S Nolen, Y Savani - Uncertainty in Artificial …, 2021 - proceedings.mlr.press
Neural architecture search (NAS) has seen a steep rise in interest over the last few years.
Many algorithms for NAS consist of searching through a space of architectures by iteratively …

FlowerFormer: Empowering Neural Architecture Encoding using a Flow-aware Graph Transformer

D Hwang, H Kim, S Kim, K Shin - Proceedings of the IEEE …, 2024 - openaccess.thecvf.com
The success of a specific neural network architecture is closely tied to the dataset and task it
tackles; there is no one-size-fits-all solution. Thus considerable efforts have been made to …

[PDF][PDF] Graph Masked Autoencoder Enhanced Predictor for Neural Architecture Search.

K **g, J Xu, P Li - IJCAI, 2022 - ijcai.org
Performance estimation of neural architecture is a crucial component of neural architecture
search (NAS). Meanwhile, neural predictor is a current mainstream performance estimation …

Pace: A parallelizable computation encoder for directed acyclic graphs

Z Dong, M Zhang, F Li, Y Chen - International conference on …, 2022 - proceedings.mlr.press
Optimization of directed acyclic graph (DAG) structures has many applications, such as
neural architecture search (NAS) and probabilistic graphical model learning. Encoding …

DiffusionNAG: predictor-guided neural architecture generation with diffusion models

S An, H Lee, J Jo, S Lee, SJ Hwang - arxiv preprint arxiv:2305.16943, 2023 - arxiv.org
Existing NAS methods suffer from either an excessive amount of time for repetitive sampling
and training of many task-irrelevant architectures. To tackle such limitations of existing NAS …

On Latency Predictors for Neural Architecture Search

Y Akhauri, M Abdelfattah - Proceedings of Machine Learning …, 2024 - proceedings.mlsys.org
Efficient deployment of neural networks (NN) requires the co-optimization of accuracy and
latency. For example, hardware-aware neural architecture search has been used to …