Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges
Most machine learning algorithms are configured by a set of hyperparameters whose values
must be carefully chosen and which often considerably impact performance. To avoid a time …
must be carefully chosen and which often considerably impact performance. To avoid a time …
Taking the human out of the loop: A review of Bayesian optimization
Big Data applications are typically associated with systems involving large numbers of
users, massive complex software systems, and large-scale heterogeneous computing and …
users, massive complex software systems, and large-scale heterogeneous computing and …
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
The conventional recipe for maximizing model accuracy is to (1) train multiple models with
various hyperparameters and (2) pick the individual model which performs best on a held …
various hyperparameters and (2) pick the individual model which performs best on a held …
Advances and open problems in federated learning
Federated learning (FL) is a machine learning setting where many clients (eg, mobile
devices or whole organizations) collaboratively train a model under the orchestration of a …
devices or whole organizations) collaboratively train a model under the orchestration of a …
Mobilenetv2: Inverted residuals and linear bottlenecks
In this paper we describe a new mobile architecture, mbox {MobileNetV2}, that improves the
state of the art performance of mobile models on multiple tasks and benchmarks as well as …
state of the art performance of mobile models on multiple tasks and benchmarks as well as …
Learning transferable architectures for scalable image recognition
B Zoph, V Vasudevan, J Shlens… - Proceedings of the …, 2018 - openaccess.thecvf.com
Develo** neural network image classification models often requires significant
architecture engineering. In this paper, we study a method to learn the model architectures …
architecture engineering. In this paper, we study a method to learn the model architectures …
AutoML: A survey of the state-of-the-art
Deep learning (DL) techniques have obtained remarkable achievements on various tasks,
such as image recognition, object detection, and language modeling. However, building a …
such as image recognition, object detection, and language modeling. However, building a …
[PDF][PDF] Hyperparameter optimization
Recent interest in complex and computationally expensive machine learning models with
many hyperparameters, such as automated machine learning (AutoML) frameworks and …
many hyperparameters, such as automated machine learning (AutoML) frameworks and …
Hyperband: A novel bandit-based approach to hyperparameter optimization
Performance of machine learning algorithms depends critically on identifying a good set of
hyperparameters. While recent approaches use Bayesian optimization to adaptively select …
hyperparameters. While recent approaches use Bayesian optimization to adaptively select …
Deep networks with stochastic depth
Very deep convolutional networks with hundreds of layers have led to significant reductions
in error on competitive benchmarks. Although the unmatched expressiveness of the many …
in error on competitive benchmarks. Although the unmatched expressiveness of the many …