[HTML][HTML] Hyperparameter optimization: Classics, acceleration, online, multi-objective, and tools

JM Tan, H Liao, W Liu, C Fan, J Huang… - Mathematical …, 2024‏ - aimspress.com
Hyperparameter optimization (HPO) has been well-developed and evolved into a well-
established research topic over the decades. With the success and wide application of deep …

Recycle: Fast and efficient long time series forecasting with residual cyclic transformers

A Weyrauch, T Steens, O Taubert… - … IEEE Conference on …, 2024‏ - ieeexplore.ieee.org
Transformers have recently gained prominence in long time series forecasting by elevating
accuracies in a variety of use cases. Regrettably, in the race for better predictive …

Short paper: Accelerating hyperparameter optimization algorithms with mixed precision

M Aach, R Sarma, E Inanc, M Riedel… - Proceedings of the SC' …, 2023‏ - dl.acm.org
Hyperparameter Optimization (HPO) of Neural Networks (NNs) is a computationally
expensive procedure. On accelerators, such as NVIDIA Graphics Processing Units (GPUs) …

Harnessing Orthogonality to Train Low-Rank Neural Networks

D Coquelin, K Flügel, M Weiel, N Kiefer, C Debus… - ECAI 2024, 2024‏ - ebooks.iospress.nl
This study explores the learning dynamics of neural networks by analyzing the singular
value decomposition (SVD) of their weights throughout training. Our investigation reveals …

AutoPQ: Automating Quantile estimation from Point forecasts in the context of sustainability

S Meisenbacher, K Phipps, O Taubert, M Weiel… - arxiv preprint arxiv …, 2024‏ - arxiv.org
Optimizing smart grid operations relies on critical decision-making informed by uncertainty
quantification, making probabilistic forecasting a vital tool. Designing such forecasting …

Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradients

K Flügel, D Coquelin, M Weiel, A Streit… - arxiv preprint arxiv …, 2024‏ - arxiv.org
The gradients used to train neural networks are typically computed using backpropagation.
While an efficient way to obtain exact gradients, backpropagation is computationally …

Resource-Adaptive Successive Doubling for Hyperparameter Optimization with Large Datasets on High-Performance Computing Systems

M Aach, R Sarma, H Neukirchen, M Riedel… - arxiv preprint arxiv …, 2024‏ - arxiv.org
On High-Performance Computing (HPC) systems, several hyperparameter configurations
can be evaluated in parallel to speed up the Hyperparameter Optimization (HPO) process …

PETNet–Coincident Particle Event Detection using Spiking Neural Networks

J Debus, C Debus, G Dissertori… - 2024 Neuro Inspired …, 2024‏ - ieeexplore.ieee.org
Spiking neural networks (SNN) hold the promise of being a more biologically plausible, low-
energy alternative to conventional artificial neural networks. Their time-variant nature makes …

Parallel and Scalable Hyperparameter Optimization for Distributed Deep Learning Methods on High-Performance Computing Systems

M Aach - 2025‏ - opinvisindi.is
The design of Deep Learning (DL) models is a complex task, involving decisions on the
general architecture of the model (eg, the number of layers of the Neural Network (NN)) and …

Design of Cluster-Computing Architecture to Improve Training Speed of the Neuroevolution Algorithm

I Omelianenko - International Congress on Information and …, 2024‏ - Springer
In this paper, we review the key features and major drawbacks of the NeuroEvolution of
Augmenting Topologies (NEAT) algorithm, such as slow training speed that limits its area of …