A survey of gradient methods for solving nonlinear optimization
I Branislav, M Haifeng, M Dijana - Electronic research archive, 2020 - aimsciences.org
The paper surveys, classifies and investigates theoretically and numerically main classes of
line search methods for unconstrained optimization. Quasi-Newton (QN) and conjugate …
line search methods for unconstrained optimization. Quasi-Newton (QN) and conjugate …
Two-phase switching optimization strategy in deep neural networks
HH Tan, KH Lim - IEEE transactions on neural networks and …, 2020 - ieeexplore.ieee.org
Optimization in a deep neural network is always challenging due to the vanishing gradient
problem and intensive fine-tuning of network hyperparameters. Inspired by multistage …
problem and intensive fine-tuning of network hyperparameters. Inspired by multistage …
On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization
A bang-bang iteration method equipped with a component-wise line search strategy is
introduced to solve unconstrained optimization problems. The main idea of this method is to …
introduced to solve unconstrained optimization problems. The main idea of this method is to …
Nonlinear Least Squares Problems Using Approximate Greatest Descent Method
Many numerical methods have been developed and modified to solve nonlinear least
squares (NLS) problems as unconstrained optimization problems. One of the challenges …
squares (NLS) problems as unconstrained optimization problems. One of the challenges …