Consensus-based optimization methods converge globally

M Fornasier, T Klock, K Riedl - SIAM Journal on Optimization, 2024 - SIAM
In this paper we study consensus-based optimization (CBO), which is a multiagent
metaheuristic derivative-free optimization method that can globally minimize nonconvex …

Revisiting sampling for combinatorial optimization

H Sun, K Goshvadi, A Nova… - International …, 2023 - proceedings.mlr.press
Sampling approaches like Markov chain Monte Carlo were once popular for combinatorial
optimization, but the inefficiency of classical methods and the need for problem-specific …

Appropriate noise addition to metaheuristic algorithms can enhance their performance

KP Choi, EHH Kam, XT Tong, WK Wong - Scientific reports, 2023 - nature.com
Nature-inspired swarm-based algorithms are increasingly applied to tackle high-
dimensional and complex optimization problems across disciplines. They are general …

Stochastic Anderson mixing for nonconvex stochastic optimization

F Wei, C Bao, Y Liu - Advances in Neural Information …, 2021 - proceedings.neurips.cc
Anderson mixing (AM) is an acceleration method for fixed-point iterations. Despite its
success and wide usage in scientific computing, the convergence theory of AM remains …

State-dependent temperature control for Langevin diffusions

X Gao, ZQ Xu, XY Zhou - SIAM Journal on Control and Optimization, 2022 - SIAM
We study the temperature control problem for Langevin diffusions in the context of
nonconvex optimization. The classical optimal control of such a problem is of the bang-bang …

Failure of smooth pasting principle and nonexistence of equilibrium stop** rules under time-inconsistency

KS Tan, W Wei, XY Zhou - SIAM journal on control and optimization, 2021 - SIAM
This paper considers time-inconsistent stop** problems in which the inconsistency arises
from a class of nonexponential discount functions called weighted discount functions. We …

Maximum entropy differential dynamic programming

O So, Z Wang, EA Theodorou - 2022 International Conference …, 2022 - ieeexplore.ieee.org
In this paper, we present a novel maximum entropy formulation of the Differential Dynamic
Programming algorithm and derive two variants using unimodal and multimodal value …

On stationary-point hitting time and ergodicity of stochastic gradient Langevin dynamics

X Chen, SS Du, XT Tong - Journal of Machine Learning Research, 2020 - jmlr.org
Stochastic gradient Langevin dynamics (SGLD) is a fundamental algorithm in stochastic
optimization. Recent work by Zhang et al.(2017) presents an analysis for the hitting time of …

Second-Order Stein Variational Dynamic Optimization

Y Aoyama, P Lehmamnn, EA Theodorou - arxiv preprint arxiv:2409.04644, 2024 - arxiv.org
We present a novel second-order trajectory optimization algorithm based on Stein
Variational Newton's Method and Maximum Entropy Differential Dynamic Programming. The …

PCA matrix denoising is uniform

XT Tong, W Wang, Y Wang - arxiv preprint arxiv:2306.12690, 2023 - arxiv.org
Principal component analysis (PCA) is a simple and popular tool for processing high-
dimensional data. We investigate its effectiveness for matrix denoising. We assume iid high …