Understanding generalization and optimization performance of deep CNNs
This work aims to provide understandings on the remarkable success of deep convolutional
neural networks (CNNs) by theoretically analyzing their generalization performance and …
neural networks (CNNs) by theoretically analyzing their generalization performance and …
Uncertainty quantification for online learning and stochastic approximation via hierarchical incremental gradient descent
WJ Su, Y Zhu - arxiv preprint arxiv:1802.04876, 2018 - arxiv.org
Stochastic gradient descent (SGD) is an immensely popular approach for online learning in
settings where data arrives in a stream or data sizes are very large. However, despite an …
settings where data arrives in a stream or data sizes are very large. However, despite an …
Local convergence properties of SAGA/Prox-SVRG and acceleration
In this paper, we present a local convergence anal-ysis for a class of stochastic optimisation
meth-ods: the proximal variance reduced stochastic gradient methods, and mainly focus on …
meth-ods: the proximal variance reduced stochastic gradient methods, and mainly focus on …
Fast rates for bandit optimization with upper-confidence frank-wolfe
We consider the problem of bandit optimization, inspired by stochastic optimization and
online learning problems with bandit feedback. In this problem, the objective is to minimize a …
online learning problems with bandit feedback. In this problem, the objective is to minimize a …
Stochastic Composite Least-Squares Regression with Convergence Rate
We consider the minimization of composite objective functions composed of the expectation
of quadratic functions and an arbitrary convex function. We study the stochastic dual …
of quadratic functions and an arbitrary convex function. We study the stochastic dual …
Optimization of smooth functions with noisy observations: Local minimax rates
We consider the problem of global optimization of an unknown non-convex smooth function
with noisy zeroth-order feedback. We propose a local minimax framework to study the …
with noisy zeroth-order feedback. We propose a local minimax framework to study the …
Higrad: Uncertainty quantification for online learning and stochastic approximation
WJ Su, Y Zhu - Journal of Machine Learning Research, 2023 - jmlr.org
Stochastic gradient descent (SGD) is an immensely popular approach for online learning in
settings where data arrives in a stream or data sizes are very large. However, despite an …
settings where data arrives in a stream or data sizes are very large. However, despite an …
Optimization of smooth functions with noisy observations: Local minimax rates
We consider the problem of global optimization of an unknown non-convex smooth function
with noisy zeroth-order feedback. We propose a local minimax framework to study the …
with noisy zeroth-order feedback. We propose a local minimax framework to study the …
Asymptotic Behaviors and Phase Transitions in Projected Stochastic Approximation: A Jump Diffusion Approach
In this paper we consider linearly constrained optimization problems and propose a loopless
projection stochastic approximation (LPSA) algorithm. It performs the projection with …
projection stochastic approximation (LPSA) algorithm. It performs the projection with …
Stochastic approximation and least-squares regression, with applications to machine learning
N Flammarion - 2017 - theses.hal.science
Many problems in machine learning are naturally cast as the minimization of a smooth
function defined on a Euclidean space. For supervised learning, this includes least-squares …
function defined on a Euclidean space. For supervised learning, this includes least-squares …