In the paper, we propose a class of accelerated zeroth-order and first-order momentum
methods for both nonconvex mini-optimization and minimax-optimization. Specifically, we …

In this paper, we consider a stochastic distributed nonconvex optimization problem with the
cost function being distributed over n agents having access only to zeroth-order (ZO) …

Abstract Distance Metric Learning aims to learn an appropriate metric that faithfully
measures the distance between two data points. Traditional metric learning methods usually …

Recently, many stochastic variance reduced alternating direction methods of multipliers
(ADMMs)(eg, SAG-ADMM and SVRG-ADMM) have made exciting progress such as linear …

In the paper, we propose a class of accelerated stochastic gradient-free and projection-free
(aka, zeroth-order Frank-Wolfe) methods to solve the constrained stochastic and finite-sum …

In this paper, we introduce a novel framework for black-box prompt tuning with a subspace
learning and selection strategy, leveraging derivative-free optimization algorithms. This …

This article studies the distributed nonconvex optimization problem with nonsmooth
regularization, which has wide applications in decentralized learning, estimation, and …

Proximal gradient method has a major role in solving nonsmooth composite optimization
problems. However, in some machine learning problems related to black-box optimization …

Alternating direction method of multipliers (ADMM) is a popular first-order method owing to
its simplicity and efficiency. However, similar to other proximal splitting methods, the …

Zeroth-order (aka, derivative-free) methods are a class of effective optimization methods for
solving complex machine learning problems, where gradients of the objective functions are …