We consider the zeroth-order optimization problem in the huge-scale setting, where the
dimension of the problem is so large that performing even basic vector operations on the …

In this study, we delve into an emerging optimization challenge involving a black-box
objective function that can only be gauged via a ranking oracle-a situation frequently …

Prompt-tuning has emerged as a promising method for adapting pre-trained models to
downstream tasks or aligning with human preferences. Prompt learning is widely used in …

First-order optimization algorithms are widely used today. Two standard building blocks in
these algorithms are proximal operators (proximals) and gradients. Although gradients can …

We consider the problem of minimizing a high-dimensional objective function, which may
include a regularization term, using only (possibly noisy) evaluations of the function. Such …

We study stochastic zeroth-order gradient and Hessian estimators for real-valued functions
in. We show that, via taking finite difference along random orthogonal directions, the …

Computing tasks may often be posed as optimization problems. The objective functions for
real-world scenarios are often nonconvex and/or nondifferentiable. State-of-the-art methods …

The paper discusses derivative-free optimization (DFO), which involves minimizing a
function without access to gradients or directional derivatives, only function evaluations …

We introduce and analyze Structured Stochastic Zeroth order Descent (S-SZD), a finite
difference approach that approximates a stochastic gradient on a set of l≤ d orthogonal …

This work considers stochastic optimization problems in which the objective function values
can only be computed by a blackbox corrupted by some random noise following an …