Zeroth-order (ZO) optimization is a subset of gradient-free optimization that emerges in many
signal processing and machine learning (ML) applications. It is used for solving optimization …

Fine-tuning language models (LMs) has yielded success on diverse downstream tasks, but
as LMs grow in size, backpropagation requires a prohibitively large amount of memory …

In many optimization problems arising from scientific, engineering and artificial intelligence
applications, objective and constraint functions are available only as the output of a black …

Large language models~(LLMs) are instruction followers, but it can be challenging to find
the best instruction for different situations, especially for black-box LLMs on which …

Current model extraction attacks assume that the adversary has access to a surrogate
dataset with characteristics similar to the proprietary data used to train the victim model. This …

Recent studies have shown that adversarial examples in state-of-the-art image classifiers
trained by deep neural networks (DNN) can be easily generated when the target model is …

In the evolving landscape of natural language processing (NLP), fine-tuning pre-trained
Large Language Models (LLMs) with first-order (FO) optimizers like SGD and Adam has …

We study derivative-free methods for policy optimization over the class of linear policies. We
focus on characterizing the convergence rate of these methods when applied to linear …

Nonsmooth nonconvex optimization problems broadly emerge in machine learning and
business decision making, whereas two core challenges impede the development of …

As application demands for zeroth-order (gradient-free) optimization accelerate, the need for
variance reduced and faster converging approaches is also intensifying. This paper …