Recent work on pruning large language models (LLMs) has shown that one can eliminate a
large number of parameters without compromising performance, making pruning a …

In this manuscript we consider the problem of generalized linear estimation on Gaussian
mixture data with labels given by a single-index model. Our first result is a sharp asymptotic …

Feature learning is thought to be one of the fundamental reasons for the success of deep
neural networks. It is rigorously known that in two-layer fully-connected neural networks …

Regularization in modern machine learning is crucial, and it can take various forms in
algorithmic design: training set, model family, error function, regularization terms, and …

Topologically associating domains (TADs) are critical structural units in three-dimensional
genome organization of mammalian genome. Dynamic reorganizations of TADs between …

We develop a stochastic differential equation, called homogenized SGD, for analyzing the
dynamics of stochastic gradient descent (SGD) on a high-dimensional random least squares …

Overparametrization often helps improve the generalization performance. This paper
presents a dual view of overparametrization suggesting that downsampling may also help …

Evaluating the performance of machine learning models under distribution shifts is
challenging, especially when we only have unlabeled data from the shifted (target) domain …

Parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), are known
to enhance training efficiency in Large Language Models (LLMs). Due to the limited …

Modern deep neural networks (DNNs) are extremely powerful; however, this comes at the
price of increased depth and having more parameters per layer, making their training and …