Abstract Large Language Models (LLMs), despite their recent impressive accomplishments,
are notably cost-prohibitive to deploy, particularly for applications involving long-content …

Large language models (LLMs) have brought significant and transformative changes in
human society. These models have demonstrated remarkable capabilities in natural …

We consider the problem of training a multi-layer over-parametrized neural network to
minimize the empirical risk induced by a loss function. In the typical setting of over …

In this work, we improved the analysis of the running time of SparseGPT [Frantar, Alistarh
ICML 2023] from $ O (d^{3}) $ to $ O (d^{\omega}+ d^{2+ a+ o (1)}+ d^{1+\omega (1, 1, a)-a}) …

Large language models (LLMs) have demonstrated exceptional performance across a wide
range of tasks. These models, powered by advanced deep learning techniques, have …

Attention mechanism is the key to large language models, and attention matrix serves as an
algorithmic and computational bottleneck for such a scheme. In this paper, we define two …

Large language models (LLMs) have numerous real-life applications across various
domains, such as natural language translation, sentiment analysis, language modeling …

Quadratic programming is a fundamental problem in the field of convex optimization. Many
practical tasks can be formulated as quadratic programming, for example, the support vector …

Large language models (LLMs), such as ChatGPT and GPT4, have shown outstanding
performance in many human life task. Attention computation plays an important role in …

Given a convex function $ f $ on $\mathbb {R}^ n $ with an integer minimizer, we show how
to find an exact minimizer of $ f $ using $ O (n^ 2\log n) $ calls to a separation oracle and …