We investigate the statistical and computational limits of latent Diffusion Transformers (DiTs)
under the low-dimensional linear latent space assumption. Statistically, we study the …

We introduce an Outlier-Efficient Modern Hopfield Model (termed $\mathrm {OutEffHop} $)
and use it to address the outlier inefficiency problem of {training} gigantic transformer-based …

Tensor Attention, a multi-view attention that is able to capture high-order correlations among
multiple modalities, can overcome the representational limitations of classical matrix …

We propose a two-stage memory retrieval dynamics for modern Hopfield models, termed
$\mathtt {U\text {-} Hop} $, with enhanced memory capacity. Our key contribution is a …

We investigate the computational limits of the memory retrieval dynamics of modern Hopfield
models from the fine-grained complexity analysis. Our key contribution is the …

The computational complexity of the self-attention mechanism in popular transformer
architectures poses significant challenges for training and inference, and becomes the …

Large Language Models (LLMs) have demonstrated remarkable capabilities across various
applications, but their performance on long-context tasks is often limited by the …

Large language models (LLMs) such as GPT-4 sometimes appear to be creative, solving
novel tasks often with a few demonstrations in the prompt. These tasks require the models to …

Large language models (LLMs) are known for their exceptional performance in natural
language processing, making them highly effective in many human life-related or even job …