Transformers with linear attention (ie, linear transformers) and state-space models have
recently been suggested as a viable linear-time alternative to transformers with softmax …

Flow models are effective at progressively generating realistic images, but they generally
struggle to capture long-range dependencies during the generation process as they …

Diffusion Transformers (DiT) have become a leading architecture in image generation.
However, the quadratic complexity of attention mechanisms, which are responsible for …

An essential component of modern recurrent sequence models is the* forget gate*. While
Transformers do not have an explicit recurrent form, we show that a forget gate can be …

Sampling operations in discrete space are widely used in different fields such as language
models, reinforcement learning, VAE, GAN, and neural architecture search. Current …