Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers

N Ma, M Goldstein, MS Albergo, NM Boffi… - … on Computer Vision, 2024 - Springer
Abstract We present Scalable Interpolant Transformers (SiT), a family of generative models
built on the backbone of Diffusion Transformers (DiT). The interpolant framework, which …

Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions

S Chen, S Chewi, J Li, Y Li, A Salim… - arxiv preprint arxiv …, 2022 - arxiv.org
We provide theoretical convergence guarantees for score-based generative models (SGMs)
such as denoising diffusion probabilistic models (DDPMs), which constitute the backbone of …

Stochastic interpolants: A unifying framework for flows and diffusions

MS Albergo, NM Boffi, E Vanden-Eijnden - arxiv preprint arxiv …, 2023 - arxiv.org
A class of generative models that unifies flow-based and diffusion-based methods is
introduced. These models extend the framework proposed in Albergo & Vanden-Eijnden …

The probability flow ode is provably fast

S Chen, S Chewi, H Lee, Y Li, J Lu… - Advances in Neural …, 2024 - proceedings.neurips.cc
We provide the first polynomial-time convergence guarantees for the probabilistic flow ODE
implementation (together with a corrector step) of score-based generative modeling. Our …

Restoration-degradation beyond linear diffusions: A non-asymptotic analysis for ddim-type samplers

S Chen, G Daras, A Dimakis - International Conference on …, 2023 - proceedings.mlr.press
We develop a framework for non-asymptotic analysis of deterministic samplers used for
diffusion generative modeling. Several recent works have analyzed stochastic samplers …

Linear convergence bounds for diffusion models via stochastic localization

J Benton, V De Bortoli, A Doucet… - arxiv preprint arxiv …, 2023 - arxiv.org
Diffusion models are a powerful method for generating approximate samples from high-
dimensional data distributions. Several recent results have provided polynomial bounds on …

Learning mixtures of gaussians using the DDPM objective

K Shah, S Chen, A Klivans - Advances in Neural …, 2023 - proceedings.neurips.cc
Recent works have shown that diffusion models can learn essentially any distribution
provided one can perform score estimation. Yet it remains poorly understood under what …

White-box transformers via sparse rate reduction

Y Yu, S Buchanan, D Pai, T Chu, Z Wu… - Advances in …, 2023 - proceedings.neurips.cc
In this paper, we contend that the objective of representation learning is to compress and
transform the distribution of the data, say sets of tokens, towards a mixture of low …

On the generalization properties of diffusion models

P Li, Z Li, H Zhang, J Bian - Advances in Neural Information …, 2023 - proceedings.neurips.cc
Diffusion models are a class of generative models that serve to establish a stochastic
transport map between an empirically observed, yet unknown, target distribution and a …

Towards faster non-asymptotic convergence for diffusion-based generative models

G Li, Y Wei, Y Chen, Y Chi - arxiv preprint arxiv:2306.09251, 2023 - arxiv.org
Diffusion models, which convert noise into new data instances by learning to reverse a
Markov diffusion process, have become a cornerstone in contemporary generative …