Exploring the frontiers of softmax: Provable optimization, applications in diffusion model, and beyond

J Gu, C Li, Y Liang, Z Shi, Z Song - arxiv preprint arxiv:2405.03251, 2024 - arxiv.org
The softmax activation function plays a crucial role in the success of large language models
(LLMs), particularly in the self-attention mechanism of the widely adopted Transformer …

IBD-PSC: Input-level backdoor detection via parameter-oriented scaling consistency

L Hou, R Feng, Z Hua, W Luo, LY Zhang… - arxiv preprint arxiv …, 2024 - arxiv.org
Deep neural networks (DNNs) are vulnerable to backdoor attacks, where adversaries can
maliciously trigger model misclassifications by implanting a hidden backdoor during model …

Unifying bayesian flow networks and diffusion models through stochastic differential equations

K Xue, Y Zhou, S Nie, X Min, X Zhang, J Zhou… - arxiv preprint arxiv …, 2024 - arxiv.org
Bayesian flow networks (BFNs) iteratively refine the parameters, instead of the samples in
diffusion models (DMs), of distributions at various noise levels through Bayesian inference …

On statistical rates of conditional diffusion transformers: Approximation, estimation and minimax optimality

JYC Hu, W Wu, YC Lee, YC Huang, M Chen… - arxiv preprint arxiv …, 2024 - arxiv.org
We investigate the approximation and estimation rates of conditional diffusion transformers
(DiTs) with classifier-free guidance. We present a comprehensive analysis for``in …

Lightweight diffusion models: a survey

W Song, W Ma, M Zhang, Y Zhang, X Zhao - Artificial Intelligence Review, 2024 - Springer
Diffusion models (DMs) are a type of potential generative models, which have achieved
better effects in many fields than traditional methods. DMs consist of two main processes …

Mix-ddpm: Enhancing diffusion models through fitting mixture noise with global stochastic offset

H Wang, D Zhai, X Zhou, J Jiang, X Liu - ACM Transactions on …, 2024 - dl.acm.org
Denoising diffusion probabilistic models (DDPM) have shown impressive performance in
various domains as a class of deep generative models. In this article, we introduce the …

Efficient Diffusion Models: A Survey

H Shen, J Zhang, B **ong, R Hu, S Chen… - arxiv preprint arxiv …, 2025 - arxiv.org
Diffusion models have emerged as powerful generative models capable of producing high-
quality contents such as images, videos, and audio, demonstrating their potential to …

Improved ddim sampling with moment matching gaussian mixtures

P Gabbur - arxiv preprint arxiv:2311.04938, 2023 - arxiv.org
We propose using a Gaussian Mixture Model (GMM) as reverse transition operator (kernel)
within the Denoising Diffusion Implicit Models (DDIM) framework, which is one of the most …

[HTML][HTML] Average entropy of Gaussian mixtures

B Joudeh, B Škorić - Entropy, 2024 - mdpi.com
We calculate the average differential entropy of aq-component Gaussian mixture in R n. For
simplicity, all components have covariance matrix σ 2 1, while the means {W i} i= 1 q are iid …

DC-DPM: A Divide-and-Conquer Approach for Diffusion Reverse Process

YJ Dong, H Yin, F Wang, Y Zhao, C Zhang, C Li… - openreview.net
Diffusion models have achieved great success in generative tasks\textblue {, with the quality
of generated samples guaranteed by their convergence properties, typically derived within …