A-vit: Adaptive tokens for efficient vision transformer

H Yin, A Vahdat, JM Alvarez, A Mallya… - Proceedings of the …, 2022 - openaccess.thecvf.com
We introduce A-ViT, a method that adaptively adjusts the inference cost of vision transformer
ViT for images of different complexity. A-ViT achieves this by automatically reducing the …

Understanding the difficulty of training transformers

L Liu, X Liu, J Gao, W Chen, J Han - ar**_Training_With_Dynamic_Re-Parameterization_CVPR_2022_paper.pdf" data-clk="hl=cs&sa=T&oi=gga&ct=gga&cd=6&d=9004725926464672087&ei=wOm3Z8juMNjGieoPx6O9mA4" data-clk-atid="V1GtYoU293wJ" target="_blank">[PDF] thecvf.com

Dyrep: Bootstrap** training with dynamic re-parameterization

T Huang, S You, B Zhang, Y Du… - Proceedings of the …, 2022 - openaccess.thecvf.com
Structural re-parameterization (Rep) methods achieve noticeable improvements on simple
VGG-style networks. Despite the prevalence, current Rep methods simply re-parameterize …

Implicit regularization of deep residual networks towards neural ODEs

P Marion, YH Wu, ME Sander, G Biau - arxiv preprint arxiv:2309.01213, 2023 - arxiv.org
Residual neural networks are state-of-the-art deep learning models. Their continuous-depth
analog, neural ordinary differential equations (ODEs), are also widely used. Despite their …

[HTML][HTML] A hybrid quantum–classical neural network with deep residual learning

Y Liang, W Peng, ZJ Zheng, O Silvén, G Zhao - Neural Networks, 2021 - Elsevier
Inspired by the success of classical neural networks, there has been tremendous effort to
develop classical effective neural networks into quantum concept. In this paper, a novel …

Adavit: Adaptive tokens for efficient vision transformer

H Yin, A Vahdat, J Alvarez, A Mallya, J Kautz… - arxiv preprint arxiv …, 2021 - arxiv.org
We introduce A-ViT, a method that adaptively adjusts the inference cost of vision transformer
(ViT) for images of different complexity. A-ViT achieves this by automatically reducing the …