A comprehensive survey on applications of transformers for deep learning tasks
Abstract Transformers are Deep Neural Networks (DNN) that utilize a self-attention
mechanism to capture contextual relationships within sequential data. Unlike traditional …
mechanism to capture contextual relationships within sequential data. Unlike traditional …
Advances in medical image analysis with vision transformers: a comprehensive review
The remarkable performance of the Transformer architecture in natural language processing
has recently also triggered broad interest in Computer Vision. Among other merits …
has recently also triggered broad interest in Computer Vision. Among other merits …
Vision transformer adapter for dense predictions
This work investigates a simple yet powerful adapter for Vision Transformer (ViT). Unlike
recent visual transformers that introduce vision-specific inductive biases into their …
recent visual transformers that introduce vision-specific inductive biases into their …
On the integration of self-attention and convolution
Convolution and self-attention are two powerful techniques for representation learning, and
they are usually considered as two peer approaches that are distinct from each other. In this …
they are usually considered as two peer approaches that are distinct from each other. In this …
Mobile-former: Bridging mobilenet and transformer
Abstract We present Mobile-Former, a parallel design of MobileNet and transformer with a
two-way bridge in between. This structure leverages the advantages of MobileNet at local …
two-way bridge in between. This structure leverages the advantages of MobileNet at local …
Vitaev2: Vision transformer advanced by exploring inductive bias for image recognition and beyond
Vision transformers have shown great potential in various computer vision tasks owing to
their strong capability to model long-range dependency using the self-attention mechanism …
their strong capability to model long-range dependency using the self-attention mechanism …
Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives
Transformer, one of the latest technological advances of deep learning, has gained
prevalence in natural language processing or computer vision. Since medical imaging bear …
prevalence in natural language processing or computer vision. Since medical imaging bear …
Vitae: Vision transformer advanced by exploring intrinsic inductive bias
Transformers have shown great potential in various computer vision tasks owing to their
strong capability in modeling long-range dependency using the self-attention mechanism …
strong capability in modeling long-range dependency using the self-attention mechanism …
Hrformer: High-resolution vision transformer for dense predict
Abstract We present a High-Resolution Transformer (HRFormer) that learns high-resolution
representations for dense prediction tasks, in contrast to the original Vision Transformer that …
representations for dense prediction tasks, in contrast to the original Vision Transformer that …
Hrformer: High-resolution transformer for dense prediction
We present a High-Resolution Transformer (HRFormer) that learns high-resolution
representations for dense prediction tasks, in contrast to the original Vision Transformer that …
representations for dense prediction tasks, in contrast to the original Vision Transformer that …