Transformers in medical imaging: A survey

F Shamshad, S Khan, SW Zamir, MH Khan… - Medical Image …, 2023 - Elsevier
Following unprecedented success on the natural language tasks, Transformers have been
successfully applied to several computer vision problems, achieving state-of-the-art results …

Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives

J Li, J Chen, Y Tang, C Wang, BA Landman… - Medical image …, 2023 - Elsevier
Transformer, one of the latest technological advances of deep learning, has gained
prevalence in natural language processing or computer vision. Since medical imaging bear …

Daformer: Improving network architectures and training strategies for domain-adaptive semantic segmentation

L Hoyer, D Dai, L Van Gool - Proceedings of the IEEE/CVF …, 2022 - openaccess.thecvf.com
As acquiring pixel-wise annotations of real-world images for semantic segmentation is a
costly process, a model can instead be trained with more accessible synthetic data and …

Do vision transformers see like convolutional neural networks?

M Raghu, T Unterthiner, S Kornblith… - Advances in neural …, 2021 - proceedings.neurips.cc
Convolutional neural networks (CNNs) have so far been the de-facto model for visual data.
Recent work has shown that (Vision) Transformer models (ViT) can achieve comparable or …

Mlp-mixer: An all-mlp architecture for vision

IO Tolstikhin, N Houlsby, A Kolesnikov… - Advances in neural …, 2021 - proceedings.neurips.cc
Abstract Convolutional Neural Networks (CNNs) are the go-to model for computer vision.
Recently, attention-based networks, such as the Vision Transformer, have also become …

Are transformers more robust than cnns?

Y Bai, J Mei, AL Yuille, C **e - Advances in neural …, 2021 - proceedings.neurips.cc
Transformer emerges as a powerful tool for visual recognition. In addition to demonstrating
competitive performance on a broad range of visual benchmarks, recent works also argue …

Intriguing properties of vision transformers

MM Naseer, K Ranasinghe, SH Khan… - Advances in …, 2021 - proceedings.neurips.cc
Vision transformers (ViT) have demonstrated impressive performance across numerous
machine vision tasks. These models are based on multi-head self-attention mechanisms that …

Vitaev2: Vision transformer advanced by exploring inductive bias for image recognition and beyond

Q Zhang, Y Xu, J Zhang, D Tao - International Journal of Computer Vision, 2023 - Springer
Vision transformers have shown great potential in various computer vision tasks owing to
their strong capability to model long-range dependency using the self-attention mechanism …

Polyp-pvt: Polyp segmentation with pyramid vision transformers

B Dong, W Wang, DP Fan, J Li, H Fu, L Shao - arxiv preprint arxiv …, 2021 - arxiv.org
Most polyp segmentation methods use CNNs as their backbone, leading to two key issues
when exchanging information between the encoder and decoder: 1) taking into account the …

Flexivit: One model for all patch sizes

L Beyer, P Izmailov, A Kolesnikov… - Proceedings of the …, 2023 - openaccess.thecvf.com
Vision Transformers convert images to sequences by slicing them into patches. The size of
these patches controls a speed/accuracy tradeoff, with smaller patches leading to higher …