Vision Transformers (ViTs) have achieved impressive performance over various computer
vision tasks. However, modeling global correlations with multi-head self-attention (MSA) …

There are two de facto standard architectures in recent computer vision: Convolutional
Neural Networks (CNNs) and Vision Transformers (ViTs). Strong inductive biases of …

Deep learning has become the cornerstone of artificial intelligence, and has fueled
breakthroughs in a number of fields. Yet, the key reasons underpinning the success of deep …

Convolutional networks are a key component of many computer vision applications.
However, convolutions have a serious flaw. It only works in a small area, hence it lacks …

Transformers have revolutionized modern applications but are costly as model sizes grow.
This thesis targets efficient training and inference of large Transformer models. We first …

Although transformer networks are recently employed in various vision tasks with
outperforming performance, extensive training data and a lengthy training time are required …

Deep neural networks have been widely adopted as state-of-the-art models of the visual
ventral stream (eg Cadieu et al., 2014; Yamins and DiCarlo, 2016; Cichy et al., 2016). Most …