Abstract Generalized Referring Expression Segmentation (GRES) extends the scope of
classic RES to refer to multiple objects in one expression or identify the empty targets absent …

This paper identifies significant redundancy in the query-key interactions within self-attention
mechanisms of diffusion transformer models, particularly during the early stages of …

The brain's connectivity is locally dense and globally sparse, forming a small-world graph—
a principle prevalent in the evolution of various species, suggesting a universal solution for …

The Transformer architecture has been widely applied in the field of image segmentation
due to its powerful ability to capture long-range dependencies. However, its ability to capture …

Abstract Vision Transformers (ViT) have emerged as the de-facto choice for numerous
industry grade vision solutions. But their inference cost can be prohibitive for many settings …

Transformers have shown superior performance on various vision tasks. Their large
receptive field endows Transformer models with higher representation power than their CNN …

As a core of Vision Transformer (ViT), self-attention has high versatility in modeling long-
range spatial interactions because every query attends to all spatial locations. Although ViT …

Recent studies have integrated convolution into transformers to introduce inductive bias and
improve generalization performance. However, the static nature of conventional convolution …

In this work, we seek to learn multiple mainstream vision tasks concurrently using a unified
network, which is storage-efficient as numerous networks with task-shared parameters can …

Abstract Vision Transformer (ViT) faces obstacles in wide application due to its huge
computational cost. Almost all existing studies on compressing ViT adopt the manner of …