Recent years have seen a phenomenal rise in the performance and applications of
transformer neural networks. The family of transformer networks, including Bidirectional …

The dynamic neural network (DNN), in contrast to the static counterpart, offers numerous
advantages, such as improved accuracy, efficiency, and interpretability. These benefits stem …

The rapid advances in Vision Transformer (ViT) refresh the state-of-the-art performances in
various vision tasks, overshadowing the conventional CNN-based models. This ignites a few …

Abstract While Vision Transformers (ViTs) have undoubtedly made impressive strides in
computer vision (CV), their intricate network structures necessitate substantial computation …

Deployment of Transformer models on edge devices is becoming increasingly challenging
due to the exponentially growing inference cost that scales quadratically with the number of …

Large Language Models (LLMs) stand out for their impressive performance in intricate
language modeling tasks. However, their demanding computational and memory needs …

With the increase in the computation intensity of the chip, the mismatch between
computation layer shapes and the available computation resource significantly limits the …

Transformer-like network has shown remarkable high performance in both natural language
processing and computer vision. However, the huge computational demands in non-linear …

The wide adoption and significant computing resource cost of attention-based transformers,
eg, Vision Transformers and large language models, have driven the demand for efficient …

Self-attention-based transformers have outperformed recurrent and convolutional neural
networks (RNN/CNNs) in many applications. Despite the effectiveness, calculating self …