Recent years have seen a phenomenal rise in the performance and applications of
transformer neural networks. The family of transformer networks, including Bidirectional …

Recent years have witnessed significant advances in image deraining due to the kinds of
effective image priors and deep learning models. As each deraining approach has …

Large language models (LLMs) have been applied in various applications due to their
astonishing capabilities. With advancements in technologies such as chain-of-thought (CoT) …

Vision transformers (ViTs) have recently received explosive popularity, but the huge
computational cost is still a severe issue. Since the computation complexity of ViT is …

DETR-like models have significantly boosted the performance of detectors and even
outperformed classical convolutional models. However, all tokens are treated equally …

Transformer-based language models have become a key building block for natural
language processing. While these models are extremely accurate, they can be too large and …

Recent advances in state-of-the-art DNN architecture design have been moving toward
Transformer models. These models achieve superior accuracy across a wide range of …

High-resolution images enable neural networks to learn richer visual representations.
However, this improved performance comes at the cost of growing computational …

In this study, we introduce adaptive KV cache compression, a plug-and-play method that
reduces the memory footprint of generative inference for Large Language Models (LLMs) …

Abstract Autoregressive Transformers adopted in Large Language Models (LLMs) are hard
to scale to long sequences. Despite several works trying to reduce their computational cost …