Attention has arguably become one of the most important concepts in the deep learning
field. It is inspired by the biological systems of humans that tend to focus on the distinctive …

Over the last several years, the field of natural language processing has been propelled
forward by an explosion in the use of deep learning models. This article provides a brief …

While recent language models have the ability to take long contexts as input, relatively little
is known about how well they use longer context. We analyze the performance of language …

Along with the massive growth of the Internet from the 1990s until now, various innovative
technologies have been created to bring users breathtaking experiences with more virtual …

Attention is an important mechanism that can be employed for a variety of deep learning
models across many different domains and tasks. This survey provides an overview of the …

Transformers do not scale very well to long sequence lengths largely because of quadratic
self-attention complexity. In the recent months, a wide spectrum of efficient, fast Transformers …

We introduce a modern Hopfield network with continuous states and a corresponding
update rule. The new Hopfield network can store exponentially (with the dimension of the …

Emotion recognition based on electroencephalography (EEG) is a significant task in the
brain-computer interface field. Recently, many deep learning-based emotion recognition …

In humans, Attention is a core property of all perceptual and cognitive operations. Given our
limited ability to process competing sources, attention mechanisms select, modulate, and …

Neural network models have been widely used in the field of natural language processing
(NLP). Recurrent neural networks (RNNs), which have the ability to process sequences of …