Abstract Transformers are Deep Neural Networks (DNN) that utilize a self-attention
mechanism to capture contextual relationships within sequential data. Unlike traditional …

The field of speech processing has undergone a transformative shift with the advent of deep
learning. The use of multiple processing layers has enabled the creation of models capable …

Transformers have revolutionized almost all natural language processing (NLP) tasks but
suffer from memory and computational complexity that scales quadratically with sequence …

Abstract Recurrent Neural Networks (RNNs) offer fast inference on long sequences but are
hard to optimize and slow to train. Deep state-space models (SSMs) have recently been …

The recent boom of linear forecasting models questions the ongoing passion for
architectural modifications of Transformer-based forecasters. These forecasters leverage …

Recently, there has been a surge of Transformer-based solutions for the long-term time
series forecasting (LTSF) task. Despite the growing performance over the past few years, we …

While originally designed for natural language processing tasks, the self-attention
mechanism has recently taken various computer vision areas by storm. However, the 2D …

Time series forecasting holds significant importance in many real-world dynamic systems
and has been extensively studied. Unlike natural language process (NLP) and computer …

As a typical problem in time series analysis, traffic flow prediction is one of the most
important application fields of machine learning. However, achieving highly accurate traffic …

A central goal of sequence modeling is designing a single principled model that can
address sequence data across a range of modalities and tasks, particularly on long-range …