Traffic prediction is crucial for transportation management and user convenience. With the
rapid development of deep learning techniques, numerous models have emerged for traffic …

Deep learning has revolutionized cancer diagnostics, shifting from pixel-based image
analysis to more comprehensive, patient-centric care. This opinion article explores recent …

The benefit of transformers in large-scale 3D point cloud perception tasks, such as 3D object
detection, is limited by their quadratic computation cost when modeling long-range …

Pretraining on large-scale, unstructured internet text enables language models to acquire a
significant amount of world knowledge. However, this knowledge acquisition is data …

Modern large language models are built on sequence modeling via next-token prediction.
While the Transformer remains the dominant architecture for sequence modeling, its …

The application of transformer-based models on time series forecasting (TSF) tasks has long
been popular to study. However, many of these works fail to beat the simple linear residual …

Recent efforts in fine-tuning language models often rely on automatic data selection,
commonly using Nearest Neighbors retrieval from large datasets. However, we theoretically …

Linear Recurrent Neural Networks (LRNNs) such as Mamba, RWKV, GLA, mLSTM, and
DeltaNet have emerged as efficient alternatives to Transformers in large language …

Linear attention Transformers and their gated variants, celebrated for enabling parallel
training and efficient recurrent inference, still fall short in recall-intensive tasks compared to …

Linear Transformers have gained attention as efficient alternatives to standard Transformers,
but their performance in retrieval and long-context tasks has been limited. To address these …