Transformer architectures have widespread applications, particularly in Natural Language
Processing and Computer Vision. Recently, Transformers have been employed in various …

Time-Series Mining (TSM) is an important research area since it shows great potential in
practical applications. Deep learning models that rely on massive labeled data have been …

Recently, transformer architectures for graphs emerged as an alternative to established
techniques for machine learning with graphs, such as (message-passing) graph neural …

Fine-tuning pretrained language models (LMs) without making any architectural changes
has become a norm for learning various language downstream tasks. However, for non …

Tabular data underpins numerous high-impact applications of machine learning from fraud
detection to genomics and healthcare. Classical approaches to solving tabular problems …

Tabular data (or tables) are the most widely used data format in machine learning (ML).
However, ML models often assume the table structure keeps fixed in training and testing …

Graph convolutional networks (GCN) have been applied in the traffic flow forecasting tasks
with the graph capability in describing the irregular topology structures of road networks …

The impact of air pollution on public health is substantial, and accurate long-term predictions
of air quality are crucial for early warning systems to address this issue. Air quality prediction …

Abstract Autoregressive and Masked Transformers are incredibly effective as generative
models and classifiers. While these models are most prevalent in NLP, they also exhibit …

Recent text and image foundation models are incredibly impressive, and these models are
attracting an ever-increasing portion of research resources. In this position piece we aim to …