Heterogeneous tabular data are the most commonly used form of data and are essential for
numerous critical and computationally demanding applications. On homogeneous datasets …

Recent work in natural language processing (NLP) has yielded appealing results from
scaling model parameters and training data; however, using only scale to improve …

Attention Model has now become an important concept in neural networks that has been
researched within diverse application domains. This survey provides a structured and …

Abstract Autoregressive Transformers adopted in Large Language Models (LLMs) are hard
to scale to long sequences. Despite several works trying to reduce their computational cost …

Large language models have shown promising results in zero-shot settings (Brown et al.,
2020; Radford et al., 2019). For example, they can perform multiple choice tasks simply by …

We introduce the sparse modern Hopfield model as a sparse extension of the modern
Hopfield model. Like its dense counterpart, the sparse modern Hopfield model equips a …

Many approaches have been proposed for time series forecasting, in light of its significance
in wide applications including business demand prediction. However, the existing methods …

Nowadays, deep neural networks (DNNs) have become the main instrument for machine
learning tasks within a wide range of domains, including vision, NLP, and speech …

Identifying cellular identities is a key use case in single-cell transcriptomics. While machine
learning has been leveraged to automate cell annotation predictions for some time, there …

Abstract The Mixture-of-Experts (MoE) architecture is showing promising results in improving
parameter sharing in multi-task learning (MTL) and in scaling high-capacity neural networks …