Abstract Machine Learning (ML) models are ubiquitous in real-world applications and are a
constant focus of research. Modern ML models have become more complex, deeper, and …

This article provides a birds-eye view on the role of decision trees in machine learning and
data science over roughly four decades. It sketches the evolution of decision tree research …

A key element in solving real-life data science problems is selecting the types of models to
use. Tree ensemble models (such as XGBoost) are usually recommended for classification …

The existing literature on deep learning for tabular data proposes a wide range of novel
architectures and reports competitive results on various datasets. However, the proposed …

Dynamic neural network is an emerging research topic in deep learning. Compared to static
models which have fixed computational graphs and parameters at the inference stage …

Fine-tuning pretrained language models (LMs) without making any architectural changes
has become a norm for learning various language downstream tasks. However, for non …

Recently, Transformer-like deep architectures have shown strong performance on tabular
data problems. Unlike traditional models, eg, MLP, these architectures map scalar values of …

Abstract The Mixture-of-Experts (MoE) architecture is showing promising results in improving
parameter sharing in multi-task learning (MTL) and in scaling high-capacity neural networks …

Recent development of deep neural networks (DNNs) for tabular learning has largely
benefited from the capability of DNNs for automatic feature interaction. However, the …

Adaptive optimization methods for deep learning adjust the inference task to the current
circumstances at runtime to improve the resource footprint while maintaining the model's …