Transformers have achieved great success in many artificial intelligence fields, such as
natural language processing, computer vision, and audio processing. Therefore, it is natural …

Active learning (AL) attempts to maximize a model's performance gain while annotating the
fewest samples possible. Deep learning (DL) is greedy for data and requires a large amount …

As the capabilities of generative language models continue to advance, the implications of
biases ingrained within these models have garnered increasing attention from researchers …

Abstract We present Scalable Interpolant Transformers (SiT), a family of generative models
built on the backbone of Diffusion Transformers (DiT). The interpolant framework, which …

ChatGPT has the ability to generate grammatically flawless and seemingly-human replies to
different types of questions from various domains. The number of its users and of its …

SGD performs worse than Adam by a significant margin on Transformers, but the reason
remains unclear. In this work, we provide an explanation through the lens of Hessian:(i) …

Context-aware STR methods typically use internal autoregressive (AR) language models
(LM). Inherent limitations of AR models motivated two-stage methods which employ an …

Vision transformer (ViT) models exhibit substandard optimizability. In particular, they are
sensitive to the choice of optimizer (AdamW vs. SGD), optimizer hyperparameters, and …

The existing literature on deep learning for tabular data proposes a wide range of novel
architectures and reports competitive results on various datasets. However, the proposed …

Transformer architecture has emerged to be successful in a number of natural language
processing tasks. However, its applications to medical vision remain largely unexplored. In …