Modern deep neural networks, particularly recent large language models, come with
massive model sizes that require significant computational and storage resources. To …

Metric learning aims to measure the similarity among samples while using an optimal
distance metric for learning tasks. Metric learning methods, which generally use a linear …

The scaling of Transformers has driven breakthrough capabilities for language models. At
present, the largest large language models (LLMs) contain upwards of 100B parameters …

The current modus operandi in adapting pre-trained models involves updating all the
backbone parameters, ie., full fine-tuning. This paper introduces Visual Prompt Tuning (VPT) …

Existing fine-tuning methods either tune all parameters of the pre-trained model (full fine-
tuning), which is not efficient, or only tune the last linear layer (linear probing), which suffers …

Abstract Sparsely-gated Mixture of Experts networks (MoEs) have demonstrated excellent
scalability in Natural Language Processing. In Computer Vision, however, almost all …

The size of vision models has grown exponentially over the last few years, especially after
the emergence of Vision Transformer. This has motivated the development of parameter …

The pretrain-then-finetune paradigm has been widely adopted in computer vision. But as the
size of Vision Transformer (ViT) grows exponentially, the full finetuning becomes prohibitive …

Robustness to distribution shifts is critical for deploying machine learning models in the real
world. Despite this necessity, there has been little work in defining the underlying …

The coronavirus (COVID-19) pandemic is putting healthcare systems across the world under
unprecedented and increasing pressure according to the World Health Organization (WHO) …