To effectively exploit the potential of large-scale models, various pre-training strategies
supported by massive data from different sources are proposed, including supervised pre …

Prompt Tuning, conditioning on task-specific learned prompt vectors, has emerged as a data-
efficient and parameter-efficient method for adapting large pretrained vision-language …

Abstract Class Incremental Learning (CIL) aims to sequentially learn new classes while
avoiding catastrophic forgetting of previous knowledge. We propose to use Masked …

Masked autoencoders (MAE) is a deep learning method based on Transformer. Originally
used for images, it has now been extended to video, audio, and some other temporal …

This paper provides an efficiency study of training Masked Autoencoders (MAE) a framework
introduced by He et. al. for pre-training Vision Transformers (ViTs). Our results surprisingly …

Self supervision and natural language supervision have emerged as two exciting ways to
train general purpose image encoders which excel at a variety of downstream tasks. Recent …

Most existing masked audio modeling (MAM) methods learn audio representations by
masking and reconstructing local spectrogram patches. However, the reconstruction loss …

In this work, we survey recent studies on masked image modeling (MIM), an approach that
emerged as a powerful self-supervised learning technique in computer vision. The MIM task …

Diabetic retinopathy (DR) is a diabetic condition that affects vision, despite the great success
of supervised learning and Conventional Neural Networks (CNNs), it's still challenging to …

The past few years have seen an increased interest in aerial image object detection due to
its critical value to large-scale geoscientific research like environmental studies, urban …