A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends

J Gui, T Chen, J Zhang, Q Cao, Z Sun… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Deep supervised learning algorithms typically require a large volume of labeled data to
achieve satisfactory performance. However, the process of collecting and labeling such data …

Self-supervised learning for videos: A survey

MC Schiappa, YS Rawat, M Shah - ACM Computing Surveys, 2023 - dl.acm.org
The remarkable success of deep learning in various domains relies on the availability of
large-scale annotated datasets. However, obtaining annotations is expensive and requires …

Videomae v2: Scaling video masked autoencoders with dual masking

L Wang, B Huang, Z Zhao, Z Tong… - Proceedings of the …, 2023 - openaccess.thecvf.com
Scale is the primary factor for building a powerful foundation model that could well
generalize to a variety of downstream tasks. However, it is still challenging to train video …

Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training

Z Tong, Y Song, J Wang… - Advances in neural …, 2022 - proceedings.neurips.cc
Pre-training video transformers on extra large-scale datasets is generally required to
achieve premier performance on relatively small datasets. In this paper, we show that video …

Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text

H Akbari, L Yuan, R Qian… - Advances in …, 2021 - proceedings.neurips.cc
We present a framework for learning multimodal representations from unlabeled data using
convolution-free Transformer architectures. Specifically, our Video-Audio-Text Transformer …

Contrastive learning for representation degeneration problem in sequential recommendation

R Qiu, Z Huang, H Yin, Z Wang - … conference on web search and data …, 2022 - dl.acm.org
Recent advancements of sequential deep learning models such as Transformer and BERT
have significantly facilitated the sequential recommendation. However, according to our …

TCTrack: Temporal contexts for aerial tracking

Z Cao, Z Huang, L Pan, S Zhang… - Proceedings of the …, 2022 - openaccess.thecvf.com
Temporal contexts among consecutive frames are far from being fully utilized in existing
visual trackers. In this work, we present TCTrack, a comprehensive framework to fully exploit …

Hard negative mixing for contrastive learning

Y Kalantidis, MB Sariyildiz, N Pion… - Advances in neural …, 2020 - proceedings.neurips.cc
Contrastive learning has become a key component of self-supervised learning approaches
for computer vision. By learning to embed two augmented versions of the same image close …

Tsmae: a novel anomaly detection approach for internet of things time series data using memory-augmented autoencoder

H Gao, B Qiu, RJD Barroso, W Hussain… - … on network science …, 2022 - ieeexplore.ieee.org
With the development of communication, the Internet of Things (IoT) has been widely
deployed and used in industrial manufacturing, intelligent transportation, and healthcare …

Spatiotemporal contrastive video representation learning

R Qian, T Meng, B Gong, MH Yang… - Proceedings of the …, 2021 - openaccess.thecvf.com
We present a self-supervised Contrastive Video Representation Learning (CVRL) method to
learn spatiotemporal visual representations from unlabeled videos. Our representations are …