On the opportunities and risks of foundation models
AI is undergoing a paradigm shift with the rise of models (eg, BERT, DALL-E, GPT-3) that are
trained on broad data at scale and are adaptable to a wide range of downstream tasks. We …
trained on broad data at scale and are adaptable to a wide range of downstream tasks. We …
Barlow twins: Self-supervised learning via redundancy reduction
Self-supervised learning (SSL) is rapidly closing the gap with supervised methods on large
computer vision benchmarks. A successful approach to SSL is to learn embeddings which …
computer vision benchmarks. A successful approach to SSL is to learn embeddings which …
Hard negative mixing for contrastive learning
Contrastive learning has become a key component of self-supervised learning approaches
for computer vision. By learning to embed two augmented versions of the same image close …
for computer vision. By learning to embed two augmented versions of the same image close …
Beyond just vision: A review on self-supervised representation learning on multimodal and temporal data
Recently, Self-Supervised Representation Learning (SSRL) has attracted much attention in
the field of computer vision, speech, natural language processing (NLP), and recently, with …
the field of computer vision, speech, natural language processing (NLP), and recently, with …
Provable guarantees for self-supervised deep learning with spectral contrastive loss
Recent works in self-supervised learning have advanced the state-of-the-art by relying on
the contrastive learning paradigm, which learns representations by pushing positive pairs, or …
the contrastive learning paradigm, which learns representations by pushing positive pairs, or …
How well do self-supervised models transfer?
Self-supervised visual representation learning has seen huge progress recently, but no
large scale evaluation has compared the many models now available. We evaluate the …
large scale evaluation has compared the many models now available. We evaluate the …
Whitening for self-supervised representation learning
Most of the current self-supervised representation learning (SSL) methods are based on the
contrastive loss and the instance-discrimination task, where augmented versions of the …
contrastive loss and the instance-discrimination task, where augmented versions of the …
An overview of deep learning techniques for epileptic seizures detection and prediction based on neuroimaging modalities: Methods, challenges, and future works
Epilepsy is a disorder of the brain denoted by frequent seizures. The symptoms of seizure
include confusion, abnormal staring, and rapid, sudden, and uncontrollable hand …
include confusion, abnormal staring, and rapid, sudden, and uncontrollable hand …
From canonical correlation analysis to self-supervised graph neural networks
We introduce a conceptually simple yet effective model for self-supervised representation
learning with graph data. It follows the previous methods that generate two views of an input …
learning with graph data. It follows the previous methods that generate two views of an input …
Intriguing properties of contrastive losses
We study three intriguing properties of contrastive learning. First, we generalize the standard
contrastive loss to a broader family of losses, and we find that various instantiations of the …
contrastive loss to a broader family of losses, and we find that various instantiations of the …