A comprehensive survey on pretrained foundation models: A history from bert to chatgpt

C Zhou, Q Li, C Li, J Yu, Y Liu, G Wang… - International Journal of …, 2024 - Springer
Abstract Pretrained Foundation Models (PFMs) are regarded as the foundation for various
downstream tasks across different data modalities. A PFM (eg, BERT, ChatGPT, GPT-4) is …

Graph prompt learning: A comprehensive survey and beyond

X Sun, J Zhang, X Wu, H Cheng, Y **ong… - arxiv preprint arxiv …, 2023 - arxiv.org
Artificial General Intelligence (AGI) has revolutionized numerous fields, yet its integration
with graph data, a cornerstone in our interconnected world, remains nascent. This paper …

Simgrace: A simple framework for graph contrastive learning without data augmentation

J **a, L Wu, J Chen, B Hu, SZ Li - … of the ACM web conference 2022, 2022 - dl.acm.org
Graph contrastive learning (GCL) has emerged as a dominant technique for graph
representation learning which maximizes the mutual information between paired graph …

Deep contrastive representation learning with self-distillation

Z **ao, H **ng, B Zhao, R Qu, S Luo… - … on Emerging Topics …, 2023 - ieeexplore.ieee.org
Recently, contrastive learning (CL) is a promising way of learning discriminative
representations from time series data. In the representation hierarchy, semantic information …

Data augmentation for deep graph learning: A survey

K Ding, Z Xu, H Tong, H Liu - ACM SIGKDD Explorations Newsletter, 2022 - dl.acm.org
Graph neural networks, a powerful deep learning tool to model graph-structured data, have
demonstrated remarkable performance on numerous graph learning tasks. To address the …

Graph contrastive learning automated

Y You, T Chen, Y Shen, Z Wang - … conference on machine …, 2021 - proceedings.mlr.press
Self-supervised learning on graph-structured data has drawn recent interest for learning
generalizable, transferable and robust representations from unlabeled graphs. Among …

Graph self-supervised learning: A survey

Y Liu, M **, S Pan, C Zhou, Y Zheng… - IEEE transactions on …, 2022 - ieeexplore.ieee.org
Deep learning on graphs has attracted significant interests recently. However, most of the
works have focused on (semi-) supervised learning, resulting in shortcomings including …

Augmentation-free self-supervised learning on graphs

N Lee, J Lee, C Park - Proceedings of the AAAI conference on artificial …, 2022 - ojs.aaai.org
Inspired by the recent success of self-supervised methods applied on images, self-
supervised learning on graph structured data has seen rapid growth especially centered on …

Towards unsupervised deep graph structure learning

Y Liu, Y Zheng, D Zhang, H Chen, H Peng… - Proceedings of the ACM …, 2022 - dl.acm.org
In recent years, graph neural networks (GNNs) have emerged as a successful tool in a
variety of graph-related applications. However, the performance of GNNs can be …

From canonical correlation analysis to self-supervised graph neural networks

H Zhang, Q Wu, J Yan, D Wipf… - Advances in Neural …, 2021 - proceedings.neurips.cc
We introduce a conceptually simple yet effective model for self-supervised representation
learning with graph data. It follows the previous methods that generate two views of an input …