A comprehensive survey on pretrained foundation models: A history from bert to chatgpt

C Zhou, Q Li, C Li, J Yu, Y Liu, G Wang… - International Journal of …, 2024 - Springer
Abstract Pretrained Foundation Models (PFMs) are regarded as the foundation for various
downstream tasks across different data modalities. A PFM (eg, BERT, ChatGPT, GPT-4) is …

Knowledge distillation on graphs: A survey

Y Tian, S Pei, X Zhang, C Zhang, N Chawla - ACM Computing Surveys, 2023 - dl.acm.org
Graph Neural Networks (GNNs) have received significant attention for demonstrating their
capability to handle graph data. However, they are difficult to be deployed in resource …

Llmrec: Large language models with graph augmentation for recommendation

W Wei, X Ren, J Tang, Q Wang, L Su, S Cheng… - Proceedings of the 17th …, 2024 - dl.acm.org
The problem of data sparsity has long been a challenge in recommendation systems, and
previous studies have attempted to address this issue by incorporating side information …

Large language models (LLMs): survey, technical frameworks, and future challenges

P Kumar - Artificial Intelligence Review, 2024 - Springer
Artificial intelligence (AI) has significantly impacted various fields. Large language models
(LLMs) like GPT-4, BARD, PaLM, Megatron-Turing NLG, Jurassic-1 Jumbo etc., have …

Graph neural prompting with large language models

Y Tian, H Song, Z Wang, H Wang, Z Hu… - Proceedings of the …, 2024 - ojs.aaai.org
Large language models (LLMs) have shown remarkable generalization capability with
exceptional performance in various language modeling tasks. However, they still exhibit …

Higpt: Heterogeneous graph language model

J Tang, Y Yang, W Wei, L Shi, L **a, D Yin… - Proceedings of the 30th …, 2024 - dl.acm.org
Heterogeneous graph learning aims to capture complex relationships and diverse relational
semantics among entities in a heterogeneous graph to obtain meaningful representations …

Learning mlps on graphs: A unified view of effectiveness, robustness, and efficiency

Y Tian, C Zhang, Z Guo, X Zhang… - … Conference on Learning …, 2022 - openreview.net
While Graph Neural Networks (GNNs) have demonstrated their efficacy in dealing with non-
Euclidean structural data, they are difficult to be deployed in real applications due to the …

Federated graph learning under domain shift with generalizable prototypes

G Wan, W Huang, M Ye - Proceedings of the AAAI Conference on …, 2024 - ojs.aaai.org
Federated Graph Learning is a privacy-preserving collaborative approach for training a
shared model on graph-structured data in the distributed environment. However, in real …

Breaking the trilemma of privacy, utility, and efficiency via controllable machine unlearning

Z Liu, G Dou, E Chien, C Zhang, Y Tian… - Proceedings of the ACM …, 2024 - dl.acm.org
Machine Unlearning (MU) algorithms have become increasingly critical due to the
imperative adherence to data privacy regulations. The primary objective of MU is to erase …

Fair graph representation learning via diverse mixture-of-experts

Z Liu, C Zhang, Y Tian, E Zhang, C Huang… - Proceedings of the …, 2023 - dl.acm.org
Graph Neural Networks (GNNs) have demonstrated a great representation learning
capability on graph data and have been utilized in various downstream applications …