The evolution of distributed systems for graph neural networks and their origin in graph processing and deep learning: A survey

J Vatter, R Mayer, HA Jacobsen - ACM Computing Surveys, 2023 - dl.acm.org
Graph neural networks (GNNs) are an emerging research field. This specialized deep
neural network architecture is capable of processing graph structured data and bridges the …

A comprehensive survey on deep graph representation learning

W Ju, Z Fang, Y Gu, Z Liu, Q Long, Z Qiao, Y Qin… - Neural Networks, 2024 - Elsevier
Graph representation learning aims to effectively encode high-dimensional sparse graph-
structured data into low-dimensional dense vectors, which is a fundamental task that has …

Do transformers really perform badly for graph representation?

C Ying, T Cai, S Luo, S Zheng, G Ke… - Advances in neural …, 2021 - proceedings.neurips.cc
The Transformer architecture has become a dominant choice in many domains, such as
natural language processing and computer vision. Yet, it has not achieved competitive …

Adan: Adaptive nesterov momentum algorithm for faster optimizing deep models

X **e, P Zhou, H Li, Z Lin, S Yan - IEEE Transactions on Pattern …, 2024 - ieeexplore.ieee.org
In deep learning, different kinds of deep networks typically need different optimizers, which
have to be chosen after multiple trials, making the training process inefficient. To relieve this …

Masked label prediction: Unified message passing model for semi-supervised classification

Y Shi, Z Huang, S Feng, H Zhong, W Wang… - arxiv preprint arxiv …, 2020 - arxiv.org
Graph neural network (GNN) and label propagation algorithm (LPA) are both message
passing algorithms, which have achieved superior performance in semi-supervised …

Training graph neural networks with 1000 layers

G Li, M Müller, B Ghanem… - … conference on machine …, 2021 - proceedings.mlr.press
Deep graph neural networks (GNNs) have achieved excellent results on various tasks on
increasingly large graph datasets with millions of nodes and edges. However, memory …

[PDF][PDF] Natural language is all a graph needs

R Ye, C Zhang, R Wang, S Xu, Y Zhang - arxiv preprint arxiv …, 2023 - yongfeng.me
The emergence of large-scale pre-trained language models, such as ChatGPT, has
revolutionized various research fields in artificial intelligence. Transformersbased large …

Are we really making much progress? revisiting, benchmarking and refining heterogeneous graph neural networks

Q Lv, M Ding, Q Liu, Y Chen, W Feng, S He… - Proceedings of the 27th …, 2021 - dl.acm.org
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but
the unique data processing and evaluation setups used by each work obstruct a full …

Nested graph neural networks

M Zhang, P Li - Advances in Neural Information Processing …, 2021 - proceedings.neurips.cc
Graph neural network (GNN)'s success in graph classification is closely related to the
Weisfeiler-Lehman (1-WL) algorithm. By iteratively aggregating neighboring node features …

Graph neural networks with heterophily

J Zhu, RA Rossi, A Rao, T Mai, N Lipka… - Proceedings of the …, 2021 - ojs.aaai.org
Abstract Graph Neural Networks (GNNs) have proven to be useful for many different
practical applications. However, many existing GNN models have implicitly assumed …