The evolution of distributed systems for graph neural networks and their origin in graph processing and deep learning: A survey

J Vatter, R Mayer, HA Jacobsen - ACM Computing Surveys, 2023 - dl.acm.org
Graph neural networks (GNNs) are an emerging research field. This specialized deep
neural network architecture is capable of processing graph structured data and bridges the …

Distributed graph neural network training: A survey

Y Shao, H Li, X Gu, H Yin, Y Li, X Miao… - ACM Computing …, 2024 - dl.acm.org
Graph neural networks (GNNs) are a type of deep learning models that are trained on
graphs and have been successfully applied in various domains. Despite the effectiveness of …

A comprehensive study on large-scale graph training: Benchmarking and rethinking

K Duan, Z Liu, P Wang, W Zheng… - Advances in …, 2022 - proceedings.neurips.cc
Large-scale graph training is a notoriously challenging problem for graph neural networks
(GNNs). Due to the nature of evolving graph structures into the training process, vanilla …

Parallel and distributed graph neural networks: An in-depth concurrency analysis

M Besta, T Hoefler - IEEE Transactions on Pattern Analysis and …, 2024 - ieeexplore.ieee.org
Graph neural networks (GNNs) are among the most powerful tools in deep learning. They
routinely solve complex problems on unstructured networks, such as node classification …

Enabling resource-efficient aiot system with cross-level optimization: A survey

S Liu, B Guo, C Fang, Z Wang, S Luo… - … Surveys & Tutorials, 2023 - ieeexplore.ieee.org
The emerging field of artificial intelligence of things (AIoT, AI+ IoT) is driven by the
widespread use of intelligent infrastructures and the impressive success of deep learning …

Neutronstar: distributed GNN training with hybrid dependency management

Q Wang, Y Zhang, H Wang, C Chen, X Zhang… - Proceedings of the 2022 …, 2022 - dl.acm.org
GNN's training needs to resolve issues of vertex dependencies, ie, each vertex
representation's update depends on its neighbors. Existing distributed GNN systems adopt …

Rsc: accelerate graph neural networks training via randomized sparse computations

Z Liu, C Shengyuan, K Zhou, D Zha… - International …, 2023 - proceedings.mlr.press
Training graph neural networks (GNNs) is extremely time consuming because sparse graph-
based operations are hard to be accelerated by community hardware. Prior art successfully …

Understanding gnn computational graph: A coordinated computation, io, and memory perspective

H Zhang, Z Yu, G Dai, G Huang… - Proceedings of …, 2022 - proceedings.mlsys.org
Abstract Graph Neural Networks (GNNs) have been widely used in various domains, and
GNNs with sophisticated computational graph lead to higher latency and larger memory …

Communication-efficient graph neural networks with probabilistic neighborhood expansion analysis and caching

T Kaler, A Iliopoulos, P Murzynowski… - Proceedings of …, 2023 - proceedings.mlsys.org
Training and inference with graph neural networks (GNNs) on massive graphs in a
distributed environment has been actively studied since the inception of GNNs, owing to the …

Survey on graph neural network acceleration: An algorithmic perspective

X Liu, M Yan, L Deng, G Li, X Ye, D Fan, S Pan… - arxiv preprint arxiv …, 2022 - arxiv.org
Graph neural networks (GNNs) have been a hot spot of recent research and are widely
utilized in diverse applications. However, with the use of huger data and deeper models, an …