The evolution of distributed systems for graph neural networks and their origin in graph processing and deep learning: A survey

J Vatter, R Mayer, HA Jacobsen - ACM Computing Surveys, 2023 - dl.acm.org
Graph neural networks (GNNs) are an emerging research field. This specialized deep
neural network architecture is capable of processing graph structured data and bridges the …

Computing graph neural networks: A survey from algorithms to accelerators

S Abadal, A Jain, R Guirado, J López-Alonso… - ACM Computing …, 2021 - dl.acm.org
Graph Neural Networks (GNNs) have exploded onto the machine learning scene in recent
years owing to their capability to model and learn from graph-structured data. Such an ability …

GossipFL: A decentralized federated learning framework with sparsified and adaptive communication

Z Tang, S Shi, B Li, X Chu - IEEE Transactions on Parallel and …, 2022 - ieeexplore.ieee.org
Recently, federated learning (FL) techniques have enabled multiple users to train machine
learning models collaboratively without data sharing. However, existing FL algorithms suffer …

[HTML][HTML] Privacy and security in federated learning: A survey

R Gosselin, L Vieu, F Loukil, A Benoit - Applied Sciences, 2022 - mdpi.com
In recent years, privacy concerns have become a serious issue for companies wishing to
protect economic models and comply with end-user expectations. In the same vein, some …

{MAST}: Global scheduling of {ML} training across {Geo-Distributed} datacenters at hyperscale

A Choudhury, Y Wang, T Pelkonen… - … USENIX Symposium on …, 2024 - usenix.org
In public clouds, users must manually select a datacenter region to upload their ML training
data and launch ML training workloads in the same region to ensure data and computation …

{nnScaler}:{Constraint-Guided} Parallelization Plan Generation for Deep Learning Training

Z Lin, Y Miao, Q Zhang, F Yang, Y Zhu, C Li… - … USENIX Symposium on …, 2024 - usenix.org
With the growing model size of deep neural networks (DNN), deep learning training is
increasingly relying on handcrafted search spaces to find efficient parallelization execution …

Chimera: efficiently training large-scale neural networks with bidirectional pipelines

S Li, T Hoefler - Proceedings of the International Conference for High …, 2021 - dl.acm.org
Training large deep learning models at scale is very challenging. This paper proposes
Chimera, a novel pipeline parallelism scheme which combines bidirectional pipelines for …

A survey of on-device machine learning: An algorithms and learning theory perspective

S Dhar, J Guo, J Liu, S Tripathi, U Kurup… - ACM Transactions on …, 2021 - dl.acm.org
The predominant paradigm for using machine learning models on a device is to train a
model in the cloud and perform inference using the trained model on the device. However …

Pruner-zero: Evolving symbolic pruning metric from scratch for large language models

P Dong, L Li, Z Tang, X Liu, X Pan, Q Wang… - ar** us to make decisions in everything
we do, even in finding our “true love” and the “significant other”. While 5G promises us high …