Graph neural networks (GNNs) are an emerging research field. This specialized deep
neural network architecture is capable of processing graph structured data and bridges the …

Efficient collective communication is crucial to parallel-computing applications such as
distributed training of large-scale recommendation systems and natural language …

The advent of switches with programmable dataplanes has enabled the rapid development
of new network functionality, as well as providing a platform for acceleration of a broad …

Artificial intelligence (AI) research and market have grown rapidly in the last few years, and
this trend is expected to continue with many potential advancements and innovations in this …

Federated learning (FL) enables multiple clients to collaboratively train a shared model, with
the help of a parameter server (PS), without disclosing their local datasets. However, due to …

Large deep learning models have recently garnered substantial attention from both
academia and industry. Nonetheless, frequent failures are observed during large model …

GPUs are critical for maximizing the throughput-per-Watt of deep neural network (DNN)
applications. However, DNN applications often underutilize GPUs, even when using large …

Deep learning (DL) workloads include throughput-intensive training tasks and latency-
sensitive inference tasks. The dominant practice today is to provision dedicated GPU …

The number of devices, from smartphones to IoT hardware, interconnected via the Internet is
growing all the time. These devices produce a large amount of data that cannot be analyzed …

A rich body of prior work has highlighted the existence of communication bottlenecks in
synchronous data-parallel training. To alleviate these bottlenecks, a long line of recent …