Machine learning models are distributed across multiple nodes using numerous parallelism
strategies. The resulting collective communication is often on the critical path due to a lack of …

As the size of deep neural network (DNN) models and datasets increases, distributed
training becomes popular to reduce the training time. However, a severe communication …

To overcome the memory capacity wall of large-scale AI and big data applications, Compute
Express Link (CXL) enables cost-efficient memory expansion beyond the local DRAM of …

Large Language Models increasingly rely on distributed techniques for their training and
inference. These techniques require communication across devices which can reduce …

Simulation offers unique values for both enumeration and extrapolation purposes, and is
becoming increasingly important for managing the massive machine learning (ML) clusters …

The evolution and revolution of technologies in artificial intelligence, internet of things, and
5G communication mark the dawn of a new fully digitalized era. Correspondingly, the …

Advancements in the field of machine learning has made deep neural networks (DNNs)
ubiquitous. Their application in the domain of natural language processing (NLP) with …

To help train DNN models more efficiently, researchers have developed a series of new
computation devices and parallel training strategies. Choosing which device and strategy to …