The emerging field of artificial intelligence of things (AIoT, AI+ IoT) is driven by the
widespread use of intelligent infrastructures and the impressive success of deep learning …

With the rapid evolution of the Internet, Internet of Things (IoT), and geographic information
systems (GIS), spatio-temporal Big Data (STBD) is experiencing exponential growth …

The increasing computational complexity of DNNs achieved unprecedented successes in
various areas such as machine vision and natural language processing (NLP), eg, the …

Fine-tuning is an increasingly common technique that leverages transfer learning to
dramatically expedite the training of huge, high-quality models. Critically, fine-tuning holds …

Since Deep Neural Networks (DNNs) are deeper and larger, performing DNNs training on
existing accelerators (eg, GPUs) is challenging due to their limited device memory capacity …

Training and fine-tuning large language models (LLMs) with hundreds of billions to trillions
of parameters requires tens of thousands of GPUs, and a highly scalable software stack. In …

In real applications, it is common to train deep neural networks (DNNs) on modest clusters.
With the continuous increase of model size and batch size, the training of DNNs becomes …

In Federated Learning (FL), the size of local models matters. On the one hand, it is logical to
use large-capacity neural networks in pursuit of high performance. On the other hand, deep …

Deep Neural Network (DNN) frameworks use distributed training to enable faster time to
convergence and alleviate memory capacity limitations when training large models and/or …

Iterative memory-bound solvers commonly occur in HPC codes. Typical GPU
implementations have a loop on the host side that invokes the GPU kernel as much as …