The rise of deep-learning (DL) has been fuelled by the improvements in accelerators. Due to
its unique features, the GPU continues to remain the most widely used accelerator for DL …

In recent years, artificial intelligence (AI) has been under rapid development, applied in
various areas. Among a vast number of neural network (NN) models, the convolutional …

Systems for training massive deep learning models (billions of parameters) today assume
and require specialized" hyperclusters": hundreds or thousands of GPUs wired with …

Deep learning (DL) has been increasingly adopted by a variety of software-intensive
systems. Developers mainly use GPUs to accelerate the training, testing, and deployment of …

Modern neural networks are increasingly bottlenecked by the limited capacity of on-device
GPU memory. Prior work explores drop** activations as a strategy to scale to larger neural …

It is known that deeper and wider neural networks can achieve better accuracy. But it is
difficult to continue the trend to increase model size due to limited GPU memory. One …

In recent years, deep learning has gained unprecedented success in various domains, the
key of the success is the larger and deeper deep neural networks (DNNs) that achieved very …

Training Graph Neural Networks (GNNs) on large graphs is a fundamental challenge due to
the high memory usage, which is mainly occupied by activations (eg, node embeddings) …

On-device learning is a promising technique for emerging privacy-preserving machine
learning paradigms. However, through quantitative experiments, we find that commodity …

Generative tasks, such as text generation and question answering, hold a crucial position in
the realm of mobile applications. Due to their sensitivity to privacy concerns, there is a …