Modern deep neural networks, particularly recent large language models, come with
massive model sizes that require significant computational and storage resources. To …

Neural network pruning typically removes connections or neurons from a pretrained
converged model; while a new pruning paradigm, pruning at initialization (PaI), attempts to …

Scaling up neural networks has led to remarkable performance across a wide range of
tasks. Moreover, performance often follows reliable scaling laws as a function of training set …

In response to recent data regulation requirements, machine unlearning (MU) has emerged
as a critical process to remove the influence of specific examples from a given model …

As the computing power of modern hardware is increasing strongly, pre-trained deep
learning models (eg, BERT, GPT-3) learned on large-scale datasets have shown their …

Vision transformers (ViTs) have recently received explosive popularity, but their enormous
model sizes and training costs remain daunting. Conventional post-training pruning often …

With graphs rapidly growing in size and deeper graph neural networks (GNNs) emerging,
the training and inference of GNNs become increasingly expensive. Existing network weight …

Works on lottery ticket hypothesis (LTH) and single-shot network pruning (SNIP) have raised
a lot of attention currently on post-training pruning (iterative magnitude pruning), and before …

Federated learning (FL) enables distribution of machine learning workloads from the cloud
to resource-limited edge devices. Unfortunately, current deep networks remain not only too …

The deployment constraints in practical applications necessitate the pruning of large-scale
deep learning models, ie, promoting their weight sparsity. As illustrated by the Lottery Ticket …