Modern deep neural networks, particularly recent large language models, come with
massive model sizes that require significant computational and storage resources. To …

Neural network pruning typically removes connections or neurons from a pretrained
converged model; while a new pruning paradigm, pruning at initialization (PaI), attempts to …

Abstract Spiking Neural Networks (SNNs) have recently emerged as a new generation of
low-power deep neural networks, which is suitable to be implemented on low-power …

Graph Neural Networks (GNNs) have shown great promise in various graph learning tasks.
However, the computational overheads of fitting GNNs to large-scale graphs grow rapidly …

Despite Graph Neural Networks demonstrating considerable promise in graph
representation learning tasks, GNNs predominantly face significant issues with over-fitting …

A deeper network structure generally handles more complicated non-linearity and performs
more competitively. Nowadays, advanced network designs often contain a large number of …

State-of-the-art deep learning models have a parameter count that reaches into the billions.
Training, storing and transferring such models is energy and time consuming, thus costly. A …

Sparsification-based pruning has been an important category in model compression.
Existing methods commonly set sparsity-inducing penalty terms to suppress the importance …

Multimodal Large Language Model (MLLM) have demonstrated strong generalization
capabilities across diverse distributions and tasks, largely due to extensive pre-training …

The recently developed sparse network training methods, such as Lottery Ticket Hypothesis
(LTH) and its variants, have shown impressive learning capacity by finding sparse sub …