Neural networks trained on distilled data often produce over-confident output and require
correction by calibration methods. Existing calibration methods such as temperature scaling …

This study introduces a novel hyperparameter in the Softmax function to regulate the rate of
gradient decay, which is dependent on sample probability. Our theoretical and empirical …

The recently developed sparse network training methods, such as Lottery Ticket Hypothesis
(LTH) and its variants, have shown impressive learning capacity by finding sparse sub …

Pruning deep neural networks is a widely used strategy to alleviate the computational
burden in machine learning. Overwhelming empirical evidence suggests that pruned …

In an age dominated by resource-intensive foundation models the ability to efficiently adapt
to downstream tasks is crucial. Visual Prompting (VP) drawing inspiration from the prompting …

Despite impressive performance, deep neural networks require significant memory and
computation costs, prohibiting their application in resource-constrained scenarios. Sparse …

Output uncertainty indicates whether the probabilistic properties reflect objective
characteristics of the model output. Unlike most loss functions and metrics in machine …

Sparse training has emerged as a promising method for resource-efficient deep neural
networks (DNNs) in real-world applications. However, the reliability of sparse models …

Modern deep learning models often exhibit overconfident predictions, inadequately
capturing uncertainty. During model optimization, the expected calibration error tends to …

Modern machine learning tasks often involve the training of over-parameterized models and
the challenge of addressing data bias. However, despite recent advances, there remains a …