Dual-Head Knowledge Distillation: Enhancing Logits Utilization with an Auxiliary Head
Traditional knowledge distillation focuses on aligning the student's predicted probabilities
with both ground-truth labels and the teacher's predicted probabilities. However, the …
with both ground-truth labels and the teacher's predicted probabilities. However, the …
Neural Collapse Inspired Knowledge Distillation
S Zhang, Z Song, K He - arxiv preprint arxiv:2412.11788, 2024 - arxiv.org
Existing knowledge distillation (KD) methods have demonstrated their ability in achieving
student network performance on par with their teachers. However, the knowledge gap …
student network performance on par with their teachers. However, the knowledge gap …
Ever Evolving Evaluator (EV3): Towards Flexible and Reliable Meta-Optimization for Knowledge Distillation
We introduce EV3, a novel meta-optimization framework designed to efficiently train
scalable machine learning models through an intuitive explore-assess-adapt protocol. In …
scalable machine learning models through an intuitive explore-assess-adapt protocol. In …
Unveiling Incomplete Modality Brain Tumor Segmentation: Leveraging Masked Predicted Auto-Encoder and Divergence Learning
Z Sun, J Li, Y Wang, J Cheng, Q Zhou, C Li - arxiv preprint arxiv …, 2024 - arxiv.org
Brain tumor segmentation remains a significant challenge, particularly in the context of multi-
modal magnetic resonance imaging (MRI) where missing modality images are common in …
modal magnetic resonance imaging (MRI) where missing modality images are common in …