Knowledge distillation (KD) is a prevalent model compression technique in deep learning,
aiming to leverage knowledge from a large teacher model to enhance the training of a …

Abstract Knowledge distillation involves transferring soft labels from a teacher to a student
using a shared temperature-based softmax function. However the assumption of a shared …

Prompt learning has emerged as a valuable technique in enhancing vision-language
models (VLMs) such as CLIP for downstream tasks in specific domains. Existing work mainly …

Abstract Knowledge Distillation (KD) has been validated as an effective model compression
technique for learning compact object detectors. Existing state-of-the-art KD methods for …

We introduce an efficient self-knowledge distillation framework, Dual Teachers for Self-
Knowledge Distillation (DTSKD), where the student receives self-supervisions by dual …

Abstract Contrastive Language-Image Pre-training (CLIP) has become a promising
language-supervised visual pre-training framework. This paper aims to distill small CLIP …

Transformer-based architectures have become the de-facto standard models for diverse
vision tasks owing to their superior performance. As the size of these transformer-based …

Large Language Models (LLMs), such as LLaMA and GPT-4, have transformed the
paradigm of natural language comprehension and generation. Despite their impressive …

Prompt learning has surfaced as an effective approach to enhance the performance of
Vision-Language Models (VLMs) like CLIP when applied to downstream tasks. However …

Objective: Histological classification is a challenging task due to the diverse appearances,
unpredictable variations, and blurry edges of histological tissues. Recently, many …