Abstract Knowledge Distillation (KD) uses the teacher's prediction logits as soft labels to
guide the student, while self-KD does not need a real teacher to require the soft labels. This …

Abstract Knowledge distillation (KD) is an effective training strategy to improve the
lightweight student models under the guidance of cumbersome teachers. However, the large …

In this paper, we present Auto-KD, the first automated search framework for optimal
knowledge distillation design. Traditional distillation techniques typically require handcrafted …

Abstract Knowledge distillation can be generally divided into offline and online categories
according to whether teacher model is pre-trained and persistent during the distillation …

Abstract Knowledge distillation (KD) has emerged as an effective technique for compressing
models that can enhance the lightweight model. Conventional KD methods propose various …

Abstract Mixed-Precision Quantization (MQ) can achieve a competitive accuracy-complexity
trade-off for models. Conventional training-based search methods require time-consuming …

The substantial success of Vision Transformer (ViT) in computer vision tasks is largely
attributed to the architecture design. This underscores the necessity of efficient architecture …

In this paper, we present SasWOT, the first training-free Semantic segmentation Architecture
Search (SAS) framework via an auto-discovery proxy. Semantic segmentation is widely used …

Artificial Intelligence (AI) has achieved significant advancements in technology and research
with the development over several decades, and is widely used in many areas including …

In this paper, we introduce Auto-GAS, the first training-free Generative Architecture Search
(GAS) framework enabled by an auto-discovered proxy. Generative models like Generative …