Uncovering the mechanisms of physics is driving a new paradigm in artificial intelligence
(AI) discovery. Today, physics has enabled us to understand the AI paradigm in a wide …

In recent years, foundation models (FMs) such as large language models (LLMs) and latent
diffusion models (LDMs) have profoundly impacted diverse sectors, including music. This …

Abstract Existing Knowledge Distillation (KD) methods typically focus on transferring
knowledge from a large-capacity teacher to a low-capacity student model achieving …

Recently, text-to-image models have been thriving. Despite their powerful generative
capacity, our research has uncovered a lack of robustness in this generation process …

The burgeoning field of Multimodal Large Language Models (MLLMs) has exhibited
remarkable performance in diverse tasks such as captioning commonsense reasoning and …

In this paper, we present DetKDS, the first framework that searches for optimal detection
distillation policies. Manual design of detection distillers becomes challenging and time …

Abstract Knowledge distillation (KD) has been applied to various tasks successfully and
mainstream methods typically boost the student model via spatial imitation losses. However …

The tremendous success of large models trained on extensive datasets demonstrates that
scale is a key ingredient in achieving superior results. Therefore, the reflection on the …

In contrast to existing complex methodologies commonly employed for distilling knowledge
from a teacher to a student, this paper showcases the efficacy of a simple yet powerful …

Deep neural networks have consistently represented the state of the art in most computer
vision problems. In these scenarios, larger and more complex models have demonstrated …