In the field of large language models (LLMs), Knowledge Distillation (KD) is a critical
technique for transferring capabilities from teacher models to student models. However …

The success of Large Language Models (LLM) has led researchers to explore Multimodal
Large Language Models (MLLM) for unified visual and linguistic understanding. However …

Knowledge distillation (KD) is a technique that compresses large teacher models by training
smaller student models to mimic them. The success of KD in auto-regressive language …

Knowledge distillation (KD) is known as a promising solution to compress large language
models (LLMs) via transferring their knowledge to smaller models. During this process, white …

In this paper, we propose a novel LLM-Neo framework that efficiently transfers knowledge
from a large language model (LLM) teacher to a compact student. Initially, we revisit the …

Low-resource event extraction presents a significant challenge in real-world applications,
particularly in domains like pharmaceuticals, military and law, where data is frequently …

Knowledge distillation (KD) has shown great promise in transferring knowledge from larger
teacher models to smaller student models. However, existing KD strategies for large …

With the passage of the Right to Be Forgotten (RTBF) regulations and the scaling up of
language model training datasets, research on model unlearning in large language models …

Causal language models have demonstrated remarkable capabilities, but their size poses
significant challenges for deployment in resource-constrained environments. Knowledge …

This paper introduces a novel approach to extract knowledge from large language models
and generate structured historical datasets. We investigate the feasibility and limitations of …