In recent years, deep neural networks have been successful in both industry and academia,
especially for computer vision tasks. The great success of deep learning is mainly due to its …

In recent years, deep learning (DL) models have demonstrated remarkable achievements
on non-trivial tasks such as speech recognition, image processing, and natural language …

Deep neural networks (DNNs) have achieved great success in many object detection tasks.
However, such DNNS-based large object detection models are generally computationally …

In prevalent knowledge distillation, logits in most image recognition models are computed by
global average pooling, then used to learn to encode the high-level and task-relevant …

Deep learning has achieved notable success across academia, medicine, and industry. Its
ability to identify complex patterns in large-scale data and to manage millions of parameters …

Abstract Knowledge distillation which learns a lightweight student model by distilling
knowledge from a cumbersome teacher model is an attractive approach for learning …

Data-free knowledge distillation (DFKD) is an effective manner to solve model compression
and transmission restrictions while retaining privacy protection, which has attracted …

Neural network quantization aims to accelerate and trim full-precision neural network
models by using low bit approximations. Methods adopting the quantization aware training …

This work presents a Fully BInarized Large Language Model (FBI-LLM), demonstrating for
the first time how to train a large-scale binary language model from scratch (not the partial …

Quantization-aware training (QAT) and Knowledge Distillation (KD) are combined to achieve
competitive performance in creating low-bit deep learning models. However, existing works …