The past decade has witnessed the great success of deep neural networks in various
domains. However, deep neural networks are very resource-intensive in terms of energy …

Over the past decade, deep-learning-based representations have demonstrated remarkable
performance in academia and industry. The learning capability of convolutional neural …

In recent years, deep neural networks have been successful in both industry and academia,
especially for computer vision tasks. The great success of deep learning is mainly due to its …

Deep neural models, in recent years, have been successful in almost every field, even
solving the most complex problem statements. However, these models are huge in size with …

Most existing distillation methods ignore the flexible role of the temperature in the loss
function and fix it as a hyper-parameter that can be decided by an inefficient grid search. In …

Remarkable achievements have been obtained by deep neural networks in the last several
years. However, the breakthrough in neural networks accuracy is always accompanied by …

In recent years, larger and deeper models are springing up and continuously pushing state-
of-the-art (SOTA) results across various fields like natural language processing (NLP) and …

Knowledge distillation (KD) has become a well established paradigm for compressing deep
neural networks. The typical way of conducting knowledge distillation is to train the student …

Semantic segmentation benchmarks in the realm of autonomous driving are dominated by
large pre-trained transformers, yet their widespread adoption is impeded by substantial …

Pre-trained language models have achieved state-of-the-art results in various Natural
Language Processing (NLP) tasks. GPT-3 has shown that scaling up pre-trained language …