Over recent years, there has been a rapid development of deep learning (DL) in both
industry and academia fields. However, finding the optimal hyperparameters of a DL model …

Recent advancements in natural language processing have given rise to numerous pre-
training language models in question-answering systems. However, with the constant …

Pre-trained language models (PLMs) have achieved great success in natural language
processing. Most of PLMs follow the default setting of architecture hyper-parameters (eg, the …

In this paper, we develop a novel Joint Model Compression (referred to as JMC) method by
combining structured pruning and dense knowledge distillation techniques to significantly …

Purpose The purpose of this paper is to describe a new approach to sentence
representation learning leading to text classification using Bidirectional Encoder …

Knowledge distillation (KD) is the process of transferring knowledge from a large model to a
small one. It has gained increasing attention in the natural language processing community …

Large language models have become a vital component in modern NLP, achieving state of
the art performance in a variety of tasks. However, they are often inefficient for real-world …

In recent years, both industry and academia have made significant advances in deep
learning (DL). However, configuring the hyperparameters of deep models typically requires …

Searching for suitable biological strategies in bio-inspired design (BID) is the first problem
that designers need to solve. Based on the biological strategy database of the AskNature, a …

Building huge and highly capable language models has been a trend in the past years.
Despite their great performance, they incur high computational cost. A common solution is to …