The field of deep learning has witnessed significant progress, particularly in computer vision
(CV), natural language processing (NLP), and speech. The use of large-scale models …

ChatGPT shows remarkable capabilities for machine translation (MT). Several prior studies
have shown that it achieves comparable results to commercial systems for high-resource …

Non-autoregressive (NAR) generation, which is first proposed in neural machine translation
(NMT) to speed up inference, has attracted much attention in both machine learning and …

The field of deep learning has witnessed significant progress in recent times, particularly in
areas such as computer vision (CV), natural language processing (NLP), and speech. The …

Fine-tuning large pretrained language models on a limited training corpus usually suffers
from poor generalization. Prior works show that the recently-proposed sharpness-aware …

Adaptive training approaches, widely used in sequence-to-sequence models, commonly
reweigh the losses of different target tokens based on priors, eg word frequency. However …

This article aims to provide the information retrieval community with some reflections on
recent advances in retrieval learning by analyzing the reproducibility of image-text retrieval …

Catastrophic Forgetting (CF) means models forgetting previously acquired knowledge when
learning new data. It compromises the effectiveness of large language models (LLMs) …

Instruction tuning (IT) is crucial to tailoring large language models (LLMs) towards human-
centric interactions. Recent advancements have shown that the careful selection of a small …

Abstract Knowledge distillation (KD) is the preliminary step for training non-autoregressive
translation (NAT) models, which eases the training of NAT models at the cost of losing …