We present a simple self-training method that achieves 88.4% top-1 accuracy on ImageNet,
which is 2.0% better than the state-of-the-art model that requires 3.5 B weakly labeled …

Abstract Knowledge distillation (KD) is the preliminary step for training non-autoregressive
translation (NAT) models, which eases the training of NAT models at the cost of losing …

We present a simple and effective pretraining strategy--bidirectional training (BiT) for neural
machine translation. Specifically, we bidirectionally update the model parameters at the …

Knowledge distillation (KD) is commonly used to construct synthetic data for training non-
autoregressive translation (NAT) models. However, there exists a discrepancy on low …

While monolingual data has been shown to be useful in improving bilingual neural machine
translation (NMT), effectively and efficiently leveraging monolingual data for Multilingual …

Self-training has proven effective for improving NMT performance by augmenting model
training with synthetic parallel data. The common practice is to construct synthetic data …

Simultaneous machine translation (SiMT) is usually done via sequence-level knowledge
distillation (Seq-KD) from a full-sentence neural machine translation (NMT) model. However …

This paper describes the University of Sydney& JD's joint submission of the IWSLT 2021 low
resource speech translation task. We participated in the Swahili-English direction and got …

This paper presents the submission of Huawei Translate Services Center (HW-TSC) to the
WMT 2021 News Translation Shared Task. We participate in 7 language pairs, including …

This paper describes Tencent's multilingual machine translation systems for the WMT22
shared task on Large-Scale Machine Translation Evaluation for African Languages. We …