Efficient methods for natural language processing: A survey

M Treviso, JU Lee, T Ji, B Aken, Q Cao… - Transactions of the …, 2023 - direct.mit.edu
Recent work in natural language processing (NLP) has yielded appealing results from
scaling model parameters and training data; however, using only scale to improve …

Token-level self-evolution training for sequence-to-sequence learning

K Peng, L Ding, Q Zhong, Y Ouyang… - Proceedings of the …, 2023 - aclanthology.org
Adaptive training approaches, widely used in sequence-to-sequence models, commonly
reweigh the losses of different target tokens based on priors, eg word frequency. However …

Meta multi-task nuclei segmentation with fewer training samples

C Han, H Yao, B Zhao, Z Li, Z Shi, L Wu, X Chen… - Medical Image …, 2022 - Elsevier
Cells/nuclei deliver massive information of microenvironment. An automatic nuclei
segmentation approach can reduce pathologists' workload and allow precise of the …

Uncertainty estimation and reduction of pre-trained models for text regression

Y Wang, D Beck, T Baldwin, K Verspoor - Transactions of the …, 2022 - direct.mit.edu
State-of-the-art classification and regression models are often not well calibrated, and
cannot reliably provide uncertainty estimates, limiting their utility in safety-critical …

Frequency-aware contrastive learning for neural machine translation

T Zhang, W Ye, B Yang, L Zhang, X Ren… - Proceedings of the …, 2022 - ojs.aaai.org
Low-frequency word prediction remains a challenge in modern neural machine translation
(NMT) systems. Recent adaptive training methods promote the output of infrequent words by …

Challenges of neural machine translation for short texts

Y Wan, B Yang, DF Wong, LS Chao, L Yao… - Computational …, 2022 - direct.mit.edu
Short texts (STs) present in a variety of scenarios, including query, dialog, and entity names.
Most of the exciting studies in neural machine translation (NMT) are focused on tackling …

Combining curriculum learning and knowledge distillation for dialogue generation

Q Zhu, X Chen, P Wu, JF Liu… - Findings of the Association …, 2021 - aclanthology.org
Curriculum learning, a machine training strategy that feeds training instances to the model
from easy to hard, has been proven to facilitate the dialogue generation task. Meanwhile …

Towards energy-preserving natural language understanding with spiking neural networks

R **ao, Y Wan, B Yang, H Zhang… - … on Audio, Speech …, 2022 - ieeexplore.ieee.org
Artificial neural networks have shown promising results in a variety of natural language
understanding (NLU) tasks. Despite their successes, conventional neural-based NLU …

Self-modifying state modeling for simultaneous machine translation

D Yu, X Kang, Y Liu, Y Zhou, C Zong - arxiv preprint arxiv:2406.02237, 2024 - arxiv.org
Simultaneous Machine Translation (SiMT) generates target outputs while receiving stream
source inputs and requires a read/write policy to decide whether to wait for the next source …