Neural machine translation for low-resource languages: A survey
Neural Machine Translation (NMT) has seen tremendous growth in the last ten years since
the early 2000s and has already entered a mature phase. While considered the most widely …
the early 2000s and has already entered a mature phase. While considered the most widely …
Scaling data-constrained language models
The current trend of scaling language models involves increasing both parameter count and
training dataset size. Extrapolating this trend suggests that training dataset size may soon be …
training dataset size. Extrapolating this trend suggests that training dataset size may soon be …
Towards multidomain and multilingual abusive language detection: a survey
Abusive language is an important issue in online communication across different platforms
and languages. Having a robust model to detect abusive instances automatically is a …
and languages. Having a robust model to detect abusive instances automatically is a …
Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation
Much recent progress in applications of machine learning models to NLP has been driven
by benchmarks that evaluate models across a wide variety of tasks. However, these broad …
by benchmarks that evaluate models across a wide variety of tasks. However, these broad …
Ext5: Towards extreme multi-task scaling for transfer learning
V Aribandi, Y Tay, T Schuster, J Rao, HS Zheng… - ar** session dataset for recommendation and text generation
Modeling customer shop** intentions is a crucial task for e-commerce, as it directly
impacts user experience and engagement. Thus, accurately understanding customer …
impacts user experience and engagement. Thus, accurately understanding customer …
Survey of low-resource machine translation
We present a survey covering the state of the art in low-resource machine translation (MT)
research. There are currently around 7,000 languages spoken in the world and almost all …
research. There are currently around 7,000 languages spoken in the world and almost all …
From zero to hero: On the limitations of zero-shot cross-lingual transfer with multilingual transformers
Massively multilingual transformers pretrained with language modeling objectives (eg,
mBERT, XLM-R) have become a de facto default transfer paradigm for zero-shot cross …
mBERT, XLM-R) have become a de facto default transfer paradigm for zero-shot cross …
Neural unsupervised domain adaptation in NLP---a survey
Deep neural networks excel at learning from labeled data and achieve state-of-the-art
resultson a wide array of Natural Language Processing tasks. In contrast, learning from …
resultson a wide array of Natural Language Processing tasks. In contrast, learning from …
Intermediate-task transfer learning with pretrained models for natural language understanding: When and why does it work?
While pretrained models such as BERT have shown large gains across natural language
understanding tasks, their performance can be improved by further training the model on a …
understanding tasks, their performance can be improved by further training the model on a …