Transfer learning has recently become the dominant paradigm of machine learning. Pre-
trained models fine-tuned for downstream tasks achieve better performance with fewer …

Large language models (LLMs) demonstrate remarkable multilingual capabilities without
being pre-trained on specially curated multilingual parallel corpora. It remains a challenging …

To what extent can neural network models learn generalizations about language structure,
and how do we find out what they have learned? We explore these questions by training …

Computational typology has gained traction in the field of Natural Language Processing
(NLP) in recent years, as evidenced by the increasing number of papers on the topic and the …

Massively multilingual models such as mBERT and XLM-R are increasingly valued in
Natural Language Processing research and applications, due to their ability to tackle the …

Large pretrained multilingual models, trained on dozens of languages, have delivered
promising results due to cross-lingual learning capabilities on variety of language tasks …

Current decoder-based pre-trained language models (PLMs) successfully demonstrate
multilingual capabilities. However, it is unclear how these models handle multilingualism …

Current voice recognition approaches use multi-task, multilingual models for speech tasks
like Automatic Speech Recognition (ASR) to make them applicable to many languages …

We find arithmetic ability resides within a limited number of attention heads, with each head
specializing in distinct operations. To delve into the reason, we introduce the Comparative …

Structural probing work has found evidence for latent syntactic information in pre-trained
language models. However, much of this analysis has focused on monolingual models, and …