Despite the recent success of deep neural networks in natural language processing and
other spheres of artificial intelligence, their interpretability remains a challenge. We analyze …

Multilingual BERT (mBERT) trained on 104 languages has shown surprisingly good cross-
lingual performance on several NLP tasks, even without explicit cross-lingual signals …

UDPipe is a trainable pipeline which performs sentence segmentation, tokenization, POS
tagging, lemmatization and dependency parsing. We present a prototype for UDPipe 2.0 …

Viable cross-lingual transfer critically depends on the availability of parallel texts. Shortage
of such resources imposes a development and evaluation bottleneck in multilingual …

We propose a practical scheme to train a single multilingual sequence labeling model that
yields state of the art results and is small and fast enough to run on a single CPU. Starting …

Multilingual Language Models (\MLLMs) such as mBERT, XLM, XLM-R,\textit {etc.} have
emerged as a viable option for bringing the power of pretraining to a large number of …

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before
fine-tuning again on the target task---often improves model performance substantially on …

Pre-trained word embeddings like ELMo and BERT contain rich syntactic and semantic
information, resulting in state-of-the-art performance on various tasks. We propose a very …

While recent work on multilingual language models has demonstrated their capacity for
cross-lingual zero-shot transfer on downstream tasks, there is a lack of consensus in the …

We recast dependency parsing as a sequence labeling problem, exploring several
encodings of dependency trees as labels. While dependency parsing by means of …