Large, pre-trained language models (PLMs) such as BERT and GPT have drastically
changed the Natural Language Processing (NLP) field. For numerous NLP tasks …

Deep neural-network-based language models (LMs) are increasingly applied to large-scale
protein sequence data to predict protein function. However, being largely black-box models …

Surprisal theory posits that less-predictable words should take more time to process, with
word predictability quantified as surprisal, ie, negative log probability in context. While …

Multilingual Language Models (\MLLMs) such as mBERT, XLM, XLM-R,\textit {etc.} have
emerged as a viable option for bringing the power of pretraining to a large number of …

Pretrained multilingual contextual representations have shown great success, but due to the
limits of their pretraining data, their benefits do not apply equally to all language varieties …

Recently, leveraging pre-trained Transformer based language models in down stream, task
specific models has advanced state of the art results in natural language understanding …

In recent years, large language models (eg, Open AI's GPT-4, Meta's LLaMa, Google's
PaLM) have become the dominant approach for building AI systems to analyze and …

We train several language models for Icelandic, including IceBERT, that achieve state-of-the-
art performance in a variety of downstream tasks, including part-of-speech tagging, named …

Multilingual language models have been a crucial breakthrough as they considerably
reduce the need of data for under-resourced languages. Nevertheless, the superiority of …

This paper focuses on text-based affective computing for Jopara, a code-switching language
that combines Guarani and Spanish. First, we collected a dataset of tweets primarily written …