Sentiment analysis automatically evaluates people's opinions of products or services. It is an
emerging research area with promising advancements in high-resource languages such as …

We introduce a new domain expert mixture (DEMix) layer that enables conditioning a
language model (LM) on the domain of the input text. A DEMix layer is a collection of expert …

What are the units of text that we want to model? From bytes to multi-word expressions, text
can be analyzed and generated at many granularities. Until recently, most natural language …

The BLOOM model is a large publicly available multilingual language model, but its
pretraining was limited to 46 languages. To extend the benefits of BLOOM to other …

Pretrained multilingual models are able to perform cross-lingual transfer in a zero-shot
setting, even for languages unseen during pretraining. However, prior work evaluating …

The performance of multilingual pretrained models is highly dependent on the availability of
monolingual or parallel text present in a target language. Thus, the majority of the world's …

We present IndoBERTweet, the first large-scale pretrained model for Indonesian Twitter that
is trained by extending a monolingually-trained Indonesian BERT model with additive …

Pretrained multilingual models (PMMs) enable zero-shot learning via cross-lingual transfer,
performing best for languages seen during pretraining. While methods exist to improve …

Transfer learning based on pretraining language models on a large amount of raw data has
become a new norm to reach state-of-the-art performance in NLP. Still, it remains unclear …

Pretrained large language models (LLMs) have emerged as a cornerstone in modern
natural language processing, with their utility expanding to various applications and …