The field of deep learning has witnessed significant progress, particularly in computer vision
(CV), natural language processing (NLP), and speech. The use of large-scale models …

Large multimodal models trained on natural documents, which interleave images and text,
outperform models trained on image-text pairs on various multimodal benchmarks …

The current trend of scaling language models involves increasing both parameter count and
training dataset size. Extrapolating this trend suggests that training dataset size may soon be …

As language models grow ever larger, the need for large-scale high-quality text datasets has
never been more pressing, especially in multilingual settings. The BigScience workshop, a 1 …

Large language models have led to remarkable progress on many NLP tasks, and
researchers are turning to ever-larger text corpora to train them. Some of the largest corpora …

In this paper, we explore the effects of language variants, data sizes, and fine-tuning task
types in Arabic pre-trained language models. To do so, we build three pre-trained language …

The driving factors behind the development of large language models (LLMs) with
impressive learning capabilities are their colossal model sizes and extensive training …

In this paper, we describe our approach to utilize pre-trained BERT models with
Convolutional Neural Networks for sub-task A of the Multilingual Offensive Language …

Pretrained multilingual language models have been shown to work well on many languages
for a variety of downstream NLP tasks. However, these models are known to require a lot of …

We illustrate how standard psychometric inventories originally designed for assessing
noncognitive human traits can be repurposed as diagnostic tools to evaluate analogous …