Preserving integrity in online social networks Page 1 92 COMMUNICATIONS OF THE ACM |
FEBRUARY 2022 | VOL. 65 | NO. 2 review articles THE GOAL OF online social networks is to …

Convolutional neural networks (CNNs) have so far been the de-facto model for visual data.
Recent work has shown that (Vision) Transformer models (ViT) can achieve comparable or …

State-of-the-art unsupervised multilingual models (eg, multilingual BERT) have been shown
to generalize in a zero-shot cross-lingual setting. This generalization ability has been …

This paper shows that pretraining multilingual language models at scale leads to significant
performance gains for a wide range of cross-lingual transfer tasks. We train a Transformer …

In this work, we present an information-theoretic framework that formulates cross-lingual
language model pre-training as maximizing mutual information between multilingual-multi …

A key factor in the success of deep neural networks is the ability to scale models to improve
performance by varying the architecture depth and width. This simple property of neural …

The success of large pretrained language models (LMs) such as BERT and RoBERTa has
sparked interest in probing their representations, in order to unveil what types of knowledge …

Multilingual BERT (mBERT) trained on 104 languages has shown surprisingly good cross-
lingual performance on several NLP tasks, even without explicit cross-lingual signals …

Massively multilingual transformers pretrained with language modeling objectives (eg,
mBERT, XLM-R) have become a de facto default transfer paradigm for zero-shot cross …

We present mSLAM, a multilingual Speech and LAnguage Model that learns cross-lingual
cross-modal representations of speech and text by pre-training jointly on large amounts of …