Federated Learning (FL) is a machine learning paradigm that allows decentralized clients to
learn collaboratively without sharing their private data. However, excessive computation and …

We provide the first exploration of sentence embeddings from text-to-text transformers (T5).
Sentence embeddings are broadly useful for language processing tasks. While T5 achieves …

Heterogeneity across clients in federated learning (FL) usually hinders the optimization
convergence and generalization performance when the aggregation of clients' knowledge …

The goal of text ranking is to generate an ordered list of texts retrieved from a corpus in
response to a query. Although the most common formulation of text ranking is search …

How to transfer the semantic information in a sentence to a computable numerical
embedding form is a fundamental problem in natural language processing. An informative …

Text generation has made significant advances in the last few years. Yet, evaluation metrics
have lagged behind, as the most popular choices (eg, BLEU and ROUGE) may correlate …

While task-specific finetuning of pretrained networks has led to significant empirical
advances in NLP, the large size of networks makes finetuning difficult to deploy in multi-task …

We show how to assess a language model's knowledge of basic concepts of morality. We
introduce the ETHICS dataset, a new benchmark that spans concepts in justice, well-being …

Although pretrained Transformers such as BERT achieve high accuracy on in-distribution
examples, do they generalize to new distributions? We systematically measure out-of …

The classic supervised machine learning paradigm is based on learning in isolation, a
single predictive model for a task using a single dataset. This approach requires a large …