A major factor in the recent success of large language models is the use of enormous and
ever-growing text datasets for unsupervised pre-training. However, naively training a model …

While deep learning models have shown remarkable performance in various tasks, they are
susceptible to learning non-generalizable _spurious features_ rather than the core features …

Local SGD is a popular optimization method in distributed learning, often outperforming mini-
batch SGD. Despite this practical success, proving the efficiency of local SGD has been …

Abstract Multi-distribution learning (MDL), which seeks to learn a shared model that
minimizes the worst-case risk across $ k $ distinct data distributions, has emerged as a …

We provide a unifying framework for the design and analysis of multi-calibrated predictors.
By placing the multi-calibration problem in the general setting of multi-objective learning …

This paper investigates group distributionally robust optimization (GDRO), with the purpose
to learn a model that performs well over $ m $ different distributions. First, we formulate …

Multi-distribution learning generalizes the classic PAC learning to handle data coming from
multiple distributions. Given a set of $ k $ data distributions and a hypothesis class of VC …

When facing data with imbalanced classes or groups, practitioners follow an intriguing
strategy to achieve best results. They throw away examples until the classes or groups are …

Multi-distribution learning is a natural generalization of PAC learning to settings with multiple
data distributions. There remains a significant gap between the known upper and lower …

Multi-distribution or collaborative learning involves learning a single predictor that works
well across multiple data distributions, using samples from each during training. Recent …