Modern deep neural networks, particularly recent large language models, come with
massive model sizes that require significant computational and storage resources. To …

Learning continuously during all model lifetime is fundamental to deploy machine learning
solutions robust to drifts in the data distribution. Advances in Continual Learning (CL) with …

The growing energy and performance costs of deep learning have driven the community to
reduce the size of neural networks by selectively pruning components. Similarly to their …

Multi-head self-attention is a key component of the Transformer, a state-of-the-art
architecture for neural machine translation. In this work we evaluate the contribution made …

Coreference resolution is an important task for natural language understanding, and the
resolution of ambiguous pronouns a longstanding challenge. Nonetheless, existing corpora …

For machine translation, a vast majority of language pairs in the world are considered low-
resource because they have little parallel data available. Besides the technical challenges …

One of the first steps in the utterance interpretation pipeline of many task-oriented
conversational AI systems is to identify user intents and the corresponding slots. Since data …

Standard machine translation systems process sentences in isolation and hence ignore
extra-sentential information, even though extended context can both prevent mistakes in …

Though machine translation errors caused by the lack of context beyond one sentence have
long been acknowledged, the development of context-aware NMT systems is hampered by …

Unsupervised pre-training has led to much recent progress in natural language
understanding. In this paper, we study self-training as another way to leverage unlabeled …