Large language models can memorize and repeat their training data, causing privacy and
copyright risks. To mitigate memorization, we introduce a subtle modification to the next …

Inference on large language models (LLMs) can be expensive in terms of thecompute and
memory costs involved, especially when long sequence lengths areused. In particular, the …

Mixture-of-Experts (MoE) is a neural network architecture that adds sparsely activated expert
blocks to a base model, increasing the number of parameters without impacting …

We study a novel language model architecture that is capable of scaling test-time
computation by implicitly reasoning in latent space. Our model works by iterating a recurrent …

The field of deep learning has witnessed a remarkable shift towards extremely compute-and
memory-intensive neural networks. These newer larger models have enabled researchers …