Large Language Models (LLMs) have the capacity to store and recall facts. Through
experimentation with open-source models, we observe that this ability to retrieve facts can …

Large language models (LLMs) have proven to be remarkably efficient, both across a wide
range of natural language processing tasks and well beyond them. However, a …

The transformer architecture has catalyzed revolutionary advances in language modeling.
However, recent architectural recipes, such as state-space models, have bridged the …

Attention mechanisms have revolutionized several domains of artificial intelligence, such as
natural language processing and computer vision, by enabling models to selectively focus …

Chain-of-Thought (CoT) is an efficient prompting method that enables the reasoning ability
of large language models by augmenting the query using multiple examples with …

Chain-of-Thought (CoT) is an efficient prompting method that enables the reasoning ability
of large language models by augmenting the query using multiple examples with multiple …

While attention-based models have demonstrated the remarkable ability of in-context
learning, the theoretical understanding of how these models acquired this ability through …

Transformers have achieved significant success in various fields, notably excelling in tasks
involving sequential data like natural language processing. Despite these achievements, the …

A key paradigm to improve the reasoning capabilities of large language models (LLMs) is to
allocate more inference-time compute to search against a verifier or reward model. This …

In recent years, transformer-based models have revolutionized deep learning, particularly in
sequence modeling. To better understand this phenomenon, there is a growing interest in …