With the increasing capabilities of large language models (LLMs), in-context learning (ICL)
has emerged as a new paradigm for natural language processing (NLP), where LLMs make …

Understanding in-context learning (ICL) capability that enables large language models
(LLMs) to excel in proficiency through demonstration examples is of utmost importance. This …

Transformers have recently revolutionized many domains in modern machine learning and
one salient discovery is their remarkable in-context learning capability, where models can …

Recent research has demonstrated that transformers, particularly linear attention models,
implicitly execute gradient-descent-like algorithms on data provided in-context during their …

Abstract Large Language Models have demonstrated remarkable performance across
various tasks exhibiting the capacity to swiftly acquire new skills such as through In-Context …

Large Language Models (LLMs) are transforming the way people generate, explore, and
engage with content. We study how we can develop LLM applications for online social …

Generalization and robustness to input variation are core desiderata of machine learning
research. Language varies along several axes, most importantly, language instance (eg …

Large language models (LLMs) have demonstrated impressive reasoning capabilities, yet
there is ongoing debate about these abilities and the potential data contamination problem …

Large monolithic generative models trained on massive amounts of data have become an
increasingly dominant approach in AI research. In this paper, we argue that we should …

The high cost of full-parameter fine-tuning (FFT) of Large Language Models (LLMs) has led
to a series of parameter-efficient fine-tuning (PEFT) methods. However, it remains unclear …