The needle-in-a-haystack (NIAH) test, which examines the ability to retrieve a piece of
information (the" needle") from long distractor texts (the" haystack"), has been widely …

Although current Multi-modal Large Language Models (MLLMs) demonstrate promising
results in video understanding, processing extremely long videos remains an ongoing …

Current long context large language models (LLMs) can process inputs up to 100,000
tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words …

Extending large language models to effectively handle long contexts requires instruction fine-
tuning on input sequences of similar length. To address this, we present LongAlign--a recipe …

This paper aims to overcome the" lost-in-the-middle" challenge of large language models
(LLMs). While recent advancements have successfully enabled LLMs to perform stable …

Large language models (LLMs) represent a groundbreaking advancement in the domain of
natural language processing due to their impressive reasoning abilities. Recently, there has …

The rapid advancement of Large Language Models (LLMs) has introduced a new frontier in
natural language processing, particularly in understanding and processing long-context …

The ability of Large Language Models (LLMs) to process and generate coherent text is
markedly weakened when the number of input tokens exceeds their pretraining length …

With large language models (LLMs) widely deployed in long content generation recently,
there has emerged an increasing demand for efficient long-sequence inference support …

The impressive performance of large language models (LLMs) on code-related tasks has
shown the potential of fully automated software development. In light of this, we introduce a …