Modern artificial intelligence (AI) systems are powered by foundation models. This paper
presents a new set of foundation models, called Llama 3. It is a herd of language models …

Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated
exceptional performance in code-related tasks. However, most existing models are solely …

In this work we systematically review the recent advancements in software engineering with
language models, covering 70+ models, 40+ evaluation tasks, 180+ datasets, and 900 …

Large Language Models (LLMs) combined with instruction tuning have made significant
progress when generalizing to unseen tasks. However, they have been less successful in …

The extrapolation capability of Large Language Models (LLMs) based on Rotary Position
Embedding is currently a topic of considerable interest. The mainstream approach to …

In this paper, we propose R $^ 3$: Learning Reasoning through Reverse Curriculum
Reinforcement Learning (RL), a novel method that employs only outcome supervision to …

As Large Language Models (LLMs) are deployed more widely, customization with respect to
vocabulary, style, and character becomes more important. In this work, we introduce model …

Recent advancements in large language models (LLMs) have greatly improved code
generation, specifically at the function level. For instance, GPT-4o has achieved a 91.0 …

Finetuning language models on domain-specific corpus is a common approach to enhance
their domain knowledge and capability. While improving performance on domain tasks, it …

Code generation plays a crucial role in various tasks, such as code auto-completion and
mathematical reasoning. Previous work has proposed numerous methods to enhance code …