Abstract Language models serve as a cornerstone in natural language processing, utilizing
mathematical methods to generalize language laws and knowledge for prediction and …

Optimizing large language models (LLMs) for downstream use cases often involves the
customization of pre-trained LLMs through further fine-tuning. Meta's open release of Llama …

Scaling the amount of compute used to train language models has dramatically improved
their capabilities. However, when it comes to inference, we often limit the amount of compute …

The growing number of cases that require digital forensic analysis raises concerns about the
ability of law enforcement to conduct investigations promptly. Consequently, this paper …

In this paper, we introduce InfiAgent-DABench, the first benchmark specifically designed to
evaluate LLM-based agents on data analysis tasks. These tasks require agents to end-to …

This work addresses the challenge of democratizing advanced Large Language Models
(LLMs) by compressing their mathematical reasoning capabilities into sub-billion parameter …

Transformer-based Large Language Models (LLMs) are pioneering advances in many
natural language processing tasks, however, their exceptional capabilities are restricted …

Large language models (LLMs) with long context windows have gained significant attention.
However, the KV cache, stored to avoid re-computation, becomes a bottleneck. Various …

Large language models (LLMs) have shown great potential as general-purpose AI
assistants in various domains. To meet the requirements of different applications, LLMs are …

Recurrent large language models that compete with Transformers in language modeling
perplexity are emerging at a rapid rate (eg, Mamba, RWKV). Excitingly, these architectures …