Redpajama: an open dataset for training large language models

M Weber, D Fu, Q Anthony, Y Oren… - Advances in …, 2025 - proceedings.neurips.cc
Large language models are increasingly becoming a cornerstone technology in artificial
intelligence, the sciences, and society as a whole, yet the optimal strategies for dataset …

Foundation models for music: A survey

Y Ma, A Øland, A Ragni, BMS Del Sette, C Saitis… - arxiv preprint arxiv …, 2024 - arxiv.org
In recent years, foundation models (FMs) such as large language models (LLMs) and latent
diffusion models (LDMs) have profoundly impacted diverse sectors, including music. This …

Opencoder: The open cookbook for top-tier code large language models

S Huang, T Cheng, JK Liu, J Hao, L Song, Y Xu… - arxiv preprint arxiv …, 2024 - arxiv.org
Large language models (LLMs) for code have become indispensable in various domains,
including code generation, reasoning tasks and agent systems. While open-access code …

Visrag: Vision-based retrieval-augmented generation on multi-modality documents

S Yu, C Tang, B Xu, J Cui, J Ran, Y Yan, Z Liu… - arxiv preprint arxiv …, 2024 - arxiv.org
Retrieval-augmented generation (RAG) is an effective technique that enables large
language models (LLMs) to utilize external knowledge sources for generation. However …

Hellobench: Evaluating long text generation capabilities of large language models

H Que, F Duan, L He, Y Mou, W Zhou, J Liu… - arxiv preprint arxiv …, 2024 - arxiv.org
In recent years, Large Language Models (LLMs) have demonstrated remarkable capabilities
in various tasks (eg, long-context understanding), and many benchmarks have been …

[PDF][PDF] Baichuan-omni technical report

Y Li, H Sun, M Lin, T Li, G Dong, T Zhang… - arxiv preprint arxiv …, 2024 - researchgate.net
The salient multimodal capabilities and interactive experience of GPT-4o highlight its critical
role in practical applications, yet it lacks a high-performing open-source counterpart. In this …

Ddk: Distilling domain knowledge for efficient large language models

J Liu, C Zhang, J Guo, Y Zhang, H Que, K Deng… - arxiv preprint arxiv …, 2024 - arxiv.org
Despite the advanced intelligence abilities of large language models (LLMs) in various
applications, they still face significant computational and storage demands. Knowledge …

Tablebench: A comprehensive and complex benchmark for table question answering

X Wu, J Yang, L Chai, G Zhang, J Liu, X Du… - arxiv preprint arxiv …, 2024 - arxiv.org
Recent advancements in Large Language Models (LLMs) have markedly enhanced the
interpretation and processing of tabular data, introducing previously unimaginable …

2 OLMo 2 Furious

T OLMo, P Walsh, L Soldaini, D Groeneveld… - arxiv preprint arxiv …, 2024 - arxiv.org
We present OLMo 2, the next generation of our fully open language models. OLMo 2
includes dense autoregressive models with improved architecture and training recipe …

RoleAgent: Building, Interacting, and Benchmarking High-quality Role-Playing Agents from Scripts

J Liu, Z Ni, H Que, N Wang, J Yang… - Advances in …, 2025 - proceedings.neurips.cc
Believable agents can empower interactive applications ranging from immersive
environments to rehearsal spaces for interpersonal communication. Recently, generative …