[HTML][HTML] A survey of GPT-3 family large language models including ChatGPT and GPT-4

KS Kalyan - Natural Language Processing Journal, 2024‏ - Elsevier
Large language models (LLMs) are a special class of pretrained language models (PLMs)
obtained by scaling model size, pretraining corpus and computation. LLMs, because of their …

Datasets for large language models: A comprehensive survey

Y Liu, J Cao, C Liu, K Ding, L ** - arxiv preprint arxiv:2402.18041, 2024‏ - arxiv.org
This paper embarks on an exploration into the Large Language Model (LLM) datasets,
which play a crucial role in the remarkable advancements of LLMs. The datasets serve as …

[PDF][PDF] A survey of large language models

WX Zhao, K Zhou, J Li, T Tang… - arxiv preprint arxiv …, 2023‏ - paper-notes.zhjwpku.com
Ever since the Turing Test was proposed in the 1950s, humans have explored the mastering
of language intelligence by machine. Language is essentially a complex, intricate system of …

Chameleon: Plug-and-play compositional reasoning with large language models

P Lu, B Peng, H Cheng, M Galley… - Advances in …, 2023‏ - proceedings.neurips.cc
Large language models (LLMs) have achieved remarkable progress in solving various
natural language processing tasks due to emergent reasoning abilities. However, LLMs …

Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

P Lu, H Bansal, T **a, J Liu, C Li, H Hajishirzi… - arxiv preprint arxiv …, 2023‏ - arxiv.org
Large Language Models (LLMs) and Large Multimodal Models (LMMs) exhibit impressive
problem-solving skills in many tasks and domains, but their ability in mathematical …

Mammoth2: Scaling instructions from the web

X Yue, T Zheng, G Zhang… - Advances in Neural …, 2025‏ - proceedings.neurips.cc
Instruction tuning improves the reasoning abilities of large language models (LLMs), with
data quality and scalability being the crucial factors. Most instruction tuning data come from …

Rest-mcts*: Llm self-training via process reward guided tree search

D Zhang, S Zhoubian, Z Hu, Y Yue… - Advances in Neural …, 2025‏ - proceedings.neurips.cc
Recent methodologies in LLM self-training mostly rely on LLM generating responses and
filtering those with correct output answers as training data. This approach often yields a low …

HelpSteer 2: Open-source dataset for training top-performing reward models

Z Wang, Y Dong, O Delalleau, J Zeng… - Advances in …, 2025‏ - proceedings.neurips.cc
High-quality preference datasets are essential for training reward models that can effectively
guide large language models (LLMs) in generating high-quality responses aligned with …

Mmlu-pro: A more robust and challenging multi-task language understanding benchmark

Y Wang, X Ma, G Zhang, Y Ni, A Chandra… - The Thirty-eight …, 2024‏ - openreview.net
In the age of large-scale language models, benchmarks like the Massive Multitask
Language Understanding (MMLU) have been pivotal in pushing the boundaries of what AI …

Scientific large language models: A survey on biological & chemical domains

Q Zhang, K Ding, T Lv, X Wang, Q Yin, Y Zhang… - ACM Computing …, 2024‏ - dl.acm.org
Large Language Models (LLMs) have emerged as a transformative power in enhancing
natural language comprehension, representing a significant stride toward artificial general …