Enhancing LLM Capabilities Beyond Scaling Up

W Yin, M Chen, R Zhang, B Zhou… - Proceedings of the …, 2024 - aclanthology.org
General-purpose large language models (LLMs) are progressively expanding both in scale
and access to unpublic training data. This has led to notable progress in a variety of AI …

Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining

D Sow, H Woisetschläger, S Bulusu, S Wang… - arxiv preprint arxiv …, 2025 - arxiv.org
Pretraining large language models (LLMs) on vast and heterogeneous datasets is crucial for
achieving state-of-the-art performance across diverse downstream tasks. However, current …