| Bloom: A 176b-parameter open-access multilingual language model BS Workshop, TL Scao, A Fan, C Akiki, E Pavlick, S Ilić, D Hesslow, ... JMLR 2023, 2022 | 1843* | 2022 |
| Beyond the imitation game: Quantifying and extrapolating the capabilities of language models A Srivastava, A Rastogi, A Rao, AAM Shoeb, A Abid, A Fisch, AR Brown, ... TMLR 2023, 2022 | 1411 | 2022 |
| StarCoder: may the source be with you! R Li, LB Allal, Y Zi, N Muennighoff, D Kocetkov, C Mou, M Marone, C Akiki, ... TMLR 2023, 2023 | 1033* | 2023 |
| A framework for few-shot language model evaluation L Gao, J Tow, S Biderman, S Black, A DiPofi, C Foster, L Golding, J Hsu, ... GitHub, 2021 | 768* | 2021 |
| Crosslingual generalization through multitask finetuning N Muennighoff, T Wang, L Sutawika, A Roberts, S Biderman, TL Scao, ... ACL 2023, 2022 | 738 | 2022 |
| MTEB: Massive text embedding benchmark N Muennighoff, N Tazi, L Magne, N Reimers EACL 2023, 2022 | 674 | 2022 |
| C-pack: Packed resources for general chinese embeddings S Xiao, Z Liu, P Zhang, N Muennighoff, D Lian, JY Nie SIGIR 2024, 2024 | 462 | 2024 |
| Kto: Model alignment as prospect theoretic optimization K Ethayarajh, W Xu, N Muennighoff, D Jurafsky, D Kiela ICML 2024 Spotlight, 2024 | 349 | 2024 |
| Olmo: Accelerating the science of language models D Groeneveld, I Beltagy, P Walsh, A Bhagia, R Kinney, O Tafjord, AH Jha, ... ACL 2024, Best Theme Paper Award, 2024 | 265* | 2024 |
| SantaCoder: don't reach for the stars! LB Allal, R Li, D Kocetkov, C Mou, C Akiki, CM Ferrandis, N Muennighoff, ... ICLR 2023 DL4C Workshop, Best Paper Award, 2023 | 251* | 2023 |
| Scaling Data-Constrained Language Models N Muennighoff, AM Rush, B Barak, TL Scao, A Piktus, N Tazi, S Pyysalo, ... NeurIPS 2023 Oral, Outstanding Paper Runner-Up Award, 2023 | 241 | 2023 |
| SGPT: GPT sentence embeddings for semantic search N Muennighoff arXiv, 2022 | 212 | 2022 |
| Starcoder 2 and the stack v2: The next generation A Lozhkov, R Li, LB Allal, F Cassano, J Lamy-Poirier, N Tazi, A Tang, ... arXiv, 2024 | 210 | 2024 |
| Octopack: Instruction tuning code large language models N Muennighoff, Q Liu, A Zebaze, Q Zheng, B Hui, TY Zhuo, S Singh, ... ICLR 2024 Spotlight, NeurIPS 2023 Instruction Workshop, 2023 | 197 | 2023 |
| Dolma: An Open Corpus of Three Trillion Tokens for Language Model Pretraining Research L Soldaini, R Kinney, A Bhagia, D Schwenk, D Atkinson, R Authur, ... ACL 2024, Best Resource Paper Award, 2024 | 190* | 2024 |
| Aya model: An instruction finetuned open-access multilingual language model A Üstün, V Aryabumi, ZX Yong, WY Ko, D D'souza, G Onilude, N Bhandari, ... ACL 2024, Best Paper Award, 2024 | 146 | 2024 |
| What Language Model to Train if You Have One Million GPU Hours? TL Scao, T Wang, D Hesslow, L Saulnier, S Bekman, MS Bari, S Bideman, ... EMNLP 2022 Findings, 2022 | 120 | 2022 |
| Generative representational instruction tuning N Muennighoff, H Su, L Wang, N Yang, F Wei, T Yu, A Singh, D Kiela ICLR 2024 AGI Workshop Oral, Best Paper Award, 2024 | 89 | 2024 |
| A Survey on Data Selection for Language Models A Albalak, Y Elazar, SM Xie, S Longpre, N Lambert, X Wang, ... TMLR 2024, 2024 | 86 | 2024 |
| Aya dataset: An open-access collection for multilingual instruction tuning S Singh, F Vargus, D Dsouza, BF Karlsson, A Mahendiran, WY Ko, ... ACL 2024, 2024 | 81 | 2024 |