A comprehensive survey on pretrained foundation models: A history from bert to chatgpt

C Zhou, Q Li, C Li, J Yu, Y Liu, G Wang… - International Journal of …, 2024 - Springer
Abstract Pretrained Foundation Models (PFMs) are regarded as the foundation for various
downstream tasks across different data modalities. A PFM (eg, BERT, ChatGPT, GPT-4) is …

Survey on factuality in large language models: Knowledge, retrieval and domain-specificity

C Wang, X Liu, Y Yue, X Tang, T Zhang… - arxiv preprint arxiv …, 2023 - arxiv.org
This survey addresses the crucial issue of factuality in Large Language Models (LLMs). As
LLMs find applications across diverse domains, the reliability and accuracy of their outputs …

Imagereward: Learning and evaluating human preferences for text-to-image generation

J Xu, X Liu, Y Wu, Y Tong, Q Li… - Advances in …, 2023 - proceedings.neurips.cc
We present a comprehensive solution to learn and improve text-to-image models from
human preference feedback. To begin with, we build ImageReward---the first general …

Expel: Llm agents are experiential learners

A Zhao, D Huang, Q Xu, M Lin, YJ Liu… - Proceedings of the AAAI …, 2024 - ojs.aaai.org
The recent surge in research interest in applying large language models (LLMs) to decision-
making tasks has flourished by leveraging the extensive world knowledge embedded in …

MTEB: Massive text embedding benchmark

N Muennighoff, N Tazi, L Magne, N Reimers - arxiv preprint arxiv …, 2022 - arxiv.org
Text embeddings are commonly evaluated on a small set of datasets from a single task not
covering their possible applications to other tasks. It is unclear whether state-of-the-art …

Text-to-sql empowered by large language models: A benchmark evaluation

D Gao, H Wang, Y Li, X Sun, Y Qian, B Ding… - arxiv preprint arxiv …, 2023 - arxiv.org
Large language models (LLMs) have emerged as a new paradigm for Text-to-SQL task.
However, the absence of a systematical benchmark inhibits the development of designing …

Large language models on graphs: A comprehensive survey

B **, G Liu, C Han, M Jiang, H Ji… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Large language models (LLMs), such as GPT4 and LLaMA, are creating significant
advancements in natural language processing, due to their strong text encoding/decoding …

BERTopic: Neural topic modeling with a class-based TF-IDF procedure

M Grootendorst - arxiv preprint arxiv:2203.05794, 2022 - arxiv.org
Topic models can be useful tools to discover latent topics in collections of documents.
Recent studies have shown the feasibility of approach topic modeling as a clustering task …

The unlocking spell on base llms: Rethinking alignment via in-context learning

BY Lin, A Ravichander, X Lu, N Dziri, M Sclar… - arxiv preprint arxiv …, 2023 - arxiv.org
The alignment tuning process of large language models (LLMs) typically involves instruction
learning through supervised fine-tuning (SFT) and preference tuning via reinforcement …

Omnivec: Learning robust representations with cross modal sharing

S Srivastava, G Sharma - Proceedings of the IEEE/CVF …, 2024 - openaccess.thecvf.com
Majority of research in learning based methods has been towards designing and training
networks for specific tasks. However, many of the learning based tasks, across modalities …