A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions
The remarkable performance of large language models (LLMs) in content generation,
coding, and common-sense reasoning has spurred widespread integration into many facets …
coding, and common-sense reasoning has spurred widespread integration into many facets …
Rethinking Uncertainty Estimation in Natural Language Generation
Large Language Models (LLMs) are increasingly employed in real-world applications,
driving the need to evaluate the trustworthiness of their generated text. To this end, reliable …
driving the need to evaluate the trustworthiness of their generated text. To this end, reliable …
Calibrating Large Language Models Using Their Generations Only
As large language models (LLMs) are increasingly deployed in user-facing applications,
building trust and maintaining safety by accurately quantifying a model's confidence in its …
building trust and maintaining safety by accurately quantifying a model's confidence in its …
Rag-check: Evaluating multimodal retrieval augmented generation performance
Retrieval-augmented generation (RAG) improves large language models (LLMs) by using
external knowledge to guide response generation, reducing hallucinations. However, RAG …
external knowledge to guide response generation, reducing hallucinations. However, RAG …
Are LLM-judges robust to expressions of uncertainty? investigating the effect of epistemic markers on LLM-based evaluation
In line with the principle of honesty, there has been a growing effort to train large language
models (LLMs) to generate outputs containing epistemic markers. However, evaluation in …
models (LLMs) to generate outputs containing epistemic markers. However, evaluation in …
Graph-based Confidence Calibration for Large Language Models
One important approach to improving the reliability of large language models (LLMs) is to
provide accurate confidence estimations regarding the correctness of their answers …
provide accurate confidence estimations regarding the correctness of their answers …
Label-Confidence-Aware Uncertainty Estimation in Natural Language Generation
Q Lin, L Zhou, Z Yang, Y Cai - arxiv preprint arxiv:2412.07255, 2024 - arxiv.org
Large Language Models (LLMs) display formidable capabilities in generative tasks but also
pose potential risks due to their tendency to generate hallucinatory responses. Uncertainty …
pose potential risks due to their tendency to generate hallucinatory responses. Uncertainty …
Large Language Model Uncertainty Measurement and Calibration for Medical Diagnosis and Treatment
T Savage, J Wang, R Gallo, A Boukil, V Patel… - medRxiv, 2024 - medrxiv.org
Introduction The inability for Large Language Models (LLMs) to communicate uncertainty is
a significant barrier to their use in medicine. Before LLMs can be integrated into patient care …
a significant barrier to their use in medicine. Before LLMs can be integrated into patient care …
Lookers-On See Most of the Game: An External Insight-Guided Method for Enhancing Uncertainty Estimation
Large Language Models (LLMs) have gained increasing attention for their impressive
capabilities, alongside concerns about the reliability arising from their potential to generate …
capabilities, alongside concerns about the reliability arising from their potential to generate …