A survey of language model confidence estimation and calibration

J Geng, F Cai, Y Wang, H Koeppl, P Nakov… - ar** loss perspective
L Chen, A Perez-Lebel, F Suchanek… - The 2024 Conference …, 2024 - hal.science
Large Language Models (LLMs), such as GPT and LLaMA, are susceptible to generating
hallucinated answers in a confident tone. While previous efforts to elicit and calibrate …

Non-exchangeable conformal language generation with nearest neighbors

D Ulmer, C Zerva, AFT Martins - arxiv preprint arxiv:2402.00707, 2024 - arxiv.org
Quantifying uncertainty in automatically generated text is important for letting humans check
potential hallucinations and making systems more reliable. Conformal prediction is an …