A survey of language model confidence estimation and calibration
Non-exchangeable conformal language generation with nearest neighbors
Quantifying uncertainty in automatically generated text is important for letting humans check
potential hallucinations and making systems more reliable. Conformal prediction is an …
potential hallucinations and making systems more reliable. Conformal prediction is an …
Prudent Silence or Foolish Babble? Examining Large Language Models' Responses to the Unknown
Large Language Models (LLMs) often struggle when faced with situations where they lack
the prerequisite knowledge to generate a sensical response. In these cases, models tend to …
the prerequisite knowledge to generate a sensical response. In these cases, models tend to …
Calibrating Large Language Models Using Their Generations Only
As large language models (LLMs) are increasingly deployed in user-facing applications,
building trust and maintaining safety by accurately quantifying a model's confidence in its …
building trust and maintaining safety by accurately quantifying a model's confidence in its …
Introspective planning: Guiding language-enabled agents to refine their own uncertainty
Large language models (LLMs) exhibit advanced reasoning skills, enabling robots to
comprehend natural language instructions and strategically plan high-level actions through …
comprehend natural language instructions and strategically plan high-level actions through …