Assessment and evaluation have long been critical challenges in artificial intelligence (AI)
and natural language processing (NLP). However, traditional methods, whether matching …

Large language models (LLMs) often generate content that contains factual errors when
responding to fact-seeking prompts on open-ended topics. To benchmark a model's long …

LLM-as-a-Judge has been widely utilized as an evaluation method in various benchmarks
and served as supervised rewards in model training. However, despite their excellence in …

We present a principled approach to provide LLM-based evaluation with a rigorous
guarantee of human agreement. We first propose that a reliable evaluation method should …

The proliferation of Vision-Language Models (VLMs) in the past several years calls for
rigorous and comprehensive evaluation methods and benchmarks. This work analyzes …

Objective: The paper introduces a framework for the evaluation of the encoding of factual
scientific knowledge, designed to streamline the manual evaluation process typically …