The open-ended nature of language generation makes the evaluation of autoregressive
large language models (LLMs) challenging. One common evaluation approach uses …

Visual Question Answering (VQA) is an important task in multimodal AI, and it is often used
to test the ability of vision-language models to understand and reason on knowledge …

Large language models (LLMs) appear to bias their survey answers toward certain values.
Nonetheless, some argue that LLMs are too inconsistent to simulate particular values. Are …

Recent research shows that pre-trained language models (PLMs) suffer from" prompt bias"
in factual knowledge extraction, ie, prompts tend to introduce biases toward specific labels …

Predictable behavior from scaling advanced AI systems is an extremely desirable property.
Although a well-established literature exists on how pretraining performance scales, the …

Multiple choice questions (MCQs) are commonly used to evaluate the capabilities of large
language models (LLMs). One common way to evaluate the model response is to rank the …

Vision and language model (VLM) decoders are currently the best-performing architectures
on multimodal tasks. Next to answers, they are able to produce natural language …

The widespread adoption of Large Language Models (LLMs) has become commonplace,
particularly with the emergence of open-source models. More importantly, smaller models …

Recent advancements in machine translation (MT) have significantly enhanced translation
quality across various domains. However, the translation of literary texts remains a …

Language models (LMs) are increasingly used as simulacra for people, yet their ability to
match the distribution of views of a specific demographic group and be\textit {distributionally …