Summary of a haystack: A challenge to long-context llms and rag systems

P Laban, AR Fabbri, C **ong, CS Wu - arxiv preprint arxiv:2407.01370, 2024 - arxiv.org
LLMs and RAG systems are now capable of handling millions of input tokens or more.
However, evaluating the output quality of such systems on long-context tasks remains …

Debate on graph: a flexible and reliable reasoning framework for large language models

J Ma, Z Gao, Q Chai, W Sun, P Wang, H Pei… - arxiv preprint arxiv …, 2024 - arxiv.org
Large Language Models (LLMs) may suffer from hallucinations in real-world applications
due to the lack of relevant knowledge. In contrast, knowledge graphs encompass extensive …

Logical Consistency of Large Language Models in Fact-checking

B Ghosh, S Hasan, NA Arafat, A Khan - arxiv preprint arxiv:2412.16100, 2024 - arxiv.org
In recent years, large language models (LLMs) have demonstrated significant success in
performing varied natural language tasks such as language translation, question-answering …

An evaluation framework for attributed information retrieval using large language models

H Djeddal, P Erbacher, R Toukal, L Soulier… - Proceedings of the 33rd …, 2024 - dl.acm.org
With the growing success of Large Language models (LLMs) in information-seeking
scenarios, search engines are now adopting generative approaches to provide answers …

Evaluation of Attribution Bias in Retrieval-Augmented Large Language Models

A Abolghasemi, L Azzopardi, SH Hashemi… - arxiv preprint arxiv …, 2024 - arxiv.org
Attributing answers to source documents is an approach used to enhance the verifiability of
a model's output in retrieval augmented generation (RAG). Prior work has mainly focused on …

Attributed Question Answering for Preconditions in the Dutch Law

F Redelaar, R Van Drie, S Verberne… - Proceedings of the …, 2024 - aclanthology.org
In this paper, we address the problem of answering questions about preconditions in the
law, eg “When can the court terminate the guardianship of a natural person?”. When …

Enhancing rag-retrieval to improve llms robustness and resilience to hallucinations

C Njeh, H Nakouri, F Jaafar - International Conference on Hybrid Artificial …, 2024 - Springer
Abstract The use of Retrieval-Augmented Generation (RAG) models has become a powerful
method for generating natural language by incorporating information retrieval. However, the …

Counter-intuitive: Large Language Models Can Better Understand Knowledge Graphs Than We Thought

X Dai, Y Hua, T Wu, Y Sheng, G Qi - arxiv preprint arxiv:2402.11541, 2024 - arxiv.org
Although the method of enhancing large language models'(LLMs') reasoning ability and
reducing their hallucinations through the use of knowledge graphs (KGs) has received …

Post-Hoc Answer Attribution for Grounded and Trustworthy Long Document Comprehension: Task, Insights, and Challenges

A Sancheti, K Goswami, BV Srinivasan - arxiv preprint arxiv:2406.06938, 2024 - arxiv.org
Attributing answer text to its source document for information-seeking questions is crucial for
building trustworthy, reliable, and accountable systems. We formulate a new task of post-hoc …

Inner-Probe: Discovering Copyright-related Data Generation in LLM Architecture

Q Ma, RJ Zhu, P Liu, R Yan, F Zhang, L Liang… - arxiv preprint arxiv …, 2024 - arxiv.org
Large Language Models (LLMs) utilize extensive knowledge databases and show powerful
text generation ability. However, their reliance on high-quality copyrighted datasets raises …