Does fine-tuning LLMs on new knowledge encourage hallucinations?

Z Gekhman, G Yona, R Aharoni, M Eyal… - arxiv preprint arxiv …, 2024 - arxiv.org
When large language models are aligned via supervised fine-tuning, they may encounter
new factual information that was not acquired through pre-training. It is often conjectured that …

Monotonic paraphrasing improves generalization of language model prompting

Q Liu, F Wang, N Xu, T Yan, T Meng… - arxiv preprint arxiv …, 2024 - arxiv.org
Performance of large language models (LLMs) may vary with different prompts or
instructions of even the same task. One commonly recognized factor for this phenomenon is …

Panda: Preference adaptation for enhancing domain-specific abilities of llms

A Liu, Z Yang, Z Zhang, Q Hu, P Li, M Yan… - arxiv preprint arxiv …, 2024 - arxiv.org
While Large language models (LLMs) have demonstrated considerable capabilities across
various natural language tasks, they often fall short of the performance achieved by domain …

Famicom: Further demystifying prompts for language models with task-agnostic performance estimation

B Li, B Zhou, X Fu, F Wang, D Roth, M Chen - arxiv preprint arxiv …, 2024 - arxiv.org
Language models have shown impressive in-context-learning capabilities, which allow them
to benefit from input prompts and perform better on downstream end tasks. Existing works …

Familiarity-aware evidence compression for retrieval augmented generation

D Jung, Q Liu, T Huang, B Zhou, M Chen - arxiv preprint arxiv:2409.12468, 2024 - arxiv.org
Retrieval Augmented Generation (RAG) improves large language models (LMs) by
incorporating non-parametric knowledge through evidence retrieval from external sources …

Gradual Learning: Optimizing Fine-Tuning with Partially Mastered Knowledge in Large Language Models

B Li, H Liang, Y Li, F Fu, H Yin, C He… - arxiv preprint arxiv …, 2024 - arxiv.org
During the pretraining phase, large language models (LLMs) acquire vast amounts of
knowledge from extensive text corpora. Nevertheless, in later stages such as fine-tuning and …

Delving into the Reversal Curse: How Far Can Large Language Models Generalize?

Z Lin, Z Fu, K Liu, L **e, B Lin, W Wang, D Cai… - arxiv preprint arxiv …, 2024 - arxiv.org
While large language models (LLMs) showcase unprecedented capabilities, they also
exhibit certain inherent limitations when facing seemingly trivial tasks. A prime example is …

[HTML][HTML] Adapting Generative Large Language Models for Information Extraction from Unstructured Electronic Health Records in Residential Aged Care: A …

D Vithanage, C Deng, L Wang, M Yin… - Journal of Healthcare …, 2025 - Springer
Abstract Information extraction (IE) of unstructured electronic health records is challenging
due to the semantic complexity of textual data. Generative large language models (LLMs) …

AmbigDocs: Reasoning across Documents on Different Entities under the Same Name

Y Lee, X Ye, E Choi - arxiv preprint arxiv:2404.12447, 2024 - arxiv.org
Different entities with the same name can be difficult to distinguish. Handling confusing entity
mentions is a crucial skill for language models (LMs). For example, given the question" …

[PDF][PDF] Evaluating machine learning approaches for multi-label classification of unstructured electronic health records with a generative large language model

D Vithanage, C Deng, L Wang, M Yin, M Alkhalaf… - 2024 - medrxiv.org
Multi-label classification of unstructured electronic health records (EHR) poses challenges
due to the inherent semantic complexity in textual data. Advances in natural language …