Benchmarking the diagnostic performance of open source LLMs in 1933 Eurorad case reports

SH Kim, S Schramm, LC Adams, R Braren… - npj Digital …, 2025 - nature.com
Recent advancements in large language models (LLMs) have created new ways to support
radiological diagnostics. While both open-source and proprietary LLMs can address privacy …

Data Extraction from Free-Text Stroke CT Reports Using GPT-4o and Llama-3.3-70B: The Impact of Annotation Guidelines

J Wihl, E Rosenkranz, S Schramm, C Berberich… - medRxiv, 2025 - medrxiv.org
Purpose To evaluate the performance of LLMs in extracting data from stroke CT reports in
the presence and absence of an annotation guideline. Methods In this study, performance of …

Boosting LLM-Assisted Diagnosis: 10-Minute LLM Tutorial Elevates Radiology Residents' Performance in Brain MRI Interpretation

SH Kim, S Schramm, J Wihl, P Raffler, M Tahedl… - medRxiv, 2024 - medrxiv.org
Purpose To evaluate the impact of a structured tutorial on the use of a large language model
(LLM)-based search engine on radiology residents' performance in LLM-assisted brain MRI …

Performance of Open-Source LLMs in Challenging Radiological Cases–A Benchmark Study on 4,049 Eurorad Case Reports

SH Kim, S Schramm, LC Adams, R Braren… - medRxiv, 2024 - medrxiv.org
Background Recent advancements in large language models (LLMs) have created new
ways to support radiological diagnostics. While both open-source and proprietary LLMs can …