Opportunities and obstacles for deep learning in biology and medicine

T Ching, DS Himmelstein… - Journal of the …, 2018‏ - royalsocietypublishing.org
Deep learning describes a class of machine learning algorithms that are capable of
combining raw inputs into layers of intermediate features. These algorithms have recently …

Vision-language models for medical report generation and visual question answering: A review

I Hartsock, G Rasool - Frontiers in Artificial Intelligence, 2024‏ - frontiersin.org
Medical vision-language models (VLMs) combine computer vision (CV) and natural
language processing (NLP) to analyze visual and textual medical data. Our paper reviews …

Medclip: Contrastive learning from unpaired medical images and text

Z Wang, Z Wu, D Agarwal, J Sun - Proceedings of the …, 2022‏ - pmc.ncbi.nlm.nih.gov
Existing vision-text contrastive learning like CLIP (Radford et al., 2021) aims to match the
paired image and caption embeddings while pushing others apart, which improves …

A medical multimodal large language model for future pandemics

F Liu, T Zhu, X Wu, B Yang, C You, C Wang, L Lu… - NPJ Digital …, 2023‏ - nature.com
Deep neural networks have been integrated into the whole clinical decision procedure
which can improve the efficiency of diagnosis and alleviate the heavy workload of …

Medklip: Medical knowledge enhanced language-image pre-training for x-ray diagnosis

C Wu, X Zhang, Y Zhang, Y Wang… - Proceedings of the …, 2023‏ - openaccess.thecvf.com
In this paper, we consider enhancing medical visual-language pre-training (VLP) with
domain-specific knowledge, by exploiting the paired image-text reports from the radiological …

Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus… - Proceedings of the AAAI …, 2019‏ - aaai.org
Large, labeled datasets have driven deep learning methods to achieve expert-level
performance on a variety of medical imaging tasks. We present CheXpert, a large dataset …

MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs

AEW Johnson, TJ Pollard, NR Greenbaum… - arxiv preprint arxiv …, 2019‏ - arxiv.org
Chest radiography is an extremely powerful imaging modality, allowing for a detailed
inspection of a patient's thorax, but requiring specialized training for proper interpretation …

Radgraph: Extracting clinical entities and relations from radiology reports

S Jain, A Agrawal, A Saporta, SQH Truong… - arxiv preprint arxiv …, 2021‏ - arxiv.org
Extracting structured clinical information from free-text radiology reports can enable the use
of radiology report information for a variety of critical healthcare applications. In our work, we …

Feasibility of using the privacy-preserving large language model Vicuna for labeling radiology reports

P Mukherjee, B Hou, RB Lanfredi, RM Summers - Radiology, 2023‏ - pubs.rsna.org
Background Large language models (LLMs) such as ChatGPT, though proficient in many
text-based tasks, are not suitable for use with radiology reports due to patient privacy …

Padchest: A large chest x-ray image dataset with multi-label annotated reports

A Bustos, A Pertusa, JM Salinas… - Medical image …, 2020‏ - Elsevier
We present a labeled large-scale, high resolution chest x-ray dataset for the automated
exploration of medical images along with their associated reports. This dataset includes …