Pre-trained language models (PLMs) have been the de facto paradigm for most natural
language processing tasks. This also benefits the biomedical domain: researchers from …

Electronic health records (EHRs), digital collections of patient healthcare events and
observations, are ubiquitous in medicine and critical to healthcare delivery, operations, and …

Contextual word embedding models such as ELMo (Peters et al., 2018) and BERT (Devlin et
al., 2018) have dramatically improved performance for many natural language processing …

Motivation Biomedical text mining is becoming increasingly important as the number of
biomedical documents rapidly grows. With the progress in natural language processing …

A large array of pretrained models are available to the biomedical NLP (BioNLP) community.
Finding the best model for a particular task can be difficult and time-consuming. For many …

Objective Neural network–based representations (“embeddings”) have dramatically
advanced natural language processing (NLP) tasks, including clinical NLP tasks such as …

In this report, we introduce SciFive, a domain-specific T5 model that has been pre-trained on
large biomedical corpora. Our model outperforms the current SOTA methods (ie BERT …

Objective This article methodically reviews the literature on deep learning (DL) for natural
language processing (NLP) in the clinical domain, providing quantitative analysis to answer …

Representing words as numerical vectors based on the contexts in which they appear has
become the de facto method of analyzing text with machine learning. In this paper, we …

Contextual word embedding models, such as BioBERT and Bio_ClinicalBERT, have
achieved state-of-the-art results in biomedical natural language processing tasks by …