ChatGPT has drawn considerable attention from both the general public and domain experts
with its remarkable text generation capabilities. This has subsequently led to the emergence …

Pre-trained Language Models (PLMs) which are trained on large text corpus via self-
supervised learning method, have yielded promising performance on various tasks in …

Pretraining large neural language models, such as BERT, has led to impressive gains on
many natural language processing (NLP) tasks. However, most pretraining efforts focus on …

Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging
and expensive. We release SciBERT, a pretrained language model based on BERT (Devlin …

Motivation Biomedical text mining is becoming increasingly important as the number of
biomedical documents rapidly grows. With the progress in natural language processing …

We introduce S2ORC, a large corpus of 81.1 M English-language academic papers
spanning many academic disciplines. The corpus consists of rich metadata, paper abstracts …

Named Entity Recognition (NER) is a key component in NLP systems for question
answering, information retrieval, relation extraction, etc. NER systems have been studied …

Despite recent advances in natural language processing, many statistical models for
processing text perform extremely poorly under domain shift. Processing biomedical and …