Large Language Models (LLMs) have recently demonstrated remarkable capabilities in
natural language processing tasks and beyond. This success of LLMs has led to a large …

This paper embarks on an exploration into the Large Language Model (LLM) datasets,
which play a crucial role in the remarkable advancements of LLMs. The datasets serve as …

Prompts for pre-trained language models (PLMs) have shown remarkable performance by
bridging the gap between pre-training tasks and various downstream tasks. Among these …

Pre-trained models have achieved state-of-the-art results in various Natural Language
Processing (NLP) tasks. Recent works such as T5 and GPT-3 have shown that scaling up …

Deep learning has become the dominant approach in addressing various tasks in Natural
Language Processing (NLP). Although text inputs are typically represented as a sequence …

Bidirectional Encoder Representations from Transformers (BERT) has shown marvelous
improvements across various NLP tasks, and consecutive variants have been proposed to …

Pre-trained language representation models, such as BERT, capture a general language
representation from large-scale corpora, but lack domain-specific knowledge. When reading …

Bidirectional Encoder Representations from Transformers (BERT) has shown marvelous
improvements across various NLP tasks, and its consecutive variants have been proposed …

Recently pre-trained models have achieved state-of-the-art results in various language
understanding tasks. Current pre-training procedures usually focus on training the model …

We present a novel language representation model enhanced by knowledge called ERNIE
(Enhanced Representation through kNowledge IntEgration). Inspired by the masking …