Recently, the emergence of pre-trained models (PTMs) has brought natural language
processing (NLP) to a new era. In this survey, we provide a comprehensive review of PTMs …

Named entity recognition (NER) is the task to identify mentions of rigid designators from text
belonging to predefined semantic types such as person, location, organization etc. NER …

For most deep learning practitioners, sequence modeling is synonymous with recurrent
networks. Yet recent results indicate that convolutional architectures can outperform …

The learning rate warmup heuristic achieves remarkable success in stabilizing training,
accelerating convergence and improving generalization for adaptive stochastic optimization …

Overparameterized transformer networks have obtained state of the art results in various
natural language processing tasks, such as machine translation, language modeling, and …

A robust evaluation metric has a profound impact on the development of text generation
systems. A desirable metric compares system output against references based on their …

Contextual word representations derived from pre-trained bidirectional language models
(biLMs) have recently been shown to provide significant improvements to the state of the art …

Large language models have recently achieved state of the art performance across a wide
variety of natural language tasks. Meanwhile, the size of these models and their latency …

An enormous amount of digital information is expressed as natural-language (NL) text that is
not easily processable by computers. Knowledge Graphs (KG) offer a widely used format for …

Learning human languages is a difficult task for a computer. However, Deep Learning (DL)
techniques have enhanced performance significantly for almost all-natural language …