With the urgent demand for generalized deep models, many pre-trained big models are
proposed, such as bidirectional encoder representations (BERT), vision transformer (ViT) …

Transformer-based pretrained language models (T-PTLMs) have achieved great success in
almost every NLP task. The evolution of these models started with GPT and BERT. These …

Recently, the emergence of pre-trained models (PTMs) has brought natural language
processing (NLP) to a new era. In this survey, we provide a comprehensive review of PTMs …

Decoder transformers have continued increasing in scale reaching hundreds of billions of
parameters. Due to their scale the same decoder sets state-of-the-art results on various …

Artificial intelligence for graphs has achieved remarkable success in modelling complex
systems, ranging from dynamic networks in biology to interacting particle systems in physics …

Pre-trained language models (PLMs) have been the de facto paradigm for most natural
language processing tasks. This also benefits the biomedical domain: researchers from …

Recent studies report that autoregressive language models can successfully solve many
NLP tasks via zero-and few-shot learning paradigms, which opens up new possibilities for …

Transformer-based pretrained language models (PLMs) have started a new era in modern
natural language processing (NLP). These models combine the power of transformers …

Abstract Representations from transformer-based unidirectional language models are
known to be effective at predicting brain responses to natural language. However, most …

Bidirectional Encoder Representations from Transformers (BERT) and BERT-based
approaches are the current state-of-the-art in many natural language processing (NLP) …