Transformer-based pretrained language models (T-PTLMs) have achieved great success in
almost every NLP task. The evolution of these models started with GPT and BERT. These …

Transfer learning is a technique utilized in deep learning applications to transmit learned
inference to a different target domain. The approach is mainly to solve the problem of a few …

Large Language Models (LLMs) have demonstrated remarkable capabilities in important
tasks such as natural language understanding and language generation, and thus have the …

Prompt tuning (PT) is a promising parameter-efficient method to utilize extremely large pre-
trained language models (PLMs), which can achieve comparable performance to full …

In recent years, the size of pre-trained language models (PLMs) has grown by leaps and
bounds. However, efficiency issues of these large-scale PLMs limit their utilization in real …

Large foundation models, including large language models (LLMs), vision transformers
(ViTs), diffusion, and LLM-based multimodal models, are revolutionizing the entire machine …

In recent years, researchers tend to pre-train ever-larger language models to explore the
upper limit of deep models. However, large language model pre-training costs intensive …

Current pre-trained language models (PLM) are typically trained with static data, ignoring
that in real-world scenarios, streaming data of various sources may continuously grow. This …

Scaling transformers has led to significant breakthroughs in many domains, leading to a
paradigm in which larger versions of existing models are trained and released on a periodic …

Multilingual large-scale Pretrained Language Models (PLMs) have been shown to store
considerable amounts of factual knowledge, but large variations are observed across …