Pre-trained Language Models (PLMs) which are trained on large text corpus via self-
supervised learning method, have yielded promising performance on various tasks in …

The goal of text-to-text generation is to make machines express like a human in many
applications such as conversation, summarization, and translation. It is one of the most …

Large language models (LLMs), such as ChatGPT and GPT4, are making new waves in the
field of natural language processing and artificial intelligence, due to their emergent ability …

The new generation of state-of-the-art computer vision systems are trained from natural
language supervision, ranging from simple object category names to descriptive captions …

Abstract Knowledge graphs (KGs) contain rich information about world knowledge, entities,
and relations. Thus, they can be great supplements to existing pre-trained language models …

Multi-task learning (MTL) has become increasingly popular in natural language processing
(NLP) because it improves the performance of related tasks by exploiting their …

Image-text contrastive learning models such as CLIP have demonstrated strong task transfer
ability. The high generality and usability of these visual models is achieved via a web-scale …

Prompt-based learning, with its capability to tackle zero-shot and few-shot NLP tasks, has
gained much attention in community. The main idea is to bridge the gap between NLP …

Knowledge in NLP has been a rising trend especially after the advent of large-scale pre-
trained models. Knowledge is critical to equip statistics-based models with common sense …

Generative commonsense reasoning (GCR) in natural language is to reason about the
commonsense while generating coherent text. Recent years have seen a surge of interest in …