Pre-trained Language Models (PLMs) which are trained on large text corpus via self-
supervised learning method, have yielded promising performance on various tasks in …

Large Language Models (LLMs) have shown extraordinary capabilities in understanding
and generating text that closely mirrors human communication. However, a primary …

Recent work has shown exciting promise in updating large language models with new
memories, so as to replace obsolete information or add specialized knowledge. However …

Despite the ability to train capable LLMs, the methodology for maintaining their relevancy
and rectifying errors remains elusive. To this end, the past few years have witnessed a surge …

Modern language models capture a large body of factual knowledge. However, some facts
can be incorrectly induced or become obsolete over time, resulting in factually incorrect …

Transformer-based language models (LMs) are at the core of modern NLP, but their internal
prediction construction process is opaque and largely not understood. In this work, we make …

Previous literature has proved that Pretrained Language Models (PLMs) can store factual
knowledge. However, we find that facts stored in the PLMs are not always correct. It …

Self-supervised protein language models have proved their effectiveness in learning the
proteins representations. With the increasing computational power, current protein language …

The Transformer architecture consists of self-attention and feed-forward networks (FFNs)
which can be viewed as key-value memories according to previous works. However, FFN …

Most of today's AI systems focus on using self-attention mechanisms and transformer
architectures on large amounts of diverse data to achieve impressive performance gains. In …