The burgeoning field of Large Language Models (LLMs), exemplified by sophisticated
models like OpenAI's ChatGPT, represents a significant advancement in artificial …

Despite achieving state-of-the-art performance on many NLP tasks, the high energy cost and
long inference delay prevent Transformer-based pretrained language models (PLMs) from …

This paper presents an overview of the Text2DT shared task 1 held in the CHIP-2022 shared
tasks. The shared task addresses the challenging topic of automatically extracting the …

Effectively scaling large Transformer models is a main driver of recent advances in natural
language processing. Dynamic neural networks, as an emerging research direction, are …

Recently, a series of works have looked into further improving the adapter-based tuning by
manually designing better adapter architectures. Understandably, these manually designed …

In this paper, we systematically evaluate the robustness of multi-exit language models
against adversarial slowdown. To audit their robustness, we design a slowdown attack that …

Parameter-efficient fine-tuning (PEFT) is widely studied for its effectiveness and efficiency in
the era of large language models. Low-rank adaptation (LoRA) has demonstrated …

Low-rank adaptation (LoRA) and its mixture-of-experts (MOE) variants are highly effective
parameter-efficient fine-tuning (PEFT) methods. However, they introduce significant latency …

Encoder-decoder transformer models have achieved great success on various vision-
language (VL) tasks, but they suffer from high inference latency. Typically, the decoder takes …

Computational complexity and overthinking problems have become the bottlenecks for pre-
training language models (PLMs) with millions or even trillions of parameters. A Flexible …