Large models represent a groundbreaking advancement in multiple application fields,
enabling remarkable achievements across various tasks. However, their unprecedented …

With the continuous growth in the number of parameters of transformer-based pretrained
language models (PLMs), particularly the emergence of large language models (LLMs) with …

With the increasing size of pre-trained language models (PLMs), fine-tuning all the
parameters in the model is not efficient, especially when there are a large number of …

Prompt-based learning has been an effective paradigm for large pretrained language
models (LLM), enabling few-shot or even zero-shot learning. Black-box prompt search has …

Fine-tuning large pre-trained foundation models, such as the 175B GPT-3, has attracted
more attention for downstream tasks recently. While parameter-efficient fine-tuning methods …

The rapid advancements in Large Language Models (LLMs) have revolutionized natural
language processing (NLP) and related fields. However, fine-tuning these models for …

We propose RoCoFT, a parameter-efficient fine-tuning method for large-scale language
models (LMs) based on updating only a few rows and columns of the weight matrices in …

While prompt tuning approaches have achieved competitive performance with high
efficiency, we observe that they invariably employ the same initialization process, wherein …

Parameter-efficient fine-tuning (PEFT) has shown its effectiveness in adapting the pre-
trained language models to downstream tasks while only updating a small number of …

For long document summarization, discourse structure is important to discern the key
content of the text and the differences in importance level between sentences. Unfortunately …