Large Language Models (LLMs) have recently demonstrated remarkable capabilities in
natural language processing tasks and beyond. This success of LLMs has led to a large …

The size of vision models has grown exponentially over the last few years, especially after
the emergence of Vision Transformer. This has motivated the development of parameter …

Fine-tuning pre-trained models has been ubiquitously proven to be effective in a wide range
of NLP tasks. However, fine-tuning the whole model is parameter inefficient as it always …

Deep model fusion/merging is an emerging technique that merges the parameters or
predictions of multiple deep learning models into a single one. It combines the abilities of …

Learning with limited labelled data, such as prompting, in-context learning, fine-tuning, meta-
learning, or few-shot learning, aims to effectively train a model using only a small amount of …

Adapters, a plug-in neural network module with some tunable parameters, have emerged as
a parameter-efficient transfer learning technique for adapting pre-trained models to …

Pre-trained large language models (LLMs) need fine-tuning to improve their responsiveness
to natural language instructions. Federated learning offers a way to fine-tune LLMs using the …

Developers deal with code-change-related tasks daily, eg, reviewing code. Pre-trained code
and code-change-oriented models have been adapted to help developers with such tasks …

The high cost of full-parameter fine-tuning (FFT) of Large Language Models (LLMs) has led
to a series of parameter-efficient fine-tuning (PEFT) methods. However, it remains unclear …

Large pretrained language models are widely used in downstream NLP tasks via task-
specific fine-tuning, but such procedures can be costly. Recently, Parameter-Efficient Fine …