Parameter-efficient fine-tuning for large models: A comprehensive survey

Z Han, C Gao, J Liu, J Zhang, SQ Zhang - arxiv preprint arxiv:2403.14608, 2024 - arxiv.org
Large models represent a groundbreaking advancement in multiple application fields,
enabling remarkable achievements across various tasks. However, their unprecedented …

Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment

L Xu, H **e, SZJ Qin, X Tao, FL Wang - arxiv preprint arxiv:2312.12148, 2023 - arxiv.org
With the continuous growth in the number of parameters of transformer-based pretrained
language models (PLMs), particularly the emergence of large language models (LLMs) with …

Dynamic adapter meets prompt tuning: Parameter-efficient transfer learning for point cloud analysis

X Zhou, D Liang, W Xu, X Zhu, Y Xu… - Proceedings of the …, 2024 - openaccess.thecvf.com
Point cloud analysis has achieved outstanding performance by transferring point cloud pre-
trained models. However existing methods for model adaptation usually update all model …

Owl: A large language model for it operations

H Guo, J Yang, J Liu, L Yang, L Chai, J Bai… - arxiv preprint arxiv …, 2023 - arxiv.org
With the rapid development of IT operations, it has become increasingly crucial to efficiently
manage and analyze large volumes of data for practical applications. The techniques of …

Continual prompt tuning for dialog state tracking

Q Zhu, B Li, F Mi, X Zhu, M Huang - arxiv preprint arxiv:2203.06654, 2022 - arxiv.org
A desirable dialog system should be able to continually learn new skills without forgetting
old ones, and thereby adapt to new domains or tasks in its life cycle. However, continually …

Exploring efficient-tuning methods in self-supervised speech models

ZC Chen, CL Fu, CY Liu, SWD Li… - 2022 IEEE spoken …, 2023 - ieeexplore.ieee.org
In this study, we aim to explore efficient tuning methods for speech self-supervised learning.
Recent studies show that self-supervised learning (SSL) can learn powerful representations …

Survey of different large language model architectures: Trends, benchmarks, and challenges

M Shao, A Basit, R Karri, M Shafique - IEEE Access, 2024 - ieeexplore.ieee.org
Large Language Models (LLMs) represent a class of deep learning models adept at
understanding natural language and generating coherent responses to various prompts or …

Group-aware parameter-efficient updating for content-adaptive neural video compression

Z Chen, L Zhou, Z Hu, D Xu - Proceedings of the 32nd ACM International …, 2024 - dl.acm.org
Content-adaptive compression is crucial for enhancing the adaptability of the pre-trained
neural codec for various contents. Though, its application in neural video compression …

Dora: Enhancing parameter-efficient fine-tuning with dynamic rank distribution

Y Mao, K Huang, C Guan, G Bao, F Mo, J Xu - arxiv preprint arxiv …, 2024 - arxiv.org
Fine-tuning large-scale pre-trained models is inherently a resource-intensive task. While it
can enhance the capabilities of the model, it also incurs substantial computational costs …

Causes and cures for interference in multilingual translation

U Shaham, M Elbayad, V Goswami, O Levy… - arxiv preprint arxiv …, 2022 - arxiv.org
Multilingual machine translation models can benefit from synergy between different
language pairs, but also suffer from interference. While there is a growing number of …