Abstract Low-Rank Adaptation (LoRA), which updates the dense neural network layers with
pluggable low-rank matrices, is one of the best performed parameter efficient fine-tuning …

Foundation models (FMs) adapt surprisingly well to downstream tasks with fine-tuning.
However, their colossal parameter space prohibits their training on resource-constrained …

Large language models (LLMs) show amazing performance on many domain-specific tasks
after fine-tuning with some appropriate data. However, many domain-specific data are …

Large language models are rapidly gaining popularity and have been widely adopted in real-
world applications. While the quality of training data is essential, privacy concerns arise …

Despite their exceptional performance on various tasks after fine-tuning, pre-trained
language models (PLMs) face significant challenges due to growing privacy concerns with …

Abstract Vision Transformers (ViT) and Visual Prompt Tuning (VPT) achieve state-of-the-art
performance with improved efficiency in various computer vision tasks. This suggests a …

Pre-trained language models (PLM) have revolutionized the NLP landscape, achieving
stellar performances across diverse tasks. These models, while benefiting from vast training …

Existing federated learning methods have effectively dealt with decentralized learning in
scenarios involving data privacy and non-IID data. However, in real-world situations, each …

Recent research in federated large language models (LLMs) has primarily focused on
enabling clients to fine-tune their locally deployed homogeneous LLMs collaboratively or on …

Recently, federated learning (FL) as a new learning paradigm allows multi-party to
collaboratively train a shared global model with privacy protection. However, vanilla FL …