Abstract Low-Rank Adaptation (LoRA), which updates the dense neural network layers with
pluggable low-rank matrices, is one of the best performed parameter efficient fine-tuning …

Large language models (LLMs) have demonstrated great capabilities in various natural
language understanding and generation tasks. These pre-trained LLMs can be further …

Foundation models (FMs) adapt well to specific domains or tasks with fine-tuning, and
federated learning (FL) enables the potential for privacy-preserving fine-tuning of the FMs …

The immense evolution in Large Language Models (LLMs) has underscored the importance
of massive, heterogeneous, and high-quality data. A data recipe is a mixture of data from …

Pre-trained large language models (LLMs) need fine-tuning to improve their responsiveness
to natural language instructions. Federated learning offers a way to fine-tune LLMs using the …

Large language models (LLMs) have significantly advanced artificial intelligence (AI) by
excelling in tasks such as understanding, generation, and reasoning across multiple …

The confluence of Federated Learning (FL) and Large Language Models (LLMs) is ushering
in a new era in privacy-preserving natural language processing. However, the intensive …

Low-rank adaptation (LoRA) is a natural method for finetuning in communication-
constrained machine learning settings such as cross-device federated learning. Prior work …

Large language models (LLMs) have emerged as important components across various
fields, yet their training requires substantial computation resources and abundant labeled …

The rapid development of large language models (LLMs) has been witnessed in recent
years. Based on the powerful LLMs, multi-modal LLMs (MLLMs) extend the modality from …