Federatedscope-llm: A comprehensive package for fine-tuning large language models in federated learning
Large language models (LLMs) have demonstrated great capabilities in various natural
language understanding and generation tasks. These pre-trained LLMs can be further …
language understanding and generation tasks. These pre-trained LLMs can be further …
Data-juicer: A one-stop data processing system for large language models
The immense evolution in Large Language Models (LLMs) has underscored the importance
of massive, heterogeneous, and high-quality data. A data recipe is a mixture of data from …
of massive, heterogeneous, and high-quality data. A data recipe is a mixture of data from …
Decentralized federated learning: A survey and perspective
Federated learning (FL) has been gaining attention for its ability to share knowledge while
maintaining user data, protecting privacy, increasing learning efficiency, and reducing …
maintaining user data, protecting privacy, increasing learning efficiency, and reducing …
Federated large language models: Current progress and future directions
Large language models are rapidly gaining popularity and have been widely adopted in real-
world applications. While the quality of training data is essential, privacy concerns arise …
world applications. While the quality of training data is essential, privacy concerns arise …
A survey of resource-efficient llm and multimodal foundation models
Large foundation models, including large language models (LLMs), vision transformers
(ViTs), diffusion, and LLM-based multimodal models, are revolutionizing the entire machine …
(ViTs), diffusion, and LLM-based multimodal models, are revolutionizing the entire machine …
On the convergence of zeroth-order federated tuning for large language models
The confluence of Federated Learning (FL) and Large Language Models (LLMs) is ushering
in a new era in privacy-preserving natural language processing. However, the intensive …
in a new era in privacy-preserving natural language processing. However, the intensive …
Towards Federated Large Language Models: Motivations, Methods, and Future Directions
Large Language Models (LLMs), such as LLaMA and GPT-4, have transformed the
paradigm of natural language comprehension and generation. Despite their impressive …
paradigm of natural language comprehension and generation. Despite their impressive …
Fwdllm: Efficient federated finetuning of large language models with perturbed inferences
Large Language Models (LLMs) are transforming the landscape of mobile intelligence.
Federated Learning (FL), a method to preserve user data privacy, is often employed in fine …
Federated Learning (FL), a method to preserve user data privacy, is often employed in fine …
Federated fine-tuning of large language models under heterogeneous language tasks and client resources
Federated Learning (FL) has recently been applied to the parameter-efficient fine-tuning of
Large Language Models (LLMs). While promising, it raises significant challenges due to the …
Large Language Models (LLMs). While promising, it raises significant challenges due to the …
On the limitations of compute thresholds as a governance strategy
S Hooker - arxiv preprint arxiv:2407.05694, 2024 - arxiv.org
At face value, this essay is about understanding a fairly esoteric governance tool called
compute thresholds. However, in order to grapple with whether these thresholds will achieve …
compute thresholds. However, in order to grapple with whether these thresholds will achieve …