Abstract Low-Rank Adaptation (LoRA), which updates the dense neural network layers with
pluggable low-rank matrices, is one of the best performed parameter efficient fine-tuning …

Model merging offers an effective strategy to combine the strengths of multiple finetuned
models into a unified model that preserves the specialized capabilities of each. Existing …

Fine-tuning large language models (LLMs) with low-rank adaptations (LoRAs) has become
common practice, often yielding numerous copies of the same LLM differing only in their …

Lately, the practice of utilizing task-specific fine-tuning has been implemented to improve the
performance of large language models (LLM) in subsequent tasks. Through the integration …

Low-Rank Adaptation (LoRA) is a computationally efficient approach for fine-tuning large
pre-trained language models, designed to reduce memory and computational overhead by …

Meta-Learning aims at extracting shared knowledge (meta-knowledge) from historical tasks
to accelerate learning on new tasks. It has achieved promising performance in various …