On-device large language models (LLMs), referring to running LLMs on edge devices, have
raised considerable interest since they are more cost-effective, latency-efficient, and privacy …

With the continuous growth in the number of parameters of transformer-based pretrained
language models (PLMs), particularly the emergence of large language models (LLMs) with …

To parameter-efficiently fine-tune (PEFT) large language models (LLMs), the low-rank
adaptation (LoRA) method approximates the model changes $\Delta W\in\mathbb …

Parameter-efficient finetuning (PEFT) methods seek to adapt large neural models via
updates to a small number of weights. However, much prior interpretability work has shown …

Abstract Low-Rank Adaptation (LoRA), which updates the dense neural network layers with
pluggable low-rank matrices, is one of the best performed parameter efficient fine-tuning …

Parameter-efficient fine-tuning (PEFT) is a popular method for tailoring pre-trained large
language models (LLMs), especially as the models' scale and the diversity of tasks increase …

Large foundation models are becoming ubiquitous, but training them from scratch is
prohibitively expensive. Thus, efficiently adapting these powerful models to downstream …

Pre-trained models have become the preferred backbone due to the expansion of model
parameters, with techniques like Parameter-Efficient Fine-Tuning (PEFTs) typically fixing the …

The large models, as predicted by scaling raw forecasts, have made groundbreaking
progress in many fields, particularly in natural language generation tasks, where they have …

Large pre-trained models (LPMs) have demonstrated exceptional performance in diverse
natural language processing and computer vision tasks. However, fully fine-tuning these …