How to implement Parameter-Efficient Fine-Tuning (PEFT) with Hugging Face Transformers?

All answers to this question.

LoRA is the way to go for large models. You need to install the peft library first. Then, define a LoraConfig specifying your rank (r) and target modules (like 'q_proj' and 'v_proj'). Use the get_peft_model(model, config) function to wrap your base model. Now, when you call trainer.train(), only the small LoRA adapters will be updated, while the base weights remain frozen. This reduces the number of trainable parameters by over 99%. I managed to fine-tune a Falcon model using this method on a single A100, which would have been impossible with a full fine-tune.

   Answered 2025-09-19 by Betty Nelson

Can I merge the LoRA weights back into the base model after training to avoid the extra latency during inference?

   Answered 2025-09-20 by Paul Rodriguez

• Yes, Paul! You can use model.merge_and_unload(). This mathematically merges the adapter weights into the main layers. After this, you can save the model as a standard Transformer model and use it with zero added latency during inference. It’s an incredibly clean way to deploy specialized models.

     Commented 2025-09-21 by Donna King

It’s amazing how we can now get near-full-fine-tune performance with such a tiny fraction of the compute.

   Answered 2025-09-22 by Brian Green

• Absolutely, Brian. PEFT has truly democratized the ability to work with massive LLMs for smaller companies.

     Commented 2025-09-23 by Betty Nelson