How do you handle fine-tuning for small models in an Agile sprint?

All answers to this question.

It's absolutely doable, but you have to break it down. In our 2023 sprints, we treated data collection, fine-tuning, and evaluation as separate stories. We used PEFT (Parameter-Efficient Fine-Tuning) techniques which allowed us to train a model on a single A100 in just a few hours. By Thursday of the first week, we’d have a candidate model. The second week was dedicated to "Red Teaming" and A/B testing. The key is automation; if your training pipeline is scripted, the actual "training" becomes just another background task in your CI/CD pipeline.

   Answered 2025-09-21 by Megan Foster

Megan, what kind of "Definition of Done" did you use for a fine-tuning story? Was it based on a specific accuracy metric?

   Answered 2025-09-23 by Andrew Morris

• Andrew, we used a benchmark suite. A model was only "Done" if it beat the previous version on our top 5 KPIs without regressing on general safety checks.

     Commented 2025-09-24 by Larry Peterson

We found that keeping a "Golden Dataset" ready made fine-tuning much faster. We just add new edge cases to the set each sprint and hit "train."

   Answered 2025-09-25 by Theresa Knight

• That’s the secret, Theresa. Continuous data curation makes the actual training part a non-event. It just becomes part of the regular dev workflow.

     Commented 2025-09-26 by Megan Foster