Can we use Small LLMs (SLMs) to generate unit tests in a CI/CD pipeline efficiently?

All answers to this question.

I’ve implemented this using a local instance of StarCoder2-3B, which is a specialized code-focused version of Small LLMs (SLMs). It’s definitely fast enough for a pre-commit hook if you keep the scope limited to the changed files. On a modern MacBook or workstation, it generates a suite of 5-10 tests in under three seconds. The key is to use a model that has been pre-trained specifically on code repositories. It understands boilerplate and edge cases for Python and JavaScript much better than a general-purpose model would, and it doesn't require a constant internet connection.

   Answered 2025-10-05 by Valerie Hudson

Do you find that the suggested tests are actually useful, or do developers end up spending more time fixing the AI's "hallucinated" test cases?

   Answered 2025-10-06 by Patrick Sullivan

• It’s about 70/30, Patrick. About 70% of the tests are solid "happy path" cases that save time. The other 30% might need a little tweak to the assertions. But even then, it's easier to edit a suggested test than to write one from scratch. We also include the model's "thinking" process in a comment, which helps the developer understand why the test was suggested in the first place, making the review process much faster.

     Commented 2025-10-07 by Craig Whitaker

Using an SLM for this is great because it won't break your budget like running GPT-4 on every single "git commit" would.

   Answered 2025-10-08 by Monica Weaver

• Exactly, Monica. Cost is a huge factor when you have a team of 50 developers committing code dozens of times a day

     Commented 2025-10-09 by Douglas Graham