Should we build our own SLMs or stick with giant LLMs like GPT-4 for niche tasks?

All answers to this question.

Have you tested whether the "context window" of the SLMs you are considering is large enough to handle a 50-page legal contract without losing track of the early clauses?

   Answered 2025-05-24 by Robert Miller

• Robert, that is exactly where we struggled with earlier 7B models. However, some of the newer 2025 "Long-Context" SLMs are now supporting up to 128k tokens, which is plenty for most legal documents. Our main focus now is whether the "reasoning density" of a smaller model can catch subtle contradictions between clauses that a human lawyer might miss. We are currently running a benchmark test comparing a fine-tuned Llama 3 variant against GPT-4o to see if the cost savings justify the potential risk of missed edge cases.

     Commented 2025-09-30 by James Anderson

For a domain as high-stakes as legal review, SLMs are becoming very attractive in 2025 because you can run them entirely "on-prem." This solves your data privacy and client confidentiality issues instantly. While a 7B model lacks the "broad trivia" of GPT-4, it can actually outperform a giant model on a narrow task if it is fine-tuned on 100,000 legal contracts. You also save a fortune on token costs. A common architecture we see now is "Router-based AI," where a cheap SLM handles 80% of the routine tasks, and only the most complex 20% of legal queries get escalated to a more expensive, high-reasoning LLM.

   Answered 2025-09-18 by Nancy Evans

Privacy is the deciding factor. If your clients don't want their data touching a third-party server, you have to go with a locally hosted SLM, regardless of the cost.

   Answered 2025-10-05 by Laura Bennett

• Laura is right. In legal and finance, the "Data Sovereignty" argument usually wins out over pure performance metrics, making SLMs the default choice for most of our projects this year.

     Commented 2025-10-08 by Brian Foster