Can Small LLMs (SLMs) effectively handle automated ticket routing in large IT organizations?

All answers to this question.

Absolutely. We found that Small LLMs (SLMs) are actually better for intent classification because you can fine-tune them on your specific company jargon. Large models often over-analyze simple requests, leading to "hallucinations" or over-categorization. An SLM like Mistral 7B, when trained on your historical ticket data, will recognize internal project codes and server names much more reliably. Plus, the cost difference is night and day; you’re looking at pennies compared to hundreds of dollars when processing thousands of short-form messages every single day.

   Answered 2025-05-24 by Danielle O'Connor

Are you worried about the model's ability to handle "edge cases" where a user describes a problem using very vague or non-technical language?

   Answered 2025-05-25 by Bradley Vance

• That is a valid concern, Bradley. However, we address that by setting a confidence threshold. If the SLM is less than 85% sure of the category, it routes the ticket to a "General Review" queue for a human to check. This hybrid approach ensures that we get the speed of automation for 90% of tickets while maintaining a high safety net for the weirder, more complex issues.

     Commented 2025-05-26 by Sean Maxwell

Fine-tuned SLMs consistently show higher F1 scores for classification tasks than zero-shot large models. It's about precision over breadth.

   Answered 2025-05-27 by Gregory Hudson

• Spot on, Gregory. In our tests, the SLM hit 94% accuracy for routing, while the larger general model was stuck at 88% due to being too "creative."

     Commented 2025-05-28 by Vanessa Jenkins