Is it better to use Small LLMs (SLMs) or traditional BERT models for Named Entity Recognition?

All answers to this question.

Transitioning from BERT to Small LLMs (SLMs) was a game-changer for our legal extraction pipeline. BERT is great for simple tagging, but it lacks the generative "understanding" that helps when an entity is split across multiple sentences or referenced implicitly. A model like Llama-3-8B (which technically sits at the upper end of SLMs) handles the nuances of "whereas" and "herein" clauses with much higher fidelity. While BERT is faster, the SLM's ability to output structured JSON directly saves us a massive amount of post-processing time and regex-based cleanup.

   Answered 2025-01-17 by Shannon Montgomery

Do you find that the generative nature of SLMs leads to more "fabricated" entities compared to the extractive nature of a BERT-based model?

   Answered 2025-01-18 by Derek Fisher

• That is the primary risk, Derek. To mitigate it, we use a constrained output format like Pydantic or a grammar-based sampler. This forces the SLM to only extract text that actually exists in the source document. While BERT is "safer" in that it only selects tokens, a properly constrained SLM is just as safe and significantly more intelligent about identifying complex relationships between those tokens.

     Commented 2025-01-19 by Marcus Thorne

For production, BERT is cheaper for high-volume CPU-only servers, but SLMs win if you need deeper context for entity classification.

   Answered 2025-01-20 by Christina Lowe

• I agree with Christina. If you have the GPU budget, the SLM provides a level of insight that BERT just can't match without extensive custom layers.

     Commented 2025-01-21 by Kimberly Dawson