Best frameworks for implementing guidance in LLMs?

All answers to this question.

The landscape is moving fast! Microsoft’s "Guidance" is excellent for its syntax, but many are shifting toward "Outlines" by the .dot team. Outlines is particularly powerful because it uses FSM (Finite State Machines) to guide the sampling, making it incredibly fast. Another one to watch is LMQL (Language Model Query Language), which allows for very complex logic nested within the prompt. If you are already deep into the LangChain ecosystem, check out their "Output Parsers," though they are more of a post-processing step than true "guidance" which happens during the token selection itself.

   Answered 2025-06-17 by Sharon Myers

Between Outlines and Guidance, which one have you found to be more stable when dealing with frequent API updates from providers like OpenAI?

   Answered 2025-06-18 by Joshua Hayes

• Joshua, that's the catch—most guidance libraries work best with local models (via Transformers or llama.cpp) where you have control over the logits. For OpenAI, you are limited to their "JSON Mode" or "Function Calling," which are their versions of guidance. If you want the full programmatic power of these libraries, you're usually better off hosting your own model on a cloud GPU to get that granular control.

     Commented 2025-06-19 by Larry Gardner

I’ve been using Outlines for a few months now for my Data Science work and it’s been a game changer for extracting structured entities from messy text.

   Answered 2025-06-20 by Cheryl Bishop

• I've heard great things about Outlines as well! It’s interesting to see how the industry is pivoting toward these more rigorous Software Development practices for AI.

     Commented 2025-06-21 by Daniel Foster