How do I configure AutoGPT to use local LLMs via Ollama to reduce API costs?

All answers to this question.

To swap to a local backend, you need to modify the OPENAI_API_BASE_URL in your configuration to point to your Ollama local port, which is usually http://localhost:11434/v1. You also need to change the SMART_LLM_MODEL and FAST_LLM_MODEL names to match the exact string of the model you have pulled in Ollama. Be warned: while this saves money, smaller local models like the 8B or 7B versions often fail to follow the complex JSON formatting required for the agent to function. I’ve found that only the 70B models or highly fine-tuned "function-calling" variants can actually complete multi-step tasks without crashing.

   Answered 2025-09-23 by Betty Nelson

Have you noticed a significant drop in the agent's ability to correct its own errors when moving from GPT-4 to a local 70B model?

   Answered 2025-09-24 by Daniel King

• Daniel, absolutely. GPT-4 has a "common sense" layer that is hard to replicate. When a local model gets stuck in a loop, it rarely realizes it. You might need to increase the "temperature" slightly or provide a more explicit system prompt to help the local model stay on track during complex reasoning steps.

     Commented 2025-09-25 by Paul Garcia

It's a great way to experiment without worrying about the bill, even if the agent is a bit "dumber" overall.

   Answered 2025-09-26 by Lisa Scott

• Agreed, Lisa. For simple file sorting or web scraping tasks, local models are more than sufficient and much more private.

     Commented 2025-09-27 by Betty Nelson