Can OpenDevin match the 13.8% SWE-bench score of Cognition's Devin?

All answers to this question.

The gap is closing much faster than people expected. While Devin had the first-mover advantage with its custom search and browser tools, OpenDevin leverages the collective power of the community to integrate with any LLM, like Claude 3.5 or GPT-4o. In my recent tests, the CodeAct architecture in OpenDevin actually performs better at multi-step terminal interactions because it isn't locked into a single provider's fine-tuned model. It’s not just about the benchmark score; it’s about the flexibility to run the agent locally on your own infrastructure for security.

   Answered 2025-11-15 by Kimberly Davis

Does the current OpenDevin implementation support RAG-based codebase indexing to reduce token costs?

   Answered 2025-11-20 by Jeffrey Moore

• Yes, Jeffrey, the latest updates include better workspace indexing. The OpenDevin project has moved toward a more modular "Event-Stream" architecture. This means the agent doesn't have to ingest the whole repo; it uses specialized tools to search for relevant classes and methods first. This drastically lowers the "hallucination" rate we saw in the earlier alpha versions of the OpenDevin framework.

     Commented 2025-11-25 by Tyler Henderson

I shifted my team to OpenDevin because we can't send our proprietary code to a closed-source cloud like Devin's.

   Answered 2025-12-05 by Megan Kelly

• That’s the deciding factor for most enterprises. OpenDevin provides that essential layer of privacy while still delivering high-level autonomy.

     Commented 2025-12-10 by Bradley Cooper