What is the best way to train OpenDevin on proprietary project management documentation?

All answers to this question.

For internal project management guidelines, RAG (Retrieval-Augmented Generation) is almost always superior to fine-tuning for OpenDevin. Your documentation likely changes weekly, and fine-tuning is too slow and expensive to keep up with that pace. By using a vector database like Pinecone or Milvus, you can have the agent search your Wiki in real-time. We found that the agent performs significantly better when we include a "Style Guide" as a persistent document in its context. This allows it to write documentation that sounds exactly like our senior architects without the need for a custom-trained model.

   Answered 2025-08-14 by Deborah Craig

Does the agent struggle with "conflicting information" if your Wiki has outdated pages alongside newer project management standards?

   Answered 2025-08-15 by Ryan Mitchell

• Yes, it does, Ryan. We solved this by adding "metadata" to our RAG chunks, such as the "Last Updated" date. We then instruct the agent to prioritize information from the most recent 12 months. Without this temporal filtering, the agent would occasionally suggest old deployment protocols that were deprecated years ago. It’s a classic "garbage in, garbage out" problem that requires a bit of data engineering on the backend to truly fix.

     Commented 2025-08-16 by Justin O'Neill

RAG is also much better for security because you can control access to specific documents based on the agent's user ID.

   Answered 2025-08-17 by Cynthia Gross

• Excellent point, Cynthia. Granular access control is impossible with a fine-tuned model where the information is "baked in" for everyone.

     Commented 2025-08-18 by Jeffrey Simons