How to build a powerful AI search engine using Haystack with hybrid retrieval for better results?

All answers to this question.

Building a hybrid system in Haystack 2.0 is remarkably modular. First, you need to set up your DocumentStore, like Elasticsearch or OpenSearch, which supports both keyword and vector storage. Second, define two separate retrievers: a BM25Retriever for lexical matching and a InMemoryEmbeddingRetriever (or similar) for semantic search. Third, use the DocumentJoiner component to connect both retrievers. You can choose a "reciprocal rank fusion" or a simple "distribution-based score" as your joining strategy. Finally, wrap these into a Pipeline object. We deployed this for a technical documentation site in late 2023, and the combination of keyword accuracy for specific error codes and semantic search for general queries significantly improved our user satisfaction metrics.

   Answered 2025-04-16 by Kimberly Davis

Does the DocumentJoiner handle the normalization of scores automatically, or do I need to write a custom component to ensure the BM25 scores don't overwhelm the vector scores?

   Answered 2025-04-20 by Steven Miller

• Steven, the standard DocumentJoiner in Haystack 2.0 includes a few built-in functions like reciprocal_rank_fusion which inherently handles different score scales by looking at rank rather than raw values. However, if you want more control, you can use the distribution_based_score_fusion which normalizes scores to a 0-1 range before merging. This is crucial when you build a powerful AI search engine using Haystack because vector and keyword scores are mathematically incomparable without that normalization step.

     Commented 2025-04-22 by Patrick Wright

The best part about Haystack is the visualization. You can call pipeline.draw() to see exactly how your data flows from the document store to the final joiner.

   Answered 2025-04-25 by Charles King

• I agree, Charles. Visualizing the pipeline graph makes it so much easier to explain the complex "hybrid" logic to stakeholders who aren't technical.

     Commented 2025-04-26 by Gregory Foster