How is the "Modern Data Stack" evolving to handle GenAI and RAG in 2024?

All answers to this question.

The "Batch" era of the Modern Data Stack is definitely feeling the pressure from GenAI. For RAG, "Freshness" is everything. If your bot is answering based on 24-hour-old data, it’s useless. We’ve seen a surge in "Streaming ETL" where tools like ByteWax or Pathway are replacing traditional batch jobs. You still need Snowflake for the long-term "Cold" storage, but your "Hot" path needs a Vector Database like Pinecone or Weaviate. The trend for 2024 is definitely "Hybrid"—keeping the simplicity of dbt for reporting, but adding a high-speed lane for AI features.

   Answered 2024-01-10 by Danielle Woods

Danielle, does adding a "high-speed lane" mean we’re doubling our infrastructure costs? Managing both a warehouse and a vector store sounds like a nightmare for a small Data Engineering team.

   Answered 2024-01-22 by Kevin Porter

• Kevin, it can be, which is why many are looking at "Integrated" solutions. Snowflake and Databricks are both adding native vector search capabilities so you don't have to manage a separate DB. The key is to evaluate if you really need sub-second latency. If a 15-minute refresh is okay, you can stay within your current stack and just optimize your Airflow triggers. Don't let the AI hype force you into a complex streaming architecture unless your use case strictly demands it.

     Commented 2024-02-02 by Tyler Moreno

It's all about "Unstructured Data" now. The old stack was built for rows and columns, but 2024 is about how you pipe PDFs and videos into a format an LLM can actually understand.

   Answered 2025-02-15 by Monica Geller

• Exactly, Monica. "Unstructured ingestion" is the biggest trend I’m seeing in cloud certs this year. The skills needed are shifting from simple SQL to complex document parsing.

     Commented 2024-02-22 by Danielle Woods