Building a RAG Chain with LangChain

LangChain provides dedicated abstractions that make building RAG pipelines significantly cleaner than assembling the pieces manually. The key components are document loaders (load data from PDFs, web pages, databases), text splitters (chunk documents), vector stores (persist and query embeddings), retrievers (interface over a vector store for similarity search), and the final RAG chain that wires everything together. LangChain's LCEL makes it easy to compose these into a single, streaming-capable pipeline.

The LangChain RAG pattern follows a standard structure: build the retriever from a vector store, define a prompt that accepts context and a question, then chain the retriever output into the prompt and pass it to the LLM. The RetrievalQA chain and the newer create_retrieval_chain function handle this plumbing automatically. For production systems, you would use a persistent vector store like Chroma with disk storage, Pinecone, pgvector, or Weaviate instead of an in-memory store.

The below example shows the complete end-to-end LangChain RAG pipeline from document ingestion to question answering, built with LCEL for full control over the retrieval and generation steps.

It gives the following output,

Indexed 5 chunks into ChromaDB.

The below example shows the LCEL RAG chain for question answering, with source citation and streaming support.

It gives the following output,

Q: How does CDC work and which tool publishes events to Kafka?
A: CDC works by reading the database transaction log to capture changes.
[Source: cdc_guide.pdf p.1] Debezium is the most popular open-source CDC
tool and publishes these change events to Apache Kafka.
------------------------------------------------------------

Q: What is the difference between Blue-Green and Strangler Fig migration?
A: Blue-Green deployment [Source: migration_guide.pdf p.5] uses two identical
environments where traffic switches from the old (Blue) environment to the
new (Green) one via a load balancer after validation. The Strangler Fig pattern
[Source: migration_guide.pdf p.3] migrates incrementally by routing traffic
to old or new systems based on feature flags, replacing legacy features one at
a time.
------------------------------------------------------------

Q: How does Flyway track which migrations have been applied?
A: Flyway tracks migrations using the flyway_schema_history table
[Source: schema_guide.pdf p.1], which records each versioned SQL script
that has been successfully applied, ensuring each script runs exactly once.
------------------------------------------------------------

LangChain RAG enhancements for production:

Conversational memory - Use ConversationBufferMemory or LangGraph to maintain chat history, allowing follow-up questions like "Tell me more about that" to work correctly in multi-turn conversations.

Re-ranking - After vector retrieval, use a cross-encoder re-ranker (like Cohere Rerank or a local cross-encoder model) to re-score the top-k chunks by relevance to the exact question before passing them to the LLM. This significantly improves answer quality.

Hybrid search - Combine vector similarity search with keyword BM25 search. Semantic search excels at conceptual queries; keyword search excels at exact term matching. The ensemble retriever in LangChain combines both for better coverage.