RAG with Sentence Window Retrieval for Better Context Capture

Sentence window retrieval solves a fundamental RAG tradeoff: small chunks embed accurately but lack surrounding context, while large chunks provide context but embed poorly. ShopMax India product manuals have this exact problem - a sentence like 'battery life is 30 hours' has a precise embedding, but without the surrounding sentences the LLM cannot tell whether this refers to headphones, a laptop, or a power bank. Sentence window retrieval embeds at the sentence level but returns a window of surrounding sentences as context.

The pattern works in two stages. At index time, split documents into individual sentences and embed each one separately. Store each sentence with metadata pointing to its parent document and its position index. At query time, retrieve the top-k matching sentences by embedding similarity, then for each matched sentence fetch the surrounding window (e.g. 2 sentences before and 2 after) from the original document. Pass this expanded window to the LLM instead of just the matched sentence.

The following example implements sentence window retrieval for ShopMax India product documentation. Sentences are embedded using a simple TF-IDF approximation, and each retrieved sentence is expanded with its neighbors before being passed to Claude.

It gives the following output,

Q: How long does the battery last and can I quick charge it?
A: The Sony WH-1000XM5 battery lasts 30 hours with noise cancelling on. Quick charging for 3 minutes provides 3 hours of playback, making it convenient for emergencies.

Q: What connectivity options does the headphone support?
A: The Sony WH-1000XM5 supports Bluetooth 5.2 with multipoint connection for two devices simultaneously. It also has a USB-C port for charging and wired listening.

Q: Is it comfortable for long use and what colors are available?
A: Yes, at 250 grams with soft synthetic leather ear cushions, it is designed for comfortable extended wear. It is available in Black and Platinum Silver.

For ShopMax India at scale, pre-compute and store sentence windows in your vector database as a separate field alongside the sentence embedding. This avoids recomputing windows at query time. Set window size based on average sentence length in your documents - technical spec sheets with short dense sentences need a window of 3-4, while narrative product descriptions with longer sentences work well with a window of 1-2. Experiment with asymmetric windows (more context after the match than before) for documents where key context tends to follow the matched sentence rather than precede it.