	In Browser
	StumbleUpon
	del.icio.us
	Google
	Google Buzz
	reddit
	LinkedIn

	Facebook
	Twitter
	Linkedin
	E-Mail

Generative AI > RAG Pipelines > RAG with Query Expansion using Synonyms and LLM Paraphrasing

RAG with Query Expansion using Synonyms and LLM Paraphrasing

Author: Venkata Sudhakar

Query expansion improves RAG retrieval by augmenting the original customer query with synonyms, related terms, and paraphrases before searching the document index. ShopMax India customers use inconsistent terminology - 'earphones', 'earbuds', 'in-ear headphones', and 'TWS' all refer to the same product category. Without query expansion, a BM25 index built with 'earbuds' in the product titles misses queries that use 'earphones'. Query expansion bridges this vocabulary mismatch without requiring multiple retrieval passes.

Two approaches implement query expansion: dictionary-based expansion uses a predefined synonym map for known product terms (fast, deterministic, zero cost), and LLM-based paraphrasing generates query variants on the fly using a small model (flexible, handles novel queries but adds latency). For e-commerce, a hybrid works best: maintain a product vocabulary synonym dictionary for common category terms, and fall back to LLM paraphrasing for queries that match no dictionary entries.

The following example implements hybrid query expansion for ShopMax India. It first checks a product synonym dictionary, then expands with LLM paraphrasing if needed, and merges all expanded terms before retrieval.

import anthropic
from rank_bm25 import BM25Okapi

client = anthropic.Anthropic(api_key="sk-ant-...")

product_synonyms = {
    "earphones": ["earbuds", "in-ear headphones", "tws", "iems"],
    "earbuds": ["earphones", "in-ear headphones", "tws"],
    "headphones": ["headset", "over-ear", "on-ear", "cans"],
    "laptop": ["notebook", "ultrabook", "computer"],
    "mobile": ["phone", "smartphone", "handset"],
    "tv": ["television", "display", "screen"]
}

product_docs = [
    "Sony WF-1000XM5 earbuds: 8-hour battery, ANC, IPX4, Rs 19990, pan-India.",
    "Samsung Galaxy Buds2 Pro: true wireless earphones, 8-hour, IPX7, Rs 17990.",
    "Apple AirPods Pro 2: in-ear headphones, 6-hour, ANC, Rs 26900.",
    "Sony WH-1000XM5 headphones: 30-hour, over-ear, ANC, Rs 29990.",
    "Dell XPS 15: laptop, 32GB RAM, Rs 135000, Delhi."
]
tokenized = [doc.lower().split() for doc in product_docs]
bm25 = BM25Okapi(tokenized)

def expand_with_synonyms(query):
    tokens = query.lower().split()
    expanded = list(tokens)
    for token in tokens:
        if token in product_synonyms:
            expanded.extend(product_synonyms[token])
    return list(set(expanded))

def expand_with_llm(query):
    msg = client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=80,
        messages=[{"role": "user", "content": f"List 3 alternative search terms for this product query. Return only the terms comma-separated, no explanation.\nQuery: {query}"}]
    )
    extras = [t.strip().lower() for t in msg.content[0].text.split(",") if t.strip()]
    return extras

def expanded_rag(query, top_k=2):
    expanded_tokens = expand_with_synonyms(query)
    if len(expanded_tokens) == len(query.lower().split()):
        expanded_tokens += expand_with_llm(query)
    expanded_query = " ".join(expanded_tokens)
    scores = bm25.get_scores(expanded_query.split())
    idx = sorted(range(len(scores)), key=lambda i: scores[i], reverse=True)[:top_k]
    context = "\n".join([product_docs[i] for i in idx])
    msg = client.messages.create(
        model="claude-opus-4-7",
        max_tokens=150,
        system="You are ShopMax India assistant. Answer using only the provided context.",
        messages=[{"role": "user", "content": f"Context:\n{context}\n\nQuestion: {query}"}]
    )
    return msg.content[0].text

queries = [
    "best earphones under Rs 20000",
    "noise cancelling TWS under Rs 25000"
]

for q in queries:
    print(f"Q: {q}")
    print(f"A: {expanded_rag(q)}")
    print()

It gives the following output,

Q: best earphones under Rs 20000
A: The Sony WF-1000XM5 earbuds at Rs 19,990 are an excellent choice. They offer 8-hour battery, ANC, and IPX4 water resistance. The Samsung Galaxy Buds2 Pro at Rs 17,990 is also great value with IPX7 protection.

Q: noise cancelling TWS under Rs 25000
A: The Samsung Galaxy Buds2 Pro (Rs 17,990, IPX7) and Sony WF-1000XM5 (Rs 19,990, IPX4) are both noise-cancelling true wireless options available under Rs 25,000 across India.

For ShopMax India, maintain the synonym dictionary as a YAML file in your repository so product and marketing teams can update it without code changes. Include common misspellings for top-selling brands (e.g. 'samsng', 'appel', 'soni') in the synonym map - these show up frequently in mobile-typed queries. Run monthly analysis on queries that returned zero results to identify missing synonym entries. The LLM paraphrasing fallback should use your smallest, fastest model to keep latency under 200ms for the expansion step.

Send your comments, suggestions or queries regarding this site to [email protected].