	In Browser
	StumbleUpon
	del.icio.us
	Google
	Google Buzz
	reddit
	LinkedIn

	Facebook
	Twitter
	Linkedin
	E-Mail

Generative AI > RAG Pipelines > RAG with Table Data - Extracting and Querying Tabular Content

RAG with Table Data - Extracting and Querying Tabular Content

Author: Venkata Sudhakar

RAG with table data extends retrieval pipelines beyond plain text to handle structured information like product pricing sheets, sales reports, and inventory tables. ShopMax India stores category-level pricing data in spreadsheets and database exports - these tabular sources contain precise numbers that LLMs must not hallucinate. Parsing tables into a retrievable format and injecting them with their row-column structure intact ensures the model can accurately answer questions like 'which laptop has the best price-to-RAM ratio under Rs 100000'.

The key challenge with tabular RAG is preserving row context during chunking. Naive text chunking splits a CSV row across chunks, losing the relationship between column headers and values. The recommended approach converts each row into a natural language sentence (e.g. 'Dell XPS 15: 32GB RAM, Rs 135000, in stock in Mumbai') and stores these as retrievable documents. For wide tables with many columns, a summary embedding plus the raw row stored as metadata gives the best retrieval accuracy.

The following example converts a ShopMax India product pricing table into sentence-format documents, retrieves relevant rows for a query, and uses Claude to answer questions about pricing and specifications from the structured data.

import anthropic
from rank_bm25 import BM25Okapi

client = anthropic.Anthropic(api_key="sk-ant-...")

product_table = [
    {"name": "Dell XPS 15 9530", "category": "Laptop", "ram_gb": 32, "storage_tb": 1, "price": 135000, "city": "Mumbai, Delhi", "stock": 8},
    {"name": "HP Spectre x360", "category": "Laptop", "ram_gb": 16, "storage_tb": 1, "price": 115000, "city": "Bangalore, Chennai", "stock": 5},
    {"name": "Lenovo ThinkPad X1", "category": "Laptop", "ram_gb": 16, "storage_tb": 0.5, "price": 98000, "city": "Hyderabad, Delhi", "stock": 12},
    {"name": "ASUS ROG Zephyrus", "category": "Laptop", "ram_gb": 32, "storage_tb": 1, "price": 145000, "city": "Mumbai, Bangalore", "stock": 3},
    {"name": "Acer Swift 5", "category": "Laptop", "ram_gb": 16, "storage_tb": 0.5, "price": 72000, "city": "Delhi, Chennai", "stock": 15}
]

def row_to_sentence(row):
    return (f"{row['name']} is a {row['category']} with {row['ram_gb']}GB RAM "
            f"and {row['storage_tb']}TB storage, priced at Rs {row['price']}, "
            f"available in {row['city']} with {row['stock']} units in stock.")

doc_sentences = [row_to_sentence(r) for r in product_table]
tokenized = [s.lower().split() for s in doc_sentences]
bm25 = BM25Okapi(tokenized)

def table_rag(query, top_k=3):
    scores = bm25.get_scores(query.lower().split())
    idx = sorted(range(len(scores)), key=lambda i: scores[i], reverse=True)[:top_k]
    context = "\n".join([doc_sentences[i] for i in idx])
    msg = client.messages.create(
        model="claude-opus-4-7",
        max_tokens=200,
        system="You are ShopMax India assistant. Answer using only the provided product table data.",
        messages=[{"role": "user", "content": f"Product data:\n{context}\n\nQuestion: {query}"}]
    )
    return msg.content[0].text

queries = [
    "Which laptops have 32GB RAM and what are their prices?",
    "What is the cheapest laptop available in Delhi?",
    "Which laptop has the most units in stock in Mumbai?"
]

for q in queries:
    print(f"Q: {q}")
    print(f"A: {table_rag(q)}")
    print()

It gives the following output,

Q: Which laptops have 32GB RAM and what are their prices?
A: Two laptops have 32GB RAM: the Dell XPS 15 9530 at Rs 135,000 and the ASUS ROG Zephyrus at Rs 145,000.

Q: What is the cheapest laptop available in Delhi?
A: The Lenovo ThinkPad X1 is available in Delhi at Rs 98,000. The Acer Swift 5 is also available in Delhi at Rs 72,000, making it the cheapest option there.

Q: Which laptop has the most units in stock in Mumbai?
A: The Dell XPS 15 9530 has the most units in stock in Mumbai with 8 units available.

For ShopMax India at scale, keep the raw table rows in a structured database and regenerate sentence embeddings whenever prices or stock levels change. Do not embed prices as static text - they change daily. Instead, fetch the current row from the database after retrieval and inject it fresh into the context. For complex analytical queries like 'best price-per-GB laptop under Rs 100000', add a calculation step before the LLM call that computes derived metrics from the raw table data and appends them to the context.

Send your comments, suggestions or queries regarding this site to [email protected].