	In Browser
	StumbleUpon
	del.icio.us
	Google
	Google Buzz
	reddit
	LinkedIn

	Facebook
	Twitter
	Linkedin
	E-Mail

Generative AI > AI Observability > LangSmith - Tracing and Debugging LangChain Applications

LangSmith - Tracing and Debugging LangChain Applications

Author: Venkata Sudhakar

ShopMax India's chatbot handles thousands of customer queries daily - order status, product compatibility, return requests. When the chatbot gives a wrong answer or takes 10 seconds to respond, the team needs to know which LLM call failed, what prompt was used, and how long each step took. LangSmith is LangChain's native observability platform that automatically captures every chain run, LLM call, tool invocation, and retrieval step as a trace. ShopMax India's engineering team uses LangSmith to debug failures, compare prompt versions, and catch latency regressions before customers notice.

LangSmith tracing works by setting two environment variables: LANGCHAIN_TRACING_V2=true and LANGCHAIN_API_KEY. Once set, every LangChain object (LLM, chain, agent, retriever) automatically sends trace data to LangSmith without code changes. Each trace shows the full input and output at every step, token counts, latency breakdown, and cost estimate. The LangSmith UI allows filtering by date, model, chain type, and error status. The langsmith Python client also lets you annotate traces with feedback scores and run dataset evaluations programmatically.

The example below sets up LangSmith tracing on a ShopMax India order support chain. It creates a simple chain that classifies customer queries and generates a response, then shows how to add custom metadata tags and retrieve recent traces programmatically using the LangSmith client.

import os
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langsmith import Client

os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = "ls-your-key-here"
os.environ["LANGCHAIN_PROJECT"] = "shopmax-india-support"
os.environ["OPENAI_API_KEY"] = "sk-your-key-here"

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

classify_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a ShopMax India support classifier. Classify the query as: ORDER_STATUS, RETURN_REQUEST, PRODUCT_QUERY, or OTHER."),
    ("user", "{query}")
])

respond_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful ShopMax India support agent. Give a brief, friendly response."),
    ("user", "Query: {query}\nCategory: {category}")
])

classify_chain = classify_prompt | llm | StrOutputParser()
respond_chain = respond_prompt | llm | StrOutputParser()

test_queries = [
    "Where is my order ORD-5042 from Mumbai?",
    "I want to return my Samsung TV bought last week",
    "Does the OnePlus 11 support 5G in Delhi?"
]

for query in test_queries:
    category = classify_chain.invoke({"query": query})
    response = respond_chain.invoke({"query": query, "category": category})
    print("Query:    " + query)
    print("Category: " + category)
    print("Response: " + response[:80] + "...")
    print()

# Retrieve recent traces via LangSmith client
client = Client()
runs = list(client.list_runs(
    project_name="shopmax-india-support",
    run_type="chain",
    limit=3
))
print("Recent traces in LangSmith:")
for run in runs:
    status = run.status
    latency = round(run.end_time.timestamp() - run.start_time.timestamp(), 2) if run.end_time else 0
    tokens = run.total_tokens if hasattr(run, "total_tokens") else "N/A"
    print("  " + str(run.name) + " | status: " + str(status) + " | latency: " + str(latency) + "s | tokens: " + str(tokens))

It gives the following output,

Query:    Where is my order ORD-5042 from Mumbai?
Category: ORDER_STATUS
Response: Thank you for reaching out! Your order ORD-5042 is currently being processed...

Query:    I want to return my Samsung TV bought last week
Category: RETURN_REQUEST
Response: We are sorry to hear that! To initiate a return for your Samsung TV, please...

Query:    Does the OnePlus 11 support 5G in Delhi?
Category: PRODUCT_QUERY
Response: Yes, the OnePlus 11 5G supports 5G connectivity and works with all major...

Recent traces in LangSmith:
  RunnableSequence | status: success | latency: 1.24s | tokens: 87
  RunnableSequence | status: success | latency: 0.98s | tokens: 72
  RunnableSequence | status: success | latency: 1.31s | tokens: 91

In production, add run_name and tags parameters to chain.invoke() calls to label traces by feature area: chain.invoke({...}, config={"run_name": "order-support", "tags": ["v2", "mumbai"]}). Use LangSmith datasets to run regression tests: create a dataset of 50 golden ShopMax India queries with expected outputs and run chain evaluations automatically on every deployment. Set up alerts in LangSmith for error rate spikes or p95 latency exceeding 3 seconds so the on-call engineer is paged before customers notice degradation.

Send your comments, suggestions or queries regarding this site to [email protected].