Callbacks and Tracing (LangSmith)

🧑‍🏫 Sabse pehle — simple mein samjho#

Jab tumhara RAG system fail hota hai aur AI galat answer deta hai, toh tumhe kaise pata chalega ki problem kahan thi? Kya Retriever ne galat document uthaya tha? Ya Prompt galat ban gaya tha? Ya LLM ko baat samajh nahi aayi? Is poore pipeline ko inspect karne ke liye humein debugging tools chahiye hote hain. LangChain ka apna ek platform hai jise LangSmith kehte hain. Ye ek dashboard hai jahan tum apni har ek API call ka xray dekh sakte ho: kitne paise lage, kya input gaya tha, kya output aaya.

1. What is LangSmith?#

LangSmith is a cloud platform built specifically for debugging, testing, evaluating, and monitoring LLM applications built on LangChain. It visually traces the exact path of your LCEL chains and Agents.

You can see:

• The exact prompt sent to OpenAI.
• The raw JSON returned by OpenAI.
• The chunks retrieved by the Vector Store.
• The exact execution time of each step.
• The token usage and estimated cost of each call.

2. Enabling LangSmith Tracing#

The best part about LangSmith is that it requires absolutely zero changes to your Python code! It is entirely configured via Environment Variables.

In your .env file, simply add:

# Enable tracing
LANGCHAIN_TRACING_V2="true"

# Add your API key (get it from smith.langchain.com)
LANGCHAIN_API_KEY="ls__your_api_key_here"

# (Optional) Group all logs for a specific project together
LANGCHAIN_PROJECT="My Custom RAG App"

Once these variables are active, every time you call .invoke() on a chain, the entire execution trace is silently sent to your LangSmith dashboard in the background.

3. Callbacks (For Streaming & UI Updates)#

While LangSmith is for debugging, Callbacks are used to update your frontend UI in real-time. The most common use case for callbacks is Streaming.

When a user asks a question, generating a long response might take 10 seconds. We don't want the user staring at a blank screen. We want to stream the text chunk-by-chunk, exactly like ChatGPT does.

Streaming with LCEL#

If you are using LCEL, you don't even need manual callbacks. You can just use the .stream() method on your chain!

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_template("Write a long poem about {topic}")
model = ChatOpenAI(model="gpt-3.5-turbo")

chain = prompt | model | StrOutputParser()

# Instead of .invoke(), we loop over .stream()
for chunk in chain.stream({"topic": "the ocean"}):
    # Print each word/token as soon as it arrives from OpenAI
    print(chunk, end="", flush=True)

Manual Callbacks (Legacy / Advanced)#

If you are building custom agents or need to trigger actions (like updating a progress bar) when specific components start/stop, you can create custom Callback Handlers.

from langchain.callbacks.base import BaseCallbackHandler

class MyCustomHandler(BaseCallbackHandler):
    # Triggers the moment the LLM starts generating text
    def on_llm_start(self, serialized, prompts, **kwargs):
        print("LLM is starting its work...")

    # Triggers every time the LLM generates a new token
    def on_llm_new_token(self, token, **kwargs):
        print(f"New token received: {token}")

    # Triggers when the LLM finishes
    def on_llm_end(self, response, **kwargs):
        print("LLM has finished generating!")

# Attach the handler to the model
model = ChatOpenAI(callbacks=[MyCustomHandler()])