Memory

🧑‍🏫 Sabse pehle — simple mein samjho#

API ke through LLM bilkul "Ghajini" ki tarah kaam karta hai. Har nayi request uske liye pehli request hoti hai. Agar tum pehle pucho "Mera naam Tarun hai", aur agli baar pucho "Mera naam kya hai?", toh wo bhool chuka hoga! LLM ko purani baatein yaad rakhne ke liye humein pichli saari baaton (chat history) ko naye sawal ke sath attach karke bhejna padta hai. Is process ko automate karne ke liye LangChain Memory components deta hai.

Why Memory is Necessary#

Large Language Models are completely stateless. They do not remember previous API calls. To have a back-and-forth conversation, you must pass the entire conversation history to the model every single time you ask a new question.

In modern LangChain (using LCEL), Memory is typically handled manually by appending messages to a list, or by using RunnableWithMessageHistory to automatically inject the history from a database (like Redis) into the prompt.

1. Managing Message History (The LCEL Way)#

Let's look at how to manually manage a conversational thread.

from langchain_openai import ChatOpenAI
from langchain_core.messages import SystemMessage, HumanMessage, AIMessage

model = ChatOpenAI(model="gpt-3.5-turbo")

# We create an array to store the ongoing conversation
chat_history = [
    SystemMessage(content="You are a helpful assistant.")
]

# Turn 1
user_input1 = "Hi, my name is Tarun and I like coding."
chat_history.append(HumanMessage(content=user_input1))

# Invoke the model with the entire history array
response1 = model.invoke(chat_history)
print(response1.content) # "Nice to meet you, Tarun! What do you like to code?"

# We MUST append the AI's response back into the history array!
chat_history.append(AIMessage(content=response1.content))

# Turn 2
user_input2 = "What is my name?"
chat_history.append(HumanMessage(content=user_input2))

# Because the history contains the first turn, the model remembers!
response2 = model.invoke(chat_history)
print(response2.content) # "Your name is Tarun."

2. The Problem with Infinite Memory#

If a user chats with your bot for 3 hours, the chat_history array will contain thousands of messages. This causes two massive problems:

1. Context Window Limits: You will exceed the maximum number of tokens the LLM can accept (e.g., GPT-4 has a limit of 128k tokens).
2. Cost: OpenAI charges you per token sent. If you send the entire 3-hour history on every single turn, your API bill will skyrocket astronomically!

3. Strategies for Memory Management#

LangChain provides several strategies to trim or compress the history before sending it to the model.

Window Memory#

Instead of keeping the entire history, you only keep the last N messages (e.g., the last 5 turns). This is cheap and effective, but the bot will forget things said at the very beginning of the conversation.

Summary Memory (The Smart Approach)#

You run a smaller, cheaper LLM (like GPT-3.5) in the background. Every few turns, you ask this background LLM to read the raw chat history and write a short summary of it. Instead of injecting 100 raw messages into the main prompt, you just inject a 2-paragraph summary (e.g., "The user is named Tarun. They like coding. We are currently discussing Python").

# A conceptual look at how Summary Memory works behind the scenes
summary_prompt = "Summarize this conversation: {chat_history}"
# ... chain invokes the summary ...
# result: "User's name is Tarun. Likes coding."

# The main prompt for the next turn now looks like this:
final_prompt = f"""
Summary of previous conversation: {summary}
User's new message: What is my name?
"""

By utilizing Summary memory, you ensure the context stays small (saving money) while retaining all the important facts from a massive conversation!