Cell: Initialize keys and call the OpenAI LLM¶

Purpose

• Initialize credentials, construct the OpenAI client (langchain_openai wrapper) and run a synchronous model invocation to get an explanatory response.

Prerequisites

• Imports are in the next cell: from keys import set_keys and from langchain_openai import OpenAI.
• keys.set_keys() must securely load/set the OpenAI API key (e.g., environment variable or secrets store).
• Network access and a valid API key are required.

Line-by-line explanation

• set_keys() — loads the API key into the environment or library config; must be called before creating the client.
• openai = OpenAI() — creates a synchronous OpenAI client wrapper (stores underlying client and credentials).
• response = openai.invoke("What is Agentic AI?") — sends the prompt to the model and returns a string response.
• print(response) — prints the model output to the notebook stdout.

Outputs and variables

• openai — instance of langchain_openai.llms.base.OpenAI (client + config).
• response — str containing the model’s answer (printed and available for further processing).

Cell: Demo — FakeListLLM deterministic response¶

Purpose

• Demonstrate using langchain_community.llms.FakeListLLM to produce a deterministic, canned response for testing or demos.

Prerequisites

• The class is available via: from langchain_community.llms import FakeListLLM
• No network or API key is required.

What the code does

• Instantiates a FakeListLLM with a list of predefined responses.
• Invokes the LLM with a prompt (“Hello”) and prints the next canned response.

Line-by-line explanation

• fake_llm = FakeListLLM(responses = [“Hello! This is a fake test response.”])
  • Creates a fake LLM that returns responses in the order provided.
• response = fake_llm.invoke(“Hello”)
  • Sends the prompt to the fake LLM; returns the next canned response as a str.
• print(response)
  • Writes the returned string to stdout.

Outputs and variables

• fake_llm: FakeListLLM instance containing the provided responses.
• response: str containing the returned canned response (expected: “Hello! This is a fake test response.”).

Usage notes

• Useful for unit tests, UI prototypes, and examples where predictable outputs are required.
• Each invoke() call consumes one response from the list; reinstantiate or provide multiple responses for repeated calls.

Cell: Sync chat invocation with ChatOpenAI¶

Purpose

• Perform a synchronous chat-style invocation against the ChatOpenAI client (gpt-5) with a SystemMessage that enforces a response-style constraint and a HumanMessage prompt asking about Agentic AI and deployment strategy.
• Print both the message content (human-consumable text) and the full underlying response object (metadata, usage, tool calls).

Prerequisites

• The following classes/objects must already be available in the notebook (imports and instantiation happen in other cells):
  • ChatOpenAI (langchain_openai.chat_models.base.ChatOpenAI)
  • SystemMessage (langchain_core.messages.system.SystemMessage)
  • HumanMessage (langchain_core.messages.human.HumanMessage)

What this cell does

• Constructs a system message that sets the assistant persona and enforces that it “always end the response with a joke.”
• Constructs a human message asking about “Agentic AI” and an enterprise deployment strategy.
• Calls chat.invoke(messages) to synchronously get an AIMessage response.
• Prints:
  1. The main textual content of the LLM response (response.content) for easy reading.
  2. The full response object for inspection of metadata (token usage, model name, tool calls, IDs, finish reason, etc.).

Line-by-line explanation

• messages = […]
  • Builds a list of two message objects: SystemMessage (assistant role + constraint) and HumanMessage (user prompt).
• response = chat.invoke(messages)
  • Sends the messages to the ChatOpenAI client synchronously and returns an AIMessage-like object containing content and metadata.
• print(f”LLM response content: \n {response.content} \n”)
  • Prints the human-readable content produced by the model.
• print(f”Full LLM response: \n {response}”)
  • Prints the full response object for debugging, auditing, or extracting usage statistics and tool-call details.

Outputs and variables

• messages (list[BaseMessage]) — the list of messages sent to the model.
• response (AIMessage-like) — contains:
  • content: str (the model’s generated text)
  • additional_kwargs/response_metadata: dict (token usage, model_name, ids, tool calls, etc.)
  • other runtime metadata useful for logging, billing, or decision-making.

Usage notes and cautions

• Cost and token usage: synchronous invocations incur token usage; inspect response metadata for cost analysis.
• Safety and prompt constraints: the system message enforces a stylistic constraint (append a joke). Adjust system prompts to match policy and safety requirements.

Quick troubleshooting

• If chat.invoke raises an authentication or network error, verify keys are set and network access is available.
• If response.content is empty, inspect response metadata for finish_reason and tool_calls to understand termination conditions.

Cell: Chat prompt → LLM chain (prime-check example)¶

Code that builds a chat-style prompt template, composes it with a ChatOpenAI runnable, invokes the chain with a question, and prints the model response.

What the code does (high level)¶

1. Creates a chat prompt template with:
   • A system message: “You are a coding assistant that helps write Python code.”
   • A user message template: “{question}” (placeholder for runtime input).
2. Instantiates a ChatOpenAI model with model_name="o3-mini" and reasoning_effort="medium".
3. Composes the prompt template and the chat model into a runnable sequence (chain = template | chat).
4. Provides a concrete question (“given a number, check if it is prime or not”) and invokes the chain with that input.
5. Prints the LLM response content.

Variables used / created¶

• template — a ChatPromptTemplate with input variable question.
• chat — ChatOpenAI instance (configured for o3-mini, medium reasoning).
• chain — RunnableSequence combining template and chat.
• question — string input to the prompt template.
• response — AIMessage-like object returned by chain.invoke(...); has content and metadata.

Expected output¶

• response.content contains the assistant’s answer: a human-readable Python implementation and explanation for checking whether a number is prime.
• response also carries metadata (token usage, model info) accessible via attributes.

Notes and usage tips¶

• To change the task, update the question string or extend the prompt template.
• reasoning_effort="medium" affects how much internal reasoning is requested; increase for harder tasks.
• Inspect response metadata for token counts and debug information if results are truncated or unexpected.

Cell: Compare factual vs. creative chat chains¶

• Create two ChatOpenAI chains with different decoding settings (low-variance factual vs. high-variance creative), run them on the same prompt, and print both outputs for comparison.

What this cell does (high level)

1. Instantiates two ChatOpenAI clients:
   • factual_chat: low temperature/top_p to favor deterministic, concise answers.
   • creative_chat: higher temperature/top_p and larger max_tokens to favor creative, expansive responses.
2. Builds a simple ChatPromptTemplate that sets a system message and a user message placeholder {question}.
3. Composes each template with the corresponding ChatOpenAI into runnable sequences:
   • factual_chain = template | factual_chat
   • creative_chain = template | creative_chat
4. Invokes each chain with the same question (“Explain AI in simple terms.”) and prints both responses to compare style and length.

Line-by-line explanation

• factual_chat = ChatOpenAI(temperature=0.1, top_p=0.2, max_tokens=256)
  • Low randomness and small output budget for succinct, factual replies.
• creative_chat = ChatOpenAI(temperature=0.8, top_p=0.9, max_tokens=512)
  • Higher randomness and larger output budget for more creative or verbose replies.
• template = ChatPromptTemplate.from_messages([…])
  • Defines a two-message prompt: a system message (“You are a helpful chat assistant…”) and a user message template “{question}”.
• question = “Explain AI in simple terms.”
  • The user-facing prompt passed to both chains.
• factual_chain = template | factual_chat; creative_chain = template | creative_chat
  • Compose prompt + model into runnable sequences.
• factual_response = factual_chain.invoke({“question”: question})
  • Synchronously runs the factual chain and returns an AIMessage-like object.
• print(f”\nFactual LLM response: {factual_response.content}”)
  • Prints the textual content of the factual response.
• creative_response = creative_chain.invoke({“question”: question})
  • Runs the creative chain.
• print(f”\nCreative LLM response: {creative_response.content}”)
  • Prints the creative reply content.

Usage notes and tips

• To test other behaviors, adjust temperature, top_p, or max_tokens on each ChatOpenAI instance.
• Change question to evaluate responses on different tasks.
• Inspect response.response_metadata or the full AIMessage object for token usage and debugging.
• For deterministic outputs, keep temperature very low (≈0.0–0.2) and reduce top_p; for creativity, increase temperature and top_p.

Cost and safety considerations

• Each invoke call consumes tokens billed by the provider; monitor response.response_metadata['token_usage'] for cost analysis.
• System prompts should be reviewed for policy compliance; avoid instructions that encourage unsafe or disallowed content.

LCEL – LangChain Expression Language¶

LCEL (LangChain Expression Language) is LangChain’s “pipeline” syntax for wiring components (prompt → model → parser/tools → etc.) together using simple operators (commonly |). It lets you build runnable chains that are easy to read, compose, and reuse, while keeping the data flow explicit (inputs/outputs are just passed along the chain). It also works well with streaming, async, and batching because the composed chain is still a first-class runnable object.

In LangChain, a “Runnable” is any component that can be executed with an input to produce an output (for example, a prompt, an LLM, a parser, a tool wrapper, or a full chain). It exposes a consistent interface like invoke() (single input), plus often batch() and stream()/astream() for bulk or streaming execution. Because everything is a Runnable, you can compose pieces cleanly (e.g., prompt | model | parser) and run the whole pipeline the same way.

Purpose

• Build a small runnable pipeline that prompts a chat LLM to produce a brief summary of a given topic, parses the LLM output to a plain string, and prints it.

Step-by-step

• template = PromptTemplate.from_template("Provide a brief summary of the following topic: {topic}")
  Creates a prompt template with a {topic} input variable.
• llm = ChatOpenAI()
  Instantiates a ChatOpenAI model (uses defaults and the notebook’s configured API key).
• parser = StrOutputParser()
  Creates a parser that converts the LLM output into a plain Python string.
• chain = template | llm | parser
  Composes a RunnableSequence: prompt -> LLM -> parser.
• topic = "Reinforcement Learning"
  Supplies the concrete input value for {topic}.
• response = chain.invoke({"topic": topic})
  Synchronously runs the chain and returns the parsed string result.
• print(f"Parsed LLM response: {response}")
  Prints the final summary produced by the model.

Outputs

• response: str containing the LLM-generated brief summary of the topic.

Notes

• Using StrOutputParser ensures the chain returns a simple string suitable for further programmatic use.

Generate and Analyse Summary for “Transformer Models” – Chaining multiple LLM calls¶

Purpose

• Generate a brief summary of “Transformer Models” with a gpt-5 ChatOpenAI, then rate and explain that summary with a gpt-4 ChatOpenAI.

Prerequisites

• ChatOpenAI, PromptTemplate, and StrOutputParser are available in the notebook.
• The variables generate_chain, analyse_chain, generate_llm, and analyse_llm will be created in this cell.

What this cell does

1. Create generate_chain = PromptTemplate(“Provide a brief summary of the following topic: {topic}”) | gpt-5 | StrOutputParser
2. Set topic = "Transformer Models"
3. Create analyse_chain = PromptTemplate(“The topic is ” + topic + “… \n{summary}”) | gpt-4 | StrOutputParser
4. Compose combined_chain = generate_chain | analyse_chain
5. Invoke combined_chain.invoke({"topic": topic}) and print the final parsed response.

Outputs

• Prints Final LLM response: containing the analyser’s rating and explanation for the generated summary.

Note

• We lose the generated summary here and only see the final analysis response. We will fix that in the next cell.

Aggregate summary + analysis with RunnablePassthrough¶

Purpose

• Run the summary generator and the analysis chain together and collect their outputs into a single dict.

What it does

• Uses RunnablePassthrough.assign(...) to map:
  • summary -> generate_chain
  • analysis -> analyse_chain
• Invokes the composed runnable with {"topic": topic} and returns a dict containing the original input and the assigned outputs (e.g., {'topic', 'summary', 'analysis'}).
• Prints the returned keys and each key/value pair.

RunnablePassthrough — why and how

• RunnablePassthrough (via .assign(...)) lets you attach named runnables to a passthrough pipeline so their outputs are produced and returned as named fields in the final dict.
• Useful for running multiple sub-chains (or models) on the same input and aggregating results without manually managing intermediate values.

Local Ollama chat invocation – Using locally hosted LLMs¶

Purpose

• Run a local Ollama-backed chat LLM to answer a user question using a simple chat prompt template.

What the cell does

1. Instantiates a local ChatOllama model:
   • local_llm = ChatOllama(model="qwen3-vl:8b") — configures the Ollama model to use.
2. Builds a chat prompt template:
   • template = ChatPromptTemplate.from_messages([...]) — system message sets assistant behavior; user message includes {question} placeholder.
3. Sets the runtime question:
   • question = "What is self-supervised learning?"
4. Composes the prompt and model into a runnable chain:
   • chain = template | local_llm — LCEL composition: prompt → model.
5. Invokes the chain and prints the result:
   • response = chain.invoke({"question": question})
   • print(f"Local LLM response: {response.content}") — prints the model’s textual reply.

Inputs and outputs

• Input: {"question": question} passed into the prompt template.
• Output: response — an AIMessage-like object; main readable output is response.content.

Note:

• We are using Ollama to serve our downloaded model locally
• Download and install Ollama from: https://ollama.com/download
• Download a model of your choice and run the Ollama server using cmd line ollama serve

Text-to-Image generation with Stable Diffusion 3.5¶

• Load and configure the Stable Diffusion 3.5 Large model for text-to-image generation and wrap it in a LangChain-compatible runnable.

Prerequisites

• Required libraries must be installed and imported:
  • diffusers (StableDiffusion3Pipeline)
  • torch (PyTorch for tensor operations and device management)
  • langchain_core.prompts (PromptTemplate – not used in this cell but available for future prompt composition)
  • langchain_core.runnables (RunnableLambda – wraps the image generation function)
  • hf_xet, accelerate, transformers (Hugging Face dependencies for model loading and optimization)

What this cell does

1. Load the Stable Diffusion 3.5 Large model:

   • pipe = StableDiffusion3Pipeline.from_pretrained("stabilityai/stable-diffusion-3.5-large", torch_dtype=torch.bfloat16)
   • Downloads/loads the pre-trained model from Hugging Face with bfloat16 precision for memory efficiency.

2. Move the pipeline to CUDA (GPU):

   • pipe = pipe.to("cuda") — transfers the model to GPU for faster inference.

3. Enable CPU offloading:

   • pipe.enable_model_cpu_offload() — offloads parts of the model to CPU when not actively in use, reducing VRAM requirements.

4. Wrap the pipeline in a LangChain Runnable:

   • image_runnable = RunnableLambda(lambda prompt: pipe(prompt, num_inference_steps=50, guidance_scale=3.5).images[0])
   • Creates a runnable that:
     • Takes a text prompt as input
     • Runs the Stable Diffusion pipeline with 50 inference steps and guidance scale of 3.5
     • Returns the first generated image (PIL Image object)

Variables created

• pipe — StableDiffusion3Pipeline instance, configured and loaded on GPU with CPU offloading enabled.
• image_runnable — RunnableLambda wrapping the image generation function, compatible with LCEL chains.

Usage notes

• Hardware requirements: Requires a CUDA-capable GPU with sufficient VRAM (recommended: 12GB+ for SD3.5 Large).
• Generation parameters:
  • num_inference_steps=50 — controls quality/speed tradeoff (higher = better quality, slower).
  • guidance_scale=3.5 — controls prompt adherence (higher = stricter prompt following).
• Integration: image_runnable can be composed with other LangChain runnables (e.g., prompt generators, image-to-text analyzers) using LCEL.
• Output: Returns a PIL Image object that can be displayed, saved, or passed to downstream processing.

Generate and display an image from text prompt using Stable Diffusion¶

• Invoke the pre-configured Stable Diffusion 3.5 image generation runnable with a detailed text prompt and display the resulting image in the notebook.

Prerequisites

• image_runnable must be defined — a RunnableLambda wrapping the StableDiffusion3Pipeline.
• IPython.display.display must be imported to render the PIL Image in the notebook output.

What this cell does

1. Import display function:

   • from IPython.display import display — enables rendering of rich objects (images, HTML, etc.) in Jupyter notebooks.

2. Invoke the image generation runnable:

   • response = image_runnable.invoke("a futuristic cityscape at sunset with flying cars and neon lights and a bot standing on a rooftop")
   • Passes the descriptive text prompt to the Stable Diffusion pipeline.
   • The pipeline generates an image matching the prompt description (50 inference steps, guidance scale 3.5).
   • Returns a PIL Image object stored in response.

3. Display the generated image:

   • display(response) — renders the PIL Image directly in the notebook cell output for visual inspection.

Variables

• response — PIL Image object containing the generated artwork.

Usage notes

• Prompt engineering: More detailed, specific prompts typically produce better results. Include style descriptors, lighting, composition details, etc.
• Generation time: Expect 10-60 seconds depending on GPU speed and model size.
• Saving images: Use response.save("output.png") to persist the generated image to disk.
• Multiple generations: Run the cell multiple times with the same prompt to see variations (Stable Diffusion has inherent randomness).

Expected output

• A high-resolution image of a futuristic cityscape at sunset featuring flying cars, neon lights, and a robot on a rooftop, rendered inline in the notebook.

Vision: Analyze generated image with local LLM¶

Purpose

• Send the previously generated image (from Stable Diffusion) to the local Ollama vision-capable LLM and get a detailed description of what the image contains.

Prerequisites

• generated_image — a PIL Image object created in the previous cell.
• local_llm — the ChatOllama instance configured earlier that supports vision/multimodal inputs.
• base64, io, and HumanMessage must be imported (happens in the cell below).

What the cell does (high level)

1. Convert the PIL image to a base64-encoded data URL:

   • Create an in-memory BytesIO buffer.
   • Save the PIL image to the buffer in PNG format.
   • Base64-encode the buffer contents and construct a data URL string (data:image/png;base64,...).

2. Construct a multimodal HumanMessage:

   • Create a HumanMessage with two content parts:
     • A text prompt: "Describe the image in detail and explain what is happening."
     • An image_url part containing the base64 data URL.

3. Invoke the local vision LLM:

   • local_llm.invoke([msg]) — sends the multimodal message to the Ollama model.
   • The model analyzes the image and generates a textual description/explanation.

4. Print the LLM response:

   • print(f"Local LLM response with image context: {response.content}") — displays the model’s detailed description of the generated futuristic cityscape image.

Variables used/created

• question — the text prompt asking for image analysis.
• buf — BytesIO buffer holding the PNG image bytes.
• image_b64 — base64-encoded string of the image.
• image_url — data URL string for embedding the image in the message.
• msg — HumanMessage containing both text and image content.
• response — AIMessage-like object returned by the local LLM; response.content contains the textual analysis.

Usage notes

• Vision model requirement: The Ollama model (qwen3-vl:8b) must support vision/multimodal inputs. Not all LLMs can process images; verify model capabilities before use.
• Image format: The cell uses PNG format and base64 encoding, which is widely supported for data URLs in multimodal APIs.
• Prompt customization: Change the question string to ask for specific aspects (e.g., colors, objects, mood, artistic style).
• Cost and latency: Vision inference is typically slower and more resource-intensive than text-only inference.

Expected output

• A detailed textual description of the generated image, explaining the futuristic cityscape, sunset lighting, flying cars, neon lights, and the robot on the rooftop.

Define EmailState — Typed state dictionary for email processing agent¶

Purpose

• Define a strongly-typed state schema (EmailState) that will be used to track and pass data between nodes in an email processing agent workflow (LangGraph).

Prerequisites

• TypedDict from typing_extensions must be imported (happens in the cell below).
• This is a preparatory cell for building a multi-step agent that analyzes and responds to emails.

What the code does

• class EmailState(TypedDict): — creates a typed dictionary class that acts as a schema/contract for the agent state.
• The state contains four fields:
  • summary: str — a brief textual summary of the email content.
  • sentiment: str — the detected sentiment/tone of the email (e.g., “positive”, “negative”, “neutral”, “urgent”).
  • needs_reply: bool — a boolean flag indicating whether the email requires a response.
  • reply: str — the generated reply text (populated only if needs_reply is True).

Why use TypedDict for agent state?

• Type safety: Provides autocomplete and type checking in IDEs and linters.
• Documentation: Self-documents the expected structure of the state as it flows through the agent graph.
• LangGraph compatibility: LangGraph uses TypedDict schemas to validate state transitions and ensure all nodes produce/consume the correct keys.
• Debugging: Makes it easier to catch missing or incorrectly typed fields during development.

Usage notes

• This state schema will be passed to LangGraph’s StateGraph constructor (in a later cell) to define the agent’s data flow.
• Each node in the graph will receive the current EmailState as input and return an updated EmailState (or a partial update dict).
• Nodes can read any field and update one or more fields; LangGraph merges updates automatically.

Expected workflow (in subsequent cells)

1. Summarizer node — reads email text, writes summary.
2. Sentiment analyzer node — reads summary, writes sentiment.
3. Reply decision node — reads summary and sentiment, writes needs_reply.
4. Reply generator node — (conditional) reads summary and sentiment, writes reply if needs_reply is True.

Next steps

• In the following cells, we will define the individual node functions (each accepting and returning EmailState) and wire them together into a LangGraph workflow.

Build email processing agent workflow with LangGraph¶

Purpose

• Define a multi-step agent workflow that automatically processes incoming emails by summarizing, analyzing sentiment, and generating replies using a local LLM.

Prerequisites

• EmailState — TypedDict schema defined in the previous cell with fields: email_text, summary, sentiment, needs_reply, reply.
• local_llm — ChatOllama instance configured earlier for text generation.
• StateGraph, START, END from langgraph.graph must be imported (happens at the top of this cell).

What the code does (high level)

1. Define three node functions that each process one step of the email workflow:

   • summarize_email(state) — reads email_text, generates a summary, writes to state["summary"].
   • analyze_sentiment(state) — reads summary, classifies sentiment (positive/negative/neutral), writes to state["sentiment"].
   • generate_reply(state) — reads email_text, generates a reply, writes to state["reply"].

2. Build a StateGraph:

   • email_workflow = StateGraph(EmailState) — creates a graph that uses EmailState as its state schema.

3. Add nodes to the graph:

   • Three nodes are registered, each wrapping one of the node functions.

4. Define the workflow execution order with edges:

   • START → summarize_email → analyze_sentiment → generate_reply → END
   • This creates a linear pipeline where each step depends on the output of the previous step.

5. Compile the graph:

   • email_graph = email_workflow.compile() — produces a runnable, executable graph that can be invoked with initial state.

Line-by-line explanation

Node functions

• def summarize_email(state): — node function that modifies state in-place.

  • email_text = state["email_text"] — reads the raw email content from state.
  • prompt = f"Summarize the following email message: \n{email_text}" — constructs prompt for LLM.
  • response = local_llm.invoke(prompt) — calls the local Ollama model synchronously.
  • state["summary"] = response.content — writes the generated summary back into state (no return needed; LangGraph reads the modified state).

• def analyze_sentiment(state): — sentiment classification node.

  • summary = state["summary"] — reads the summary produced by the previous node.
  • prompt = f"Look at the following email summary and determine the sentiment as positive, negative, or neutral: \n{summary}\nOnly respond with one word: positive, negative, or neutral." — instructs LLM to output a single sentiment label.
  • response = local_llm.invoke(prompt) — generates sentiment classification.
  • state["sentiment"] = response.content.strip().lower() — normalizes and writes sentiment to state.

• def generate_reply(state): — reply generation node.

  • email_text = state["email_text"] — reads the original email text.
  • prompt = f"Generate a reply to the following email: \n{email_text}" — asks LLM to draft a response.
  • response = local_llm.invoke(prompt) — generates reply text.
  • state["reply"] = response.content — writes reply into state.

Graph construction

• email_workflow = StateGraph(EmailState) — creates a new state graph using EmailState as the schema; all state updates must conform to this type.
• email_workflow.add_node("summarize_email", summarize_email) — registers the summarize function as a node named "summarize_email".
• email_workflow.add_node("analyze_sentiment", analyze_sentiment) — registers sentiment analysis node.
• email_workflow.add_node("generate_reply", generate_reply) — registers reply generation node.

Edge definitions (workflow order)

• email_workflow.add_edge(START, "summarize_email") — entry point: workflow starts by running summarize_email.
• email_workflow.add_edge("summarize_email", "analyze_sentiment") — after summarization, run sentiment analysis.
• email_workflow.add_edge("analyze_sentiment", "generate_reply") — after sentiment analysis, generate a reply.
• email_workflow.add_edge("generate_reply", END) — after reply generation, workflow terminates.

Compilation

• email_graph = email_workflow.compile() — compiles the graph into an executable runnable. The compiled graph validates the state schema, checks for cycles, and prepares the execution plan.

Variables created

• summarize_email, analyze_sentiment, generate_reply — node functions (callables that accept and modify EmailState).
• email_workflow — StateGraph instance (builder object for the workflow).
• email_graph — CompiledStateGraph (executable runnable that can be invoked with initial state).

Expected behavior

• Given an input email, the graph will:
  1. Generate a concise summary.
  2. Classify the sentiment of that summary.
  3. Draft a reply to the original email.
• All three outputs (summary, sentiment, reply) will be available in the returned EmailState dict.

Run email processing agent on sample emails and display results¶

Purpose

• Invoke the compiled email processing agent (email_graph) on four sample emails with varying sentiments and display the generated summary, sentiment classification, and reply for each.

Prerequisites

• email_graph — the compiled LangGraph workflow defined in the previous cell.
• email_examples — a dictionary of four sample emails.

What this cell does

1. Define sample email dataset:

   • Creates email_examples dictionary containing four realistic email scenarios:
     • email_1 — Negative/urgent: missed deliverables and accountability request
     • email_2 — Negative/frustrated: feedback on unprepared meeting
     • email_3 — Positive/collaborative: follow-up on successful brainstorming
     • email_4 — Positive/appreciative: praise for smooth product launch

2. Process each email through the agent workflow:

   • Iterates over each email in email_examples.items().
   • Invokes email_graph.invoke({"email_text": example}) for each email.
   • The graph runs all three nodes (summarize → sentiment → reply) and returns the updated EmailState.

3. Display results:

   • Prints the summary, sentiment, and generated reply for each email in a structured format for easy comparison.

Line-by-line explanation

• email_examples = {...} — dictionary mapping email identifiers (email_1, email_2, etc.) to multiline email text strings.
• for key, example in email_examples.items(): — iterates over each email, unpacking the identifier and text.
• response = email_graph.invoke({"email_text": example}) — runs the full agent workflow:
  • Passes the email text as initial state.
  • Returns a complete EmailState dict with summary, sentiment, and reply populated.
• print(f"{key} Summary:\n{response['summary']}\n") — displays the generated summary for the current email.
• print(f"{key} Sentiment:\n{response['sentiment']}\n") — displays the detected sentiment.
• print(f"{key} Generated Reply:\n{response['reply']}\n") — displays the draft reply.

Expected outputs

• For each of the four emails, the cell will print:
  • Summary: A 1-2 sentence concise summary of the email content.
  • Sentiment: One word — “positive”, “negative”, or “neutral”.
  • Reply: A contextually appropriate draft response generated by the LLM.

Usage notes and insights

• Agent performance evaluation: By running the workflow on multiple emails with different tones, you can evaluate how well the LLM:
  • Captures key information in summaries.
  • Detects subtle sentiment cues (e.g., frustration vs. disappointment vs. enthusiasm).
  • Generates contextually appropriate, professional replies.
• Sentiment classification: The agent should correctly identify:
  • email_1, email_2 as negative (urgent/frustrated tone).
  • email_3, email_4 as positive (collaborative/appreciative tone).
• Reply quality: Generated replies should:
  • Match the tone of the original email (e.g., apologetic/action-oriented for negative emails, warm/encouraging for positive ones).
  • Address specific points raised in the email.
  • Maintain professionalism and clarity.

Add conditional reply logic to email processing workflow¶

Purpose

• Extend the email processing agent to intelligently decide whether an email requires a reply before generating one, reducing unnecessary reply generation for informational or appreciation-only emails.

Prerequisites

• EmailState — TypedDict schema with email_text, summary, sentiment, needs_reply, and reply fields.
• local_llm — ChatOllama instance for text generation.
• email_workflow, summarize_email, analyze_sentiment, generate_reply — previously defined workflow components.
• StateGraph, START, END — LangGraph imports.

What this cell does (high level)

1. Define a new decision node (needs_reply) that uses the LLM to determine if a reply is necessary based on the email summary and sentiment.
2. Define a conditional routing function (needs_reply_condition) that directs the workflow to either generate a reply or skip directly to END.
3. Rebuild the workflow with the new decision node and conditional edge.
4. Recompile and visualize the updated workflow graph.

Line-by-line explanation

Decision node function

• def needs_reply(state): — node function that determines if the email requires a response.
  • summary = state["summary"] — reads the email summary from state.
  • sentiment = state["sentiment"] — reads the detected sentiment.
  • prompt = f"Based on the following email summary: '{summary}' and its sentiment: '{sentiment}', determine if a reply is necessary. Respond only with 'yes' or 'no'." — instructs LLM to make a binary decision.
  • response = local_llm.invoke(prompt) — calls the local LLM to evaluate whether a reply is needed.
  • return {"needs_reply": response.content.strip().lower() == "yes"} — returns a boolean: True if LLM says “yes”, False otherwise.

Conditional routing function

• def needs_reply_condition(state: EmailState): — routing logic that determines the next node based on state.
  • if state["needs_reply"]: — checks the boolean flag set by the needs_reply node.
  • return ["generate_reply"] — if True, route to the reply generation node.
  • else: return END — if False, skip reply generation and end the workflow.

Workflow reconstruction

• email_workflow = StateGraph(EmailState) — creates a fresh StateGraph instance.
• Four nodes are added:
  • "summarize_email" — generates email summary.
  • "analyze_sentiment" — classifies sentiment.
  • "needs_reply" — NEW: decides if reply is necessary.
  • "generate_reply" — generates reply text (now conditionally executed).

Edge definitions (updated workflow)

• email_workflow.add_edge(START, "summarize_email") — entry point: run summarization.
• email_workflow.add_edge(START, "analyze_sentiment") — also run sentiment analysis in parallel.
• email_workflow.add_edge(["summarize_email", "analyze_sentiment"], "needs_reply") — after both summarization and sentiment analysis complete, run the decision node.
• email_workflow.add_edge("generate_reply", END) — if reply is generated, end workflow.
• email_workflow.add_conditional_edges("needs_reply", needs_reply_condition, ["generate_reply", END]) — conditional branching: from needs_reply, route to either generate_reply OR directly to END based on the needs_reply_condition function output.

Compilation and visualization

• email_graph = email_workflow.compile() — recompiles the graph with the new conditional logic.
• display(Image(email_graph.get_graph().draw_mermaid_png())) — renders the workflow graph as a Mermaid diagram, showing the conditional branch visually.

Expected behavior changes

• Before: All emails always get a reply generated.
• After: The agent now intelligently skips reply generation for emails that don’t require a response (e.g., thank-you notes, FYI messages, or positive acknowledgments).
• The needs_reply field in the returned EmailState will be True or False depending on the LLM’s assessment.
• The reply field will only be populated if needs_reply is True.

Use cases and benefits

• Efficiency: Reduces unnecessary LLM invocations by skipping reply generation when not needed.
• Realism: Mirrors human email triage behavior — not every email warrants a response.
• Customization: The decision logic can be refined by adjusting the prompt to the needs_reply node (e.g., considering urgency, sender role, or organizational policies).

Test conditional email workflow on all sample emails¶

Purpose

• Execute the updated email processing agent (with conditional reply logic) on all four sample emails and display the results, including whether each email triggered reply generation.

Prerequisites

• email_graph — the recompiled LangGraph workflow with conditional reply logic (defined in the previous cell).
• email_examples — dictionary of four sample emails with varying sentiments and reply requirements.
• EmailState — state schema including the needs_reply boolean field.

What this cell does

1. Iterate over all sample emails:

   • Loops through each email in email_examples.items(), unpacking the identifier (key) and email text (example).

2. Invoke the conditional workflow:

   • Calls email_graph.invoke({"email_text": example}) for each email.
   • The workflow now runs:
     • Summary generation (in parallel with sentiment analysis)
     • Sentiment classification (in parallel with summary)
     • Reply necessity decision (needs_reply node)
     • Conditionally generates reply only if needs_reply is True

3. Display comprehensive results:

   • Prints the summary, sentiment, and needs_reply decision for every email.
   • Conditionally prints the generated reply: only displays reply text if needs_reply is True; otherwise shows “No reply generated.”

Expected outputs and behavior

• email_1 (missed deliverables, negative/urgent):

  • needs_reply: True — requires acknowledgment and action plan.
  • Reply generated: Yes (apologetic, action-oriented).

• email_2 (unprepared meeting feedback, negative/frustrated):

  • needs_reply: True — requires response to address concerns.
  • Reply generated: Yes (acknowledges issue, commits to improvement).

• email_3 (positive brainstorming follow-up, collaborative):

  • needs_reply: Likely True — contains a request (send top three priorities by tomorrow).
  • Reply generated: Yes (confirms receipt, commits to deadline).

• email_4 (appreciation for smooth launch, positive):

  • needs_reply: Likely False — pure appreciation message, no action or question.
  • Reply generated: No — workflow skips generate_reply node and prints “No reply generated.”

Key insights and validation

• Workflow correctness: This cell validates that the conditional logic works as intended:

  • Emails requiring responses (questions, requests, negative feedback) trigger reply generation.
  • Emails that are purely informational or appreciative skip reply generation, saving compute and avoiding unnecessary responses.

• Agent intelligence: The needs_reply decision demonstrates the agent’s ability to triage emails realistically, mirroring human judgment about when a response is appropriate.

• Cost efficiency: By conditionally skipping reply generation, the agent reduces LLM invocations by 25-50% (depending on email mix), directly lowering API costs and latency.

Usage notes

• Debugging unexpected decisions: If needs_reply produces unexpected results, inspect the LLM’s reasoning by printing the raw response.content from the needs_reply node before converting to boolean.

• Handling missing replies: The conditional print statement safely handles cases where response['reply'] is not populated (when needs_reply is False), preventing KeyError exceptions.

Add error handling, logging, and retry logic to email workflow¶

Purpose

• Enhance the email processing agent with production-grade resilience by adding error handling, logging, and automatic retry logic to all node functions and the LLM client.

Prerequisites

• logging — Python’s standard logging module (imported in the cell).

What this cell does (high level)

1. Configure logging: Creates a logger instance to record errors and debug information.
2. Add retry logic to the LLM client: Wraps the local LLM with automatic retry behavior for transient failures.
3. Rewrite all node functions with error handling: Each node now catches exceptions, logs errors, and returns safe fallback values instead of crashing.
4. Rebuild and recompile the workflow: Reconstructs the workflow graph with the hardened node functions.
5. Visualize the updated workflow: Displays the graph diagram to confirm structure.

Line-by-line explanation

Logging setup

• logger = logging.getLogger() — creates a root logger instance for recording errors and warnings.

LLM client with retry logic

• local_llm = ChatOllama(model = "qwen3-vl:8b") — instantiates the local Ollama chat model.
• local_llm = local_llm.with_retry(...) — wraps the LLM client with automatic retry behavior:
  • retry_if_exception_type = (ValueError, RuntimeError) — only retry on these exception types (e.g., transient network errors, model loading failures).
  • wait_exponential_jitter = True — use exponential backoff with random jitter between retries to avoid thundering herd problems.
  • stop_after_attempt = 3 — give up after 3 failed attempts and raise the exception.

Hardened node functions

Each node function now follows this pattern:

• Wrap the core logic in a try block
• Catch Exception to handle any error
• Log the error with logger.error(...) for debugging and monitoring
• Return a safe fallback value that allows the workflow to continue gracefully

summarize_email(state) — with error handling

• try: — attempts to generate email summary.
• except Exception as e: — catches any error (network timeout, model crash, etc.).
• logger.error(f"Error in summarize_email: {e}") — logs the error with context.
• return {"summary": "Error generating summary."} — returns a fallback summary so downstream nodes can still execute.

analyze_sentiment(state) — with error handling

• Same pattern as summarize_email.
• Fallback: return {"sentiment": "Error analyzing sentiment."}

needs_reply(state) — with error handling

• Same pattern.
• Fallback: return {"needs_reply": False} — defaults to “no reply needed” to avoid generating unnecessary replies on errors.

generate_reply(state) — with error handling

• Same pattern.
• Fallback: return {"reply": "Error generating reply."} — produces an error message as the reply content.

needs_reply_condition(state) — routing logic (unchanged)

• No error handling needed here since it’s just reading a boolean from state and making a routing decision.

Workflow reconstruction

• The workflow is rebuilt identically to the previous version, but now uses the hardened node functions.
• Graph structure remains the same:
  • Parallel execution of summarize_email and analyze_sentiment
  • Decision node needs_reply
  • Conditional edge to either generate_reply or END

Expected behavior and benefits

Before (without error handling)

• If any LLM call fails (network timeout, rate limit, model crash), the entire workflow crashes with an unhandled exception.
• No visibility into which node failed or why.
• Requires manual restart and debugging.

After (with error handling + retry + logging)

• Automatic retries: Transient errors (network glitches, temporary model unavailability) are retried up to 3 times with exponential backoff.
• Graceful degradation: If retries fail, nodes return safe fallback values (error messages) and the workflow continues, allowing partial results to be returned.
• Observability: All errors are logged with context, making it easy to diagnose issues in production.
• Production readiness: The agent can now handle real-world failures (API rate limits, network issues, model timeouts) without crashing.

Usage notes and best practices

Logging configuration

• In production, configure the logger with a proper handler and level:

  logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
  

• This will write timestamped logs to stdout for monitoring and debugging.

Retry configuration tuning

• Adjust stop_after_attempt based on expected failure rates and latency tolerance.
• For highly reliable APIs, 2 attempts may suffice; for flaky local models, 5+ may be needed.
• wait_exponential_jitter prevents retry storms when many requests fail simultaneously.

Fallback value design

• Fallback values should be chosen to minimize downstream errors:
  • summary: “Error generating summary.” — clearly indicates failure but doesn’t break sentiment analysis.
  • sentiment: “Error analyzing sentiment.” — downstream logic should handle non-standard sentiment values.
  • needs_reply: False — conservative default; avoids generating potentially incorrect replies.
  • reply: “Error generating reply.” — makes the error visible to the user.

Error handling scope

• The current implementation catches all exceptions with Exception.
• For finer control, catch specific exception types (e.g., NetworkError, TimeoutError, ModelNotFoundError) and handle them differently.

Test hardened email workflow with error handling¶

Purpose

• Execute the updated email processing agent (now with error handling, retry logic, and logging) on all four sample emails to validate that the resilience improvements work correctly and the workflow produces expected results.

Prerequisites

• email_graph — the recompiled LangGraph workflow with error-handling-enhanced node functions (defined in the previous cell).
• email_examples — dictionary of four sample emails with varying sentiments and reply requirements.
• EmailState — state schema including email_text, summary, sentiment, needs_reply, and reply fields.

What this cell does

1. Iterate over all sample emails:

   • Loops through each email in email_examples.items(), unpacking the identifier (key) and email text (example).

2. Invoke the hardened workflow:

   • Calls email_graph.invoke({"email_text": example}) for each email.
   • The workflow now includes:
     • Automatic retry logic on transient LLM failures
     • Error logging for debugging
     • Graceful fallback values if all retries fail
   • Executes the full pipeline: summary → sentiment → needs_reply decision → conditional reply generation

3. Display comprehensive results:

   • Prints the summary, sentiment, and needs_reply decision for every email.
   • Conditionally prints the generated reply:
     • If needs_reply is True, displays the reply content.
     • If needs_reply is False, displays “No reply generated.”

Expected behavior and validation

Normal operation (no errors)

• All four emails should process successfully:
  • email_1 (missed deliverables): needs_reply=True, reply generated
  • email_2 (meeting feedback): needs_reply=True, reply generated
  • email_3 (brainstorming follow-up): needs_reply=True (contains request), reply generated
  • email_4 (appreciation): needs_reply=False, no reply generated

Error scenarios (if LLM fails)

• If the LLM fails after retries, the workflow returns fallback values:
  • summary: “Error generating summary.”
  • sentiment: “Error analyzing sentiment.”
  • needs_reply: False (safe default)
  • reply: Not generated (since needs_reply is False)
• The workflow completes without crashing, and errors are logged for investigation.

Differences from previous version

• Identical user-facing behavior: The output format and logic are the same as the previous version.

• Enhanced reliability: The workflow now handles failures gracefully instead of crashing, making it suitable for production use.

• Observability: Errors are logged to the logger, allowing monitoring and debugging of failures.

• Testing error handling: To test the error handling logic, you can temporarily:

  • Stop the Ollama server to trigger connection errors
  • Use an invalid model name to trigger model loading errors
  • Set stop_after_attempt=1 to see fallback behavior more frequently

from langchain_core.messages import trim_messages from langchain_openai import ChatOpenAI

Chat Memory Management with Message Trimming¶

Purpose

• Demonstrate LangChain’s chat memory management capabilities using InMemoryChatMessageHistory and trim_messages to maintain conversational context while limiting memory usage.

Prerequisites

• InMemoryChatMessageHistory — in-memory storage for chat message history per session.
• trim_messages — utility function to limit the number of messages retained in conversation history.
• RunnableWithMessageHistory — wrapper that automatically manages chat history for a runnable chain.
• BaseCallbackHandler — base class for creating custom callbacks to monitor chain execution.
• local_llm — ChatOllama instance (qwen3-vl:8b) for generating responses.

What this cell does (high level)

1. Instantiate the local LLM: Creates a ChatOllama client for conversational AI.
2. Define a custom callback: Creates InputMessagesCallback to monitor and log the number of messages sent to the LLM on each invocation.
3. Set up session-based chat history storage: Creates a dictionary to store separate conversation histories for different users/sessions.
4. Configure message trimming: Sets up a trimmer that keeps only the last 3 messages to prevent unbounded memory growth.
5. Build a conversational chain: Composes the message trimmer and LLM into a chain that automatically manages conversation history.

Line-by-line explanation

LLM instantiation

• local_llm = ChatOllama(model = "qwen3-vl:8b") — creates the local Ollama chat model instance that will generate responses.

Custom callback for monitoring

• class InputMessagesCallback(BaseCallbackHandler): — defines a custom callback handler that extends LangChain’s base callback class.
• def on_chat_model_start(self, serialized, messages, **kwargs): — callback method that executes before the chat model processes input.
• print(f"Number of input messages to llm: {len(messages[0])}") — logs the count of messages being sent to the LLM, useful for debugging and understanding context window usage.

Session-based history management

• sessions = {} — dictionary that maps session IDs to their respective chat history objects, enabling multi-user conversation tracking.
• def get_session_history(session_id: str): — factory function that retrieves or creates a chat history for a given session.
• if session_id not in sessions: — checks if this is a new session.
• sessions[session_id] = InMemoryChatMessageHistory() — creates a new in-memory history store for the session.
• return sessions[session_id] — returns the history object for the session.

Message trimming configuration

• messages_trimmer = trim_messages(...) — creates a trimmer that limits conversation history to prevent memory bloat and token limit issues.
• max_tokens = 3 — despite the parameter name, with token_counter=len, this actually means “keep the last 3 messages” (not 3 tokens).
• strategy = "last" — keeps the most recent messages and discards older ones.
• token_counter = len — important: uses Python’s len() function to count messages (not actual tokens), so max_tokens=3 means 3 messages.
• include_system = True — system messages (if present) are counted toward the 3-message limit.
• start_on = "human" — when trimming, prefer to start the retained history with a human message (maintains conversation flow).

Conversational chain composition

• chain = RunnableWithMessageHistory(messages_trimmer | local_llm, get_session_history) — creates a stateful conversational chain:
  • messages_trimmer | local_llm — LCEL composition: first trim messages, then send to LLM.
  • get_session_history — callback function to retrieve/store history per session.
  • RunnableWithMessageHistory wrapper automatically:
    • Retrieves conversation history for the session before each invocation.
    • Appends the new user message and LLM response to the history after each invocation.
    • Applies the trimmer to limit history size.

Key concepts and benefits

Why use message trimming?

• Token limits: LLMs have maximum context window sizes (e.g., 4k, 8k, 128k tokens). Long conversations can exceed these limits.
• Cost control: More messages = more tokens = higher API costs.
• Latency: Larger context windows increase inference time.
• Memory management: Unbounded history growth can cause memory issues in long-running applications.

Message counting vs. token counting

• The code uses token_counter=len, which counts messages, not actual tokens.
• To count actual tokens, replace len with a tokenizer function:

  messages_trimmer = trim_messages(
      max_tokens=1000,  # Now actually means 1000 tokens
      strategy="last",
      token_counter=ChatOpenAI().get_num_tokens_from_messages,
      include_system=True
  )
  

Session isolation

• Each session_id gets its own independent conversation history.
• Useful for multi-user applications, parallel conversations, or A/B testing different conversation flows.

Callback usage

• The InputMessagesCallback helps debug and monitor how many messages are being sent to the LLM after trimming.
• In production, callbacks can be used for logging, metrics collection, cost tracking, or request tracing.

Expected behavior (demonstrated in the next cell)

• The next cell will invoke the chain multiple times with a session ID.
• After each invocation, the callback will print the number of messages sent to the LLM.
• Once the conversation exceeds 3 messages, older messages will be automatically trimmed, keeping only the last 3 (most recent) messages.
• The conversation history will be maintained across invocations within the same session, allowing the LLM to reference previous exchanges.

Usage notes and best practices

Choosing max_tokens value

• For short-term memory (focused conversations): 3-5 messages
• For medium-term memory (task-oriented conversations): 10-20 messages
• For long-term memory (comprehensive context): 50-100 messages or token-based counting with a large limit

Trimming strategies

• "last" — keeps most recent messages (best for ongoing conversations)
• "first" — keeps oldest messages (useful for preserving initial context/instructions)

System message handling

• If you have a system message that sets assistant behavior, set include_system=True to ensure it’s always retained (if within the message limit).