DSPy Masterclass — 5 Real-World Use Cases for AI Engineers

ZazenCodes

Introduction to DSPy: A Higher-Level Language for AI Programming [0:00]

This video introduces DSPy as a declarative framework designed for AI software, offering a higher-level language for AI programming. The core idea is to move beyond manual text-based prompt engineering towards a more programmatic and structured approach to building complex AI applications. The tutorial specifically uses DSPy version 3.0.1. The presenter encourages viewers to download and run the provided source code to solidify their understanding of the concepts.

1. Structured Output Extraction [3:17]

This section demonstrates how to convert messy, unstructured support emails into structured JSON tickets.

• Defining the Task Signature: [3:38]
  • This image shows the Python code defining the SupportEmail signature using dspy.Signature and instantiating dspy.Predict(SupportEmail) to extract structured information from emails.
  • A Python class SupportEmail is defined using dspy.Signature.
  • It specifies email: str as the input field using dspy.InputField().
  • Output fields are defined with types and descriptions:
    • subject: str = dspy.OutputField(desc="Subject line of the email")
    • priority: Literal['low', 'medium', 'high'] = dspy.OutputField()
    • product: str = dspy.OutputField(desc="The product(s) referenced. Output an empty string if unknown.")
    • negative_sentiment: bool = dspy.OutputField(desc="True/False")
• Instantiating a Prediction Module: [4:09]
  • A dspy.Predict module named extract_ticket is created by passing the SupportEmail signature: extract_ticket = dspy.Predict(SupportEmail).
  • This module is then called with sample email text to perform the extraction.
• Execution and Output: [5:43]
  • The system quickly processes sample emails, outputting structured tickets with fields like subject, priority, product, and negative_sentiment.
  • For example, an email about a "cracked screen" is correctly identified with "high" priority, "AlphaTab II tablet" product, and "True" for negative sentiment.
• Underlying Mechanism and Cost: [9:37]
  • The video shows the actual prompts sent to the OpenAI platform. DSPy automatically constructs a detailed system message that defines the input/output fields, types, and desired structure, along with the user's input email.
  • The output from the LLM also adheres to this structured format, which DSPy then parses into a Python prediction object.
  • Consideration: While simplifying development, DSPy adds extra tokens to the prompts, which can increase API costs. The presenter sets temperature=1 and max_tokens=128000 for the gpt-3.5-turbo model.

2. Chain of Thought [11:30]

This section introduces the "Chain of Thought" reasoning pattern, where the LLM is prompted to explain its thought process step-by-step.

• Goal: Build a financial risk checker that explains why a loan application is approved or rejected.
• Defining the Task Signature: [12:26]
  • The image displays the Python code for the LoanRisk signature and the dspy.ChainOfThought module used to create a financial risk checker that provides step-by-step reasoning.
  • A LoanRisk signature is defined with applicant_profile: str as input.
  • Output fields include loan_risk: Literal['low', 'medium', 'high'] and approved: bool.
• DSPy ChainOfThought Module: [12:50]
  • Instead of dspy.Predict, dspy.ChainOfThought is used: risk_checker = dspy.ChainOfThought(LoanRisk).
  • This explicitly instructs the LLM to provide a reasoning trace before producing the final answer.
• Execution and Reasoning: [13:28]
  • The risk_checker processes a sample applicant profile (Jane Diaz, credit score, income, debt, loan purpose).
  • The output includes a reasoning field with a detailed explanation of the decision-making process, including factors like credit score, debt-to-income ratio, and requested loan amount.
  • The final loan_risk is "high" and approved is "False" for the sample applicant.
• Under the Hood: [13:52]
  • The system message generated by DSPy now explicitly requests a "reasoning" string as one of the output fields, in addition to the loan_risk and approved fields.
  • This demonstrates how ChainOfThought modifies the prompt to elicit step-by-step explanations from the LLM.
• Signature Syntax and Flexibility: [16:03]
  • The video touches upon the flexibility of DSPy signatures, allowing both simple string representations ("applicant_profile -> loan_risk") or explicit field definitions.
  • It highlights a potential pitfall: providing incorrect types (e.g., a string instead of a boolean for an is_male argument) will not necessarily raise a clear validation error immediately, as DSPy tries to accommodate. Explicit field definitions are generally preferred for clarity and validation.

3. RAG Bot [19:09]

This section details building a Retrieval-Augmented Generation (RAG) bot to answer questions based on a provided knowledge base (employee handbook).

• Goal: Implement a RAG system to answer HR-related questions using a short employee handbook.
• Corpus and Embeddings: [19:28]
  • The employee handbook is provided as a list of strings (corpus).
  • SentenceTransformers (specifically "all-MiniLM-L6-v2") is used to generate embeddings for each string in the corpus.
  • dspy.Embedder wraps the SentenceTransformers model to create an embedder.
• Building the Retriever: [20:04]
  • dspy.Retrieve is used with the created embedder and corpus to build a search function.
  • k=3 is set to retrieve the top 3 most relevant passages.
• RAG Module Implementation: [22:09]
  • A custom RAG module is defined, inheriting from dspy.Module.
  • __init__: Initializes a dspy.ChainOfThought module, which will be responsible for generating the answer.
  • forward: Takes a question as input.
    1. It calls search(question) to retrieve relevant ctx_passages from the corpus.
    2. These passages are concatenated into a single context string.
    3. Finally, it calls the internal dspy.ChainOfThought module (self.respond) with the context, question, and an empty answer field to generate the final response.
• Execution: [25:07]
  • This image shows the console output of the RAG bot, providing a detailed reasoning and answer based on the retrieved context for a question about PTO days.
  • A question like "How many PTO days do we get each year?" is posed to the RAG bot.
  • The output provides reasoning based on the retrieved context, correctly stating "Employees receive 18 PTO days each year."

4. ReAct Agent [30:03]

This section focuses on building a ReAct (Reasoning and Acting) agent, which uses internal "thoughts" and external "tools" to solve problems.

• Goal: Create an expense assistant agent that can fetch exchange rates and perform calculations.
• Defining Tools: [30:30]
  • get_exchange_rate(currency_code: str) -> float: A Python function that fetches the USD conversion rate for a given currency code.
  • calculate_expression(expression: str, yolo: bool = False) -> float: A Python function that evaluates a mathematical expression. (Caution: The presenter warns against using eval() in production).
• Wrapping Tools for DSPy: [31:20]
  • dspy.Tool is used to wrap each Python function, providing a name and the function itself. This makes the tools accessible to the DSPy agent.
• Creating the ReAct Agent: [31:45]
  • expense_agent = dspy.ReAct(question=answer, tools=[exchange_tool, calc_tool]) instantiates a dspy.ReAct agent, passing the defined tools.
  • The question -> answer signature defines the agent's primary input and output.
• Execution and Trajectory: [32:46]
  • The image illustrates the ReAct agent's multi-step thought process and tool calls (FX for exchange rate, Calc for calculation) to answer an expense-related query.
  • A complex query is given: "I spent 130 EUR on a client dinner. What is that in USD and is it under the $75 per-person limit?"
  • The agent executes a multi-step "trajectory" of thoughts, tool calls, and observations:
    1. Thought: "I'll fetch the current EUR->USD conversion rate."
    2. Tool Call (FX): Calls get_exchange_rate(currency_code="EUR").
    3. Observation: Receives the rate (e.g., 1 EUR = 1.07 USD).
    4. Thought: "I need to convert 130 EUR to USD and then compare to the $75 limit. I need to know how many people attended."
    5. Tool Call (Calc): Calls calculate_expression(expression="130 * 1.07").
    6. Observation: Result is $139.10.
    7. Thought: The agent then reasons about the $75 limit, considering if the number of people attending is needed. It asks the user to clarify how many people attended the dinner.
  • This demonstrates the agent's ability to break down problems, use tools, and interact with the user for missing information.
• Swapping LLMs: [37:05]
  • This screenshot from the OpenAI platform logs demonstrates the detailed prompt structure generated by DSPy for the ReAct agent, including the trajectory of thoughts and tool calls.
  • The video illustrates that the underlying LLM can be easily swapped (e.g., from OpenAI's gpt-3.5-turbo to Anthropic's claude-instant-1.2) without changing the DSPy agent's logic. This highlights DSPy's model-agnostic nature.

5. Self-Improving Pipeline [38:14]

This final section demonstrates how DSPy can be used to create self-improving pipelines that optimize the performance of other DSPy modules.

• Goal: Optimize the RAG bot from Use Case 3 using a Bayesian optimization technique.
• Dataset and Evaluation: [39:07]
  • The image displays the raw_eval dataset of question-answer pairs used to evaluate the RAG bot's initial performance, showing a baseline accuracy of 0/4.
  • A "raw_eval" dataset of question-answer pairs is used.
  • An evaluate_bot helper function runs the RAG bot on this dataset and calculates accuracy by comparing predicted answers to gold answers.
• DSPy Optimizer (MIPROv2): [39:26]
  • optimizer = dspy.teleprompt.MIPRO(metric=exact_match) initializes MIPROv2, a Bayesian optimization algorithm designed for instruction optimization.
  • num_trials=10, num_bootstrap_demos=6, and max_bootstrapped_demos=4 are set for the optimization process.
• Optimization Process: [40:40]
  • This image provides a textual explanation of DSPy's MIPROv2 optimization process, detailing steps like bootstrapping few-shot examples, proposing instruction candidates, and optimizing their combination.
  • Step 1: Bootstrap Few-Shot Examples: MIPROv2 samples examples from the training set and uses them to create few-shot example candidates for the LLM.
  • Step 2: Propose Instruction Candidates: Generates candidate instructions (prompts) for the LLM based on the training dataset and previously bootstrapped examples.
  • Step 3: Optimized Combination: Uses Bayesian optimization to select the best combination of few-shot examples and instructions for the LLM program.
  • Iteration: This process runs for num_trials, evaluating new sets of prompts against a validation set at each trial.
• Results and Impact: [49:49]
  • The image shows the console output after optimization, where the optimized_bot achieves a perfect 4/4 accuracy, along with the sophisticated, automatically generated 'proposed instruction' for the RAG bot.
  • The initial (baseline) RAG bot has an accuracy of 0/4 on the small dataset.
  • After running the optimization process, the optimized_bot shows a perfect accuracy of 4/4.
  • The generated "proposed instruction" (the new optimized prompt) is significantly more detailed and includes explicit instructions for step-by-step reasoning. This highlights how DSPy can automatically fine-tune prompts to achieve better performance, even generating sophisticated reasoning processes.
  • Cost & Maintainability: The optimization process is computationally intensive and incurs API costs. The highly specific and complex nature of the optimized prompt, while effective, underscores the challenge of manual prompt engineering and the benefit of automated optimization. The generated prompt for the RAG bot is tailored to produce succinct answers, demonstrating how DSPy enables control over LLM output characteristics.