How Pocket Flow, an LLM Framework, Needs Only 100 Lines, Its Core Principles, and Applications

Zachary Huang

Introduction to Pocket Flow: Minimalism in LLM Frameworks [0:00]

The video argues that building AI applications doesn't require massive, convoluted systems or extensive documentation, and that many existing LLM frameworks are unnecessarily complex.

• The Problem with Existing Frameworks:
  • Require reading textbook-like documentation.
  • Feature confusing "spiderweb diagrams."
  • Lead to "dependency hell."

    

• The Solution: Pocket Flow [0:50]
  • A minimalist LLM framework.
  • Built on just 100 lines of code.
  • Focuses on stripping away non-essential components to master fundamentals.
  • Popularity indicated by rapidly growing GitHub stars.
• Core Philosophy: Zero dependencies and a tiny codebase (100 lines, 56KB) compared to hundreds of megabytes and tangled codebases of competitors (e.g., LangChain, CrewAI, SmolAgent, LangGraph, AutoGen).

  

• Underlying Principle: All LLM applications are essentially simple graphs.
  • Core abstractions (Node, Batch, Flow, Async, Branch, Looping, Nesting) can build complex patterns (Workflow, Chat, RAG, Map-Reduce, Chat Memory, Agent, Multi-Agents, Supervisor).

    

    

• Learning Approach: The video promises a masterclass in first principles, teaching how agents and RAG truly work from the ground up, requiring only basic Python knowledge.

The 3 Core Ideas of Pocket Flow [3:03]

The entire Pocket Flow framework is built upon three fundamental concepts: the Node, the Shared Store, and the Flow.

1. The Node: The Reliable Worker [3:26]

• Definition: The smallest, most basic unit of work. Analogous to a hyper-focused worker at a single station on an assembly line.
• Three-Step Routine:
  • Prep: "Prepare" phase; reads specific data from the Shared Store needed for its job. [3:53]
  • Exec: "Execute" phase; performs the core task in isolation, receiving only the data from prep. This enforces a clean separation of data-fetching and computation logic, making code clean and testable (e.g., performing math, calling APIs, prompting LLMs). [4:14]
  • Post: "Clean up and serve" phase; writes results back to the Shared Store and returns an "action" string to guide the workflow (e.g., "normal," "error"). [4:45]

    

• Built-in Fault Tolerance:
  • Nodes include retry logic (max_retries, wait) that wraps the exec code in a loop. [5:59]
  • If execution fails after all retries, exec_fallback is called, allowing graceful error handling (e.g., logging, returning default values).
  • Example: SummarizeFile Node:
    • prep: Gets file_content from shared. [7:07]
    • exec: Calls LLM to summarize file_content. [7:22]
    • exec_fallback: Handles LLM call failures (e.g., prints error, returns a friendly message). [7:42]
    • post: Saves the summary to shared['summary'] and returns "next". [8:00]

2. The Shared Store: The Central Whiteboard [8:50]

• Definition: A simple Python dictionary that all Nodes in the system can access. [9:03]
• Functionality:
  • Facilitates communication between Nodes (Nodes do not talk directly).
  • prep functions read from it, post functions write to it.
  • Serves as the single source of truth for the entire workflow.
• Benefits: Beautifully simple, incredibly easy to debug (can print the dictionary state at any time).

3. The Flow: The Master Recipe [9:50]

• Definition: The orchestrator that tells each worker (Node) what to do and where to go next. [9:59]
• Connecting Nodes:
  • Simple transitions: node_a >> node_b (after node_a is done, go to node_b). [10:04]
  • Named transitions for branching logic: review_node - "approved" >> payment_node (uses the "action" string returned by the post method).

    

• Implementation: Each Node has a self.successors address book; connecting nodes adds entries to this book. The Flow orchestrator uses the action string from a node's post method to look up the next node in the address book. [10:38]
• Flow is a Node: A Flow itself can be bundled and treated as a single Node. [11:09]
  • Allows for powerful nested workflows, building complex applications from small, reusable parts without losing simplicity.
  • Flows can connect to other nodes or other flows.

    

LLM Integration: No Vendor Lock-in [11:43]

• Philosophy: Pocket Flow deliberately avoids built-in "wrappers" for specific LLM APIs (OpenAI, Anthropic, Google).
  • Avoids vendor lock-in.
  • Prevents maintenance nightmares (APIs change, wrappers become outdated).
• Approach: Users write their own simple utility functions (e.g., 10 lines of code) to call desired LLM APIs.
  • Provides complete control and flexibility to switch models easily.

Building Applications with Pocket Flow

Chatbot: A Self-Looping Node [13:20]

• Structure: A chatbot is fundamentally one ChatNode talking to itself in an infinite loop, with the conversation ending when the user exits.

  

• ChatNode Implementation:
  • prep: Initializes messages history in the shared store (if first run), prompts user for input, adds user message to history, returns messages for LLM. [13:47]
  • exec: Passes the message list to call_llm function. [14:12]
  • post: Prints LLM response, appends assistant's response to messages history in shared store, returns "continue" action to self-loop. [14:27]
• Conversation Walkthrough:
  • Initial state: Empty shared store.
  • prep: User input added to shared['messages'].
  • exec: LLM processes messages (shared store unchanged).
  • post: LLM response added to shared['messages'], ensures context for next turn.

    

• Key takeaway: A chatbot is 1 Node + 1 Loop + 1 Dictionary. [17:15]

  

Structured Output: Robust Data Extraction [17:35]

• Challenge: LLMs often provide rambling, unstructured text instead of clean, machine-readable data, making parsing brittle.
• Solution: Explicitly ask the AI for the desired format.
• Pro Tip: Use YAML, Not JSON. [18:54]
  • JSON is fragile due to strict escaping requirements for quotes and newlines, which LLMs struggle with.
  • YAML is more human-readable and robust, handling strings and newlines gracefully.

    

• Building a Resume Parser Node:
  • Prompt Design: Give the LLM a clear instruction ("Output ONLY the requested information in YAML format") and a perfect few-shot example of the desired YAML structure. [20:32]

    

  • Parsing and Validation: The exec method attempts to parse the LLM's YAML response. If invalid, the Node's retry logic triggers, giving the LLM another chance.
  • Output: The post step saves the clean, structured Python dictionary to the shared store.

    

Batch Processing: Handling Bulk Data [22:23]

• Challenge: Processing thousands of items (e.g., resumes) one by one is inefficient.
• Solution: The BatchNode: A special Node designed for bulk operations.
  • Differences from Standard Node:
    • prep: Returns an iterable (e.g., a list) of all items to be processed.
    • exec: Called once for each item in the list sequentially.
    • post: Receives a list of all individual results from exec calls.

      

• Implementation: The BatchNode's _exec method is simply a for loop that calls the standard _exec method for each item. It's "dumb simple" but effective for organizing bulk data. [24:04]
• Batch Node Walkthrough: prep reads all resumes, _exec loop processes each sequentially, post receives all results.

  

Parallel Processing: Unleashing Speed with Async [26:52]

• Problem with BatchNode: Still sequential (e.g., 10 resumes * 10 seconds each = 100 seconds). The "villain" is waiting during API calls.
• Solution: Asynchronous programming using Python's async and await keywords.
  • Smart Chef Analogy: A smart chef starts multiple tasks concurrently, making progress on one while another is waiting (e.g., cooking toast while eggs cook). [27:33]

    

• Asynchronous LLM Calls:
  • Convert call_llm to call_llm_async by using an async-compatible library (e.g., AsyncOpenAI) and await the API call. [29:44]

    

• Upgrading the Batch Node to Parallel:
  • Change parent class from BatchNode to AsyncParallelBatchNode. [30:43]
  • Add async keyword to prep, exec, post methods (e.g., prep_async).
  • Inside exec_async, use await call_llm_async.

    

• Implementation: The AsyncParallelBatchNode's _exec method replaces the simple for loop with await asyncio.gather(*tasks). This allows all tasks to be fired off concurrently. [31:21]

  

• Performance Payoff: Processing 10 resumes (10 seconds each) drops from 100 seconds to just over 10 seconds (time of the longest task). [32:04]

  

• Caveats: Be mindful of API rate limits and ensure tasks are independent.

Multi-Step Workflows: Task Decomposition for Quality [32:55]

• Problem: Asking a single LLM call to perform a complex task (e.g., write a comprehensive blog post) often results in generic, low-quality output.
• Solution: Task Decomposition. Break down a big task into a series of smaller, focused steps (workflow).
• Example: Article Writing Workflow: A three-station assembly line.
  • Outliner Node: Takes a topic, creates a structured list of section titles.
  • BatchDrafter Node: Receives the list of titles and drafts all sections simultaneously (leveraging parallel processing).
  • Combiner Node: Takes all drafted sections and assembles them into a polished, coherent article.

    

• Workflow Walkthrough:
  • Initial shared: {"topic": "AI Safety"}.
  • Outliner runs: shared now includes section_titles (e.g., ["Introduction", "Key Risks", "Solutions"]). [35:40]
  • BatchDrafter runs: shared now includes section_drafts (drafted content for each section). [35:52]
  • Combiner runs: shared now includes final_article. [36:04]

    

• Result: By breaking one complex request into three simple requests, the output quality is significantly higher. [36:17]

Agent: A Flow with a Loop and a Branch [36:44]

• Demystifying Agents: Agents are not magic or sentient; they are a design pattern, specifically a flow with a loop and a branch. [37:57]

  

• Agent Blueprint: A central "thinking" node (DecideNode) that determines the next step based on a goal and current context, outputting a simple "action" command.

  

• Building a Search Agent:
  • DecideAction Node (The Brain):
    • prep: Gathers question and existing context.
    • exec: Prompts LLM to choose between 'search' or 'answer' based on context, returning a structured YAML decision.
    • post: Parses LLM's decision; if 'search', sets search_query in shared and returns 'search' action; if 'answer', returns 'answer' action. [38:47]
  • SearchWeb Node (The Tool):
    • prep: Gets search_query.
    • exec: Performs web search (e.g., using Google Search API).
    • post: Adds search results to shared['context'], returns 'decide' action to loop back to DecideAction node. [39:32]
  • DirectAnswer Node (The Exit):
    • prep: Gathers question and all collected context.
    • exec: Prompts LLM to synthesize a final answer using the research.
    • post: Delivers the answer to the user, returns 'done' action to end the mission. [40:00]
• Live Run Walkthrough (Who won 2024 Physics Nobel Prize?):
  • Initialization: shared contains question. [40:33]
  • Round 1 (Decide): DecideAction runs, LLM decides to search. shared gets search_query. Flow goes to SearchWeb. [40:41]
  • Round 2 (Search): SearchWeb runs, finds results, adds summary to shared['context']. Flow returns 'decide' to DecideAction. [41:07]
  • Round 3 (Decide with Context): DecideAction runs again, sees new context. LLM decides to answer. Flow goes to DirectAnswer. [41:21]
  • Final Step (Answer): DirectAnswer generates and delivers the final answer. Flow completes. [41:54]

    

Conclusion: The Power of Simplicity [42:22]

• Summary of what was built: Chatbots, Batch Processing, Parallel Workflows, and Agents – all from a 100-line foundation.

  

• No Black Boxes: The framework reveals core concepts: the Node, the Shared Store, and the Flow.
• PocketFlow's Real-World Impact: Example project "Codebase Knowledge Builder" (built with PocketFlow) became more popular than the framework itself, demonstrating the power of the simple design.
• Final Reveal: The video's script, structure, and knowledge breakdown were also generated by a PocketFlow agent.