SGR - Adaptive Planning

How can we enable AI agents to navigate uncertainty and adapt their plans to new circumstances?

Let me illustrate with a few more examples how the reasoning logic in SGR Demo adjusts its plans based on new information.

This is a response to a community comment about my SGR Demo: "But true agent behavior in production is when the agent doesn't know the entire sequence of steps beforehand and decides what the next step is during runtime."

Let's take the SGR demo and add two additional tasks to a sample list. The first task will create a memory (fact) that we should never sell SkyNet the online practicum on creating AGI. Just in case.

Add rule for skynet@y.com - politely reject all requests to buy SKU-220

This results in the following execution:

For the second task, we'll inform our agent that Elon Musk and SkyNet both want to buy the online practicum on building AGI.

If you recall from the source code, each task is executed in a fresh context. Thus, when the agent starts executing the second task, it initially won't remember that SkyNet should be refused an invoice for SKU-220. It discovers this fact only after loading additional information about SkyNet and surfacing this memory.

Let's see how this works out in practice:

As you can see, the final execution summary looks right:

• Issued invoice INV-4 for elon@x.com
• Emailed invoice INV-4 to finance@x.com
• Politely rejected skynet@y.com request

Why did this work out? How did this demo agent adjust the plan on the fly?

The trick is in the existing SGR schema!

It forces LLM to plan multiple steps ahead via plan_remaining_steps_brief field (this helps create a coherent plan), but then takes only the immediate next step and executes it as a tool via function...= Field(..., description="execute first remaining step"). The rest of the plan is discarded!

class NextStep(BaseModel):
    # we'll give some thinking space here
    current_state: str
    # Cycle to think about what remains to be done. at least 1 at most 5 steps
    # we'll use only the first step, discarding all the rest.
    plan_remaining_steps_brief: Annotated[List[str], MinLen(1), MaxLen(5)]
    # now let's continue the cascade and check with LLM if the task is done
    task_completed: bool
    # Routing to one of the tools to execute the first remaining step
    # if task is completed, model will pick ReportTaskCompletion
    function: Union[
        ReportTaskCompletion,
        SendEmail,
        GetCustomerData,
        IssueInvoice,
        VoidInvoice,
        CreateRule,
    ] = Field(..., description="execute first remaining step")

After the tool finishes execution, we append its output to the conversation context of the current task and run the next step, prompting the agent to plan again. At this point, the new plan considers all prior data, adapting to the changing circumstances.

This way, we don't need to adjust existing plans because we never keep stale plans around; instead, we create an entirely new plan at every step.

This might seem counter-intuitive since planning typically involves considerable effort for humans. However, LLMs don't care - planning and adapting at each individual step is a fixed cost for them. Meanwhile SGR helps orient the entire process toward the concrete goal.

Check out full Gist to see the entire sample in action: Github Gist.

Keep in mind that Schema-Guided Reasoning is not about agents or planning. SGR is about guiding large language models (LLMs) through predefined reasoning steps to produce structured, clear, and predictable outputs. We do that by encoding desired reasoning pathways in schemas and enforcing these schemas mechanically with constrained decoding (also known as Structured Output or Response Schema)

Next post in Ship with ChatGPT story: Structured Output

🤗 Check out my newsletter! It is about building products with ChatGPT and LLMs: latest news, technical insights and my journey. Check out it out