New Paper (Dec 2025)

SPIRAL Agent

Symbolic Planning via Integrated Reasoning And Learning. A framework embedding three specialized LLM agents (Planner, Simulator, Critic) into a Monte Carlo Tree Search (MCTS) loop.

Read Paper (ArXiv)

What is SPIRAL?

SPIRAL solves a classic problem of LLMs: acting immediately without planning. It treats reasoning as a search process where an agent explores options, simulates consequences, and reflects critically before deciding. It consistently outperforms Chain-of-Thought and other state-of-the-art agents (83.6% accuracy on DailyLifeAPIs).

Planner (π)

Decides what action to try next. Expands the search tree by proposing creative steps.

Simulator (W)

Predicts realistic outcomes without executing in the real world. It "imagines" the future.

Critic (C)

Evaluates strategic quality. Provides a dense reward signal (ρ_pref) to guide the search.

How It Works (The Loop)

Expansion

The Planner proposes next possible actions from current state sₜ.

"Search park", "Ask neighbors"

Simulation & Reflection

The Simulator generates an observation oₜ₊₁ (e.g., "No one saw the dog").
The Critic scores this outcome (ρ_pref) asking "Does this get us closer?".

Backpropagation

The score is propagated up the tree. The system learns which paths are promising.
R_t = α · R_base + (1-α) · ρ_pref

Conceptual Implementation

Python (Toy Example)

A simplified non-LLM version demonstrating the flow: Planner → Simulator → Critic → Backprop.

class Node:
    def __init__(self, state, parent=None, action=None):
        self.state = state; self.parent = parent; self.action = action
        self.children = []; self.value = 0.0; self.visits = 0

    def update(self, reward):
        self.visits += 1
        self.value += reward

# 1. Planner (Generates Options)
class Planner:
    def propose_actions(self, state):
        actions = {
            "start": ["search_park", "ask_neighbors"],
            "park": ["check_shelter"],
            "neighbors": ["check_cameras"]
        }
        return actions.get(state, [])

# 2. Simulator (Predicts Outcome - World Model)
class Simulator:
    def simulate(self, state, action):
        transitions = {
            ("start", "search_park"): "park",
            ("start", "ask_neighbors"): "neighbors",
            ("park", "check_shelter"): "found",
            ("neighbors", "check_cameras"): "found"
        }
        return transitions.get((state, action), state)

# 3. Critic (Evaluates Progress)
class Critic:
    def score(self, state):
        # In reality, this would be an LLM prompt: "Is this state closer to goal?"
        scores = {"start": 0.1, "park": 0.4, "neighbors": 0.5, "found": 1.0}
        return scores.get(state, 0.0)

# 4. Main Loop (MCTS-like)
def spiral_search():
    root = Node(state="start")
    planner = Planner(); simulator = Simulator(); critic = Critic()

    for _ in range(5):  # Search iterations
        current = root
        
        # Expansion
        actions = planner.propose_actions(current.state)
        if not actions: continue
        action = actions[0] # Simplified selection
        
        # Simulation
        next_state = simulator.simulate(current.state, action)
        child = Node(next_state, parent=current, action=action)
        current.children.append(child)
        
        # Reflection (Critic)
        reward = critic.score(next_state)
        
        # Backpropagation
        node = child
        while node:
            node.update(reward)
            node = node.parent

    return root

# Result: Agent 'finds' the path with highest value

Comparison

Agent Type	Planning Method	Self-Correction
ReAct / CoT	Linear (Step-by-step)	Hard to recover from early errors
Tree of Thoughts	Tree Search (BFS/DFS)	Resource intensive, no separation of roles
SPIRAL	MCTS + Simulator	High (Explores & backtracks)