NLP & Routing

NLU & Intent Recognition

Modern NLU has evolved beyond simple keyword matching or rigid BERT classifiers. Semantic Routers and LLM-based Routing now enable dynamic, zero-shot intent recognition with minimal training data.

The Routing Paradigm Shift

Traditional (BERT/Rasa)

Requires thousands of labeled examples per intent. Fast inference, but rigid. Hard to add new intents without retraining.

Semantic Router

Superfast

Uses Vector Embeddings. Intents are clusters in vector space. Matches user input to the nearest cluster. Zero training, just examples.

LLM Routing (Agentic)

Smartest

Asks an LLM (GPT-4, Claude) to decide. "You are a router. Classify this request." Extremely flexible but slower and costlier.

Hybrid Routing Architecture

graph TD U["User Input"] --> SR{"Semantic Router"} SR -->|"High Similarity Score"| SC["Static Response / Tool"] SR -->|"Low Score (Unknown)"| LLM{"LLM Router"} LLM -->|"Complex Query"| A["Agency Workflow"] LLM -->|"Chit chat"| C["Conversational Model"] style SR stroke:#06b6d4,stroke-width:2px style LLM stroke:#3b82f6,stroke-width:2px

Code: Semantic Router

Using the semantic-router library (by Aurelio AI) for microsecond-latency decision making.

from semantic_router import Route, RouteLayer
from semantic_router.encoders import OpenAIEncoder

# 1. Define distinct routes with example utterances
politics = Route(
    name="politics",
    utterances=[
        "isn't politics the best thing ever",
        "why don't you tell me about your political opinions",
        "don't you just love the president",
    ],
)

coding = Route(
    name="coding",
    utterances=[
        "how do i write a python function",
        "explain recursion to me",
        "what is the difference between list and tuple",
    ],
)

# 2. Compile the RouteLayer (uses Embeddings)
encoder = OpenAIEncoder()
rl = RouteLayer(encoder=encoder, routes=[politics, coding])

# 3. Fast Inference
print(rl("how do i define a class in java?").name) 
# Output: 'coding' (Matches semantic meaning, not just keywords)

Code: Zero-Shot LLM Routing

For when you need reasoning. Using Pydantic for structured output guarantees valid routing decisions.

from pydantic import BaseModel
from enum import Enum
import openai

class Intent(str, Enum):
    REFUND = "refund"
    TECHNICAL_SUPPORT = "technical_support"
    SALES = "sales"
    OTHER = "other"

class RouterDecision(BaseModel):
    intent: Intent
    reasoning: str

completion = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful customer support router."},
        {"role": "user", "content": "I bought this laptop yesterday but the screen is flickering and I want my money back."}
    ],
    response_format={ "type": "json_object" }, # Or structured output
    functions=[{
        "name": "route_ticket",
        "parameters": RouterDecision.model_json_schema()
    }],
    function_call={"name": "route_ticket"}
)
# Output: intent='refund' reasoning='User mentions flickering screen and explicitly asks for money back.'