Agentic RAG

Autonomous RAG Pipelines with Intelligent Query Reformulation

🎧

Listen to this Explanation

Enjoy a clear, AI-narrated audio version (Solo Mode).

Overview

Agentic RAG represents a sophisticated approach to Retrieval-Augmented Generation where the system autonomously analyzes incoming queries, determines what reformulation strategy is needed, and executes that strategy without explicit user instruction. Unlike traditional RAG systems that take queries as-is, Agentic RAG intelligently preprocesses queries to bridge the gap between user intent and optimal retrieval.

"The agentic nature lies in the system's ability to autonomously analyze, reformulate, and orchestrate queries—combining multi-turn context, query expansion, and query decomposition to obtain the most robust RAG results."

Agentic RAG Architecture

Architecture showing autonomous query reformulation and multi-component RAG pipeline

Motivation

Traditional RAG systems often struggle with ambiguous, context-lacking, or complex queries:

Ambiguous Queries

Short or vague queries lead to poor retrievals without context

Complex Multi-faceted Queries

Queries requiring reasoning across multiple unrelated documents fail

Autonomous Reformulation

Agentic RAG automatically expands, decomposes, or contextualizes queries

Production-Ready Components

Parser, Reranker, GLM, and LMUnit work together for robust results

Query Reformulation Strategies

Multi-turn Context

Adds iterative dialogue context from previous conversation turns to maintain continuity.

Example: "What about last quarter?" → includes previous Q4 context

Query Expansion

Adds additional context to short queries to help retrieve optimal results.

Example: "Revenue?" → "What is the company's revenue breakdown?"

Query Decomposition

Breaks complex multi-faceted queries into several focused sub-queries.

Example: "Compare Q1-Q4" → 4 separate quarterly sub-queries

Four Key RAG Components

1. Document Parser

Enterprise-grade parsing that handles complex tables, charts, figures, and multi-page documents with hierarchical structure understanding.

Tables Charts Hierarchy

2. Instruction-Following Reranker

Dynamically selects the most relevant evidence with customizable prioritization rules for handling conflicting information.

Example: "Prioritize internal sales documents over market analysis. Recent documents weighted higher."

3. Grounded Language Model (GLM)

Generates factual, source-backed responses with minimal hallucinations. Engineered specifically for RAG use cases where accuracy is critical.

Grounded Low Hallucination Attributable

4. LMUnit (Language Model Unit Tests)

Natural language unit testing for evaluating accuracy, grounding, and reliability of agent responses. Brings software engineering rigor to LLM evaluation.

Accuracy Grounding Reliability

Implementation Example

# Agentic RAG Implementation with Contextual AI
from contextual import ContextualAI

# Initialize client
client = ContextualAI(api_key=API_KEY)

# Create datastore for documents
datastore = client.datastores.create(name="Financial_Demo")

# Ingest documents (handles tables, charts, figures)
with open("report.pdf", "rb") as f:
    client.datastores.documents.ingest(datastore.id, file=f)

# Create agent with query reformulation enabled
agent = client.agents.create(
    name="Financial Analyst",
    datastore_ids=[datastore.id],
    agent_configs={
        "global_config": {
            "enable_multi_turn": True  # Enable multi-turn context
        }
    },
    system_prompt="""You are a helpful AI assistant.
    Only use information from provided documentation.
    Keep answers concise and relevant."""
)

# Query with automatic reformulation
result = client.agents.query.create(
    agent_id=agent.id,
    messages=[{
        "content": "What was NVIDIA's annual revenue FY22-25?",
        "role": "user"
    }]
)

print(result.message.content)

# Evaluate with LMUnit
evaluation = client.lmunit.create(
    query="What was revenue?",
    response=result.message.content,
    unit_test="Does the response accurately extract numerical data?"
)
print(f"Quality Score: {evaluation.score}/5")

Benefits of Agentic RAG

Autonomous Operation

System determines reformulation strategy without user intervention.

Minimized Hallucinations

Grounded Language Model ensures responses are source-backed.

Complex Query Handling

Breaks down multi-faceted queries for accurate multi-document reasoning.

Built-in Quality Assurance

LMUnit provides continuous validation and performance metrics.

Enterprise Document Handling

Parses complex tables, charts, and multi-page documents with hierarchy.

Configurable Prioritization

Instruction-following reranker handles conflicting information sources.

Ideal Use Cases

Financial Analysis

Quantitative reasoning from earnings reports, financial statements, and market data.

Enterprise Knowledge

Internal documentation, policies, and procedures across multiple systems.

Compliance & Legal

Regulatory documents requiring precise, attributable responses with source verification.

Test Your Knowledge

Score 8/10 or higher to pass