Safety

Hallucination Guard

Detect, prevent, and mitigate LLM hallucinations. Build trustworthy AI systems with grounding, verification, and guardrails that ensure factual accuracy.

🤔 What is Hallucination?

LLM hallucination occurs when a model generates text that is fluent and confident but factually incorrect, fabricated, or unsupported by the provided context.

Factual Hallucination

Incorrect facts: wrong dates, names, numbers, or events that contradict reality.

Fabrication

Invented citations, fake sources, non-existent papers, or made-up URLs.

Context Unfaithfulness

Answers that ignore or contradict the provided context documents.

⚠️ Important: Hallucinations are especially dangerous in high-stakes domains: medical advice, legal documents, financial reports, and customer-facing applications.

🔍 Detection Methods

Method	How It Works	Accuracy	Speed	Cost
Self-Consistency	Sample multiple responses, check agreement	Medium	Slow	High
LLM-as-Judge	Second LLM verifies against context	High	Medium	Medium
NLI-based	Check entailment between answer & context	Good	Fast	Low
Uncertainty Estimation	Analyze token probabilities/confidence	Medium	Fast	Low
Knowledge Base Lookup	Verify claims against trusted sources	High	Medium	Medium

🛡️ Prevention Strategies

Grounding (RAG)

Anchor responses to retrieved documents:

• Provide relevant context in prompt
• Instruction: "Only use information from the context"
• Include citations in output format

Temperature Control

Lower temperature reduces creativity/risk:

• Factual tasks: temp 0.0 - 0.3
• Balanced: temp 0.5 - 0.7
• Creative: temp 0.8 - 1.0

Prompt Engineering

Design prompts to reduce hallucination:

• "If unsure, say 'I don't know'"
• "Do not make up information"
• "Cite your sources"

Structured Output

Constrain output format:

• JSON Schema validation
• Required "confidence" field
• Mandatory source citations

💻 Implementation Example

Python Hallucination Check with NLI

from transformers import pipeline

# Load NLI model for entailment checking
nli = pipeline("text-classification", 
               model="facebook/bart-large-mnli")

def check_hallucination(context: str, answer: str) -> dict:
    """Check if answer is supported by context."""
    
    # Format for NLI: premise (context) vs hypothesis (answer)
    result = nli(f"{context}", candidate_labels=[answer])
    
    # Extract entailment probability
    entailment_score = result['scores'][0]
    
    return {
        "is_grounded": entailment_score > 0.7,
        "confidence": entailment_score,
        "verdict": "PASS" if entailment_score > 0.7 else "FAIL"
    }

# Example usage
context = "The Eiffel Tower was built in 1889 in Paris."
answer = "The Eiffel Tower was constructed in 1889."

result = check_hallucination(context, answer)
# {'is_grounded': True, 'confidence': 0.92, 'verdict': 'PASS'}

Python LLM-as-Judge Verification

VERIFICATION_PROMPT = """
You are a fact-checker. Given the CONTEXT and ANSWER, determine if 
the answer is fully supported by the context.

CONTEXT:
{context}

ANSWER:
{answer}

Respond with JSON:
{{"supported": true/false, "reason": "brief explanation"}}
"""

def verify_with_llm(context: str, answer: str) -> dict:
    response = llm.generate(
        VERIFICATION_PROMPT.format(context=context, answer=answer),
        temperature=0,  # Deterministic for verification
        response_format={"type": "json_object"}
    )
    return json.loads(response)

🛠️ Guardrails Tools

NeMo Guardrails

NVIDIA's framework for adding guardrails to LLM apps.

Open Source Colang DSL

Guardrails AI

Validation framework with pre-built validators.

Open Source Pydantic

RAGAS

RAG evaluation metrics including faithfulness scores.

Open Source Evaluation

TruLens

Evaluation and feedback functions for LLM apps.

Open Source Observability

✅ Best Practices

Do's

Always ground responses in context (RAG)
Use low temperature for factual tasks
Require citations in high-stakes domains
Monitor hallucination metrics (faithfulness)
Add human-in-the-loop for critical decisions

Don'ts

Trust LLM output without verification
Use high temperature for factual Q&A
Deploy without hallucination testing
Ignore user reports of incorrect answers
Expose raw LLM output in medical/legal apps