Arize Phoenix

Open-Source AI Observability for LLM Applications

What is Arize Phoenix?

Arize Phoenix is an open-source AI observability platform built for evaluating, troubleshooting, and experimenting with LLM applications. Developed by Arize AI, the company behind enterprise ML observability, Phoenix brings production-grade monitoring capabilities to the open-source community.

Phoenix stands out with its powerful embedding visualization, RAG analysis, and LLM evaluation capabilities. It runs locally with a single command and provides an intuitive notebook-style interface for interactive exploration.

Apache

Open Source

OTel

Native Standard

15+

LLM Evaluators

10K+

GitHub Stars

Core Features

OpenTelemetry-Native Tracing

Core

Capture complete execution traces using OpenInference, an open standard built on OpenTelemetry. See every LLM call, retrieval step, and tool invocation with full context.

OpenInference Span Visualization Latency Breakdown Token Tracking

RAG Analysis & Debugging

Specialized

Deep dive into your retrieval pipeline. Visualize what documents were retrieved, their relevance scores, and how they influenced the final response.

Retrieval Metrics Document Relevance Context Window NDCG/MRR

Embedding Visualization

Unique

Explore your embedding space with interactive 3D UMAP projections. Identify clusters, detect drift, and understand how your data is distributed in vector space.

UMAP/t-SNE Cluster Analysis Drift Detection Point Cloud

LLM Evaluation Suite

15+ Evals

Ready-to-use evaluation templates for hallucination detection, QA correctness, summarization quality, toxicity, and custom metrics using LLM-as-a-Judge.

Hallucination QA Correctness Toxicity Relevance

Experiments & Datasets

Iterative

Run structured experiments with versioned datasets. Compare prompts, models, and configurations side-by-side with statistical significance testing.

A/B Testing Version Control Comparison Export

Quick Start

Terminal - Installation & Launch

# Install Phoenix
pip install arize-phoenix

# Launch the UI (opens at http://localhost:6006)
python -m phoenix.server.main serve

# Or launch in a notebook
import phoenix as px
px.launch_app()

Python - Tracing with OpenAI

import phoenix as px
from phoenix.otel import register
from openinference.instrumentation.openai import OpenAIInstrumentor
from openai import OpenAI

# Launch Phoenix UI
px.launch_app()

# Register tracer with Phoenix
tracer_provider = register(
    project_name="my-llm-app",
    endpoint="http://localhost:6006/v1/traces"
)

# Instrument OpenAI (auto-captures all calls)
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

# Now all OpenAI calls are traced automatically
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)

# View traces at http://localhost:6006
print(response.choices[0].message.content)

LangChain Integration

Python - Tracing LangChain RAG Pipeline

import phoenix as px
from phoenix.otel import register
from openinference.instrumentation.langchain import LangChainInstrumentor
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain.chains import RetrievalQA

# Launch Phoenix and register
px.launch_app()
tracer_provider = register(project_name="rag-app")

# Auto-instrument LangChain
LangChainInstrumentor().instrument(tracer_provider=tracer_provider)

# Build RAG pipeline (all steps traced)
embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_texts(
    texts=["Phoenix is an observability tool"],
    embedding=embeddings
)

qa_chain = RetrievalQA.from_chain_type(
    llm=ChatOpenAI(model="gpt-4"),
    retriever=vectorstore.as_retriever()
)

# Query (traces show retrieval + LLM call)
result = qa_chain.invoke("What is Phoenix?")

LLM Evaluation

Python - Hallucination Detection

from phoenix.evals import (
    HallucinationEvaluator,
    QAEvaluator,
    OpenAIModel,
    run_evals
)
import pandas as pd

# Initialize evaluator model
eval_model = OpenAIModel(model="gpt-4")

# Create evaluators
hallucination_eval = HallucinationEvaluator(eval_model)
qa_eval = QAEvaluator(eval_model)

# Sample data (could be from traces)
df = pd.DataFrame({
    "input": ["What is the capital of France?"],
    "output": ["The capital of France is Paris."],
    "reference": ["Paris is the capital of France."]
})

# Run evaluations
hallucination_results = run_evals(
    dataframe=df,
    evaluators=[hallucination_eval],
    provide_explanation=True
)

print(f"Hallucination Score: {hallucination_results}")

Built-in Evaluators

Evaluator	Purpose	Input Required
HallucinationEvaluator	Detect factual inconsistencies	output, reference
QAEvaluator	Assess response correctness	input, output, reference
RelevanceEvaluator	Check retrieval relevance	query, document
ToxicityEvaluator	Detect harmful content	output
SummarizationEvaluator	Assess summary quality	input, output
SQLEvaluator	Validate SQL generation	query, generated_sql

Framework Integrations

OpenAI

Anthropic

LangChain

LlamaIndex

DSPy

Haystack

LangGraph

AutoGen

Deployment Options

Local

Run locally with a single pip install. Perfect for development and experimentation.

Docker

Self-host with Docker for team access and production workloads.

Arize Cloud

Enterprise-grade managed service with advanced features and support.

Phoenix vs Langfuse

Feature	Arize Phoenix	Langfuse
Embedding Visualization	Advanced 3D UMAP	Limited
Prompt Management	Basic	Advanced
RAG Analysis	Deep insights	Basic
Notebook Integration	Native	API only
Best For	Data scientists, RAG tuning	Production monitoring

Resources

Documentation

Comprehensive guides & tutorials

GitHub

Source code & examples

Live Demo

Try Phoenix in your browser

Slack Community

Get help from the community

Test Your Knowledge

Score 8/10 or higher to pass