GenAIHub
← Back to Technical Section

Langfuse

Open-Source LLM Engineering Platform for Observability, Tracing & Evaluation

What is Langfuse?

Langfuse is an open-source LLM engineering platform that provides comprehensive observability for LLM applications. It enables developers to debug, analyze, and optimize their AI applications through detailed tracing, prompt management, and automated evaluation.

Unlike proprietary solutions, Langfuse can be self-hosted for complete data control, or used as a managed cloud service. With native OpenTelemetry support and integrations with popular frameworks like LangChain, LlamaIndex, and OpenAI, it seamlessly fits into any LLM stack.

MIT
Open Source
20+
Integrations
v3.0
Latest Version
30K+
GitHub Stars

Core Capabilities

Tracing & Debugging

Core Feature

Capture every step of your LLM application: inputs, outputs, latencies, costs, and metadata. Nested traces show the complete execution path through agents, tools, and chains.

Nested Spans Auto-capture Metadata User Sessions Cost Tracking

Prompt Management

Version Control

Manage prompts in one central location. Version control, A/B testing, and deployment without code changes. Collaborate on prompts with your team through the UI.

Version History LLM Playground A/B Testing Hot-swap Prompts

Evaluation & Scoring

LLM-as-Judge

Automatically evaluate outputs using LLM-as-a-Judge, manual annotation, or custom scoring functions. Build datasets from production traces for continuous improvement.

Automated Scoring Human Annotation Custom Metrics Datasets

Analytics & Dashboards

Insights

Monitor cost, latency, token usage, and quality metrics across all your LLM applications. Customizable dashboards, alerting, and detailed breakdowns by model, user, or feature.

Cost Analytics Latency Tracking Token Usage Custom Dashboards

How Langfuse Works

flowchart LR subgraph App["Your LLM Application"] A["LangChain / LlamaIndex"] B["OpenAI SDK"] C["Custom Code"] end subgraph SDK["Langfuse SDK"] D["Python / JS SDK"] E["OpenTelemetry"] end subgraph Platform["Langfuse Platform"] F["Trace Storage"] G["Analytics Engine"] H["Prompt Registry"] I["Evaluation"] end A --> D B --> D C --> D D --> F E --> F F --> G F --> I H --> A H --> B H --> C

Quick Start

Terminal - Installation
# Install Langfuse Python SDK
pip install langfuse

# Or with OpenAI integration
pip install langfuse openai
Python - Basic Tracing Example
from langfuse import Langfuse
from langfuse.decorators import observe, langfuse_context
from openai import OpenAI

# Initialize Langfuse (auto-reads LANGFUSE_* env vars)
langfuse = Langfuse()
client = OpenAI()

@observe()
def generate_story(topic: str) -> str:
    """Generate a story with automatic tracing."""
    
    # Add custom metadata to the trace
    langfuse_context.update_current_observation(
        metadata={"topic": topic}
    )
    
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "You are a creative storyteller."},
            {"role": "user", "content": f"Write a short story about {topic}"}
        ]
    )
    
    return response.choices[0].message.content

# Every call is automatically traced
story = generate_story("a robot learning to paint")
print(story)

LangChain Integration

Python - LangChain with Langfuse Callback
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langfuse.callback import CallbackHandler

# Initialize Langfuse callback handler
langfuse_handler = CallbackHandler(
    public_key="pk-...",
    secret_key="sk-...",
    host="https://cloud.langfuse.com"  # or your self-hosted URL
)

# Create LangChain components
llm = ChatOpenAI(model="gpt-4")
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("user", "{input}")
])

chain = prompt | llm

# Run with Langfuse tracing enabled
result = chain.invoke(
    {"input": "Explain quantum computing simply"},
    config={"callbacks": [langfuse_handler]}
)

print(result.content)

Prompt Management

Python - Fetch & Use Managed Prompts
from langfuse import Langfuse
from openai import OpenAI

langfuse = Langfuse()
client = OpenAI()

# Fetch a prompt from Langfuse (version controlled in UI)
prompt = langfuse.get_prompt("customer-support-v2")

# Compile the prompt with variables
compiled_prompt = prompt.compile(
    customer_name="John",
    issue="billing inquiry",
    product="Enterprise Plan"
)

# Use compiled prompt with OpenAI
response = client.chat.completions.create(
    model=prompt.config["model"],  # Model defined in prompt config
    messages=[{"role": "user", "content": compiled_prompt}],
    temperature=prompt.config.get("temperature", 0.7)
)

# Link generation to the prompt for analytics
langfuse.generation(
    name="customer-support-response",
    prompt=prompt,
    output=response.choices[0].message.content
)

Native Integrations

OpenAI
Anthropic
Google Gemini
LangChain
LlamaIndex
AutoGen
LangGraph
OpenTelemetry

Deployment Options

Langfuse Cloud

Managed service with no infrastructure to maintain. SOC 2 Type II certified with data residency options in US and EU.

  • Free tier available
  • Auto-scaling
  • Managed updates

Self-Hosted

Deploy on your own infrastructure with Docker Compose or Kubernetes. Complete data sovereignty and customization options.

  • Full data control
  • Docker & Kubernetes
  • MIT License
Docker - Self-Hosting Quick Start
# Clone the repository
git clone https://github.com/langfuse/langfuse.git
cd langfuse

# Start with Docker Compose
docker compose up -d

# Access at http://localhost:3000

Langfuse vs Alternatives

Feature Langfuse LangSmith Arize Phoenix
Open Source MIT License Proprietary Apache 2.0
Self-Hosted Full support Cloud only Full support
Prompt Management Built-in LangChain Hub Limited
LLM Evaluation LLM-as-Judge Built-in evals LLM evals
OpenTelemetry Native Via adapters Native

Resources

Related Topics

Test Your Knowledge

Score 8/10 or higher to pass