Langfuse
Open-Source LLM Engineering Platform for Observability, Tracing & Evaluation
What is Langfuse?
Langfuse is an open-source LLM engineering platform that provides comprehensive observability for LLM applications. It enables developers to debug, analyze, and optimize their AI applications through detailed tracing, prompt management, and automated evaluation.
Unlike proprietary solutions, Langfuse can be self-hosted for complete data control, or used as a managed cloud service. With native OpenTelemetry support and integrations with popular frameworks like LangChain, LlamaIndex, and OpenAI, it seamlessly fits into any LLM stack.
Core Capabilities
Tracing & Debugging
Core FeatureCapture every step of your LLM application: inputs, outputs, latencies, costs, and metadata. Nested traces show the complete execution path through agents, tools, and chains.
Prompt Management
Version ControlManage prompts in one central location. Version control, A/B testing, and deployment without code changes. Collaborate on prompts with your team through the UI.
Evaluation & Scoring
LLM-as-JudgeAutomatically evaluate outputs using LLM-as-a-Judge, manual annotation, or custom scoring functions. Build datasets from production traces for continuous improvement.
Analytics & Dashboards
InsightsMonitor cost, latency, token usage, and quality metrics across all your LLM applications. Customizable dashboards, alerting, and detailed breakdowns by model, user, or feature.
How Langfuse Works
Quick Start
# Install Langfuse Python SDK
pip install langfuse
# Or with OpenAI integration
pip install langfuse openai
from langfuse import Langfuse
from langfuse.decorators import observe, langfuse_context
from openai import OpenAI
# Initialize Langfuse (auto-reads LANGFUSE_* env vars)
langfuse = Langfuse()
client = OpenAI()
@observe()
def generate_story(topic: str) -> str:
"""Generate a story with automatic tracing."""
# Add custom metadata to the trace
langfuse_context.update_current_observation(
metadata={"topic": topic}
)
response = client.chat.completions.create(
model="gpt-4",
messages=[
{"role": "system", "content": "You are a creative storyteller."},
{"role": "user", "content": f"Write a short story about {topic}"}
]
)
return response.choices[0].message.content
# Every call is automatically traced
story = generate_story("a robot learning to paint")
print(story)
LangChain Integration
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langfuse.callback import CallbackHandler
# Initialize Langfuse callback handler
langfuse_handler = CallbackHandler(
public_key="pk-...",
secret_key="sk-...",
host="https://cloud.langfuse.com" # or your self-hosted URL
)
# Create LangChain components
llm = ChatOpenAI(model="gpt-4")
prompt = ChatPromptTemplate.from_messages([
("system", "You are a helpful assistant."),
("user", "{input}")
])
chain = prompt | llm
# Run with Langfuse tracing enabled
result = chain.invoke(
{"input": "Explain quantum computing simply"},
config={"callbacks": [langfuse_handler]}
)
print(result.content)
Prompt Management
from langfuse import Langfuse
from openai import OpenAI
langfuse = Langfuse()
client = OpenAI()
# Fetch a prompt from Langfuse (version controlled in UI)
prompt = langfuse.get_prompt("customer-support-v2")
# Compile the prompt with variables
compiled_prompt = prompt.compile(
customer_name="John",
issue="billing inquiry",
product="Enterprise Plan"
)
# Use compiled prompt with OpenAI
response = client.chat.completions.create(
model=prompt.config["model"], # Model defined in prompt config
messages=[{"role": "user", "content": compiled_prompt}],
temperature=prompt.config.get("temperature", 0.7)
)
# Link generation to the prompt for analytics
langfuse.generation(
name="customer-support-response",
prompt=prompt,
output=response.choices[0].message.content
)
Native Integrations
Deployment Options
Langfuse Cloud
Managed service with no infrastructure to maintain. SOC 2 Type II certified with data residency options in US and EU.
- Free tier available
- Auto-scaling
- Managed updates
Self-Hosted
Deploy on your own infrastructure with Docker Compose or Kubernetes. Complete data sovereignty and customization options.
- Full data control
- Docker & Kubernetes
- MIT License
# Clone the repository
git clone https://github.com/langfuse/langfuse.git
cd langfuse
# Start with Docker Compose
docker compose up -d
# Access at http://localhost:3000
Langfuse vs Alternatives
| Feature | Langfuse | LangSmith | Arize Phoenix |
|---|---|---|---|
| Open Source | MIT License | Proprietary | Apache 2.0 |
| Self-Hosted | Full support | Cloud only | Full support |
| Prompt Management | Built-in | LangChain Hub | Limited |
| LLM Evaluation | LLM-as-Judge | Built-in evals | LLM evals |
| OpenTelemetry | Native | Via adapters | Native |
Resources
Related Topics
Test Your Knowledge
Score 8/10 or higher to pass
You need to be logged in to take this quiz.
Login to Continue