What is Haystack?
Haystack is an open-source framework by deepset for building production-ready LLM applications, semantic search, and RAG pipelines. With its modular, component-based architecture, Haystack makes it easy to create, customize, and scale NLP workflows that combine document retrieval with language model generation.
"Haystack is an end-to-end NLP framework that enables you to build pipelines for different use cases. From semantic search to question answering to RAG—Haystack handles the complexity so you can focus on building."
Pipelines
Modular workflows
Document Stores
Vector DB support
Components
Plug & play
Production-Ready
Scale with ease
Core Concepts
Components
Components are the building blocks of Haystack. Each component performs a specific task (embedding, retrieval, generation) and can be connected to form pipelines.
Pipelines
Pipelines connect components into a DAG (Directed Acyclic Graph). Data flows through
components in sequence. Haystack 2.0 uses Pipeline
class with add_component() and
connect() methods.
Document Stores
Document Stores hold your indexed documents and enable retrieval. Haystack supports many backends:
Generators
Generators are components that produce text using LLMs. Haystack integrates with OpenAI, Hugging Face, Cohere, Anthropic, and local models via Ollama.
Quick Start: Simple RAG Pipeline
Build a basic RAG pipeline with Haystack 2.0:
# Install Haystack
# pip install haystack-ai
from haystack import Pipeline, Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator
# 1. Create document store and add documents
document_store = InMemoryDocumentStore()
documents = [
Document(content="Haystack is an open-source NLP framework by deepset."),
Document(content="RAG combines retrieval with generation for accurate answers."),
Document(content="Pipelines connect components in a directed graph."),
]
# 2. Embed and store documents
doc_embedder = SentenceTransformersDocumentEmbedder()
doc_embedder.warm_up()
documents_with_embeddings = doc_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)
# 3. Build RAG pipeline
template = """Answer based on context:
Context:
Question:
Answer:"""
rag_pipeline = Pipeline()
rag_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
rag_pipeline.add_component("retriever", InMemoryEmbeddingRetriever(document_store))
rag_pipeline.add_component("prompt_builder", PromptBuilder(template=template))
rag_pipeline.add_component("llm", OpenAIGenerator(model="gpt-4o-mini"))
# Connect components
rag_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder", "llm")
# 4. Run the pipeline
result = rag_pipeline.run({
"text_embedder": {"text": "What is Haystack?"},
"prompt_builder": {"query": "What is Haystack?"}
})
print(result["llm"]["replies"][0])Expected Output
"Haystack is an open-source NLP framework developed by deepset.
It allows you to build pipelines that connect components in a
directed graph for tasks like semantic search and RAG."
Example: Indexing Pipeline
Create a pipeline to process and index documents from files:
from haystack import Pipeline
from haystack.components.converters import TextFileToDocument
from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
# Create document store
document_store = InMemoryDocumentStore()
# Build indexing pipeline
indexing_pipeline = Pipeline()
indexing_pipeline.add_component("converter", TextFileToDocument())
indexing_pipeline.add_component("cleaner", DocumentCleaner())
indexing_pipeline.add_component("splitter", DocumentSplitter(split_by="sentence", split_length=3))
indexing_pipeline.add_component("embedder", SentenceTransformersDocumentEmbedder())
indexing_pipeline.add_component("writer", DocumentWriter(document_store))
# Connect components
indexing_pipeline.connect("converter", "cleaner")
indexing_pipeline.connect("cleaner", "splitter")
indexing_pipeline.connect("splitter", "embedder")
indexing_pipeline.connect("embedder", "writer")
# Run indexing
indexing_pipeline.run({"converter": {"sources": ["docs/file1.txt", "docs/file2.txt"]}})
print(f"Indexed {document_store.count_documents()} documents")Expected Output
Indexed 24 documents
Example: Hybrid Search
Combine keyword (BM25) and semantic search for better retrieval:
from haystack import Pipeline
from haystack.components.joiners.document_joiner import DocumentJoiner
from haystack.components.rankers import TransformersSimilarityRanker
from haystack.components.retrievers.in_memory import (
InMemoryBM25Retriever,
InMemoryEmbeddingRetriever
)
# Hybrid retrieval pipeline
hybrid_pipeline = Pipeline()
# Add both retrievers
hybrid_pipeline.add_component("bm25_retriever", InMemoryBM25Retriever(document_store, top_k=5))
hybrid_pipeline.add_component("embedding_retriever", InMemoryEmbeddingRetriever(document_store, top_k=5))
hybrid_pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
# Join results and re-rank
hybrid_pipeline.add_component("joiner", DocumentJoiner())
hybrid_pipeline.add_component("ranker", TransformersSimilarityRanker(top_k=3))
# Connect
hybrid_pipeline.connect("text_embedder.embedding", "embedding_retriever.query_embedding")
hybrid_pipeline.connect("bm25_retriever", "joiner")
hybrid_pipeline.connect("embedding_retriever", "joiner")
hybrid_pipeline.connect("joiner", "ranker")
# Run
result = hybrid_pipeline.run({
"bm25_retriever": {"query": "NLP framework"},
"text_embedder": {"text": "NLP framework"},
"ranker": {"query": "NLP framework"}
})
for doc in result["ranker"]["documents"]:
print(f"Score: {doc.score:.3f} - {doc.content[:50]}...")Haystack vs Other Frameworks
| Aspect | Haystack | LangChain | LlamaIndex |
|---|---|---|---|
| Focus | NLP Pipelines & Search | General LLM workflows | Data indexing & RAG |
| Architecture | DAG-based pipelines | Chains/Agents | Query engines |
| Enterprise | deepset Cloud | LangSmith | LlamaCloud |
| Hybrid Search | Built-in | Via integrations | Via integrations |
| Best For | Search systems, QA | Agents, chatbots | Document Q&A |
Haystack 2.x Features
New Pipeline API
Haystack 2.0 introduced a new component-based pipeline architecture with explicit connections.
Query Expansion
QueryExpander and MultiQueryRetriever components boost recall by reformulating queries.
Multimodal RAG
Process PDFs with images, tables, and text using multimodal embeddings.
Agentic Pipelines
Build pipelines with fallback mechanisms and dynamic web search.
deepset Cloud & Studio
Enterprise Solution
deepset Cloud provides a hosted environment for building, deploying, and managing Haystack pipelines. deepset Studio offers a visual interface for pipeline development.