What is LlamaIndex?
LlamaIndex is an open-source data orchestration framework designed to connect your custom data with Large Language Models (LLMs). Often described as a "precision scalpel" for RAG, it specializes in data ingestion, indexing, and retrieval—making it the go-to choice for building context-aware AI applications.
"LlamaIndex is a data framework for LLM-based applications to ingest, structure, and access private or domain-specific data. Think of it as the bridge between your data and your AI."
Data Ingestion
Load any source
Indexing
Vector & more
Retrieval
Semantic search
Query Engines
Natural language
Core Concepts
Documents & Nodes
Documents are containers for your data (PDFs, text files, web pages). Nodes are chunks of documents that get indexed and retrieved. LlamaIndex handles the splitting automatically.
Indexes
Indexes organize your data for efficient retrieval. Each type is optimized for different scenarios:
Semantic similarity search (most common)
Sequential search, summarization
Hierarchical data structures
Exact keyword matching
Retrievers
Retrievers fetch the most relevant nodes given a query. Use index.as_retriever()
to create one from any index. Configure similarity_top_k
to control how many results to return.
Query Engines
Query engines combine retrieval with LLM response generation. Use index.as_query_engine()
to ask natural language questions about your data and get synthesized answers.
Quick Start: Simple RAG
Build a document Q&A system in just a few lines of code:
# Install LlamaIndex
# pip install llama-index llama-index-llms-openai
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
# 1. Load documents from a folder
documents = SimpleDirectoryReader("./data").load_data()
# 2. Create a vector index (embeddings + storage)
index = VectorStoreIndex.from_documents(documents)
# 3. Create a query engine
query_engine = index.as_query_engine()
# 4. Ask questions!
response = query_engine.query("What is the main topic of these documents?")
print(response)Expected Output
# Assuming documents about AI safety: "The main topic of these documents is AI safety and alignment, covering techniques for ensuring AI systems behave as intended, risks of advanced AI, and proposed solutions from researchers."
Example: Persist Index to Disk
Save your index to avoid re-indexing every time:
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext, load_index_from_storage
# First time: create and save
documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
index.storage_context.persist(persist_dir="./storage")
print("Index saved!")
# Later: load from disk (no re-indexing!)
storage_context = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(storage_context)
print("Index loaded from disk!")
# Query as usual
query_engine = index.as_query_engine()
response = query_engine.query("Summarize the key points")
print(response)Expected Output
Index saved!
Index loaded from disk!
"Key points: 1) AI alignment is critical for safety,
2) Current approaches include RLHF and constitutional AI,
3) More research needed on interpretability..."
Example: Using Chroma Vector Store
For production, use an external vector database like Chroma, Pinecone, or Qdrant:
# pip install llama-index-vector-stores-chroma chromadb
import chromadb
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.core import StorageContext
# Setup Chroma client
chroma_client = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = chroma_client.get_or_create_collection("my_documents")
# Create vector store and storage context
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
# Load and index documents
documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)
# Query
query_engine = index.as_query_engine()
response = query_engine.query("What are the main findings?")
print(response)LlamaIndex vs LangChain
| Aspect | LlamaIndex | LangChain |
|---|---|---|
| Primary Focus | Data indexing & retrieval (RAG) | General LLM workflows & agents |
| Strength | Optimized data ingestion pipelines | Complex multi-step workflows |
| Index Types | Vector, Summary, Tree, Keyword, KG | Relies on external stores |
| Query Engines | Built-in, specialized | Via chains/agents |
| Best For | Document Q&A, knowledge bases | Chatbots, autonomous agents |
💡 Pro Tip: Many production systems use both—LlamaIndex for data ingestion and retrieval, LangChain for orchestration and agents.
LlamaIndex Ecosystem (2024)
LlamaCloud
Enterprise solution for managed indexing, parsing, and storage of complex documents.
LlamaParse
Advanced PDF and document parsing with Premium Mode for complex layouts and tables.
LlamaDeploy
Turn your agents into production-ready microservices with one command.
Workflows
Event-driven architecture for building multi-agent applications with concurrent processing.
Data Connectors (LlamaHub)
LlamaIndex supports 100+ data sources via LlamaHub: