GenAIHub
← Back to Technical Section

CrewAI

Role-Based AI Agent Teams for Complex Tasks

What is CrewAI?

CrewAI is an open-source Python framework for orchestrating collaborative, autonomous AI agents. It enables the creation of agent teams with defined roles, goals, and backstories that work together like a real team to accomplish complex tasks. CrewAI mirrors human team dynamics to achieve shared objectives efficiently.

"CrewAI is designed to enable AI agents to assume roles, share goals, and operate in a cohesive unit—much like a well-oiled crew."

Role-Based

Clear agent roles

Task-Driven

Structured tasks

Crew Teams

Collaborative agents

Memory

Short & long-term

Core Concepts

Agents

Autonomous units with a role, goal, and backstory. Each agent's personality and expertise influences their behavior and decision-making.

Agent(
    role="Senior Data Analyst",
    goal="Analyze data and provide insights",
    backstory="10 years at top consulting firms...",
    tools=[analysis_tool, csv_tool]
)

Tasks

Specific units of work with a description, expected output, and assigned agent. Tasks can depend on other tasks and share context.

Task(
    description="Analyze Q4 sales data...",
    expected_output="Detailed report with trends",
    agent=analyst_agent,
    context=[previous_task]  # Task dependencies
)

Crews

The collaborative group that brings agents and tasks together. Manages execution strategy (sequential, parallel, hierarchical) and orchestrates agent interactions.

Crew(
    agents=[analyst, writer, reviewer],
    tasks=[research_task, write_task, review_task],
    process=Process.sequential,  # or hierarchical
    memory=True
)

Quick Start: Research Team

Create a simple research and writing team:

# Install CrewAI
# pip install crewai crewai-tools

from crewai import Agent, Task, Crew, Process
from crewai_tools import SerperDevTool

# Create tools
search_tool = SerperDevTool()

# Define agents with roles
researcher = Agent(
    role="Senior Research Analyst",
    goal="Find comprehensive information about AI trends",
    backstory="""You are an expert researcher with 10 years of experience 
    in technology analysis. You are known for thorough research.""",
    tools=[search_tool],
    verbose=True
)

writer = Agent(
    role="Tech Content Writer",
    goal="Create engaging content from research findings",
    backstory="""You are a skilled writer specializing in making 
    complex tech topics accessible to general audiences.""",
    verbose=True
)

# Define tasks
research_task = Task(
    description="Research the latest trends in AI agents for 2024",
    expected_output="A detailed summary of top 5 AI agent frameworks",
    agent=researcher
)

writing_task = Task(
    description="Write a blog post based on the research findings",
    expected_output="A 500-word blog post about AI agents",
    agent=writer,
    context=[research_task]  # Uses output from research
)

# Create and run the crew
crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, writing_task],
    process=Process.sequential,
    verbose=True
)

result = crew.kickoff()
print(result)

Expected Output

[Agent: Senior Research Analyst] Starting research...
[Tool: SerperDevTool] Searching for AI agent trends...
[Agent: Senior Research Analyst] Found 5 frameworks...

[Agent: Tech Content Writer] Creating blog post...
[Agent: Tech Content Writer] Draft complete!

Final Output:
"# The Rise of AI Agents in 2024
AI agents are transforming how we interact with..."

Example: Hierarchical Process

Use a manager to dynamically assign tasks to agents:

from crewai import Agent, Task, Crew, Process
from langchain_openai import ChatOpenAI

# Manager LLM (needs to be capable)
manager_llm = ChatOpenAI(model="gpt-4o")

# Create specialized agents
coder = Agent(
    role="Python Developer",
    goal="Write clean, efficient Python code",
    backstory="Expert Python developer with FastAPI experience"
)

tester = Agent(
    role="QA Engineer",
    goal="Write comprehensive tests and find bugs",
    backstory="Experienced in pytest and test automation"
)

reviewer = Agent(
    role="Code Reviewer",
    goal="Review code for quality and best practices",
    backstory="Senior engineer focused on code quality"
)

# Single high-level task
project_task = Task(
    description="""Build a REST API for user management with:
    - CRUD endpoints
    - Input validation  
    - Unit tests
    - Code review""",
    expected_output="Complete, tested API code with review notes"
)

# Hierarchical crew - manager assigns work
crew = Crew(
    agents=[coder, tester, reviewer],
    tasks=[project_task],
    process=Process.hierarchical,
    manager_llm=manager_llm,
    verbose=True
)

result = crew.kickoff()

Built-in Tools

CrewAI provides many built-in tools via crewai-tools:

SerperDevTool

Web search

ScrapeWebsiteTool

Web scraping

PDFSearchTool

PDF analysis

CodeInterpreter

Code execution

CSVSearchTool

CSV analysis

DirectoryReadTool

File system

GithubSearchTool

GitHub repos

YoutubeSearchTool

Video search

Execution Processes

Sequential

Tasks run one after another. Each task can use the output of previous tasks as context.

Process.sequential

Hierarchical

A manager agent dynamically assigns tasks, reviews outputs, and coordinates the team.

Process.hierarchical

Parallel (Coming)

Independent tasks run concurrently for faster execution when there are no dependencies.

Process.parallel

Memory System

Short-Term Memory

Maintains context within the current execution. Agents remember previous interactions and task outputs during the crew's run.

Long-Term Memory

Persists across executions. Agents can learn from past experiences and improve their performance over time.

# Enable memory for the crew
crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, writing_task],
    memory=True,  # Enables short & long-term memory
    embedder={
        "provider": "openai",
        "config": {"model": "text-embedding-3-small"}
    }
)

CrewAI vs Other Frameworks

Aspect CrewAI AutoGen LangGraph
Paradigm Roles & Tasks Conversations State graphs
Agent Definition Role + Goal + Backstory System message Node functions
Task Dependencies Built-in context Via messages Graph edges
Memory Short & long-term Learning agents Checkpointing
Best For Role-based teams Collaborative coding Complex workflows

CrewAI Enterprise

Production-Ready Platform

CrewAI Enterprise provides a managed platform for deploying, monitoring, and scaling AI agent teams. Features include observability dashboards, team management, and enterprise-grade security.

Learn More

Resources & References

Related Topics