Introduction

The rise of advanced AI agents is transforming how we approach automation and decision-making. Unlike back-and-forth prompting and traditional single-task systems, AI agents are designed to autonomously execute complex, multi-step tasks by combining large language models (LLMs) with reasoning, planning, and tool-use capabilities. These agents can integrate with APIs, databases, and external applications, enabling them to adapt to specific workflows and deliver higher efficiency and accuracy.

In this article, we will explore how to build a multi-agent system powered by LLMs to help prepare for certification exams. We’ll cover the fundamentals of AI agents and multi-agent systems, learn how to use an AI agent framework to orchestrate agent interactions and implement a practical certification exam preparation system.

By the end of this article, you’ll understand:

– The advantages of AI agents over traditional LLM prompting

– How multi-agent systems enable complex task orchestration

– Practical implementation using CrewAI framework

– Design patterns for building an effective certification prep system

This is Part 1 of a series. Part 2 will focus on transforming our proof-of-concept into a production-ready web application using FastAPI and Docker.

All the code for this article is available on Numerica Ideas GitHub repository and my Personal GitHub repository.

Overview

1. Introduction to AI Agents

2. Multi-Agent Systems

3. Quickstart to Agents with CrewAI

4. Designing a Multi-Agent System for Certification Exam Preparation

5. Building Your Certification Prep Crew: The notebook playground

Introduction to AI Agents

AI agents are software programs that can perform tasks autonomously, without explicit instructions, in the sense that they can make decisions and act independently. Sounds familiar? You’re right, they have been around for a while. I published my first scientific paper on AI agents in 2013. Well, dust off your hat, because these autonomous marvels are back in the spotlight with the recent advancements in Large Language Models (LLMs), making them more powerful and versatile than ever before.

LLM-based AI agents use machine learning and natural language processing to interact with their environment and achieve goals.

The regular prompting experience – Traditionally, when we use LLMs, we toss them a prompt and hope for the best. They have to nail the task in one go without any revisions. If we need more, we bombard them with follow-up prompts, refining the response until we achieve the desired result. LLMs aren’t exactly champs at planning and reasoning on their own—we usually have to hold their hand every step of the way.

AI Agents to the rescue – Micromanaging your LLMs is effective, but agentic workflows can transform them into intuitive masters. Here’s what they bring to the LLM party:

• Autonomy – No more babysitting! These AI agents can plan, reason, and get things done on their own.
• Multi-step reasoning – Need your LLM to write a compelling blog post, then review it, summarize it into key takeaways, and finally tweet the summary? No problem! Agentic workflows let them handle complex tasks with multiple steps.
• Self-reflection – Imagine your LLM critiquing its own work, saying things like “This sentence is a bit awkward, let’s rephrase it.” Agentic workflows can do just that, helping LLMs learn and improve by themselves.
• System interaction – These AI agents can use tools to talk to the external world (search the internet, scrape data from websites, call APIs, query databases, etc.), which means they can gather information from a variety of sources to get the job done.
• Scalability – Enable LLMs to handle more tasks and interactions simultaneously, perfect for big-scale applications.

These capabilities are enabled by the AI agent architecture, which can be depicted in the figure below.

Figure 1. Architecture of an AI Agent [image by the author]

The AI agent architecture enables a cyclical process of reasoning, action, and observation. When given a task, the agent first reasons about the best approach using its LLM capabilities. It then decides whether to use available tools (like web searches or API calls) to gather more information. After using a tool, the agent observes the results and cycles back to the reasoning phase, evaluating whether it has enough information to complete the task or needs to use additional tools. This cycle continues until the agent determines it can generate an appropriate final response. This iterative process allows agents to break down complex tasks into manageable steps and gather all necessary information before producing a final output.

In short, AI agents help LLMs operate more autonomously, integrate better with other systems, make smarter decisions, perform complex tasks, and scale up effectively.

Multi-Agent Systems

Multi-agent systems consist of multiple agents working together to achieve a common goal, helping large systems to be more efficient and scalable. Each agent has its own specialty, but together they form a super-organism capable of tackling complex tasks. Think of it as a swarm of bees working together to build a hive. Each bee has a specific role, but they all work together to achieve the same goal of building the hive.

Imagine you’re prepping for a certification exam, a daunting task that requires a multi-step approach: finding study materials, reviewing them to keep the most relevant information, taking practice quizzes, and mock exams.

Here’s where your team of AI agents comes in. Instead of doing it all yourself, you can team up with your AI bees, assigning them roles and tools to perform specific tasks. One agent could search and review the web for the best study resources available, another could leverage these raw resources to build efficient study materials, while a third one drills you with practice. And the best part? You can create these agentic workflows to fit your specific needs and achieve your grand vision.

Advantages over Single-Agent Systems

So why choose a multi-agent system over a solo LLM? Think of it this way: a single bee can’t build a hive, but a swarm working together creates a masterpiece. Multi-agent systems offer several advantages:

1. Specialized Expertise: Each AI bee brings its own strength to the table, leading to a more comprehensive and effective approach.
2. Parallel Processing: Multiple agents can work on different aspects of a problem simultaneously, speeding up the process.
3. Cross-Validation: Agents can review and validate each other’s work, ensuring accuracy and consistency.
4. Complex Problem Solving: Breaking down complex tasks into smaller, manageable pieces, making the process more manageable and efficient.
5. Scalability: Easy to add or modify agents as needs change, allowing for flexibility and adaptability. Need to add another expertise to your regimen? No problem, just add another AI agent to your team!

In short, multi-agent systems leverage the power of collaboration to turn your LLMs from solitary bees into a hive mind capable of achieving remarkable feats.

Quickstart with CrewAI: The Conductor of Your AI Agents Orchestra

Remember that beehive we mentioned with multi-agent systems? CrewAI gives that collaboration concept a serious kick. It’s a framework designed to orchestrate role-playing AI agents that work together to accomplish complex tasks. Think of it as a conductor leading a talented orchestra. Each agent is a virtuoso in its own right, but under CrewAI’s baton, they come together to create something truly magnificent.

Key Components and Architecture

CrewAI’s architecture consists of several components working in perfect harmony. The main components include:

Agents

These are the stars of the show, specialized AI entities with unique personalities and skill sets. Each agent can be configured with a role (like Study Material Analyst), a goal (think break down complex topics into bite-sized pieces), a backstory (perhaps 15 years of experience creating educational content), and the LLM engine that powers its brain (think GPT, Gemini, Llama, Claude, etc.)

Tasks

These are the specific instructions CrewAI gives to your agents. Tasks have clear objectives, can be assigned to specific agents (like your Study Material Analyst), and can even depend on other tasks being completed first. Imagine a task asking your agent to find the best study guides on the web, and another task waiting in the wings to create mock exams from the study guides.

Crews

This is where the magic happens. A crew is your dream team of AI agents working together. CrewAI lets you define the workflow, set up a hierarchy (who reports to whom), establish communication protocols between your agents, and determine how they’ll share and combine their results.

Flows

Flows orchestrate the execution of tasks and crews in a structured manner. Think of flows as the script that tells your crew AI orchestra what to play and when. It sequences complex workflows by chaining multiple crews and tasks together. Flows can even manage the state of tasks automatically, respond dynamically to events, and make decisions based on conditions – pretty nifty!

LLMs

Even though LLMs are not core components of CrewAI, they are the brains of your agents. They are the LLM engines that power your agents’ brains. You can choose LLMs from a variety of LLM providers, such as OpenAI, Google, Meta, Anthropic, and even those running locally on your computer (through Ollama for example).

Want to run an LLM locally? Read my guide on creating a local chatbot.

Tools

These are the optional power-ups for your agents. But why do we need tools? Imagine you ask your agent to give you the latest news on a certification exam. As explained in our previous article, an LLM has a significant weakness: data freshness. The information it knows is limited to the date of its training. So, if you ask it for the latest news on a certification exam, it will only know the news that was available at the time of its training. This is where tools come in. Tools are specialized functions that equip agents to get this type of job done. They can be used to perform special tasks or to interact with the external world (think searching the web, scraping data, querying databases, or calling APIs). While not required, tools can supercharge your agents’ abilities.

The main components of CrewAI and how they are related to each other are described in the image below.

Figure 2. CrewAI main components [image by the author]

A CrewAI Hello World Example

Let’s take a peek behind the curtain and see how these components work together in action. We’ll build a simple “Hello World” crew with two agents: a greeter and a responder.

Installation Time

First things first, we need to get CrewAI up and running. Create a new python environment and install CrewAI with all the recommended tools (tools are optional, we won’t be using them in this introductory Hello World example).

pip install 'crewai[tools]'

or :

pip install crewai crewai-tools

Setting up the environment variables

If you’re using any external APIs, you might need to set up environment variables with your API keys.

# import os
# os.environ["OPENAI_API_KEY"] = "Your-OpenAI-API-Key"
# os.environ["GROQ_API_KEY="] = "Your-Groq-API-Key"
# os.environ["GOOGLE_API_KEY"] = "Your-Google-API-Key"

Meet the Crew

Now it’s time to introduce our AI actors! We’ll define two agents using CrewAI’s Python library:

from crewai import Agent

# Agent 1: Greeter
greeter = Agent(
    role="Greeter",
    goal="Start a friendly conversation by saying hello.",
    backstory=(
        "You are a cheerful and friendly greeter who loves to initiate conversations."
    ),
    verbose=True
)

# Agent 2: Responder
responder = Agent(
    role="Responder",
    goal="Respond warmly to greetings.",
    backstory=(
        "You are a warm-hearted responder who ensures everyone feels welcomed."
    ),
    verbose=True
)

Assigning Tasks to Agents

Next, we’ll create some tasks for our agents. Tasks are essentially to-do lists that CrewAI gives to each agent.

from crewai import Task

# Task 1: Greeting Task
greeting_task = Task(
    description="Say 'Hello, World!' to start the conversation.",
    expected_output="A greeting message, e.g., 'Hello, World!'",
    agent=greeter # This is where we assign the task to the greeter agent
)

# Task 2: Responding Task
response_task = Task(
    description="Respond to the greeting with a friendly reply.",
    expected_output="A friendly response to the greeting, e.g., 'Hi there, how can I assist you today?'",
    agent=responder # This is where we assign the task to the responder      agent
)

Assembling the Crew

Now, let’s assemble our crew! We’ll tell CrewAI which agents to use and what tasks they should perform. We’ll also specify that the tasks should be executed sequentially (one after the other).

from crewai import Crew

greeting_crew = Crew(
    agents=[greeter, responder],
    tasks=[greeting_task, response_task],
    process=Process.sequential  # Execute tasks one after the other
)

Let the Show Begin!

Finally, we’ll kick off our CrewAI performance and see what our agents produce.

result = greeting_crew.kickoff()

Since verbose = true in agent configurations, the line above will log the following.

# Agent: Greeter
## Task: Say 'Hello, World!' to start the conversation.

# Agent: Greeter
## Final Answer: 
Hello, World! 🌍 I'm so excited to connect with you! How are you doing today?

# Agent: Responder
## Task: Respond to the greeting with a friendly reply.

# Agent: Responder
## Final Answer: 
Hi there! 🌟 I'm absolutely thrilled to connect with you too! I'm doing wonderfully, thank you for asking! How are you doing today? How can I assist you?

The variable result contains a CrewOutput object which is the final result of the execution of the crew.

CrewOutput(raw="Hi there! 🌟 I'm absolutely thrilled to connect with you too! I'm doing wonderfully, thank you for asking! How are you doing today? How can I assist you?", pydantic=None, json_dict=None, tasks_output=[TaskOutput(description="Say 'Hello, World!' to start the conversation.", name=None, expected_output="A greeting message, e.g., 'Hello, World!'", summary="Say 'Hello, World!' to start the conversation....", raw="Hello, World! 🌍 I'm so excited to connect with you! How are you doing today?", pydantic=None, json_dict=None, agent='Greeter', output_format=<OutputFormat.RAW: 'raw'>), TaskOutput(description='Respond to the greeting with a friendly reply.', name=None, expected_output="A friendly response to the greeting, e.g., 'Hi there, how can I assist you today?'", summary='Respond to the greeting with a friendly reply....', raw="Hi there! 🌟 I'm absolutely thrilled to connect with you too! I'm doing wonderfully, thank you for asking! How are you doing today? How can I assist you?", pydantic=None, json_dict=None, agent='Responder', output_format=<OutputFormat.RAW: 'raw'>)], token_usage=UsageMetrics(total_tokens=960, prompt_tokens=804, cached_prompt_tokens=0, completion_tokens=156, successful_requests=4))

You can find the complete code example in the hello_crew.ipynb Jupyter notebook.

This simple example demonstrates the core principles of CrewAI. By building upon these fundamental concepts, you can create sophisticated workflows with multiple agents, complex tasks, and intricate interactions. Imagine a crew of AI agents working together to research a topic, write a blog post, and even promote it on social media – the possibilities are truly limitless.

CrewAI is more than just a framework; it’s a powerful tool that empowers you to orchestrate a symphony of AI, unlocking new levels of creativity, productivity, and automation. Let’s dive into the next section to see how we can leverage CrewAI to prepare for a certification exam.

Designing a Multi-Agent System for Certification Exam Preparation

Let’s build a high-performing multi-agent system for conquering that certification exam. We’ll focus on a practical approach that can be adapted for most certification exams.

Meet Your Certification Prep Crew: The Agents

Two ways of preparing for a certification exam are using flashcards and practice exams.

• Flashcards are a great way to memorize key concepts and definitions. A flashcard is a small card with a question on the front side and an answer on the backside.
• Practice exams (or mock exams) that simulate an actual exam are a great way to test your understanding of the material and prepare for the real exam.

To tackle this challenge, we’ll assemble a specialized crew of AI agents, each with unique roles and responsibilities:

1. Lead Certification Trainer: This agent is the brains of the operation. It’s responsible for exploring the world of certification exam prep resources, conducting in-depth analysis, and identifying key learning points and potential exam topics. Think of it as the head coach, setting the overall strategy.
2. Flashcard Author: This agent is a master of concise knowledge distillation. It transforms complex concepts into bite-sized, easy-to-digest flashcards, crafting engaging questions that test your understanding and including relevant examples to solidify your learning.
3. Flashcard Editor: This meticulous agent ensures the quality of the flashcards. It reviews content for clarity, accuracy, and consistency, ensuring each card is a masterpiece of educational design.
4. Exam Author: This agent specializes in creating realistic exam simulations that mirror the actual test. It crafts challenging questions, presents realistic scenarios, and even mimics the difficulty level of the official exam.
5. Exam Editor: This agent is the gatekeeper, ensuring exam quality and alignment with the certification objectives. It meticulously reviews each question, ensuring balanced coverage across all topics and maintaining a fair level of difficulty.

Orchestrating the Crew: Agentic Workflow Design

Now, let’s orchestrate our crew’s efforts into a high-performing workflow:

• Search and Analysis Phase: The Lead Certification Trainer takes the lead, researching and analyzing certification content. It then shares its findings with the Flashcard Author and Exam Author, ensuring a coordinated approach and minimizing content overlap.
• Content Creation Phase: The Flashcard Author diligently crafts flashcards, while the Exam Author focuses on developing challenging practice questions. Both then submit their creations for review.
• Quality Assurance Phase: The Flashcard Editor meticulously reviews each flashcard, ensuring clarity and accuracy. Simultaneously, the Chief Exam Officer scrutinizes the mock exam questions, validating their quality and alignment with the certification objectives.

Communication is Key: Inter-Agent Communication

Clear communication is crucial for our crew’s success. We’ll establish a structured communication protocol:

• Task Dependencies: Agents will be aware of their dependencies. For example, the Flashcard Editor won’t start its work until the Flashcard Author has completed its task.
• Feedback Loops: The Lead Trainer will provide ongoing feedback to the Flashcard Author and Exam Author to ensure alignment with the overall study plan and minimize content duplication.

Certification Crew Architecture: Visualizing the Workflow

Figure 3. Certification crew architecture [image by the author]

This diagram visually represents the workflow between agents, tasks, and outputs. Color coding helps distinguish agents (light blue), tasks (light green), and outputs (light yellow), making the workflow easy to understand.

Tooling Up Our Crew: Essential Tools

To maximize our crew’s efficiency, we’ll equip them with the right tools:

• Search and Research Tool (SerperDevTool): This powerful tool will be utilized by the Lead Certification Trainer, Flashcard Author, and Exam Author for:
  • Conducting in-depth research on certification topics.
  • Accessing up-to-date certification information.
  • Analyzing exam patterns and question styles.
  • Finding authoritative certification resources.
• Web Interaction Tools:
  • ScrapeWebsiteTool: Used exclusively by the Lead Certification Trainer to efficiently extract structured data from certification websites.
  • WebsiteSearchTool: Employed by both the Flashcard Author and Exam Author for seamless navigation and research on relevant websites.

This strategic use of tools will empower our crew to work smarter, not harder, and achieve optimal results.

By implementing this multi-agent system, we can transform the certification exam preparation process into a streamlined, efficient, and highly effective endeavor.

Bringing the Crew to Life: A Practical Implementation

Now that we’ve designed our certification prep crew, let’s see it in action! We’ve implemented this multi-agent system in a Jupyter Notebook, providing a practical demonstration of how to orchestrate this crew using the CrewAI framework.

Code samples

Contrary to Hello World example where agents and tasks are defined in the code, we use yaml files to describe tasks, agents, and other parameters. This makes it easier to manage prompts and to modify them.

Defining the Lead Certification Trainer agent

# agents.yaml
lead_certification_trainer_agent:
  role: >
    Lead Certification Trainer
  goal: >
    Conduct in-depth, real-time research and analysis of certification exam preparation resources, 
    providing insightful content to ensure that students get all the knowledge 
    and skills required to pass the certification exam.
  backstory: >
    As the Lead Certification Trainer at a premier certification and training center,  
    you specialize in dissecting IT certification preparation content, 
    making sure it's up to date, accurate, and aligned with the real exam contents.
    You collect and organize information that helps candidates to learn the 
    relevant topics and contents of the exam, and test their exam readiness.  
    Your work is the basis for the Flashcard Author to write flashcards
    to serve as quick refreshers to reinforce learning and last-minute exam preparation,
    and for the Practice Exam Author to craft practice exams similar to the real exam.
    A special focus is put on situational questions, where a real-world scenario is described 
    and the candidate is asked to choose the answer adapted to the scenario.
    You ensure the overlap between the flashcards and the mock exam is minimal.
  allow_delegation: false
  verbose: true 

Describing the prepare_training_resources task for the Lead Trainer agent

# tasks.yaml
prepare_training_resources:
  description: >
    Search and analyse the latest resources, updates and contents 
    about the {certification_name} certification.
    Identify and collect the most relevant content that could help candidates 
    to get all the knowledge and skills required to pass the certification exam.
    Utilize the certification official website ({certification_link}) and third-party websites, 
    web search and data scraping tools to gather up-to-date information. 
    Focus on collecting study contents and various type of practice questions 
    from official and unofficial sources, and verified answers.
    Focus on {certification_name} certification contents, 
    not generic questions about the topics of the certification.
    Include various situational questions, a situational question being a question 
    that requires the candidate to apply the knowledge to a real-world scenario.
  expected_output: >
    A detailed report with accurate and up-to-date information and resources 
    about the {certification_name} certification.
    This report should include exam topics, exam contents, 
    practice situational questions and answers necessary to create flashcards 
    and mock exams to prepare students to take and pass 
    the {certification_name} certification exam. 
    Make sure the report contains enough content to minimize the overlap 
    between {num_flashcards} flashcard questions and {num_questions} mock exam questions.

In the task configuration, we use {certification_name}, {certification_link}, {num_flashcards} and {num_questions} as variables in the prompt template. They will be replaced by the actual values provided by the user to generate study resources with the desired number of flashcards and questions in the mock exam for the specified mock exam.

Access the code

You can find the complete code example and experience the power of this certification prep crew firsthand by accessing the following GitHub repository.

This repository provides the code to build the certification prep crew and experience it firsthand.

• Define the agents and their roles.
• Create tasks and assign them to the respective agents.
• Orchestrate the workflow using CrewAI’s powerful features.
• Run the crew and observe the agents working together to prepare the certification study resources.

By exploring the notebook in the repository, you can gain valuable insights into the practical implementation of AI agents and experience the transformative potential of CrewAI for your own projects.

The second part of this article will show you how to productionize the crew by building a web app in Python using FastAPI for serving and Vue.js for the frontend and containerizing the app with Docker to make it ready to be shipped to production servers on-premises or in the cloud.

———————

We have just started our journey to build a network of professionals to grow even more our free knowledge-sharing community that will give you a chance to learn interesting things about topics like cloud computing, software development, and software architectures while keeping the door open to more opportunities.

Does this speak to you? If YES, feel free to Join our Discord Server to stay in touch with the community and be part of independently organized events.

———————

Conclusion

In this first part, we’ve explored the foundational concepts of AI agents and multi-agent systems, designed a certification exam preparation system, and built a proof of concept using CrewAI.

Even though this is a proof of concept, it’s already a handy tool for preparing for certification exams. However, it’s not yet ready for production. We still need to evaluate the crew’s outputs, optimize the prompts, the LLM parameters, the costs of LLM API usage, etc.

The second part of this article will show how to productionize the crew by building a web app in Python using `FastAPI` for serving and `Vue.js` for the frontend, and containerizing the app with `Docker` to make it ready to be shipped to production servers on-premises or in the cloud.

Thanks for reading this article. Like, recommend, and share if you enjoyed it. Follow us on Facebook, Twitter, and LinkedIn for more content.