NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.

com
Who Am I?

Paulo Dichone

Software, Cloud, AI Engineer

and Instructor
 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
What Is This Course About?
● AutoGen
○ AutoGen Deep dive
■ Build and customize multi-agent systems
■ Agentic design patterns
○ Effectively implement multi-agent systems
○ Implement agents with different roles that collaborate to accomplish tasks
○ Best practices
○ Real-world, enterprise use cases
 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Who Is This Course For

Machine
Learning
Engineers

AI Engineers

Developers

Open
Data minded
Scientists learners :)
 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Course Prerequisites
1. Know programming (highly preferred… at least the basics)
a. We will be using Python

2. Basics of AI, ML, LLM

3. This is not a programming course
4. Willingness to learn :)
 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Course Structure

Theory (Fundamental Concepts)

Mixture of both

Hands-on
 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Development Environment setup
● Python
● VS Code (or any other code editor)
● OpenAI Account and an OpenAI API Key
 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Set up OpenAI API Account

** Please note that you will need an API key to use OpenAI
services, and there may be some costs associated with using
the API. However, these costs should be minimal.
Dev Environment Setup

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Python (Win, Mac, Linux)

https://kinsta.com/knowledgebase/install-python/
 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
● What is it?
AI Agent ● Why (motivation)?

Crash course ● Advantages

What is an (AI) agent (Motivation)?

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

LLM
How old is
the Universe
Query

knowledge-base

Answer
 What is an (AI) agent (Motivation)?

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

LLM
How old is my my dog,
June?
User/Query

knowledge-base

Generic
Answer/or wrong
answer
 What is an (AI) agent (Motivation)?

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

LLM getJuneAge()
How old is my my dog,
June?
User/Query

knowledge-base

Generic
Answer/or wrong
answer
Agent
Answer
What is an (AI) agent?

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Search the web

Agents
Query a SQL Database

Query
Interact with a CSV File

answer Call a tool…

 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
What is an agent & what they can do?
Personalized recommendations
Browse history
Previous vacations and activities…

Tools

Agents API…
GoogleSearch
Wikipedia
…
 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Motivation Behind AI Agents: Solving Real-World
Problems

Prioritization
Task Agent
Queue
Agent
“Write me a report on
last month Profits and
Loss for our company”

Observer Executor
Agent Agent
Events Response
/actions

Memory/ Tools
context
 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
● What is it?
● Why (motivation)?
AutoGen ● How it works?

Deep Dive ●
●
Key concepts
Hands-on
AutoGen

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

An open-source programming framework used for building AI
agents that can communicate and cooperate with other agents to
solve tasks.
AutoGen Main Features

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Enables building next-gen LLM applications easier- we can build these
applications based on multi-agent conversations
● Simplifies:
○ The orchestration
○ Automation
○ Optimization of complex LLM workflows
○ Heightens LLM models performance
○ Overcomes their weaknesses

Supports diverse conversation patterns (for complex workflows):

● Has customizable and conversational agents
● Developers can use AutoGen to build various types of conversation
patterns
Provides a collection of working systems already:
● Developers can tap into systems that span a wide range of applications
AutoGen Building Blocks

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

AutoGen Agent - an entity that can send and receive message to
and from other agents.
AutoGen Building Blocks

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Example of a AutoGen agent - ConversableAgent

ConversableAgent
Has the following components:
● A list of LLMs
● A code executor
● A function and tool executor
● A component for keeping
human-in-the-loop
AutoGen Installation & Agent - Hands on

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Must have Python set up in your environment
AutoGen Building Blocks

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Multi-agents conversations
Conversation patterns
Multi-agents conversations

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

AutoGen (as a framework)

Human
LLM Agents Tool Agents
Agents
(e.g., GPT-3) (e.g.,APIs)
(user input)

Chat

Task Coordination

Perform Task (with or without

human input)
AutoGen

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

…makes it easy to create advanced applications using multiple agents,
including AI models, tools and humans.

● Organize and automate - easily set up, automate, improve

complex workflows that use LLMs
● Boost performance - enhances performance of LLMs and helps
overcome their weaknesses
● Easy to use - minimal effort to build next-gen LLM applications
● Flexible conversations - support various conversation styles
● Versatile Systems - provides ready-to-use systems
Agents in AutoGen

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

AutoGen abstracts out all intricacies about agents and how they work,
and implements agents that communicate with each other to solve
tasks.

These agents have two features:

● Conversable - agents can send and receive messages

● Customizable - agents can integrate with AI models, tools,
humans or a combination of all.
Built-in Agents in AutoGen

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

ConversableAgent

Human_input_mode = “NEVER”
Group_chat = [ ]

Human_input_mode = “ALWAYS”

AssistantAgent UserProxyAgent GroupChatManager

 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
ConversableAgent - Hands on

Set up a ConversableAgent
Multi-agent Conversation Framework Flow

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Image source: https://microsoft.github.io/autogen/docs/Getting-Started/
Multi-agent Conversation: Diverse Application

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Implementation

Image source: https://microsoft.github.io/autogen/docs/Use-Cases/agent_chat

Code Executors

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Code executor processes messages containing
code:
● Executes the code
● Returns results

There are two built in:

● Command Line Code Executor: runs code
in cmd env (Unix shell)
● Jupyter Executor: runs in a Jupyter env.
Code Executors

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

AutoGen can execute code:
● Locally - runs code directly on the host system (development)
● Docker container - runs code in an isolated container

Local OS (Operating System)

Messages with
Messages with
code blocks
execution output
Agent

Code Executor

Read
Write

Execute Console
Code File Output
 Code Executors & execution environments

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

AutoGen can execute code:
● Locally - runs code directly on the host system (development)
● Docker container - runs code in an isolated container (must
have Docker installed and set up)

Code Executor Environment Platform

LocalCommandLineCodeExecutor Shell Local

DockerCommandLineCodeExecutor Shell Docker

jupyter.JupyterCodeExecutor Jupyter Kernel (e.g., python3) Local/Docker

Docker Execution

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Local OS (Operating System)

Messages with
code blocks
Docker Container

Write Code File

Agent

Code Executor Execute

Console
Read Output

Messages with
execution output
NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Hands-on: running Python code using code
executor
UserProxyAgent and AssistantAgent

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

ConversableAgent

UserProxyAgent AssistantAgent

human_input_mode=ALWAYS human_input_mode=NEVER
llm_config=False code_execution_config=False

Requests human input

Never requests human input
For every message,
And doesn’t execute code
Doesn’t use LLM

Good for for creating code

Good for for creating writer w/o execution
code executor agent capabilities
NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Hands-on: using UserProxyAgent &
AssistantAgent
Best practice

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

UserProxyAgent and AssystantAgent

● Always customize your agent’s “system_message”

instructions for your specific use case.

● Avoid subclassing “UserProxyAgent” and “AssistantAgent”

Human Feedback in Agents

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Humans in-the-loop with agents

ConversableAgent

Messages Human Auto-reply

-in-the- Skip Components
loop (LLM, Code,
Human Reply Executor…

Auto Reply

Terminate
Human Input Modes

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

AutoGen supports 3 modes for human input:
● NEVER - human input is never requested
● TERMINATE (default) - human input is only
requested when a termination condition is met
● ALWAYS - human input is always requested.
Human skip and trigger an auto-reply.
NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Hands-on: Human input modes
Summary

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Deep dive into AutoGen
● Building blocks and key features
● Create our first Agent
● Agent types
● UserProxyAgent and AssistantAgent
● Multi-conversation Framework
● Code Executors in AutoGen
● Adding Human input
● Best practices for UserProxyAgent and
AssistantAgent
LLM Caching

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

AutoGen API Request Caching

AutoGen supports caching API requests for cost reduction and

reusability of previously issued requests

Benefits:
● Reproducibility: consistent results when experiments are
repeated
● Cost savings: reduce expenses associated with repeated
API calls.
LLM Caching

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

assistant = AssistantAgent(
"coding_agent",
llm_config={
"cache_seed": None,
"config_list": OAI_CONFIG_LIST,
"max_tokens": 1024,
},
)

None = disable caching

41 = default
AutoGen and Tools

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

We have no control over what an
agent writes (code).
Local OS (Operating System)

Tools in AutoGen:
● Pre-defined
Write Code File functions
Agent
● Controlled actions
CodeExecutor ● Controlled availability

Tools
Summary

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

AutoGen and tools
● What are tools
● How to use them with Agents
● Hands-on
Conversation patterns

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

● Two-agent chat: simplest conversation pattern with two
agents chatting (used the initiate_chat method)

Agent 1 Agent 2

AutoGen offers more:

● Sequential chat: a series of two-agents linked
by a carryover mechanism
● Group chat: involves more than two agents
● Nested chat: combines a workflow into a single
agent for reuse in larger workflows

But let’s first delve deeper into two-agent chats and their intricacies,
then we’ll explore the other ones…
Two-agent chat - how it works?

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Message

Agent 1

Initial
Message History
Initializer Chat Summarizer Chat Result

Agent 2
Two-Agent Chat

Context
NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Hands-on - Travel Planner Agents
Sequential Chats

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Context Context Context
Message Message Message

Agent 1 Agent 3 Agent 5

Chat Chat Chat

Carryover Carryover
Agent 2 Agent 2 Agent 2

Carryover
Carryover
 NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Hands-on - Sequential Chats Document Processing
Agents
Group Chat

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

GroupChat - a conversation pattern involving more than two
agent, directed by an agent called GroupChatManager

Group
ChatManager

Agent: 1 Agent: 2 Agent: 3 Agent: 4

Group Chat

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Group
ChatManager Step 1: Select Speaker
Step 3: Broadcast Message

Group Chat
Manager
Agent: 1 Agent: 2 Agent: 3 Agent: 4 Message to All Agents

(Select Agent 2)
Agent: 1 Agent: 2 Agent: 3 Agent: 4
Step 2: Agent Speak

Group
ChatManager (Repeat steps 1 to 3)

Message from Agent B

Agent: 1 Agent: 2 Agent: 3 Agent: 4

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Hands-on - GroupChat
NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Hands-on - GroupChat in a Sequential Chat
 Nested Chat

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Context,
Message

Human-in-the-loop
Sequential Chats

Trigger Agent 1 Agent 1 Agent 1

Chat Chat Chat

Nested Chat
Reply Agent 2 Agent 2 Agent 2

Nested Chats
Handler

Auto-reply
components(code, Nested Chats
executor, LLMs…)

Agent A
NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Hands-on - Nested Chats
Summary

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Conversation patterns:
● Two-agent chat
● Sequential chat
● Group chat
● Nested chat

Use these patterns to compose blocks to create complex

workflows.
Hands-on: Customer Support Automation

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

inquiries Customer
inquiry Agent

Response
Agent

Knowledge
Base Agent

Troubleshooting
Agent

Human Support
Escalation Agent
Agent
Hands-on: Automated Financial Report Generation and Review

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Data
aggregation
Agent

Report
Generation
Agent

Accuracy
Review Agent

Compliance
Review Agent

Summary
Feedback Agent
Generation
Agent
Hands-on: Automating Research Paper Analysis and

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Visualization
Agent Initiates
Chat

Assis. Agent
(Finds paper
on topic)

Assis. Agent
Analyzes
specific
applications

Assis. Agent
Generates Bar
Chart and Saves
AutoGen and RAG (Retrieval-Augmented

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

Generation

● A way of combining language models with external knowledge

retrieval to improve the quality and relevance of generated response.

We can do that by:

● Constructing agent chats with RetrieveAssistantAgent
● REtrieveUerProxyAgent classes
AutoGen Studio

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

● A way of combining language models with external knowledge
retrieval to improve the quality and relevance of generated response.

We can do that by:

● Constructing agent chats with RetrieveAssistantAgent
● REtrieveUerProxyAgent classes
NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
You made it to the end!
● Next steps…
Congratulations!
Course Summary

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

● AutoGen
● AutoGen Deep dive
● Build and customize multi-agent systems
● Agentic design patterns
○ Two-Agent Chat
○ Sequential Chat
○ Group Chat
○ Nested Chat
● Effectively implement multi-agent systems
● Implement agents with different roles that collaborate to accomplish
tasks
● Best practices
● Real-world, enterprise use cases (hands-on)
Wrap up - Where to Go From Here?

NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com

● Keep learning
○ Extend the projects we worked on in this course
○ Design and implement your own agents
● https://microsoft.github.io/autogen/docs/Getting-Started
NOT FOR DISTRIBUTION ©Paulo Dichone www.vincibits.com
Thank you!