Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions examples/ai-mcp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# ai-mcp — Azure OpenAI + MCP sample

A Teams bot powered by **Azure OpenAI** (via the `openai` SDK's `AzureOpenAI` client) and the **Model Context Protocol** SDK. Demonstrates streaming responses, per-conversation memory, a local clarification-card tool, remote MCP server tools, inline citations, follow-up suggestions, and custom feedback.

This is the TypeScript counterpart to the .NET [`ExtAIBot`](https://github.com/microsoft/teams.net/pull/486) sample. The structural shape matches: a single bot process owns the AI plumbing, runs the tool-call loop in-process, and connects to MCP servers directly. The auto-loop comes from `openai.chat.completions.runTools()` — the OpenAI SDK's helper that auto-executes each tool's `function` callback and feeds the result back to the model until it produces final text.

> **Provider scope.** This sample is bound to the OpenAI chat-completions wire protocol — Azure OpenAI works; vanilla OpenAI works; non-OpenAI providers do not. (See the .NET sample for an `IChatClient` abstraction that's provider-agnostic — TS has no equivalent today.)

## Features

- **Streaming** — `runner.on('content', delta => stream.emit(delta))` forwards text token-by-token
- **Conversation memory** — each conversation keeps its own `Map<conv, ChatCompletionMessageParam[]>` in process
- **Local tool** — `request_clarification`: a `RunnableToolFunction` whose `function` callback pushes an Adaptive Card into a per-turn bucket and returns a placeholder string. The agent discards its wrap-up text and sends only the card.
- **MCP client** — connects to the [Microsoft Learn docs MCP server](https://learn.microsoft.com/api/mcp) at startup. Each MCP tool is wrapped as a `RunnableToolFunction` whose `function` callback invokes the server and feeds the raw result into the citation collector before returning it to the model.
- **Inline citations** — `CitationCollector` parses MCP tool results for `{ contentUrl, title, content }` records; `[N]` markers in the final reply become clickable Teams citation entities.
- **Follow-up suggestions** — a separate non-streaming `chat.completions.create({ response_format: { type: 'json_schema', ... }})` call produces two short prompts shown as suggested-action chips.
- **Custom feedback** — every text reply enables `addFeedback('custom')`; clicking thumbs up/down opens a bot-rendered task module, and submissions hit `message.submit.feedback`.

## Prerequisites

- Node.js 20+
- An **Azure OpenAI resource** with a deployed model (e.g. `gpt-4o`) and an API key. No Foundry project required.
- A Teams bot registration (App ID + secret).

## Setup

Create a `.env` in this directory:

```env
AZURE_OPENAI_ENDPOINT=https://<your-resource>.openai.azure.com
AZURE_OPENAI_API_KEY=<your-api-key>
AZURE_OPENAI_MODEL_DEPLOYMENT_NAME=<deployment-name>
AZURE_OPENAI_API_VERSION=2024-10-21

# Optional — defaults to the public MS Learn MCP endpoint.
MCP_SERVER_URL=https://learn.microsoft.com/api/mcp
```

`AZURE_OPENAI_MODEL_DEPLOYMENT_NAME` is the **deployment name** on your Azure OpenAI resource, not the base model name.

## Running

```bash
npm install
npm run dev --workspace=@examples/ai-mcp
```

The bot connects to the MS Learn MCP server at startup and lists its tools before accepting messages. If the MCP server is unreachable, startup fails — by design, since the sample is meant to demonstrate the MCP path.

## Example interactions

- `Tell me about streaming` — ambiguous: the agent calls `request_clarification`; the bot replies with a clarification card. Pick an option → that choice arrives as the next user turn and the agent answers based on it.
- `How do I stream in teams.ts?` — agent calls an MS Learn search tool, replies with a docs-grounded answer and inline citations, plus two follow-up chips.
- `How do I list users with Microsoft Graph?` — same MCP search path, lands on Graph docs.

## How the pieces fit

```
src/index.ts — AzureOpenAI client + MCP init + Agent + handler registration
src/agent.ts — per-conv chat history, runTools auto-loop, follow-ups
src/local-tools.ts — request_clarification RunnableToolFunction + card builder
src/mcp-tools.ts — MCP client lifecycle, tool listing, wraps each tool
as a RunnableToolFunction
src/citation-collector.ts — parses MCP results, attaches Teams citations to the reply
src/handlers.ts — message, card.action.clarification, message.fetch-task,
message.submit.feedback
```

### Tool loop

`openai.chat.completions.runTools({ stream: true, tools })` does the heavy lifting:

1. Sends the request with our tool definitions to Azure OpenAI.
2. If the model emits a `tool_calls` choice, the runner invokes the matching tool's `function` callback (passing parsed args).
3. The tool's return value is appended as a `role: 'tool'` message and the model is re-prompted.
4. Steps 2-3 repeat until the model produces a `content` message instead of a tool call.
5. Throughout, `content` events fire for each text delta — we forward them straight to the Teams stream.

Each tool function is responsible for its own side effects:

- `request_clarification` pushes the card into `pendingCards[]`.
- Each MCP tool invokes the server, feeds the raw result into `CitationCollector.tryExtract`, and returns the text to the model.

After `runner.done()`, the agent inspects `pendingCards` and the citation collector to assemble the final Teams activity.

### Clarification flow

1. User: ambiguous question.
2. Model calls `request_clarification`. The callback builds the card, pushes it onto `pendingCards`, returns `"Clarification card attached."`.
3. The model is re-prompted with that tool result and produces a brief wrap-up (e.g. "I've asked for clarification — please pick an option"). We discard this — `pendingCard` is set, so `replyText` is forced to `''`.
4. Bot sends an attachment-only message containing only the card.
5. User picks an option, submits → `card.action.clarification` invoke → handler calls `agent.run(conv, choice, stream)` exactly as if the choice were a new user message.
6. Model now has full context (its own previous tool call + the user's choice) and answers based on it.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
"manifestVersion": "1.20",
"id": "${{TEAMS_APP_ID}}",
"name": {
"short": "ai-sample",
"full": "ai-sample"
"short": "ai-mcp",
"full": "AI + MCP sample bot"
},
"developer": {
"name": "Microsoft",
Expand All @@ -15,8 +15,8 @@
"termsOfUseUrl": "https://www.microsoft.com/legal/terms-of-use"
},
"description": {
"short": "Sample bot that repeats back what you say",
"full": "Sample bot that repeats back what you say"
"short": "Teams bot powered by Azure OpenAI + MS Learn MCP",
"full": "A Teams bot that streams answers grounded in MS Learn docs via the MCP protocol, with inline citations, follow-up chips, a clarification card tool, and custom feedback."
},
"icons": {
"outline": "outline.png",
Expand All @@ -43,10 +43,7 @@
"supportsFiles": false
}
],
"validDomains": [
"${{BOT_DOMAIN}}",
"*.botframework.com"
],
"validDomains": ["${{BOT_DOMAIN}}", "*.botframework.com"],
"webApplicationInfo": {
"id": "${{BOT_ID}}",
"resource": "api://botid-${{BOT_ID}}"
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes.
15 changes: 7 additions & 8 deletions examples/ai/package.json → examples/ai-mcp/package.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"name": "@examples/ai",
"name": "@examples/ai-mcp",
"version": "0.0.6",
"private": true,
"license": "MIT",
Expand All @@ -21,21 +21,20 @@
"dev:teamsfx:launch-testtool": "npx env-cmd --silent -f env/.env.testtool teamsapptester start"
},
"dependencies": {
"@microsoft/teams.ai": "*",
"@microsoft/teams.api": "*",
"@microsoft/teams.apps": "*",
"@microsoft/teams.cards": "*",
"@microsoft/teams.common": "*",
"@microsoft/teams.dev": "*",
"@microsoft/teams.openai": "*",
"fuse.js": "^7.1.0"
"@modelcontextprotocol/sdk": "^1.25.2",
"openai": "^4.104.0"
},
"devDependencies": {
"rimraf": "^6.0.1",
"@microsoft/teams.config": "*",
"@types/node": "^22.5.4",
"dotenv": "^16.4.5",
"env-cmd": "latest",
"rimraf": "^6.0.1",
"tsx": "^4.20.6",
"typescript": "^5.4.5",
"env-cmd": "latest"
"typescript": "^5.4.5"
}
}
201 changes: 201 additions & 0 deletions examples/ai-mcp/src/agent.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
import { IStreamer } from '@microsoft/teams.apps';
import { AdaptiveCard } from '@microsoft/teams.cards';
import { ILogger } from '@microsoft/teams.common';


import { CitationCollector } from './citation-collector';
import { buildClarificationTool } from './local-tools';
import { McpToolSet } from './mcp-tools';

import type { AzureOpenAI } from 'openai';
import type { ChatCompletionMessageParam } from 'openai/resources/chat/completions';

const SYSTEM_PROMPT = `\
You are a Teams docs assistant that can search Microsoft Learn (Teams, .NET, TypeScript, Microsoft Graph, Azure)
and explain bot concepts (streaming, Adaptive Cards, citations, feedback).

When you use information from a search tool, cite your sources inline using a 1-based numeric marker for each result
you reference (e.g. [1], [2]). Use the same number consistently for the same source within a reply.
Do not add a references or sources list at the end of your response — citations are displayed separately in the UI.

If the user's request is ambiguous or could mean two or more things, call the request_clarification tool with a short
question and 2-4 candidate interpretations rather than guessing.`;

const FOLLOW_UPS_PROMPT = `\
Produce 2 specific prompts the user might want to ask next, based on the conversation so far.

Each prompt MUST:
- Be phrased in the first person, as the user would type.
- Stay under 8 words.

Drill into a concrete topic, API, or concept that just came up — or, if the conversation just started, suggest
prompts that showcase what you can help with.`;

const FOLLOW_UPS_SCHEMA = {
type: 'object',
properties: {
prompt1: { type: 'string' },
prompt2: { type: 'string' },
},
required: ['prompt1', 'prompt2'],
additionalProperties: false,
} as const;

export type AgentRunResult = {
fullText: string;
pendingCard: AdaptiveCard | null;
citations: CitationCollector;
followUps: string[];
};

export type AgentOptions = {
client: AzureOpenAI;
deploymentName: string;
mcpTools: McpToolSet;
log: ILogger;
};

/**
* Agent for one Teams bot. Holds the Azure OpenAI client, the MCP tool set,
* and per-conversation chat history.
*
* The tool-call loop is driven by the OpenAI SDK's `runTools()` helper,
* which auto-executes each `function` callback we declare on the tools and
* re-prompts the model with the result until the model produces final text.
* Per-turn side effects (pushing the clarification card, extracting MCP
* citations) ride along inside those tool callbacks via closure-captured
* buckets.
*
* Per-conversation history mutation is serialized by chaining each turn
* onto the previous turn's promise — concurrent submits (e.g. clarification
* race) would otherwise interleave assistant/tool messages.
*/
export class Agent {
private readonly _client: AzureOpenAI;
private readonly _deployment: string;
private readonly _mcpTools: McpToolSet;
private readonly _log: ILogger;
private readonly _histories = new Map<string, ChatCompletionMessageParam[]>();
private readonly _locks = new Map<string, Promise<unknown>>();

constructor(opts: AgentOptions) {
this._client = opts.client;
this._deployment = opts.deploymentName;
this._mcpTools = opts.mcpTools;
this._log = opts.log;
}

async run(teamsConvId: string, userText: string, stream: IStreamer): Promise<AgentRunResult> {
const previous = this._locks.get(teamsConvId) ?? Promise.resolve();
const turn = previous.then(() => this._runTurn(teamsConvId, userText, stream));
this._locks.set(
teamsConvId,
turn.catch(() => {})
);
return turn;
}

private async _runTurn(
Comment thread
MehakBindra marked this conversation as resolved.
teamsConvId: string,
userText: string,
stream: IStreamer
): Promise<AgentRunResult> {
const history = this._getOrCreateHistory(teamsConvId);
history.push({ role: 'user', content: userText });

const citations = new CitationCollector();
const pendingCards: AdaptiveCard[] = [];

stream.update('Thinking...');

const fullText = await this._streamWithTools(history, pendingCards, citations, stream);

const pendingCard = pendingCards[0] ?? null;
// Card-only reply: discard the wrap-up text the model produced after
// the clarification tool returned. The card itself is the bot's reply.
const replyText = pendingCard ? '' : fullText;
const followUps = pendingCard ? [] : await this._generateFollowUps(history);

return { fullText: replyText, pendingCard, citations, followUps };
}

/**
* Drives one chat-completions run with auto tool-calling and streaming.
* Returns the accumulated assistant text. History is updated in place by
* the runner; we copy `runner.messages` back into our map.
*/
private async _streamWithTools(
history: ChatCompletionMessageParam[],
pendingCards: AdaptiveCard[],
citations: CitationCollector,
stream: IStreamer
): Promise<string> {
const tools = [
buildClarificationTool(pendingCards, this._log),
...this._mcpTools.asRunnableTools(citations),
];

const runner = this._client.chat.completions.runTools({
model: this._deployment,
messages: history,
tools,
stream: true,
});

let fullText = '';
runner.on('content', (delta: string) => {
fullText += delta;
stream.emit(delta);
});

await runner.done();

// Sync our history with the runner's view: it includes the system +
// user + every tool_call / tool result / assistant message added
// during the auto-loop, so the next turn sees the full prior context.
const ran = runner.messages as ChatCompletionMessageParam[];
history.splice(0, history.length, ...ran);

return fullText;
}

private _getOrCreateHistory(teamsConvId: string): ChatCompletionMessageParam[] {
let history = this._histories.get(teamsConvId);
if (!history) {
history = [{ role: 'system', content: SYSTEM_PROMPT }];
this._histories.set(teamsConvId, history);
}
return history;
}

/**
* Generates two follow-up prompts via a separate non-streaming call with
* a strict JSON schema. Any parse/network failure silently degrades to
* no chips so the main reply still ships.
*/
private async _generateFollowUps(history: ChatCompletionMessageParam[]): Promise<string[]> {
try {
const completion = await this._client.chat.completions.create({
model: this._deployment,
messages: [...history, { role: 'system', content: FOLLOW_UPS_PROMPT }],
response_format: {
type: 'json_schema',
json_schema: {
name: 'follow_ups',
strict: true,
schema: FOLLOW_UPS_SCHEMA,
},
},
});

const raw = completion.choices[0]?.message?.content ?? '';
const parsed = JSON.parse(raw) as { prompt1?: string; prompt2?: string };
return [parsed.prompt1, parsed.prompt2].filter(
(s): s is string => typeof s === 'string' && s.length > 0
);
} catch (err) {
this._log.warn(`Follow-up generation failed: ${(err as Error).message}`);
return [];
}
}
}
Loading
Loading