Skip to content

Commit d034d90

Browse files
authored
feat(frameworks): Add Mastra (#15076)
Hello 👋 This adds Mastra as a framework preset to the frameworks package. We currently have this guide (https://mastra.ai/guides/deployment/vercel) and it works great! Our `VercelDeployer()` package outputs a `.vercel/output` directory conforming to Vercel's [Build Output API](https://vercel.com/docs/build-output-api/v3) so that you can deploy Mastra as serverless functions on Vercel. With this PR I want to add some polish to this experience by providing logos, descriptions and preset build commands to people deploying to Vercel. While `mastra build` creates a `.mastra` folder the VercelDeployer() is doing the work. Let me know if I need to adjust the output directory or we can keep it at `.mastra`. <!-- VADE_RISK_START --> > [!NOTE] > Low Risk Change > > This PR adds a new Mastra framework preset with example code, logos, documentation, and integration test - all additive changes with no modifications to existing security, auth, or business logic. > > - Adds new framework preset configuration in frameworks.ts with build/dev commands > - Adds complete example project with agents, tools, workflows, and scorers > - Adds integration test, logos, and documentation files > > <sup>Risk assessment for [commit b43f307](https://github.com/vercel/vercel/commit/b43f307848ced0d26ff6ef5f0d5770ec79f3a5ee).</sup> <!-- VADE_RISK_END -->
1 parent 4b6a9b6 commit d034d90

File tree

16 files changed

+786
-51
lines changed

16 files changed

+786
-51
lines changed

.changeset/thick-bananas-happen.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@vercel/frameworks': minor
3+
---
4+
5+
Add Mastra framework preset
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
import { deployExample } from '../test-utils';
2+
it('[examples] should deploy mastra', async () => {
3+
await deployExample('mastra');
4+
});

examples/mastra/.env.example

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
OPENAI_API_KEY=your-api-key

examples/mastra/.gitignore

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
output.txt
2+
node_modules
3+
dist
4+
.mastra
5+
.env.development
6+
.env
7+
*.db
8+
*.db-*
9+
.netlify
10+
.vercel

examples/mastra/README.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# mastra
2+
3+
Welcome to your new [Mastra](https://mastra.ai/) project! We're excited to see what you'll build.
4+
5+
## Getting Started
6+
7+
Start the development server:
8+
9+
```shell
10+
npm run dev
11+
```
12+
13+
Open [http://localhost:4111](http://localhost:4111) in your browser to access [Mastra Studio](https://mastra.ai/docs/getting-started/studio). It provides an interactive UI for building and testing your agents, along with a REST API that exposes your Mastra application as a local service. This lets you start building without worrying about integration right away.
14+
15+
You can start editing files inside the `src/mastra` directory. The development server will automatically reload whenever you make changes.
16+
17+
## Learn more
18+
19+
To learn more about Mastra, visit our [documentation](https://mastra.ai/docs/). Your bootstrapped project includes example code for [agents](https://mastra.ai/docs/agents/overview), [tools](https://mastra.ai/docs/agents/using-tools), [workflows](https://mastra.ai/docs/workflows/overview), [scorers](https://mastra.ai/docs/evals/overview), and [observability](https://mastra.ai/docs/observability/overview).
20+
21+
If you're new to AI agents, check out our [course](https://mastra.ai/course) and [YouTube videos](https://youtube.com/@mastra-ai). You can also join our [Discord](https://discord.gg/BTYqqHKUrf) community to get help and share your projects.
22+
23+
## Deploy on Mastra Cloud
24+
25+
[Mastra Cloud](https://cloud.mastra.ai/) gives you a serverless agent environment with atomic deployments. Access your agents from anywhere and monitor performance. Make sure they don't go off the rails with evals and tracing.
26+
27+
Check out the [deployment guide](https://mastra.ai/docs/deployment/overview) for more details.

examples/mastra/package.json

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"name": "mastra",
3+
"version": "1.0.0",
4+
"description": "",
5+
"main": "index.js",
6+
"scripts": {
7+
"test": "echo \"Error: no test specified\" && exit 1",
8+
"dev": "mastra dev",
9+
"build": "mastra build",
10+
"start": "mastra start"
11+
},
12+
"keywords": [],
13+
"author": "",
14+
"license": "ISC",
15+
"type": "module",
16+
"engines": {
17+
"node": ">=22.13.0"
18+
},
19+
"dependencies": {
20+
"@mastra/core": "^1.12.0",
21+
"@mastra/deployer-vercel": "^1.1.3",
22+
"@mastra/evals": "^1.1.2",
23+
"@mastra/libsql": "^1.7.0",
24+
"@mastra/loggers": "^1.0.2",
25+
"@mastra/memory": "^1.7.0",
26+
"@mastra/observability": "^1.4.0",
27+
"zod": "^4"
28+
},
29+
"devDependencies": {
30+
"@types/node": "^22.19.0",
31+
"mastra": "^1.3.9",
32+
"typescript": "^5.9.3"
33+
}
34+
}
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
import { Agent } from '@mastra/core/agent';
2+
import { Memory } from '@mastra/memory';
3+
import { weatherTool } from '../tools/weather-tool';
4+
import { scorers } from '../scorers/weather-scorer';
5+
6+
export const weatherAgent = new Agent({
7+
id: 'weather-agent',
8+
name: 'Weather Agent',
9+
instructions: `
10+
You are a helpful weather assistant that provides accurate weather information and can help planning activities based on the weather.
11+
12+
Your primary function is to help users get weather details for specific locations. When responding:
13+
- Always ask for a location if none is provided
14+
- If the location name isn't in English, please translate it
15+
- If giving a location with multiple parts (e.g. "New York, NY"), use the most relevant part (e.g. "New York")
16+
- Include relevant details like humidity, wind conditions, and precipitation
17+
- Keep responses concise but informative
18+
- If the user asks for activities and provides the weather forecast, suggest activities based on the weather forecast.
19+
- If the user asks for activities, respond in the format they request.
20+
21+
Use the weatherTool to fetch current weather data.
22+
`,
23+
model: 'openai/gpt-5.4',
24+
tools: { weatherTool },
25+
scorers: {
26+
toolCallAppropriateness: {
27+
scorer: scorers.toolCallAppropriatenessScorer,
28+
sampling: {
29+
type: 'ratio',
30+
rate: 1,
31+
},
32+
},
33+
completeness: {
34+
scorer: scorers.completenessScorer,
35+
sampling: {
36+
type: 'ratio',
37+
rate: 1,
38+
},
39+
},
40+
translation: {
41+
scorer: scorers.translationScorer,
42+
sampling: {
43+
type: 'ratio',
44+
rate: 1,
45+
},
46+
},
47+
},
48+
memory: new Memory(),
49+
});
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
2+
import { Mastra } from '@mastra/core/mastra';
3+
import { PinoLogger } from '@mastra/loggers';
4+
import { LibSQLStore } from '@mastra/libsql';
5+
import { Observability, DefaultExporter, CloudExporter, SensitiveDataFilter } from '@mastra/observability';
6+
import { VercelDeployer } from "@mastra/deployer-vercel";
7+
import { weatherWorkflow } from './workflows/weather-workflow';
8+
import { weatherAgent } from './agents/weather-agent';
9+
import { toolCallAppropriatenessScorer, completenessScorer, translationScorer } from './scorers/weather-scorer';
10+
11+
export const mastra = new Mastra({
12+
workflows: { weatherWorkflow },
13+
agents: { weatherAgent },
14+
scorers: { toolCallAppropriatenessScorer, completenessScorer, translationScorer },
15+
storage: new LibSQLStore({
16+
id: "mastra-storage",
17+
// stores observability, scores, ... into persistent file storage
18+
url: "file:./mastra.db",
19+
}),
20+
logger: new PinoLogger({
21+
name: 'Mastra',
22+
level: 'info',
23+
}),
24+
deployer: new VercelDeployer(),
25+
observability: new Observability({
26+
configs: {
27+
default: {
28+
serviceName: 'mastra',
29+
exporters: [
30+
new DefaultExporter(), // Persists traces to storage for Mastra Studio
31+
new CloudExporter(), // Sends traces to Mastra Cloud (if MASTRA_CLOUD_ACCESS_TOKEN is set)
32+
],
33+
spanOutputProcessors: [
34+
new SensitiveDataFilter(), // Redacts sensitive data like passwords, tokens, keys
35+
],
36+
},
37+
},
38+
}),
39+
});
Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
import { z } from 'zod';
2+
import { createToolCallAccuracyScorerCode } from '@mastra/evals/scorers/prebuilt';
3+
import { createCompletenessScorer } from '@mastra/evals/scorers/prebuilt';
4+
import {
5+
getAssistantMessageFromRunOutput,
6+
getUserMessageFromRunInput,
7+
} from '@mastra/evals/scorers/utils';
8+
import { createScorer } from '@mastra/core/evals';
9+
10+
export const toolCallAppropriatenessScorer = createToolCallAccuracyScorerCode({
11+
expectedTool: 'weatherTool',
12+
strictMode: false,
13+
});
14+
15+
export const completenessScorer = createCompletenessScorer();
16+
17+
// Custom LLM-judged scorer: evaluates if non-English locations are translated appropriately
18+
export const translationScorer = createScorer({
19+
id: 'translation-quality-scorer',
20+
name: 'Translation Quality',
21+
description:
22+
'Checks that non-English location names are translated and used correctly',
23+
type: 'agent',
24+
judge: {
25+
model: 'openai/gpt-5.4',
26+
instructions:
27+
'You are an expert evaluator of translation quality for geographic locations. ' +
28+
'Determine whether the user text mentions a non-English location and whether the assistant correctly uses an English translation of that location. ' +
29+
'Be lenient with transliteration differences and diacritics. ' +
30+
'Return only the structured JSON matching the provided schema.',
31+
},
32+
})
33+
.preprocess(({ run }) => {
34+
const userText = getUserMessageFromRunInput(run.input) || '';
35+
const assistantText = getAssistantMessageFromRunOutput(run.output) || '';
36+
return { userText, assistantText };
37+
})
38+
.analyze({
39+
description:
40+
'Extract location names and detect language/translation adequacy',
41+
outputSchema: z.object({
42+
nonEnglish: z.boolean(),
43+
translated: z.boolean(),
44+
confidence: z.number().min(0).max(1).default(1),
45+
explanation: z.string().default(''),
46+
}),
47+
createPrompt: ({ results }) => `
48+
You are evaluating if a weather assistant correctly handled translation of a non-English location.
49+
User text:
50+
"""
51+
${results.preprocessStepResult.userText}
52+
"""
53+
Assistant response:
54+
"""
55+
${results.preprocessStepResult.assistantText}
56+
"""
57+
Tasks:
58+
1) Identify if the user mentioned a location that appears non-English.
59+
2) If non-English, check whether the assistant used a correct English translation of that location in its response.
60+
3) Be lenient with transliteration differences (e.g., accents/diacritics).
61+
Return JSON with fields:
62+
{
63+
"nonEnglish": boolean,
64+
"translated": boolean,
65+
"confidence": number, // 0-1
66+
"explanation": string
67+
}
68+
`,
69+
})
70+
.generateScore(({ results }) => {
71+
const r = (results as any)?.analyzeStepResult || {};
72+
if (!r.nonEnglish) return 1; // If not applicable, full credit
73+
if (r.translated)
74+
return Math.max(0, Math.min(1, 0.7 + 0.3 * (r.confidence ?? 1)));
75+
return 0; // Non-English but not translated
76+
})
77+
.generateReason(({ results, score }) => {
78+
const r = (results as any)?.analyzeStepResult || {};
79+
return `Translation scoring: nonEnglish=${r.nonEnglish ?? false}, translated=${r.translated ?? false}, confidence=${r.confidence ?? 0}. Score=${score}. ${r.explanation ?? ''}`;
80+
});
81+
82+
export const scorers = {
83+
toolCallAppropriatenessScorer,
84+
completenessScorer,
85+
translationScorer,
86+
};
Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
import { createTool } from '@mastra/core/tools';
2+
import { z } from 'zod';
3+
4+
interface GeocodingResponse {
5+
results: {
6+
latitude: number;
7+
longitude: number;
8+
name: string;
9+
}[];
10+
}
11+
interface WeatherResponse {
12+
current: {
13+
time: string;
14+
temperature_2m: number;
15+
apparent_temperature: number;
16+
relative_humidity_2m: number;
17+
wind_speed_10m: number;
18+
wind_gusts_10m: number;
19+
weather_code: number;
20+
};
21+
}
22+
23+
export const weatherTool = createTool({
24+
id: 'get-weather',
25+
description: 'Get current weather for a location',
26+
inputSchema: z.object({
27+
location: z.string().describe('City name'),
28+
}),
29+
outputSchema: z.object({
30+
temperature: z.number(),
31+
feelsLike: z.number(),
32+
humidity: z.number(),
33+
windSpeed: z.number(),
34+
windGust: z.number(),
35+
conditions: z.string(),
36+
location: z.string(),
37+
}),
38+
execute: async (inputData) => {
39+
return await getWeather(inputData.location);
40+
},
41+
});
42+
43+
const getWeather = async (location: string) => {
44+
const geocodingUrl = `https://geocoding-api.open-meteo.com/v1/search?name=${encodeURIComponent(location)}&count=1`;
45+
const geocodingResponse = await fetch(geocodingUrl);
46+
const geocodingData = (await geocodingResponse.json()) as GeocodingResponse;
47+
48+
if (!geocodingData.results?.[0]) {
49+
throw new Error(`Location '${location}' not found`);
50+
}
51+
52+
const { latitude, longitude, name } = geocodingData.results[0];
53+
54+
const weatherUrl = `https://api.open-meteo.com/v1/forecast?latitude=${latitude}&longitude=${longitude}&current=temperature_2m,apparent_temperature,relative_humidity_2m,wind_speed_10m,wind_gusts_10m,weather_code`;
55+
56+
const response = await fetch(weatherUrl);
57+
const data = (await response.json()) as WeatherResponse;
58+
59+
return {
60+
temperature: data.current.temperature_2m,
61+
feelsLike: data.current.apparent_temperature,
62+
humidity: data.current.relative_humidity_2m,
63+
windSpeed: data.current.wind_speed_10m,
64+
windGust: data.current.wind_gusts_10m,
65+
conditions: getWeatherCondition(data.current.weather_code),
66+
location: name,
67+
};
68+
};
69+
70+
function getWeatherCondition(code: number): string {
71+
const conditions: Record<number, string> = {
72+
0: 'Clear sky',
73+
1: 'Mainly clear',
74+
2: 'Partly cloudy',
75+
3: 'Overcast',
76+
45: 'Foggy',
77+
48: 'Depositing rime fog',
78+
51: 'Light drizzle',
79+
53: 'Moderate drizzle',
80+
55: 'Dense drizzle',
81+
56: 'Light freezing drizzle',
82+
57: 'Dense freezing drizzle',
83+
61: 'Slight rain',
84+
63: 'Moderate rain',
85+
65: 'Heavy rain',
86+
66: 'Light freezing rain',
87+
67: 'Heavy freezing rain',
88+
71: 'Slight snow fall',
89+
73: 'Moderate snow fall',
90+
75: 'Heavy snow fall',
91+
77: 'Snow grains',
92+
80: 'Slight rain showers',
93+
81: 'Moderate rain showers',
94+
82: 'Violent rain showers',
95+
85: 'Slight snow showers',
96+
86: 'Heavy snow showers',
97+
95: 'Thunderstorm',
98+
96: 'Thunderstorm with slight hail',
99+
99: 'Thunderstorm with heavy hail',
100+
};
101+
return conditions[code] || 'Unknown';
102+
}

0 commit comments

Comments
 (0)