Skip to content

Commit 345c5f6

Browse files
DellaBittaDellaBitta
authored
feat(ai) Google Maps Grounding Support (#9458)
### Discussion Implement new tools to use Grounding with Google Maps, similar to how Grounding with Google Search works. ### Testing Integration tests. Unit tests. Node standalone app. Updated Quickstart React-based app (not part of this PR). ### API Changes - Grounding with Google Maps support, see internal doc: go/fal-maps-grounding-api - TemplateToolConfig, see internal doc: go/firebase-ai-server-prompt-template
1 parent 8e384c9 commit 345c5f6

20 files changed

Lines changed: 940 additions & 22 deletions

‎.changeset/quick-cars-obey.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
'@firebase/ai': minor
3+
'firebase': minor
4+
---
5+
6+
AI Logic : Feature : Added support for Grounding with Google Maps.

‎common/api-review/ai.api.md‎

Lines changed: 44 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -690,6 +690,25 @@ export interface GoogleAIGenerateContentResponse {
690690
usageMetadata?: UsageMetadata;
691691
}
692692

693+
// @public
694+
export interface GoogleMaps {
695+
// (undocumented)
696+
enableWidget?: boolean;
697+
}
698+
699+
// @public
700+
export interface GoogleMapsGroundingChunk {
701+
placeId?: string;
702+
text?: string;
703+
title?: string;
704+
uri?: string;
705+
}
706+
707+
// @public
708+
export interface GoogleMapsTool {
709+
googleMaps: GoogleMaps;
710+
}
711+
693712
// @public
694713
export interface GoogleSearch {
695714
}
@@ -701,11 +720,13 @@ export interface GoogleSearchTool {
701720

702721
// @public
703722
export interface GroundingChunk {
723+
maps?: GoogleMapsGroundingChunk;
704724
web?: WebGroundingChunk;
705725
}
706726

707727
// @public
708728
export interface GroundingMetadata {
729+
googleMapsWidgetContextToken?: string;
709730
groundingChunks?: GroundingChunk[];
710731
groundingSupports?: GroundingSupport[];
711732
// @deprecated (undocumented)
@@ -987,6 +1008,12 @@ export interface LanguageModelPromptOptions {
9871008
responseConstraint?: object;
9881009
}
9891010

1011+
// @public
1012+
export interface LatLng {
1013+
latitude?: number;
1014+
longitude?: number;
1015+
}
1016+
9901017
// @beta
9911018
export interface LiveGenerationConfig {
9921019
contextWindowCompression?: ContextWindowCompressionConfig;
@@ -1224,6 +1251,12 @@ export const ResponseModality: {
12241251
// @beta
12251252
export type ResponseModality = (typeof ResponseModality)[keyof typeof ResponseModality];
12261253

1254+
// @public
1255+
export interface RetrievalConfig {
1256+
languageCode?: string;
1257+
latLng?: LatLng;
1258+
}
1259+
12271260
// @public (undocumented)
12281261
export interface RetrievedContextAttribution {
12291262
// (undocumented)
@@ -1480,8 +1513,8 @@ export class TemplateGenerativeModel {
14801513
constructor(ai: AI, requestOptions?: RequestOptions);
14811514
// @internal (undocumented)
14821515
_apiSettings: ApiSettings;
1483-
generateContent(templateId: string, templateVariables: Record<string, unknown>, singleRequestOptions?: SingleRequestOptions): Promise<GenerateContentResult>;
1484-
generateContentStream(templateId: string, templateVariables: Record<string, unknown>, singleRequestOptions?: SingleRequestOptions): Promise<GenerateContentStreamResult>;
1516+
generateContent(templateId: string, templateVariables: Record<string, unknown>, singleRequestOptions?: SingleRequestOptions, templateToolConfig?: TemplateToolConfig): Promise<GenerateContentResult>;
1517+
generateContentStream(templateId: string, templateVariables: Record<string, unknown>, singleRequestOptions?: SingleRequestOptions, templateToolConfig?: TemplateToolConfig): Promise<GenerateContentStreamResult>;
14851518
requestOptions?: RequestOptions;
14861519
startChat(params: StartTemplateChatParams): TemplateChatSession;
14871520
}
@@ -1507,6 +1540,12 @@ export interface TemplateRequestInternal extends Omit<TemplateGenerateContentReq
15071540
// @beta
15081541
export type TemplateTool = TemplateFunctionDeclarationsTool;
15091542

1543+
// @public
1544+
export interface TemplateToolConfig {
1545+
// (undocumented)
1546+
retrievalConfig?: RetrievalConfig;
1547+
}
1548+
15101549
// @public
15111550
export interface TextPart {
15121551
// (undocumented)
@@ -1546,12 +1585,14 @@ export const ThinkingLevel: {
15461585
export type ThinkingLevel = (typeof ThinkingLevel)[keyof typeof ThinkingLevel];
15471586

15481587
// @public
1549-
export type Tool = FunctionDeclarationsTool | GoogleSearchTool | CodeExecutionTool | URLContextTool;
1588+
export type Tool = FunctionDeclarationsTool | GoogleMapsTool | GoogleSearchTool | CodeExecutionTool | URLContextTool;
15501589

15511590
// @public
15521591
export interface ToolConfig {
15531592
// (undocumented)
15541593
functionCallingConfig?: FunctionCallingConfig;
1594+
// (undocumented)
1595+
retrievalConfig?: RetrievalConfig;
15551596
}
15561597

15571598
// @beta

‎docs-devsite/_toc.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,12 @@ toc:
9898
path: /docs/reference/js/ai.generativemodel.md
9999
- title: GoogleAIBackend
100100
path: /docs/reference/js/ai.googleaibackend.md
101+
- title: GoogleMaps
102+
path: /docs/reference/js/ai.googlemaps.md
103+
- title: GoogleMapsGroundingChunk
104+
path: /docs/reference/js/ai.googlemapsgroundingchunk.md
105+
- title: GoogleMapsTool
106+
path: /docs/reference/js/ai.googlemapstool.md
101107
- title: GoogleSearch
102108
path: /docs/reference/js/ai.googlesearch.md
103109
- title: GoogleSearchTool
@@ -142,6 +148,8 @@ toc:
142148
path: /docs/reference/js/ai.languagemodelmessagecontent.md
143149
- title: LanguageModelPromptOptions
144150
path: /docs/reference/js/ai.languagemodelpromptoptions.md
151+
- title: LatLng
152+
path: /docs/reference/js/ai.latlng.md
145153
- title: LiveGenerationConfig
146154
path: /docs/reference/js/ai.livegenerationconfig.md
147155
- title: LiveGenerativeModel
@@ -178,6 +186,8 @@ toc:
178186
path: /docs/reference/js/ai.promptfeedback.md
179187
- title: RequestOptions
180188
path: /docs/reference/js/ai.requestoptions.md
189+
- title: RetrievalConfig
190+
path: /docs/reference/js/ai.retrievalconfig.md
181191
- title: RetrievedContextAttribution
182192
path: /docs/reference/js/ai.retrievedcontextattribution.md
183193
- title: SafetyRating
@@ -224,6 +234,8 @@ toc:
224234
path: /docs/reference/js/ai.templategenerativemodel.md
225235
- title: TemplateImagenModel
226236
path: /docs/reference/js/ai.templateimagenmodel.md
237+
- title: TemplateToolConfig
238+
path: /docs/reference/js/ai.templatetoolconfig.md
227239
- title: TextPart
228240
path: /docs/reference/js/ai.textpart.md
229241
- title: ThinkingConfig

‎docs-devsite/ai.googlemaps.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
Project: /docs/reference/js/_project.yaml
2+
Book: /docs/reference/_book.yaml
3+
page_type: reference
4+
5+
{% comment %}
6+
DO NOT EDIT THIS FILE!
7+
This is generated by the JS SDK team, and any local changes will be
8+
overwritten. Changes should be made in the source code at
9+
https://github.com/firebase/firebase-js-sdk
10+
{% endcomment %}
11+
12+
# GoogleMaps interface
13+
Specifies the Google Maps configuration.
14+
15+
<b>Signature:</b>
16+
17+
```typescript
18+
export interface GoogleMaps
19+
```
20+
21+
## Properties
22+
23+
| Property | Type | Description |
24+
| --- | --- | --- |
25+
| [enableWidget](./ai.googlemaps.md#googlemapsenablewidget) | boolean | |
26+
27+
## GoogleMaps.enableWidget
28+
29+
<b>Signature:</b>
30+
31+
```typescript
32+
enableWidget?: boolean;
33+
```

‎docs-devsite/ai.googlemapsgroundingchunk.md‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
Project: /docs/reference/js/_project.yaml
2+
Book: /docs/reference/_book.yaml
3+
page_type: reference
4+
5+
{% comment %}
6+
DO NOT EDIT THIS FILE!
7+
This is generated by the JS SDK team, and any local changes will be
8+
overwritten. Changes should be made in the source code at
9+
https://github.com/firebase/firebase-js-sdk
10+
{% endcomment %}
11+
12+
# GoogleMapsGroundingChunk interface
13+
A grounding chunk from Google Maps.
14+
15+
Important: If using Grounding with Google Maps, you are required to comply with the [Service Specific Terms](https://cloud.google.com/terms/service-terms) for "Grounding with Google Maps".
16+
17+
<b>Signature:</b>
18+
19+
```typescript
20+
export interface GoogleMapsGroundingChunk
21+
```
22+
23+
## Properties
24+
25+
| Property | Type | Description |
26+
| --- | --- | --- |
27+
| [placeId](./ai.googlemapsgroundingchunk.md#googlemapsgroundingchunkplaceid) | string | This Place's resource name, in <code>places/{place_id}</code> format. This can be used to look up the place in the Google Maps API. |
28+
| [text](./ai.googlemapsgroundingchunk.md#googlemapsgroundingchunktext) | string | The text of the place answer. |
29+
| [title](./ai.googlemapsgroundingchunk.md#googlemapsgroundingchunktitle) | string | The title of the place. |
30+
| [uri](./ai.googlemapsgroundingchunk.md#googlemapsgroundingchunkuri) | string | The URI of the place. |
31+
32+
## GoogleMapsGroundingChunk.placeId
33+
34+
This Place's resource name, in `places/{place_id}` format. This can be used to look up the place in the Google Maps API.
35+
36+
<b>Signature:</b>
37+
38+
```typescript
39+
placeId?: string;
40+
```
41+
42+
## GoogleMapsGroundingChunk.text
43+
44+
The text of the place answer.
45+
46+
<b>Signature:</b>
47+
48+
```typescript
49+
text?: string;
50+
```
51+
52+
## GoogleMapsGroundingChunk.title
53+
54+
The title of the place.
55+
56+
<b>Signature:</b>
57+
58+
```typescript
59+
title?: string;
60+
```
61+
62+
## GoogleMapsGroundingChunk.uri
63+
64+
The URI of the place.
65+
66+
<b>Signature:</b>
67+
68+
```typescript
69+
uri?: string;
70+
```

‎docs-devsite/ai.googlemapstool.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
Project: /docs/reference/js/_project.yaml
2+
Book: /docs/reference/_book.yaml
3+
page_type: reference
4+
5+
{% comment %}
6+
DO NOT EDIT THIS FILE!
7+
This is generated by the JS SDK team, and any local changes will be
8+
overwritten. Changes should be made in the source code at
9+
https://github.com/firebase/firebase-js-sdk
10+
{% endcomment %}
11+
12+
# GoogleMapsTool interface
13+
A tool that allows a Gemini model to connect to Google Maps to access and incorporate location-based information into its responses.
14+
15+
Important: If using Grounding with Google Maps, you are required to comply with the "Grounding with Google Maps" usage requirements for your chosen API provider: [Gemini Developer API](https://ai.google.dev/gemini-api/terms#grounding-with-google-maps) or Vertex AI Gemini API (see [Service Terms](https://cloud.google.com/terms/service-terms) section within the Service Specific Terms).
16+
17+
<b>Signature:</b>
18+
19+
```typescript
20+
export interface GoogleMapsTool
21+
```
22+
23+
## Properties
24+
25+
| Property | Type | Description |
26+
| --- | --- | --- |
27+
| [googleMaps](./ai.googlemapstool.md#googlemapstoolgooglemaps) | [GoogleMaps](./ai.googlemaps.md#googlemaps_interface) | Specifies the Google Maps configuration.<!-- -->When using this feature, you are required to comply with the "Grounding with Google Maps" usage requirements for your chosen API provider: [Gemini Developer API](https://ai.google.dev/gemini-api/terms#grounding-with-google-maps) or Vertex AI Gemini API (see [Service Terms](https://cloud.google.com/terms/service-terms) section within the Service Specific Terms). |
28+
29+
## GoogleMapsTool.googleMaps
30+
31+
Specifies the Google Maps configuration.
32+
33+
When using this feature, you are required to comply with the "Grounding with Google Maps" usage requirements for your chosen API provider: [Gemini Developer API](https://ai.google.dev/gemini-api/terms#grounding-with-google-maps) or Vertex AI Gemini API (see [Service Terms](https://cloud.google.com/terms/service-terms) section within the Service Specific Terms).
34+
35+
<b>Signature:</b>
36+
37+
```typescript
38+
googleMaps: GoogleMaps;
39+
```

‎docs-devsite/ai.groundingchunk.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,19 @@ export interface GroundingChunk
2222

2323
| Property | Type | Description |
2424
| --- | --- | --- |
25+
| [maps](./ai.groundingchunk.md#groundingchunkmaps) | [GoogleMapsGroundingChunk](./ai.googlemapsgroundingchunk.md#googlemapsgroundingchunk_interface) | Contains details if the grounding chunk is from a Google Maps source. |
2526
| [web](./ai.groundingchunk.md#groundingchunkweb) | [WebGroundingChunk](./ai.webgroundingchunk.md#webgroundingchunk_interface) | Contains details if the grounding chunk is from a web source. |
2627

28+
## GroundingChunk.maps
29+
30+
Contains details if the grounding chunk is from a Google Maps source.
31+
32+
<b>Signature:</b>
33+
34+
```typescript
35+
maps?: GoogleMapsGroundingChunk;
36+
```
37+
2738
## GroundingChunk.web
2839

2940
Contains details if the grounding chunk is from a web source.

‎docs-devsite/ai.groundingmetadata.md‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,12 @@ https://github.com/firebase/firebase-js-sdk
1212
# GroundingMetadata interface
1313
Metadata returned when grounding is enabled.
1414

15-
Currently, only Grounding with Google Search is supported (see [GoogleSearchTool](./ai.googlesearchtool.md#googlesearchtool_interface)<!-- -->).
15+
Currently, only Grounding with Google Search and Grounding with Google Maps are supported (see [GoogleSearchTool](./ai.googlesearchtool.md#googlesearchtool_interface) and [GoogleMapsTool](./ai.googlemapstool.md#googlemapstool_interface)<!-- -->, respectively).
1616

1717
Important: If using Grounding with Google Search, you are required to comply with the "Grounding with Google Search" usage requirements for your chosen API provider: [Gemini Developer API](https://ai.google.dev/gemini-api/terms#grounding-with-google-search) or Vertex AI Gemini API (see [Service Terms](https://cloud.google.com/terms/service-terms) section within the Service Specific Terms).
1818

19+
Important: If using Grounding with Google Maps, you are required to comply with the "Grounding with Google Maps" usage requirements for your chosen API provider: [Gemini Developer API](https://ai.google.dev/gemini-api/terms#grounding-with-google-maps) or Vertex AI Gemini API (see [Service Terms](https://cloud.google.com/terms/service-terms) section within the Service Specific Terms).
20+
1921
<b>Signature:</b>
2022

2123
```typescript
@@ -26,12 +28,23 @@ export interface GroundingMetadata
2628

2729
| Property | Type | Description |
2830
| --- | --- | --- |
31+
| [googleMapsWidgetContextToken](./ai.groundingmetadata.md#groundingmetadatagooglemapswidgetcontexttoken) | string | Resource name of the Google Maps widget context token that can be used with the <code>PlacesContextElement</code> widget in order to render contextual data. Only populated in the case that grounding with Google Maps is enabled. |
2932
| [groundingChunks](./ai.groundingmetadata.md#groundingmetadatagroundingchunks) | [GroundingChunk](./ai.groundingchunk.md#groundingchunk_interface)<!-- -->\[\] | A list of [GroundingChunk](./ai.groundingchunk.md#groundingchunk_interface) objects. Each chunk represents a piece of retrieved content (for example, from a web page). that the model used to ground its response. |
3033
| [groundingSupports](./ai.groundingmetadata.md#groundingmetadatagroundingsupports) | [GroundingSupport](./ai.groundingsupport.md#groundingsupport_interface)<!-- -->\[\] | A list of [GroundingSupport](./ai.groundingsupport.md#groundingsupport_interface) objects. Each object details how specific segments of the model's response are supported by the <code>groundingChunks</code>. |
3134
| [retrievalQueries](./ai.groundingmetadata.md#groundingmetadataretrievalqueries) | string\[\] | |
3235
| [searchEntryPoint](./ai.groundingmetadata.md#groundingmetadatasearchentrypoint) | [SearchEntrypoint](./ai.searchentrypoint.md#searchentrypoint_interface) | Google Search entry point for web searches. This contains an HTML/CSS snippet that must be embedded in an app to display a Google Search entry point for follow-up web searches related to a model's "Grounded Response". |
3336
| [webSearchQueries](./ai.groundingmetadata.md#groundingmetadatawebsearchqueries) | string\[\] | A list of web search queries that the model performed to gather the grounding information. These can be used to allow users to explore the search results themselves. |
3437

38+
## GroundingMetadata.googleMapsWidgetContextToken
39+
40+
Resource name of the Google Maps widget context token that can be used with the `PlacesContextElement` widget in order to render contextual data. Only populated in the case that grounding with Google Maps is enabled.
41+
42+
<b>Signature:</b>
43+
44+
```typescript
45+
googleMapsWidgetContextToken?: string;
46+
```
47+
3548
## GroundingMetadata.groundingChunks
3649

3750
A list of [GroundingChunk](./ai.groundingchunk.md#groundingchunk_interface) objects. Each chunk represents a piece of retrieved content (for example, from a web page). that the model used to ground its response.

0 commit comments

Comments
 (0)