10.

How to Rate Results

This chapter shows how to rate a variety of examples.

For each result, you will need to answer questions about:

• Relevance
• Name accuracy
• Address accuracy
• Pin accuracy

10.1. Specific Address

When the query is an address, and the user explicitly states their location intent by including the
name of a locality, this is an explicit location query and the user’s viewport and location are irrelevant.
The query refers to a unique location, so a result for this exact location should be rated Navigational.

Query Results Ratings and Explanation

Is there a Navigational Result for this Query? Yes

[12112 Relevance Navigational

sugarloaf key Name Accuracy n/a
st tampa fl
33626] Address Accuracy Correct

Pin Accuracy Perfect

12112 Sugarloaf Key St

12112 Sugarloaf Key St
Tampa, FL 33626
Category: n/a

This address belongs to an apartment complex,

where all buildings have the same address.

The pins falls on one of the rooftops of this

mutiple-rooftop result and is rated Perfect.

179
 Query Results Ratings and Explanation
Relevance Navigational
[12112 Camden Westchase Park
sugarloaf key Apartments Name Accuracy Correct
st tampa fl 12112 Sugarloaf Key St
33626] Tampa, FL 33626 Address Accuracy Correct
Category: Apartments Pin Accuracy Wrong

The result includes the name of the business,

which is located at this address.

The pin falls outside the boundaries of the

apartment complex.

12112 Sugarloaf Key St Relevance Navigational

12112 Sugarloaf Key St
Tampa, FL Name Accuracy n/a
Category: n/a Address Accuracy Incorrect - Postal Code
Pin Accuracy Perfect

The result is missing a postal code so Address

Accuracy is Incorrect (with Postal Code
specified as the incorrect component). Postal
code is a mandatory component for a
residential address result. The pin falls on one
of the rooftops of this multiple-rooftop POI.

Sugarloaf Key St Acceptable (User Intent

Relevance
Sugarloaf Key St, Tampa, FL Issue)
33626
Category: n/a Name Accuracy n/a
Address Accuracy Correct
Pin Accuracy Perfect

The result is a whole street result instead of a

specific address. As the user query is for a
specific, full address, the information provided
by the result is incomplete. The user would be
able to find the street, but not the exact address
s/he is looking for. The pin is dropped on the
street mentioned in the result and is Perfect.

180
 Query Results Ratings and Explanation

11244 Windsor Pl Relevance Bad (User Intent Issue)

11244 Windsor Pl Cir Name Accuracy n/a
Tampa, FL 33626
[12112 Category: n/a Address Accuracy Correct
sugarloaf key
st tampa fl Pin Accuracy Perfect
33626]
The result does not satisfy the query and is not
considered relevant. However, the address and
pin placement are Correct and Perfect.
Remember, the data and relevance of the result
are considered separately. The data can be
correct even when the result’s relevance is
Bad.
[220 yonge st Is there a Navigational Result for this Query? Yes
toronto]
CF Toronto Eaton Centre Relevance Navigational
User in Toronto, 220 Yonge St
Toronto ON M5B 2H1 Name Accuracy Correct
Canada
Category: Shopping Center Address Accuracy Correct
Pin Accuracy Perfect

This query is the address of the best-known

mall in downtown Toronto. Even though many
businesses have this address as their official
address, the mall is rated Navigational due to
its prominence.

The result includes the correct address plus the

name of the mall, which is useful additional
information. The pin is dropped on the mall’s
rooftop.

181
 Query Results Ratings and Explanation
Indigo Relevance Good
220 Yonge St
Name Accuracy Correct
[220 yonge st Toronto, ON M5B 2H1
toronto] Category: Bookstore Address Accuracy Correct

User in Toronto, Pin Accuracy Perfect

Canada

This large, well-known bookstore is inside the

Eaton Centre and shares the 220 Yonge Street
address. It is demoted to Good because it is
less prominent than the Eaton Centre.

The name and address are Correct. Based on

the best available evidence (including interior
maps of the mall available online) the exact
position of the bookstore under the rooftop can
be found. A pin falling on the green area
(bookstore’s position) is Perfect. The rest of the
mall is Approximate.

182
 Query Results Ratings and Explanation
Moneysworth & Best Acceptable (Distance/
Relevance
220 Yonge St Prominence)
[220 yonge st Toronto, ON M5B 2H1 Name Accuracy Correct
toronto] Category: Shoe Repair Service
Address Accuracy Correct
User in Toronto,
Canada Pin Accuracy Perfect

Most people come to the mall to shop, so this

shoe repair service, also at 220 Yonge Street,
is much less prominent and a much less likely
intent than other stores in the mall. Its
relevance is demoted to Acceptable.

The store is very small and located

underground. When comparing the official map
provided by the mall with the satellite, the
store’s location can be established inside the
green area.

183
 Query Results Ratings and Explanation

[220 yonge st Subway Relevance Bad (User Intent)

toronto] 260 Yonge St
Toronto, ON M5B 2H1, CAN Name Accuracy Correct
User in Toronto, Category: Sandwich Shop Address Accuracy Correct
Canada
Pin Accuracy Perfect

This fast-food restaurant is not located at 220

Yonge Street. The name and address in the
result are Correct and the pin placement is
Perfect.

The official venue map shows the precise

location of the restaurant, but finding the
perfect area is a little tricky. Given all evidence,
the Perfect location (green area) can be
estimated generously.

184
 Query Results Ratings and Explanation
[1141 Swiss Chalet Relevance Excellent
Highbury Ave, 1141 Highbury Ave
London, London, ON N5Y 1A5 CAN
Name Accuracy Correct
Ontario]

User in London, Address Accuracy Correct

Canada
Pin Accuracy Approximate
All four of these businesses:

1. Swiss Chalet
2. Wings Kitchen
3. Fast Photo
4. Globally Local

share the 1141 Highbury Ave address and are

of similar prominence. They are all equally
likely to be the intent of a user with this query.
This means that all results can be rated
Excellent.

Compare this example to the Eaton Centre

example, above, where there is a single most
prominent POI at the address that is the
dominant user intent.

In the diagram at left, the pin is in the parking

lot and would be rated Approximate for any of
the results.

185
Address queries can also be just the name of a street, neighborhood, intersection, or city. Results for
streets, neighborhoods, intersection, and city queries are eligible for Navigational.

The expectation is that we return the requested entity with the same level of broadness that the user
requested, which may not include all of the elements of a full address.

Query Results Ratings & Explanation

Is there a Navigational Result for this Query? Yes
Relevance Navigational
Name Accuracy n/a
Address Accuracy Correct
Pin Accuracy Perfect

Stevens Creek Blvd

Stevens Creek Blvd
[stevens Cupertino, CA
creek blvd
Category: n/a
cupertino]

The query is for a street in Cupertino, CA. The

postal code is optional for such results, but if
the postal code is returned, any portion of the
street has to belong to that postal code area.
The pin appears to fall outside the street
boundaries in the vector view. However, in
satellite it is still within the boundaries.
Whenever there is a discrepancy the most
favorable layer should be used to determine
pin accuracy.

186
 Query Results Ratings & Explanation
Relevance Bad (User Intent)
Name Accuracy n/a
Cupertino, CA
Cupertino, CA Address Accuracy Correct
Category: n/a Pin Accuracy Perfect

This result for an entire city does not satisfy

user intent when the query was for a street in
that city. Pin is dropped within the boundaries
of the city of Cupertino and is rated Perfect.

[stevens
creek blvd
cupertino] Relevance Bad (User Intent)
Name Accuracy n/a
Address Accuracy Correct
Pin Accuracy Perfect

19062 Stevens Creek Blvd

19062 Stevens Creek Blvd
Cupertino, CA, 95014
Category: n/a

The result includes a street number when the

query is just for a street, making this result too
specific for the requested query. The pin is
dropped on the result address.

187
 Query Results Ratings & Explanation
Relevance Bad (User Intent)
Happy Lamb Hot Pot
19062 Stevens Creek Blvd Name Accuracy Correct
Cupertino, CA, 95014
Address Accuracy Correct
Category: Chinese
Pin Accuracy Perfect

The result is for a business on the street.

When the query is for an entire street,
returning a business is too specific a result for
the requested query.

[stevens
creek blvd
cupertino]
Stevens Creek and De Anza Relevance Bad (User Intent)
Blvd. Stop
Cupertino CA 95014
Name Accuracy Correct
Category: Bus Stop
Address Accuracy Correct

Pin Accuracy Perfect

The result is for a bus stop on the street. This

random result does not satisfy the user's intent
and lacks prominence compared to the very
long street.
[upper east Is there a Navigational Result for this Query? Yes
side]
Relevance Navigational
Name Accuracy n/a
Upper East Side
Upper East Side Address Accuracy Correct
New York, NY 10021
Pin Accuracy Perfect
Category: n/a

The result is the requested neighborhood and

therefore is rated n/a for Name Accuracy. The
postal code is correct and belongs to the
requested neighborhood. A Perfect pin can
drop anywhere within the boundaries of the
neighborhood.

188
Query Results Ratings & Explanation
Relevance Good (User Intent)
Incorrect (Category
Name Accuracy
Issue)
Address Accuracy Correct
Metropolitan Museum of Art
1000 5th Ave Pin Accuracy Perfect
New York, NY 10028 Research shows the Metropolitan Museum,
Category: Art Gallerry one of the most famous museums in the world,
is located on the Upper East Side. Its
extremely high international prominence
qualifies it for a Good rating due to secondary
intent for this query.

The Name Accuracy rating for this result is

Incorrect due to the misspelling of the
category.

Relevance Bad (User Intent)

Boqueria Upper East Side Name Accuracy Correct

1460 2nd Ave
New York, NY 10075 Address Accuracy Correct
Category: Spanish
Pin Accuracy Perfect

The result is for a business within the queried

neighborhood. Even though the business
name contains the query terms, this result
does not satisfy user Intent, and is not
internationally prominent enough to be
considered relevant for the query.

189
 Query Results Ratings & Explanation
Relevance Bad (User Intent)
Name Accuracy n/a

Address Accuracy Correct

1596 2nd Ave
1596 2nd Ave Pin Accuracy Perfect
New York, NY 10028
Category: n/a

The result is for a specific address within the

neighborhood. This result is too specific for the
broader query. The address is not
internationally prominent enough to satisfy the
intent for the query.

Is there a Navigational Result for this Query? Yes

Relevance Navigational
Name Accuracy n/a
[Cork]
Address Accuracy Correct

Cork Pin Accuracy Perfect

Cork, Ireland
Category: n/a

The result is for the city of Cork in Ireland. A

perfect pin can drop anywhere within the city
boundaries.

190
Query Results Ratings & Explanation
Relevance Good (User Intent)
Name Accuracy Correct
Cork Airport Address Accuracy Correct
Kinsale Rd
Cork, Ireland Pin Accuracy Perfect
Category: Airport

The result is for a transit POI, which satisfies a

secondary query intent. The address and pin
are correct.

Relevance Bad (User Intent)

Name Accuracy Correct
Address Accuracy Correct
Pin Accuracy Perfect

Cork City Library

57-61 Grand Parade
Cork, Ireland
Category: Library

The result is for a POI within the city. This does

not satisfy secondary intent for the query as it
is neither internationally prominent nor a transit
POI.

191
 Query Results Ratings & Explanation
[Stevens Is there a Navigational Result for this Query? Yes
Creek and
De Anza] Relevance Navigational
Name Accuracy n/a

Address Accuracy Correct

Pin Accuracy Perfect

Stevens Creek and De Anza

Blvd
Stevens Creek and De Anza Blvd
Cupertino CA 95014
Category: n/a

The query is for an intersection and the

specific intersection is returned.

Good/Acceptable (User
Relevance
Intent)
Stevens Creek and De Anza
Blvd. Stop Name Accuracy Correct
Cupertino CA 95014 Address Accuracy Correct
Category: Bus Stop
Pin Accuracy Perfect

The query is for an intersection and a bus stop

with the same name is returned. In some
countries, bus stops carry the same name as
the intersection where they are located and
can satisfy a secondary intent for the query.
The pin falls where a user would wait for the
bus.

192
Query Results Ratings & Explanation
Relevance Bad (User Intent)
Name Accuracy n/a
Address Accuracy Correct
Pin Accuracy Perfect

Stevens Creek Blvd

Stevens Creek Blvd
Cupertino CA 95014
Category: n/a

The query is for an intersection and we return

just one of the streets from the query. This
does not satisfy user intent.

[95051]
Is there a Navigational Result for this Query? Yes

95051 Relevance Navigational

Santa Clara, CA, 95051 Name Accuracy n/a
Category: n/a
Address Accuracy Correct

Pin Accuracy Perfect

The expectation for this result is to include
locality, state and postal code. Other
components are optional, but if present they
need to be correct.

The pin must drop within the boundaries of the

locality presented in the result.

193
10.2. Non-Specific Address

For queries where the location intent is not explicitly stated in the query, you must infer the location
intent from the user’s viewport and the user’s location. One difficulty in evaluating address queries
has to do with partial addresses, which could refer to multiple locations.
Generally, for partial addresses, users are looking for the location closest to their position or within/
near the fresh viewport. Some general considerations:
• Exact location that exists returned very close to the user is eligible for Navigational, unless
other addresses that satisfy the user intent are also very close.
• Exact location returned within a fresh viewport is eligible for Navigational, unless other
addresses that satisfy the user intent are also very close.
• Results that are farther away from an ideal result but can potentially satisfy the user’s intent
can be rated as high as Good (Distance/Prominence)
• Results are demoted based on distance.
• Results are demoted based on the density of potential results in a given area
• Results that are too far away will be rated Bad (Distance/Prominence)
Consider the query [154 orchard st] in different scenarios:

194
Scenario 1: Fresh viewport over Midtown Manhattan in New York City.

Query User and Viewport

User and fresh viewport over Midtown Manhattan in
[154 orchard st]
New York City
User Intent: The query does not specify in which locality the address is expected. Use the
location intent given by the user and viewport to rate the results.
Is there a Navigational Result for this Query? Yes

195
 Results Ratings and Explanation

154 Orchard St Relevance Navigational

154 Orchard St, New York, NY 10002 Name Accuracy n/a
Category: n/a
Address Accuracy Correct
Pin Accuracy Perfect

The result is in Lower Manhattan, very close to the user

location.

154 Orchard St Relevance Good (Distance/Prominence)

154 Orchard St, Newark, NJ 07102
Category: n/a Name Accuracy n/a
Address Accuracy Correct
Pin Accuracy Perfect

The result is farther away from the viewport, but still

matches the query.

196
 Results Ratings and Explanation

154 Orchard St Relevance Good (Distance/Prominence)

154 Orchard St, Garfield, NJ 07026
Category: n/a Name Accuracy n/a
Address Accuracy Correct
Pin Accuracy Perfect

The result is farther away from the viewport, but still

matches the query.

154 Orchard Dr Relevance Bad (Distance/Prominence)

154 Orchard Dr, Levittown, Pennsylvania
19054 Name Accuracy n/a
Category: n/a
Address Accuracy Correct

Pin Accuracy Wrong

The result is in a different state and very far away from

the user’s viewport. The street type does also not match
the query.

The pin lands on the house with the street number 114,
which can be verified via street imagery. Since this is
several houses down from the intended address, it is
incorrect.

197
Scenario 2: Viewport/user’s location is in Maine. There is no address in the vicinity that contains [154
Orchard St]. All locations with this address are far from the location intent of the query.

Query User and Viewport

[154 orchard st] User and viewport in Maine
User Intent: Since there is no matching result close to the location intent given by the user and
viewport, the closest address is rated without demotion for user intent or distance/prominence.
Is there a Navigational Result for this Query? No

198
 Results Ratings and Explanation

154 Orchard St Relevance Excellent

154 Orchard St, New York, NY 10002 Name Accuracy n/a
Category: n/a
Address Accuracy Correct
Pin Accuracy Perfect

This location is among the closest possible results to

the viewport/user’s location.

154 Orchard St Relevance Excellent

154 Orchard St, Newark, NJ 07102
Name Accuracy n/a
Category: n/a
Address Accuracy Correct
Pin Accuracy Perfect

This location is among the closest possible results to

the viewport/user’s location.

154 Orchard St Relevance Excellent

154 Orchard St, Garfield, NJ 07026
Name Accuracy n/a
Category: n/a
Address Accuracy Correct
Pin Accuracy Perfect

This location is among the closest possible results to

the viewport/user’s location.

199
 Results Ratings and Explanation

154 Orchard Dr Relevance Bad (User Intent)

154 Orchard Dr, Levittown, Pennsylvania
Name Accuracy n/a
19054
Category: n/a Address Accuracy Correct
Pin Accuracy Wrong
This location is farther from the viewport/user location
than the result in New York and therefore demoted for
distance. The user is also looking for a Street, but has
been presented with a Drive.

The pin lands on the house with the street number 114
which can be verified via street imagery. Since this is
several houses down from the intended address, it is
Wrong.

200
Here is another example of an address query with implicit location intent. Although this is a common
address in US cities, it overwhelmingly refers to the location of the White House and should be
considered the primary intent of the query unless an additional location modifier is provided (i.e. state,
city, or postal code) or a different 1600 Pennsylvania Ave that’s not the White House is already in or
very near the location intent.

Scenario 1: Fresh viewport on Los Angeles, CA (not near Richmond, CA):

Query Results Ratings and Explanation

Is there a Navigational Result for this Query? Yes
The White House Relevance Navigational
1600 Pennsylvania Ave,
Washington, DC, 20500 United Name Accuracy Correct
States Address Accuracy Correct
Category: Landmark
Pin Accuracy Perfect

The POI associated with this address is

[1600 overwhelmingly prominent and this is the
pennsylvania most likely intent.
ave]

Fresh viewport
on Los Angeles,
1600 Pennsylvania Ave Acceptable/Bad
CA (not near Relevance
1600 Pennsylvania Ave, (Distance/Prominence)
Richmond, CA)
Richmond, CA 94801
Name Accuracy n/a
Category: n/a
Address Accuracy Correct
Pin Accuracy Perfect

Technically a match, but considering the

user’s viewport it is less likely that the user is
looking for this location.

201
Scenario 2: The fresh viewport is in San Francisco, CA (near Richmond, CA). The proximity of the
user viewport to the Richmond location makes this location just as relevant as the prominent location
in Washington DC.
Query Results Ratings & Explanation
Is there a Navigational Result for this Query? No
The White House Relevance Excellent
1600 Pennsylvania Ave,
Washington, DC, 20500 Name Accuracy Correct
Category: Landmark Address Accuracy Correct
Pin Accuracy Perfect

[1600 Both results are equally valid, so neither can

pennsylvania get Navigational and should instead be rated
ave] Excellent.

Fresh viewport
on San 1600 Pennsylvania Ave Relevance Excellent
Francisco, CA
1600 Pennsylvania Ave,
(near Richmond, Name Accuracy n/a
CA)
Richmond, CA 94801
Category: n/a Address Accuracy Correct
Pin Accuracy Perfect

Both results are equally valid, so neither can

get Navigational and should instead be rated
Excellent.

10.3. Query Address Does Not Exist

To be useful to the user, address results need to point to either a building with that address or a plot
of land that has been officially assigned the address. Sometimes your research will show that a full
query address does not exist (or no major search engine will be able to determine its exact location).
In cases like this, you will see one of three types of results:
• The closest verified address (on the same street and in the same city and state): Rate
relevance as Excellent.
• The same address as the query address: Rate relevance as Excellent and address as
Incorrect – Address does not exist. The pin will be rated Can’t Verify.
• The queried street without a street number: Rate relevance as Acceptable.

When a queried address does not exist, the answer to the question “Is there a navigational result for
this query” will always be No.

202
 Query Results Ratings and Explanation
[2001 Duncan Is there a Navigational Result for this Query? No
St, San
Francisco] 1099 Duncan St Relevance Excellent
1099 Duncan St., San
Francisco, 94131, CA Name Accuracy n/a
Category: n/a Address Accuracy Correct
Pin Accuracy Perfect
If 2001 does not actually exist then selecting
1099, the closest address on the same street in
the same city and state that does exist, is
considered correct.
For a locale that uses the rule of even numbers
on one side of the street and the odd numbers
on the other, we need to return the closest
existing address on the same side of the street
as the query. If the rule of odds and evens does
not apply then the closest numeric number will
be considered correct.
The pin has to be correctly dropped on the
address presented in the result to be rated
Perfect.

2001 Duncan St Relevance Excellent

2001 Duncan St., San Name Accuracy n/a
Francisco, 94131, CA
Category: n/a Incorrect – Address
Address Accuracy
does not exist
Pin Accuracy Can’t Verify
Duncan Street only goes up to number 1099,
but the query requests a non-existent number.
The result returned is the same as the queried
address and should be rated as Excellent.
The pin rating for addresses that do not exist is
Can’t Verify.

203
 Query Results Ratings and Explanation
Duncan St Relevance Acceptable (User Intent)
Duncan St, San Francisco,
94131, CA Name Accuracy n/a
Category: n/a Address Accuracy Correct
Pin Accuracy Perfect

The result returns only the street with a pin

dropped within the boundaries of the street.
This result technically satisfies the user intent,
but this intent is unlikely, so relevance should
be rated Acceptable.

10.4. Point of Interest

A correct location result should be rated Navigational for queries referring to a point of interest.
Consider the query [mount rushmore], the famous US monument. Although the query terms do not
contain an explicit location, the Mount Rushmore monument is so prominent and unambiguous that
its location should be considered the primary location intent. The user viewport and user location are
irrelevant for this query because the user is looking for the place of interest no matter where they
currently are.

Query Results Ratings and Explanation

Is there a Navigational Result for this Query? Yes
[mount
rushmore] Relevance Navigational
Name Accuracy Correct
Address Accuracy Correct
Mount Rushmore National Pin Accuracy Perfect
Memorial
13000 S Dakota 244, Keystone,
SD, United States
Category: Landmark

The pin can drop anywhere within the

boundaries of the park to be rated Perfect.

204
Query Results Ratings and Explanation
Mount Rushmore KOA at
Relevance Bad (User Intent)
Palmer Gulch
12620 Highway 244
Hill City, SD 57745 Name Accuracy Correct
Category: Campground Address Accuracy Correct
Pin Accuracy Perfect

This result is a campground within the park

and is not the park the user asked for.

Mount Rushmore Gift Shop

Relevance Bad (User Intent)
13000 SD-244, Keystone, SD
57751
Name Accuracy Correct
Category: Gift Shop
Address Accuracy Correct
Pin Accuracy Perfect
This result is a gift shop inside Mount
Rushmore National Park and is not the park
the user asked for. Both the Name and
Address Accuracy are verified and rated
Correct. The pin falls on the shop and is
Perfect.

205
 Query Results Ratings and Explanation
Is there a Navigational Result for this Query? Yes
Route 66 Relevance Navigational
Seligman, AZ 86337
United States Name Accuracy n/a
Category: n/a Address Accuracy Correct
Pin Accuracy Perfect

This result represents Route 66 and is rated

Navigational regardless of user location,
viewport, or the locality in the address. The
pin can be dropped anywhere along the route.

Route 66 Indian buffet all 7 Relevance Bad (User Intent)

[route 66] days
Fresh viewport 397 Las Colinas Blvd, Partially Correct (Name
Name Accuracy
on Amarillo, TX Irving, TX 75039 Issue)
Category: Indian Incorrect - Unit/Apt,
Address Accuracy
Street Name
Pin Accuracy Perfect
The result is a randomly chosen business far
from the user whose name includes the query
terms. The result name differs from the official
name variations used on the website: “Route
66 Fine Indian and American Cuisines” or
“Route 66 Indian Cuisine”. However, the
name is still recognizable and is rated
Partially Correct. The address on the official
website is “397 East Las Colinas Blvd”
whereas the result address excludes “East”.
Also, the official site lists a unit number, 180,
which is missing from the address in the
result.

206
 Query Results Ratings and Explanation
Is there a Navigational Result for this Query? Yes
Machu Picchu Relevance Navigational
Aguas Calientes, Peru
Name Accuracy Correct
Address Accuracy Correct
Pin Accuracy Perfect

The result is an internationally known historical

site and the most relevant result, making it the
primary user intent.
[Machu
Picchu]

Fresh viewport
over Relevance Good (User Intent)
Massachusetts,
some distance Machu Picchu Restaurant Name Accuracy Correct
away from 307 Somerville Ave, Somerville, MA
Somerville, MA
Address Accuracy Correct
02143
Pin Accuracy Perfect

Since the result is reasonably close to the

fresh user viewport, it will be considered
secondary intent for this query.

207
10.5. Understanding Multiple Query Interpretations: Beyond Viewport and User Location

Many address/POI queries have multiple interpretations. Understanding which interpretation is most
likely often requires extra research in addition to considering the viewport and user’s location. It is
critical to fully understand the query through research rather than guessing.

Query Results Ratings and Explanation

Is there a Navigational Result for this Query? Yes
New York City Relevance Navigational
New York City, NY, US
Name Accuracy n/a
Category: n/a
Address Accuracy Correct
Pin Accuracy Perfect

This is a very prominent result that satisfies

the user intent.

New York State Relevance Good (User Intent)

New York, US
Name Accuracy n/a
Category: n/a
[new york] Address Accuracy Correct
Pin Accuracy Perfect
User and
viewport
somewhere in This result refers to the state of New York and
France satisfies a secondary intent of the query.

Avenue de New York Relevance Good (User Intent)

Avenue de New York, Paris,
France Name Accuracy n/a
Category: n/a Address Accuracy Correct
Pin Accuracy Perfect

208
 Query Results Ratings and Explanation

The result is within the fresh viewport, but is

not a very prominent street. Although this is
still a possible interpretation, it is not a perfect
match to the query and is overshadowed by
the prominence of New York City.

Is there a Navigational Result for this Query? Yes

Amsterdam Cafe Relevance Navigational
481 Amsterdam Ave
New York, NY 10024 Name Accuracy Correct
Category: Cafe Address Accuracy Correct
[amsterdam
cafe] Pin Accuracy Perfect

User and fresh

viewport in New The user and fresh viewport are close to New
York, NY York, NY this cafe is an exact match, and can
therefore be considered the main intent.

The pin lands on the edge of the building, so

by applying the tennis rule the pin will be
rated as Perfect.

Is there a Navigational Result for this Query? No

Café de Jaren Relevance Excellent
Nieuwe Doelenstraat 20-22, 1012
CP Amsterdam, Netherlands Name Accuracy Correct
[amsterdam
cafe] Category: Café Address Accuracy Correct

User and Pin Accuracy Perfect

viewport
somewhere in
Europe The user and viewport are in Europe, so
cafes in the city of Amsterdam are the
expected results.

Is there a Navigational Result for this Query? No

Café de Jaren Relevance Excellent

209
 Query Results Ratings and Explanation
Café de Jaren
Nieuwe Doelenstraat 20-22, 1012 Name Accuracy Correct
CP Amsterdam, Netherlands
Address Accuracy Correct
Category: Café
[amsterdam Pin Accuracy Perfect
cafe]

User and fresh

The user and viewport are in a rural area and
viewport in
Maine, US there is no relevant result in the area, so the
strongest matches are cafes in the city of
Amsterdam.

Is there a Navigational Result for this Query? Yes

Relevance Navigational
Name Accuracy n/a
Europe
Europe Address Accuracy Correct
[europe]
Category: n/a Pin Accuracy Perfect
User and
The query is clearly for the European
viewport
somewhere in continent, so returning the continent should
US be rated Navigational. If the name of a local
business exactly matches the query with no
extra words then that business may be a
reasonable interpretation of the query and
should be rated based on distance from the
user or viewport. All other businesses are
Bad.
Is there a Navigational Result for this Query? Yes
Relevance Navigational
Name Accuracy n/a
Paris
Paris, France Address Accuracy Correct
Category: n/a Pin Accuracy Perfect

The query [paris] can mean either the capital

of France or one of the more than 10 towns
across the USA with the same name.
[paris] Because of the small population and lack of
tourism in these small towns, the city in
User and fresh France should be rated Navigational unless
viewport the query’s location intent is within or very
somewhere in close to one of these small US cities.
San Francisco,
CA
210
 Query Results Ratings and Explanation
San Francisco,
CA Paris Acceptable/Bad (User
Relevance
Texas, USA intent)
Category: n/a
Name Accuracy n/a
Address Accuracy Correct
Pin Accuracy Perfect

This result is unlikely to satisfy the user intent.

Is there a Navigational Result for this Query? No

Relevance Excellent

Brandy Ho's Hunan Food Name Accuracy Correct

217 Columbus Ave, San Address Accuracy Correct
[hunan] Francisco, CA 94133
Category: Hunan Pin Accuracy Perfect
User and fresh
viewport The query [hunan] represents both the
somewhere in seventh most populous province in China and
San Francisco, a popular type of cuisine. In this case, both
CA user location and viewport are in the San
Francisco Bay area. An online maps search
done by a user in this area shows that the
dominant intent of this query is for the cuisine.
The province is a less likely, but still viable,
intent that can be rated Acceptable.

10.6. Business Queries

Before you rate a business query, consider:

• Multiple interpretations: Are there ways to interpret the query terms other than as a
business name? French Laundry is both a famous restaurant in Napa and a type of
laundromat/cleaner.
• Number of locations: How many locations of this business exist? If there are many of them
(like Starbucks) then the user expects to find locations in or very close to the their location
intent.
• Population density: Is the area of the business results urban, suburban, or rural? City
dwellers are less willing to travel longer distances than suburban and rural users.

211
10.6.1. Non-Chain Businesses

Some businesses have only one location, so treat and rate them as navigational queries. The
correct location should be rated Navigational and any other results should receive Bad (User Intent).

Query Results Ratings and Explanation

Is there a Navigational Result for this Query? Yes
Klein High School Relevance Navigational
16715 Stuebner Airline Rd
Spring, TX 77379 Name Accuracy Correct
Category: High School Address Accuracy Correct
Pin Accuracy Perfect

This is a perfect result for the query. While

the city on the official website is ‘Klein’,
because USPS validates the full address, as
[klein high returned, in the city as ‘Spring’, we can rate
school] the address as correct.

User and Fresh

Viewport in Relevance Bad (User Intent)
Klein Independent School
Spring TX District Central Office Name Accuracy Correct
7200 Spring Cypress Road
Klein, TX 77379 Address Accuracy Correct
Category: School District Office Pin Accuracy Perfect

The school district office is not relevant for

the query.

212
 Query Results Ratings and Explanation
Is there a Navigational Result for this Query? Yes
Restaurant Gary Danko Relevance Navigational
800 North Point St, San Francisco,
California 94109 Name Accuracy Correct
Category: American (New) Address Accuracy Correct
Pin Accuracy Perfect

This is a nationally famous restaurant in San

Francisco with no other possible results.

800 North Point Relevance Bad (User Intent)

800 North Point St, San Francisco,
California 94109 Name Accuracy n/a
[Gary Danko Address Accuracy Correct
800 North
Point St, San Pin Accuracy Perfect
Francisco,
When a user types in the name of a business
California
or a POI along with its address, they expect
94109]
to see the business mentioned in the query
as part of the result. If the result is the
address alone and does not mention the
business, the user cannot be certain that the
query and the result refer to the same thing.
This is why the result’s relevance is Bad.
Garry Lee, DDS Relevance Bad (User Intent)
1259 Mason St, San Francisco,
California 94108 Name Accuracy Correct
Category: Dentist Address Accuracy Correct
Pin Accuracy Perfect

Result is unrelated to the queried restaurant.

213
Sometimes there are multiple unrelated businesses with the same name in the same area. In this
case, rate each business independently considering its relevance and distance from the location
intent of the query.

Query User and Viewport

User inside a fresh viewport over in Dearborn, MI (a
[lighthouse]
suburb of Detroit)
User Intent: When there are multiple unrelated businesses with the same name in the same area,
rate each business independently considering its relevance and distance from the location intent of
the query.
Is there a Navigational Result for this Query? No

214
 Results Ratings and Explanation

The Lighthouse Relevance Excellent

4474 W Jefferson Ave Name Accuracy Correct
Ecorse, MI 48229
Address Accuracy Correct
Category: Pub
Pin Accuracy Perfect
This is one of the closest results that fits the user’s
query. This result should receive a relevance rating of
Excellent.

Both the Name and Address Accuracy are verified and

rated Correct.

The Pin lands on the correct rooftop.

Lighthouse Ministries of Free Relevance Good (User Intent)

Methodist Church
2069 Chandler Ave Name Accuracy Correct
Lincoln Park, MI 48146
Address Accuracy Correct
Category: Church
Pin Accuracy Perfect
This is one of the closest results and partially fits the
query intent. This result is about the same distance from
the user’s location as the result above (The Lighthouse),
but is less likely to be the user's intent based on the
query. Therefore, this result should be demoted based
on secondary intent to Good.

Both the Name and Address Accuracy are verified and

rated Correct.

The pin lands on the correct rooftop and is Perfect.

10.6.2. Chain Businesses

A query for a chain business is generally not considered an explicit query and is therefore not eligible
a for Navigational rating unless the query contains a location modifier that points to a single unique
location (see Chain Business with Location Modifier).

Chain businesses are businesses that have more than one location, and include everything from
national chains like Starbucks, Target, Auchan, Albert Heijn, and Boots down to small local chains.

Distance evaluations should be based on chain business locations in the real world, not only the
returned results.

215
 • The more locations the chain business has in the area, the closer the results should be in
order to be considered relevant.
• The fewer locations the chain business has in the area, the more you can widen the distance
for relevant results.

Note: Go to the chain business’ official website and check the address of the specific branch you’re
rating. For each branch of a chain business, research to verify that it actually exists at the address
listed in the result.

216
 Query User and Viewport
[whole foods] User and fresh viewport in Irvine, California

User Intent: The user is clearly looking for the grocery chain Whole Foods close to their location
or at least within the viewport.
Is there a Navigational Result for this Query? No

217
 Results Ratings and Explanation
Is there a Navigational Result for this Query? No
Relevance Excellent
Name Accuracy Correct
Address Accuracy Correct
Whole Foods
The District, 2847 Park Ave, Tustin, CA Pin Accuracy Approximate
92782
Category: Grocery

This is a Whole Foods store within the user’s viewport

as well as the closest Whole Foods location to Irvine.

The pin is dropped within the shared parking lot of the

result, making it Approximate.

Whole Foods Relevance Good (Distance/Prominence)

Fashion Island, 415 Newport Center Dr,
Newport Beach, CA 92660 Name Accuracy Correct
Category: Grocery Address Accuracy Correct
Pin Accuracy Perfect

This is a Whole Foods store in the same metro area,

but is outside the user’s viewport.

218
 Results Ratings and Explanation

Whole Foods Acceptable (Distance/

Relevance
Prominence)
Aliso Village Shopping Center, 23932
Aliso Creek Rd, Laguna Niguel, CA Name Accuracy Correct
92677
Address Accuracy Correct
Category: Grocery
Pin Accuracy Perfect

This is a Whole Foods store in a neighboring city,

outside of the viewport and a considerable distance
away. As such this result is less relevant to the user and
is rated Acceptable.

10.6.3. Chain Business With Location Modifier

If the query for a chain business includes a location modifier that points to a single unique branch of
that business, that specific branch can be eligible for a Navigational rating.

When there are multiple results available for the location modifier, the highest possible rating for all
results will be Excellent. Remember:

• User and viewport location should always be ignored when there is a location modifier
• Ratings must always be based on possible real-world results

There are two types of location modifiers:

• General location modifier (locality or postal code)

• Specific location modifier (street or full address)

10.6.3.1. General Loca@on Modifier

Location modifiers in the form of localities or areas are considered to be general location modifiers.

Results inside the requested location

When the location modifier is general, results inside the specified location are not demoted for
distance.

219
 Quantity of results
Location of
inside the requested Rating and Explanation
results
location
If there is only one result present in the requested
location, it should receive a rating of Navigational.
One
It is promoted because it is the only result that satisfies
Inside the the user intent
requested When there are multiple results for the chain business in
location the requested location, they should all receive a rating of
Excellent.
More than one
Ignore user and viewport location. The user has already
told you exactly where they are looking for results.

Results outside the requested location

When the location modifier is general, results outside the requested location are demoted based on
how many results there are inside it.

Location of Results inside the

Rating and Explanation
results requested location

If there are no results for the chain business inside the

requested location, results outside the area may be
Outside the demoted less severely depending on how close they are
requested to the named location and how many branches of the
location chain exist in the real world. Ratings may range from
Excellent to Bad.
None
This is because when there are no locations of a chain
business inside the requested area, results that are
outside it can still be relevant. However, it's also
possible for results to be so far away from the location
that they have no relevance at all.

220
Location of Results inside the
Rating and Explanation
results requested location
If there are few results for the chain business inside the
requested location, results outside the area may be
demoted less severely depending on how close they are
to the named location and how many branches of the
chain exist in the real world. Ratings may range from
Good to Bad.
Few
This is because when there are fewer locations of a
chain business inside the requested area, results that
are outside it can still be relevant. However, it’s also
possible for results to be so far away from the location
that they have no relevance at all.
When there are many results for the chain business
inside the requested location, results outside the area
Many are demoted to Bad. This is because there are multiple
results that would satisfy user intent inside the
requested area, so there’s no need to look outside it.

221
 Query User and Viewport
[Starbucks san francisco] User and fresh viewport in San Francisco, CA
User Intent: The user is seeking a Starbucks in a specific location. The fresh viewport and user
location can be ignored since the user has an area in mind for results. Because there are so many
Starbucks within the named area, results outside it are rated Bad.
Is there a Navigational Result for this Query? No

222
 Results Ratings and Explanation

Starbucks Relevance Excellent

2165 Polk St, San Francisco, CA Name Accuracy Correct
94102
Address Accuracy Correct
Category: Coffee
Pin Accuracy Perfect

This result is one of many Starbucks in the specified

location, San Francisco, and is rated Excellent. All results
within San Francisco are eligible for the highest possible
initial rating of Excellent.

All the data for the result is correct.

Relevance Excellent
STARBUCKS
Name Accuracy Correct
1231 Market St, San Francisco, CA
94103 Address Accuracy Correct
Category: Coffee
Pin Accuracy Perfect
This result is one of many Starbucks in the specified
location, San Francisco, and is rated Excellent. All results
within San Francisco are eligible for the highest possible
initial rating of Excellent.

The STARBUCKS name is written in capital letters.

Because this is how the business refers to itself on
storefront signs, it is Correct. The address is Correct and
the Perfect pin lands on the correct rooftop.

Starbucks Relevance Bad (Distance/Prominence)

1555 40th St. Emeryville, CA 94608
Category: Coffee Name Accuracy Correct
Address Accuracy Correct
Pin Accuracy Approximate
This Starbucks is outside of the boundaries of the specified
location and should be rated Bad since there are so many
Starbucks to choose from in San Francisco.

The name and address are Correct. Using the best

available evidence — in this case, street imagery — the
location of the Starbucks under the rooftop can be found.
Since the pin falls on the correct rooftop, but not in the
correct location, it is rated Approximate.

223
 Results Ratings and Explanation

Starbucks Relevance Bad (Distance/Prominence)

601 Westlake Center, Daly City, CA
94015 Name Accuracy Correct
Category: Coffee
Address Accuracy Correct
Pin Accuracy Perfect

This Starbucks is outside of the boundaries of the specified

location and should be rated Bad since there are so many
Starbucks to choose from in San Francisco.

All the data for the result is correct.

Starbucks Relevance Bad (Distance/Prominence)

12 Manor Plaza, Pacifica, CA 94044
Category: Coffee Name Accuracy Correct
Address Accuracy Correct
Pin Accuracy Approximate

This Starbucks is outside of the boundaries of the specified

location and should be rated Bad since there are so many
Starbucks to choose from in San Francisco.

The pin drops within the Half n' Half rule boundaries of the
shared parking lot.

224
 Query User and Viewport
[Aldi waco tx] User and fresh viewport in Houston, TX
User Intent: The user asked for a specific chain business in a specific location. The fresh viewport
can be ignored since the user has a specific area in mind for results. Because there are very few
stores within the specified area, results outside it are gradually demoted.
Is there a Navigational Result for this Query? Yes

225
 Results Ratings and Explanation
Relevance Navigational
Name Accuracy Correct
Address Accuracy Correct
ALDI Pin Accuracy Approximate
1220 N Valley Mills Dr, Waco, TX
76710
Category: Grocery

ALDI is a well-known chain business. Research confirms

that the location modifier in the query narrows possible
results to a single unique location. This is the only location
eligible for a Navigational rating.

The pin drops within the shared parking lot.

ALDI Relevance Good (Distance/Prominence)

3310 S 31st St, Temple, TX 76502
Category: Grocery Name Accuracy Correct
Address Accuracy Correct
Pin Accuracy Perfect
Although this result is not in Waco, research shows that,
unlike the many Starbucks in San Francisco, there are very
few Aldi stores in the Waco area. That is why this nearby
result should be demoted -2 for distance from the highest
possible initial rating of Navigational and rated Good.

The pin drops on the edge of the appropriate rooftop and

can be rated Perfect.

226
 Results Ratings and Explanation
Acceptable (Distance/
Relevance
ALDI Prominence)
3623 W State Highway 31, Corsicana, Name Accuracy Correct
TX
Address Accuracy Incorrect – Postal Code
Category: Grocery
Pin Accuracy Perfect
Although this result is not in Waco, research shows that,
unlike the many Starbucks in the San Francisco example
above, there are very few Aldi stores in the Waco area.
This result has been demoted -3 from the highest possible
initial rating of Navigational to Acceptable because it is
significantly farther from the area of expected results.

The postal code, which is a mandatory address

component, is missing from the result address.

10.6.3.2. Specific Loca@on Modifier

Location modifiers in the form of streets or full addresses are referred to as specific location
modifiers.

A specific location modifier might also be the name of a POI, as seen here: [university of kentucky
starbucks] in a query about the location of the coffee chain at a particular university.

When the a query includes a specific location modifier, results that satisfy the location intent should
not be demoted for distance:

Number of results
Location of
inside the requested Rating and Explanation
results
location

Consider the location modifier a general one and

None demote results based on distance to the specified
location.

If there is a only a single result present in the specified

location it should receive a rating of Navigational.
One
Inside the It is promoted because it is the only result that satisfies
requested the user intent
location

227
 Number of results
Location of
inside the requested Rating and Explanation
results
location

When there are multiple results in the location, they

should all receive a rating of Excellent.
More than one
Ignore user and viewport location. The user has already
told you exactly where they are looking for results.

Query Results Ratings and Explanation

Is there a Navigational Result for this Query? Yes

Kate Spade New York Relevance Navigational

[kate spade 789 789 Madison Ave, New York, NY
Name Accuracy Correct
Madison Ave, 10065
New York] Category: Accessories Address Accuracy Correct

User and large, Pin Accuracy Perfect

fresh viewport in
This chain business query includes a specific
New York, NY
location modifier in the form of a full address.
Research confirms that this is the address of
queried business, so this result is eligible for
a Navigational rating.

All the data for the result is correct and the

pin is Perfect.

Kate Spade New York Relevance Bad (User Intent)

205 Columbus Ave, New York, Name Accuracy Correct
NY 10023
Category: Accessories Address Accuracy Correct
Pin Accuracy Perfect

This is a very specific query and the result

does not satisfy the specific location modifier
and would not be useful to the user
searching only for the location at the
specified address. This result is rated Bad.

All the data for the result is correct.

228
 Query User and Viewport
[tacodeli burnet rd] User and fresh Viewport in Austin, TX

User Intent: Since there are two Tacodelis located on the named street, both are primary intent.
Any other location is rated Bad.
Is there a Navigational Result for this Query? No

229
 Results Ratings and Explanation
Relevance Excellent
Tacodeli
12001 Burnet Rd, Austin, TX 78757 Name Accuracy Correct
Category: Mexican Address Accuracy Correct
Pin Accuracy Approximate
At first glance, this chain business query with a specific
location modifier may suggest the user is looking for a
unique location. However, research indicates that there
are two branches of this restaurant on the same road.
Both are therefore eligible for a rating of Excellent.

The pin drops off the edge of the appropriate rooftop

and should be rated Approximate.
Relevance Excellent
Tacodeli
7301 Burnet Rd, Austin, TX 78758 Name Accuracy Correct
Category: Mexican Address Accuracy Correct
Pin Accuracy Perfect
At first glance, this chain business query with a specific
location modifier may suggest the user is looking for a
unique location. However, research indicates that there
are two branches of this restaurant on the same road.
Both are eligible for a rating of Excellent.

All the data for the result is correct.

Relevance Bad (User Intent Issue)
Name Accuracy Correct
Tacodeli
Address Accuracy Correct
4200 N. Lamar Blvd, Austin, TX 78756
Category: Mexican Pin Accuracy Perfect

This result does not satisfy the specific location modifier

and would not be useful to the user searching only for
locations along Burnet Road and is therefore rated Bad.

However, if there are no results present on this road,

consider the location modifier as a general one and
demote results based on distance to the road

All the data for the result is correct.

230
When a query is for a POI on a named street, results for this POI that are on the named street but
have an official address on another street can also be rated Excellent.

10.6.4. Back Office and Businesses with No Physical Location

These type of results are most often rated Bad because they are either not useful to the user or
have no maps intent at all.

Business Type Example Rating and Explanation

Businesses with no physical location, like mobile
No Physical location Mobile locksmith locksmiths or mobile dog groomers, have no relevance for
maps and are rated Bad.
Back offices are usually not open to the public. Rate them
Administrative
with a lower prominence. In most cases the relevance
Back Office offices of a
rating will be Bad unless these offices match the specific
company
user intent.
Unless your Country Specific guidelines state otherwise,
give businesses running out of a home very low
prominence.

These businesses see customers by appointment only

and have no regular opening hours or brick and mortar
Home based
Home Business businesses (like a store with regular hours where
calligrapher
customers can drop by and choose from a large selection
of merchandise).

The most likely relevance rating is Bad, unless there is a

likely user intent or the suggestion is similar to a bed and
breakfast or other business that a user may want to visit.

For these types of results, rate the address accuracy according to the official website or consensus of
three sources, otherwise rate Can’t Verify. If the entire address is missing, rate Can’t Verify. If an
address is present, rate the pin according to the address in the result. If no address is present, rate
the pin Can’t Verify.

231
 Query Results Ratings and Explanation
Is there a Navigational Result for this Query? No
Relevance Bad (User Intent)
Name Accuracy Correct
Wendy's Flower Affair
1175 E 930 N, Provo, UT 84604 Address Accuracy Correct
Category: Wedding Planning Pin Accuracy Perfect
[flower shop]
Research shows that this is a business run
User and fresh out of a residential home, not a showroom full
viewport in of flowers. The relevance rating for this
Provo, UT location is Bad. We have been able to confirm
the address and pin for this business and they
will be rated Correct.

This business will not be rated Business/POI

is closed/does not exist because we want to
reserve that option for businesses and POIs
that truly do not exist.

232
10.6.5. Business/POI Does Not Exist

There are situations where a result returned is for a business that does not exist. You must still rate
relevance as if the business were open.

Query Results Ratings and Explanation

Is there a Navigational Result for this Query? No
Ocean Blue Sushi Club Business/POI is closed/does
1010 E El Camino Real, not exist
☑
Sunnyvale, CA 94087
Relevance Excellent
Category: Sushi
[sushi]
Research shows that this business is closed.
User and fresh
The checkbox for Business/POI is closed/
viewport in
Sunnyvale, CA
does not exist should be selected. Name,
address and pin accuracy questions will not
appear. The relevance rating for this location
is Excellent because if this business were
open, it would be a very good result for the
query.

Is there a Navigational Result for this Query? No

North Shore Hotel Business/POI is closed/does
99300 Sea View Drive, Hwy 11 not exist
☑
North Shore, CA
Relevance Excellent
[hotel] Category: Hotel

Fresh viewport
over North
Shore, CA The North Shore hotel no longer exists at this
address, but its relevance is judged as if it
were still in business.

233
10.7. Category Queries

Category queries are the broadest query type because they can return the greatest variety of results.
Give prominence more weight than distance when rating them.

The highest relevance rating most category query results will receive is Excellent.

Category queries with a location modifier are the only category queries eligible for a Navigational
rating (see Navigational Results for Category Queries).

You’ll need to determine the user’s intent based on the query:

• Primary Intent refers to the businesses or POI the user is expecting to see for the given
category.
• Secondary Intent refers to businesses that fit the category intent but would not be the user's
first choice. Secondary intent results will receive a demotion of -1 or an initial rating of Good.
• Unlikely Intent refers to results that technically match the query terms or category intent, but
woud not be an obvious, or even secondary, intent for most users. They receive a demotion of
-2 or an initial rating of Acceptable.

Significant Increase in Distance

Given how many results are often within close proximity to the location intent, results for category
queries are not demoted for distance simply because there is something closer (in the result set or in
real life). Instead, use significant increases in distance to group possible results for the appropriate
demotions.

It is important to understand the real-world distribution of all

possible results when assessing a significant increase in
distance.

The image on the left shows possible results around the user
as purple dots. In this example, demotion for distance starts
approximately 500m away from the user (around 3 to 6 blocks
away) due to the high number of possible results. This distance
will vary depending on how many or few results are available in
a given area.

Please note that no results inside the fresh Viewport should be

demoted to Bad for distance alone.

234
10.7.1.Demotion for Distance with User Inside Viewport

Query User and Viewport

[pizza] User and fresh viewport in Campbell, CA
User Intent: The user is looking for a pizza place.
Is there a Navigational Result for this Query? No

235
 Results Ratings and Explanation
Relevance Excellent
Domino’s Pizza At about 1 km away, this pizza place is among the
930 W Hamilton Ave Campbell, CA closest results to the user, who is in an area with many
Category: Pizza pizza options. Other results in the same general range
would also receive this rating.
Relevance Good (Significant Distance)
Luigi's Pizza and Pasta
2495 S Winchester Blvd, Campbell Luigi’s is significantly further away from the user is and
is demoted for distance, as are all other results in this
Category: Pizza general range.

Relevance Acceptable (Significant Distance)

Pizza My Dear
2590 S Bascom Ave, San Jose, CA Pizza My Dear is over 4km away from the user, which is
significantly further than many other results. It is
Category: Pizza demoted -2 for distance.

236
10.7.2. Demotion for Distance with User Outside Viewport

Query User and Viewport

Fresh viewport in New Braunfels, TX, with user north of
[guns]
San Antonio
User Intent: The user is looking for guns.
Is there a Navigational Result for this Query? No

Results Ratings and Explanation

EDC TX Relevance Excellent

964 N Walnut Ave, New Braunfels, Tx This result is inside the very specific viewport and
Category: Gun Store carries weapons and other accessories.

Relevance Excellent
Freddy’s Pawn & Jewelry
671 S Seguin Ave, New Braunfels, TX The pawn shop is very close to the fresh Viewport and
Category: Pawn Store sells guns and related items.

237
 Results Ratings and Explanation
Relevance Good (Significant Distance)
Guntrap
850 Schneider, Cibolo, TX This weapons store satisfies the user intent but is
significantly further away from the fresh viewport in a
Category: Gun Store more rural area, with fewer such stores close by.
Relevance Good (Significant Distance)
River City Pawn & Jewelry
417 Main St, Scherz, TX The pawn shop sells guns and other related equipment
but is also significantly further away from the fresh
Category: Pawn Store viewport.

238
10.7.3. Demotion for Distance with Fewer Possible Results

Query User and Viewport

[bicycle path] User and fresh viewport in Kennesaw, GA
User Intent: The user is looking for places to ride a bike. These could include trails, parks,
and other paths that are frequented by bicyclists.
Is there a Navigational Result for this Query? No

Results Ratings and Explanation

Relevance Excellent
Deerfield Park Deerfield Park is among the closest results to the user.
2401 Deerfield Dr, Kennesaw, GA There are other results that are within an acceptable
Category: Park range given the number of other possible results that
are within 13km.
Relevance Good (Significant Distance)
Allatoona Creek These mountain bike trails are about 25km away from
Pitner Rd NW, Acworth, GA 30101 the user and significantly more distant than the closest
Category: MTB Trails group of results. The result, and others in the same
general range, is demoted by -1 for distance.

239
 Results Ratings and Explanation
Acceptable (Significant
Relevance
Distance)
Iron Hill Trail Head
Cartersville, GA 30121 Iron Hill Trail Head is about 30km away. It is still within
the Acceptable group in terms of distance to the location
Category: Hiking Area intent given the smaller number of possible results in the
area, and gets a demotion of -2 for distance.

240
10.7.4. Category Queries and User Intent

Query Suggestion Ratings & Explanation

The Art Institute Relevance Excellent
[art] of Chicago
111 S Michigan Ave, This result is an internationally famous
User in Chicago, IL Chicago art museum in Chicago very close to the
(Viewport is stale) Category: Museum user's location.

Good
(User Intent &
Relevance
Arts & Artisans Distance/
321 N Michigan Ave Prominence)
#1, Chicago This art gallery should be demoted due
Category: Art Gallery to distance to user as there are many
other possible art-related matches in the
user’s area.
The Art of Pizza Relevance Bad (User Intent)
727 S State St,
Chicago
Category: Pizza

Even though it has the word “art” in its

name, this pizzeria does not meet the
user’s intent and should be rated Bad.

241
10.7.5. Category Queries and Prominence

Query Suggestion Ratings & Explanation

Solomon R. Relevance Excellent
[museum] Guggenheim
Museum This result is a highly prominent
No user location (viewport is fresh) 1071 5th Ave museum likely to fulfill the user’s primary
New York intent.
Category: Museum

Relevance Excellent
9/11 Memorial &
Museum This is another highly prominent result
180 Greenwich St likely to meet the user’s the primary
New York intent.
Category: Museum

Houdini Museum Good/Acceptable

421 7th Ave Relevance (Distance/
New York Prominence)
Category: Museum
This result is a small museum that’s
much less prominent than other
museums in this location, like the
Guggenheim.
Museum of the
Bad (Distance/
Moving Image Relevance
Prominence)
36-1 35th Ave
Astoria
This result is also much less prominent
Type: BUSINESS than other possible results in the area. It
Category: Museum is located outside of the viewport while
there are multiple other options available
inside of the viewport.

10.7.6. Category Query with Location Modifier

If the query contains a location modifier, all results within the specified location should receive the
highest possible initial rating of Excellent. Results outside the expected location should be demoted
based on their distance from the location modifier and the number of possible real-world results within
it. User and viewport location should be disregarded. Further demotions can be applied based on how
well the result fits the intent of the category.

Note that location modifiers for category queries can take many forms, from the names of cities,
neighborhoods, and streets to the names of POIs, like [Aeropuerto barcelona gasolinera], a query
about gas stations at or near Barcelona’s airport.

242
 Query User and Viewport
[Food montgomery tx] User and viewport outside Montgomery, TX
User Intent: Any food provider or restaurant within the named locality is considered primary
intent. Given the large number of possible results within the explicit area, results outside it are
rated Bad.
Is there a Navigational Result for this Query? No

Results Ratings and Explanation

Relevance Excellent
Pizza Shack
20821 Eva St, Montgomery, TX Name Accuracy Correct
Category: Pizza Address Accuracy Incorrect – Postal Code
Pin Accuracy Approximate
All results that satisfy the ‘Food’ intent located in
Montgomery are eligible for an initial rating of
Excellent.

The postal code, which is a mandatory address

component, is missing from the result address.
The pin drops into the shared parking lot.

243
 Results Ratings and Explanation

Taco Bell Relevance Bad (Distance/Prominence)

3207 W Davis St, Conroe, TX 77304 Name Accuracy Correct
Category: Fast Food
Address Accuracy Correct
Pin Accuracy Perfect

Given the large number of places to get food in

Montgomery, results outside of it are rated Bad.

All the data for the result is correct.

10.7.7. Navigational Results for Category Queries

Sometimes it’s not clear whether a query is for a category or a specific business/POI. What first
appears to be a category query with a location modifier may lead to a single prominent result which
can be rated Navigational.

Due to the ambiguity of these queries, even if there is a Navigational result, the query should still be
treated as a potential category query and other results considered relevant.

When deciding if a category query could have a Navigational result, consider:

• The actual wording of the query

• Possible results in the real world
• Local knowledge
• Prominence

244
 Query Results Ratings and Explanation
Is there a Navigational Result for this Query? Yes
Relevance Navigational
Hotel Kilkenny Name Accuracy Correct
College Road, Kilkenny R95
KP08 Address Accuracy Correct
Ireland Pin Accuracy Perfect
Category: Hotel

This result is an exact match to the query. The

wording of the query leads to a single result,
so it’s likely the user intent is this specific
hotel.

The address is correct without a street

number because it matches what's found on
the official website. The pin landing in the
hotel's roof is Perfect.
[hotel
kilkenny]
Relevance Excellent
Name Accuracy Correct
Newpark Hotel
Castlecomer Road, Kilkenny R95 Address Accuracy Correct
KP63, Ireland Pin Accuracy Perfect
Category: Hotel
Although the wording of the query suggests a
single result, there is still a possibility that the
intent is for any hotel in Kilkenny. Since this
could be a category query with a location
modifier, all hotels in this locality should be
rated as Excellent.

The address is correct without a street

number because it matches the address
shown on the official website. The pin is
Perfect because it lands on the hotel’s
rooftop.

245
 Query Results Ratings and Explanation
Is there a Navigational Result for this Query? No
Relevance Excellent

Hotel Kilkenny Name Accuracy Correct

College Road, Kilkenny R95
Address Accuracy Correct
KP08
Ireland Pin Accuracy Perfect
Category: Hotel
There is no result that matches the exact
wording of the query, so this is more likely to
be a category query with a location modifier.
Any hotel result returned in the locality of
Kilkenny would be rated Excellent.

The address is correct without a street

number because it matches what appears on
the official website. The pin landing the hotel's
roof is Perfect.

Relevance Excellent
Name Accuracy Correct
[Kilkenny
hotel] Address Accuracy Correct
Pin Accuracy Perfect

Newpark Hotel
Castlecomer Road, Kilkenny R95
KP63, Ireland
Category: Hotel

Any hotel result returned in the locality of

Kilkenny would be rated Excellent,
completely satisfying the ‘Category with
Location Modifier’ query

The address is correct without a street

number because it matches the address
shown on the official website. The pin is
Perfect because it lands on the hotel’s
rooftop.

246
 Query Results Ratings and Explanation
Is there a Navigational Result for this Query? No
Relevance Excellent
Name Accuracy Correct
Address Accuracy Correct
Berlin Tegel Airport Pin Accuracy Perfect
13405 Berlin, Germany
Category: Airport
There are two equally prominent airports in
Berlin. Since there is more than one possible
result for this query in the real world, it is likely
to be a category query with a location
modifier. All airports within the queried locality
should be rated as Excellent.

The address is correct without a street

number and street name because this is a
POI without an expected address so a locality
and postal code are appropriate. The pin is
Perfect because it lands on the airport’s
rooftop.

[airport berlin]
Relevance Excellent
Name Accuracy Correct
Address Accuracy Correct
Pin Accuracy Perfect
Berlin Schönefeld Airport
12521 Berlin, Germany
Category: Airport
There are two equally prominent airports in
Berlin. Since there is more than one possible
result for this query in the real world, it is likely
to be a category query with a location
modifier. All airports within the queried locality
should be rated as Excellent.

The address is correct without a street

number and street name because this is a
POI without an expected address so a locality
and postal code are appropriate. The pin is
Perfect because it lands on the airport’s
rooftop.

247
10.7.8. Clear Categories

When a query is clearly for a straightforward category, results must also belong to that category

Query Results Ratings and Explanation

Is there a Navigational Result for this Query? No
Relevance Bad (User Intent)
Name Accuracy Correct
Sephora
2855 Stevens Creek Blvd, Address Accuracy Correct
Santa Clara, CA 95050 Pin Accuracy Approximate
Category: Beauty Supplies
[mall] The query intent is for a mall. The result is a
store inside Westfield Valley Fair mall and is
User and fresh within the viewport. Since the query is a
viewport in category and the result is a specific store,
Santa Clara, which the user did not ask for, rate Bad due to
CA user intent. If the result had been the entire
mall, the result would be rated Excellent.

The name and address can be confirmed on

the official website. The pin is Approximate
because it does not land on Sephora's location
(shown in green) according to the mall’s official
map.
Is there a Navigational Result for this Query? No
Relevance Bad (User Intent)
Name Accuracy Correct
Terminal 8 Address Accuracy Correct
JFK International Airport
Queens, NY 11430 Pin Accuracy Perfect
Category: Airport
[airports]
The result is a terminal that's part of JFK
User and fresh
International Airport. Since the query is a
viewport in New
category and the result is a specific terminal
York, NY
and not the airport itself, rate Bad due to user
intent.

The name and address can be confirmed on

the official website. The pin is Perfect because
it lands on one of the rooftops of Terminal 8
according to the official airport map.

248
10.7.9. Soft Categories

Some category queries are straightforward and may have results that satisfy an intent exactly, like
[italian restaurant]. However, sometimes categories are very small or have multiple interpretations,
so several different types of results could satisfy the user’s intent. These are called “soft categories.”

These category queries will produce a variety of results, each of which could satisfy the query intent
to a greater or lesser degree. How well a result satisfies a category query will vary depending on the
services typically offered by the result entities in that market and the customs of the specific market.

It’s often difficult to define which businesses or chains belong in any one soft category, so you’ll have
to research the individual results provided to make your rating decisions. For instance, the query [ski
shop] is very specific and is associated with the more general Sporting Goods category.

249
 Query User and Viewport
[ski shop] User and fresh viewport in Denver, Co
User Intent: The user is looking for places that sell ski clothing and equipment. Any store that
carries such items is consider primary intent.
Is there a Navigational Result for this Query? No

250
 Results Ratings and Explanation
Relevance Excellent
Meier Skis
970 Yuma St, Suite 190 Name Accuracy Correct
Denver, CO 80204 Address Accuracy Correct
Category: Ski Equipment
Pin Accuracy Perfect

Business is a shop dedicated to skiing, which would

satisfy the primary intent of this category.

Patagonia Relevance Good (User Intent)

1431 15th St Name Accuracy Correct
Denver, CO 80202
Address Accuracy Correct
Category: Sports Wear
Pin Accuracy Perfect

Stores that sell ski clothing would be considered

secondary intent for the category query [ski shop].

251
 Query Results Ratings and Explanation
Is there a Navigational Result for this Query? No
Relevance Excellent
Name Accuracy Correct
Riva's Italian Restaurant
1117 Missouri St Address Accuracy Correct
Houston, TX 77006
Category: Italian Pin Accuracy Perfect

The most likely user intent for this query is

Italian restaurants. This result satisfies the
user’s intent.

Cafe Dolce Gelato Relevance Good (User Intent)

5135 W Alabama St, Ste 7315
Houston, Texas 77056 Correct
Category: Ice Cream Name Accuracy

Address Accuracy Correct

[italian] Pin Accuracy Approximate

User and fresh

viewport in
Houston, TX
An Italian ice cream café is mostly likely a
secondary intent for this query.

The official mall map shows the exact location

of the cafe. The pin lands on another part of
the mall and is rated Approximate.

252
Query Results Ratings and Explanation
Prada Houston Galleria
Relevance Bad (User Intent)
5015 Westheimer, Ste 2285A
Houston, Texas 77057
Name Accuracy Correct
Category: Accessories
Address Accuracy Incorrect – Postal Code

Pin Accuracy Perfect

An Italian clothing store is an unlikely intent for

this query.

The name is Correct, but the postal code

should be 77056.

The official mall map shows the exact location

of the store. The pin lands on this location and
is rated Perfect.

253
10.7.10. Category Query: Parking

"Parking" generally refers to:

• Parking lots
• Parking garages
• Parking decks

Example
Query Results Ratings and Explanation
Type
Is there a Navigational Result for this Query? No
Relevance Excellent
Parking
123 O’Farrell St, San Francisco, Partially Correct
Name Accuracy
CA 94102 (Name Issue)
[parking] Category: Parking Garage Address Accuracy Correct
Specific User and Pin Accuracy Perfect
Parking viewport in
San
Francisco, Research reveals that the official
CA name for this parking location is Ellis-
O'Farrell Garage. Since an official
name exists, the generic name
“Parking” is demoted to Partially
Correct.

Is there a Navigational Result for this Query? No

Parking Relevance Excellent
121 Spear St, San Francisco, CA
Name Accuracy Correct
[parking] 94105
Category: Parking Garage Address Accuracy Correct
User and
Generic fresh Pin Accuracy Perfect
Parking viewport in
San
Francisco, This result is the parking garage for
CA the Rincon Center, which does not
have a specific name according to
official resources. Accept the generic
name “Parking” as Correct.

254
Example
Query Results Ratings and Explanation
Type
Is there a Navigational Result for this Query? No
Relevance Excellent
Rincon Center Name Accuracy Correct
121 Spear St, San Francisco, CA
94105 Address Accuracy Correct
[parking] Category: Parking Garage
Pin Accuracy Perfect
User and
Paid vs. fresh
This result is the parking garage for
free viewport in
San the Rincon Center. It’s a parking
Francisco, garage accessible to the public for a
CA fee. Free and paid parking are equally
relevant. There should be no
relevance demotion if the parking lot/
garage requires payment and the
query does not specifically ask for free
parking.

Is there a Navigational Result for this Query? No

Relevance Excellent
Name Accuracy Correct
Parking
3705 El Camino Real, Santa Address Accuracy Correct
[parking Clara, CA 95051
Pin Accuracy Perfect
free] Category: Parking

User and
The query specifically requests free
Paid vs. fresh
parking, so the results should reflect
free viewport in
Santa the user’s request. This parking lot is
Clara, CA free and should be rated as Excellent.

If the result were a parking lot that

requires payment, then the correct
relevance rating would be Bad, as it
does not satisfy the user’s intent.

255
Example
Query Results Ratings and Explanation
Type
Is there a Navigational Result for this Query? No
Stevens Creek Parking Garage Relevance Excellent
2899 Stevens Creek Blvd, San
Jose, CA 95050 Name Accuracy Correct
Category: Parking Address Accuracy Correct
[parking]
Affiliate Pin Accuracy Perfect
parking User and
that has a Fresh
specific Viewport in
name San Jose,
CA The name of the parking garage can
be confirmed on the official site of the
POI to which it belongs, therefore the
name is Correct.

Is there a Navigational Result for this Query? No

Relevance Excellent
Hamilton/Waverly Lot Name Accuracy Correct
375 Hamilton Ave, Palo Alto, CA
94301 Address Accuracy Correct
[parking] Category: Parking
Pin Accuracy Perfect
Generic User and
fresh
Parking
viewport in
Palo Alto,
CA The name of the parking lot can be
confirmed on the official site of the city
of Palo Alto.

256
Example
Query Results Ratings and Explanation
Type
Is there a Navigational Result for this Query? No
Parking Relevance Excellent
San Francisco, CA United States
Name Accuracy Correct
Category: Parking
Address Accuracy Correct
[parking] Pin Accuracy Perfect
User and
fresh
Airport This result refers to the parking lot of
viewport in
Parking San Francisco International Airport.
San
Francisco, The generic name “Parking” is
CA considered Correct. Pin must be
dropped on the SFO parking lot. This
is a POI without an expected address
so just a locality is appropriate and
rated Correct without a street number
and street name.

Is there a Navigational Result for this Query? No

Parking Relevance Bad (User Intent)
3600 Flora Vista Ave, Santa Clara,
CA 95051 Name Accuracy Correct
[parking] Category: Parking Address Accuracy Correct
Private Pin Accuracy Perfect
User and
vs. public
viewport in
parking
Santa
Clara, CA The result in this example is a
residents-only parking lot. This should
be rated Bad because the lot is not
useful for the general public.

257
10.8. Rating Results with PERMANENT_CLOSURE Status

When rating results whose status is PERMANENT_CLOSURE, be sure to understand whether the
results are expected or unexpected. For more information on the differences between expected and
unexpected results, see section 5.19. Rating Relevance When Result Status is
PERMANENT_CLOSURE.

10.8.1. Expected PERMANENT_CLOSURE Status

When a result with PERMANENT_CLOSURE status is expected, choose the Business/POI is

closed or does not exist checkbox and rate as if the location were open.

Query Results Ratings and Explanation

Is there a Navigational Result for this Query? Yes
Relevance Navigational
Business Closed/does not
Yes
Gaslight Grill exist
PERMANENT_CLOSURE Name Accuracy no ratings needed
[gaslight
5020 W 137th St, Overland
grill]
Park, KS 66224 Address Accuracy no ratings needed
Fresh Pin Accuracy no ratings needed
viewport in
Kansas City, Research shows that the result for this navigational
KS query is permanently closed, and it displays the
status PERMANENT_CLOSURE. Showing this
closed result is expected since no other result can
be returned in the area of location intent.

Select Business is closed or does not exist and

rate the business as if it did exist. No further POI
data rating will be needed.
10.8.2. Unexpected PERMANENT_CLOSURE Status

When many branches of a chain are available in the area of intent, we would not expect to see any
locations with a status of PERMANENT_CLOSURE, whether that status is correct or not.

Query User and Viewport

[mcdonalds] User and fresh viewport in Sunnyvale, CA

259
 Suggestion Rating and Explanation
Relevance Acceptable
Business/POI closed/
Yes
McDonald's does not exist
PERMANENT_CLOSURE
This permanently closed restaurant (confirmed by research) has
615 N Mathilda Ave,
its status set to PERMANENT_CLOSURE. It is considered
Sunnyvale, CA 94085
unexpected because there are many other locations of this chain
in the area. This means the highest rating it can get is
Acceptable. Name, address, and pin will not be rated.
Relevance Excellent
Business/POI closed/
No
does not exist
McDonald's Name Accuracy Correct
550 Lawrence Expy,
Sunnyvale, CA 94086 Address Accuracy Correct
Pin Accuracy Perfect
This open chain location is among the closest to the user and
rated Excellent.
Relevance Acceptable
Business/POI closed/
No
does not exist
Name Accuracy Correct
Address Accuracy Correct
Pin Accuracy Perfect

McDonald's
PERMANENT_CLOSURE
556 E El Camino Real, This chain location is showing a status of
Sunnyvale, CA 94086 PERMANENT_CLOSURE. Research shows that it is, in fact,
open. Its relevance is rated independent of whether it is really
open or closed in the real world. The result is considered
unexpected and rated Acceptable.

Do not mark this result as Business/POI closed/does not exist,

and continue rating the result for relevance plus name, address,
and pin details.

260
 Suggestion Rating and Explanation
Relevance Good (Distance)
Business/POI closed/
Yes
does not exist
McDonald's
1082 E El Camino Real, The chain location does not exist according to official resources
Sunnyvale, CA 94087 and should be rated as Business is closed or does not exist.
The result does not show the status as PERMANENT_CLOSURE,
so it should be rated as if it were open, which is Good given the
distance.

10.9.Other Query Types

Queries can sometimes be less obvious or even ambiguous.

10.9.1. Routing Queries

When a query mentions two distinct locations that are not near each other, the user was likely looking
for driving directions, so returning either of the two distinct locations used for the route is the best
experience Search can offer. Each location should be rated Excellent for Relevance.

Query Results Ratings and Explanation

Is there a Navigational Result for this Query? No
Relevance Excellent
Name Accuracy n/a
Oxford Address
Oxford, United Kingdom Correct
Accuracy
Category: n/a
Pin Accuracy Perfect

The query includes two separate locations,

which are very far away from each other, so
the user was likely looking for routing
directions. Returning either of the individual
locations is expected and should be rated
Excellent.
[66 Chandos
Pl, London The pin should be dropped within the
WC2N 4HG boundaries of the result locality.
Oxford]
261
 Query Results Ratings and Explanation
Relevance Excellent
66 Chandos Pl
66 Chandos Pl Name Accuracy n/a
London WC2N 4HG Address
Category: n/a Correct
Accuracy
Pin Accuracy Perfect
The query includes two separate locations,
so the user was likely looking for routing
directions. Returning either of the individual
locations is expected and should be rated
Excellent.

The pin should be dropped on the address

provided in the result.

Is there a Navigational Result for this Query? Yes

Oxford Relevance Navigational

Oxford, United Kingdom
Name Accuracy n/a
Category: n/a
Address
Correct
Accuracy
[route to
Oxford] Pin Accuracy Perfect

This routing query contains only one

location.

262
10.9.2.Coordinate and “My Location” Queries

Note: Results for coordinate and ‘’my location” queries (or any variation of these, like “current
location,” “where I am,” and so on) are rated differently from the other results described in these
guidelines. Please read this section carefully.

Pin and Relevance are rated separately using a 50m radius around the queried coordinates or the
user location (for “my location” queries):
• Relevance:
• A result that exists and is within the 50m radius is rated Excellent.
• A result that does not exist or is outside of the 50m radius is rated Bad.
• There will be no Navigational results.
• Pin Accuracy:
• A pin that falls within 50m radius is rated Perfect.
• A pin that falls outside the 50m radius is rated Wrong.
• Name and Address Accuracy:
• Rate the name and address accuracy according the regular guidelines.

The graphic below gives an overview. These concepts are explained further on the following pages.

263
If no result can be found within 50m, then the closest address or street up to 100m away
will be accepted. If no address or street can be found within 100m, depending on the
market, a locality or sub-locality will also be accepted.

Query Pin Rating and Explanation

Draw an imaginary circle with a 50m radius around the

user’s location. Rate any pin inside the circle Perfect and
any pin outside the circle Wrong.

Draw an imaginary circle with a 50m radius around the

queried coordinates. Rate any pin inside the circle Perfect
and any pin outside the circle Wrong.

264
 Results Relevance Rating and Explanation

Use an imaginary circle with a 50m radius around the queried

location to rate relevance. A result that has any part of its
Perfect area within the circle is rated Excellent. If a result
does not have any Perfect area inside the circle, rate Bad.

The Perfect area usually refers to the rooftop or section of the

rooftop, but can include the parcel for parks and other features
rated perfect for the parcel.

Result Relevance Rating and Explanation

1: 3495 El Camino Real Part of the POI is inside the

Excellent
Santa Clara, CA 95051 50m radius

2: 3530 El Camino Real

Part of the POI is inside the
Santa Clara, CA 95051 Excellent
50m radius

The street is inside the 50m

3: El Camino Real (street name) Excellent
radius
4: United Furniture Club
No part of the POI is within
3503 El Camino Real Bad
the 50m radius
Santa Clara, CA 95051
5: 3482 El Camino Real Unit B Research shows this address
Bad
Santa Clara, CA 95051 does not exist

Note that the pin and result title do not need to match. For example, if a result for United Furniture
Club is returned, but the pin is placed at the queried coordinates or user location, the Relevance
Rating will still be Bad, but the pin rating will be Perfect even though it is far from United Furniture’s
location.

10.10. Unclear Results

There may be times when it is difficult to determine exactly what the result is because the business
name can’t be identified with certainty or the business does not correspond with the given location
and it cannot be confirmed as closed. When this happens, use all provided information (including
information that is not rated, such as URL and phone number – never call a business) to reach a
conclusion.

265
If there is a tie in the evidence or you are rating Search Relevance only, rate Business/POI is
closed/does not exist, then rate relevance as if the entity did exist.

Query Results Ratings and Explanation

Is there a Navigational Result for this Query? No
Caribou Coffee Relevance Excellent
227 West Market
Bloomington, NM 55425 Name Accuracy Correct
Category: Coffee Shop Incorrect – Street
Address Accuracy Number and Street
Additional info: Name
URL: cariboucoffee.com
Phone: 1-952-854-7828 Pin Accuracy Perfect
[caribou]
The result name and URL are pointing to a
User and fresh
Caribou location. The address listed belongs
viewport in
to the Nike store located in the same
Bloomington,
shopping center. There are multiple Caribou
NM
Coffee locations inside the Mall of America but
the phone number belongs to the store on 60
East Broadway S-380 per official website or
380 South Ave per mall website. The pin is
located in the same area as shown on the
mall website. Because a business with the
same name and phone number is found in the
immediate surrounding area, we will consider
the address incorrect for this result.

266
 Query Results Ratings and Explanation

Is there a Navigational Result for this Query? No

Relevance Excellent

Pottery Barn Name Accuracy Correct

1350 Fashion Valley Rd, San Incorrect – Street
Diego, CA 92108 Address Accuracy Number and Street
Category: Furniture Store Name

[furniture Additional info: Pin Accuracy Approximate

store] URL: www.potterybarn.com
The result name and URL point to a Pottery
Phone: 1-619-296-8014
User and Fresh
Barn location. The address listed is
San Diego, CA associated with a nearby business, Wells
Fargo Bank. After researching the nearby
area, a Pottery Barn with the same phone
number as our result is found at 7007 Friars
Road, San Diego, CA 92108, within the
Fashion Valley Mall. The pin lands on the
Fashion Valley Mall. Because a business with
the same name and phone number is found in
the immediate surrounding area, consider the
address incorrect for this result.
Is there a Navigational Result for this Query? No
The Lot Bar Relevance Excellent
2333 S St
Sacramento, CA 95816 Name Accuracy Incorrect (Name Issue)
Category: Dive Bar Address Accuracy Correct

[bar] Additional info: Pin Accuracy Perfect

267
 Query
[bar] Results
Additional info: Ratings and Explanation
Phone: 1-916-451-4682
User and Fresh Research on the result information shows that
Viewport in the address belongs to Round Corner Tavern.
Sacramento, CA The result phone number also belongs to the
Round Corner Tavern and the pin is located at
the result address. Finally, there is no ‘Lot Bar’
located in the immediate area. Because all
result information except for the name listed
points to the Round Corner Tavern, we will
consider the name incorrect for this result.

Is there a Navigational Result for this Query? No

Holiday Inn Business/POI is closed/does not
☑
300 N Harbor Drive exist
Redondo Beach, CA 90277
Category: Hotels and Events Relevance Excellent
Use all the information provided , even
Additional Info: information that's not rated, like URL and
URL: http://www.ihg.com/ phone number, to research the result.
holidayinn
Phone: 1-310-750-1149 In this case, research shows that the address
belongs to a Sonesta Hotel, not a Holiday Inn.
Street imagery also shows that this building is
[holiday inn] a Sonesta.

User and fresh The result phone number does not belong to
viewport in Santa any business and the given URL does not
Ana, CA point to any particular Holiday Inn.

Investigation of the surrounding area shows

that there is no Holiday Inn in Redondo
Beach, so this is not a case where the
business has been misplaced and the result
contains an incorrect street name or number.

Because the result information does not

overwhelmingly point to a Holiday Inn at this
location, rate the result Closed/Does not
exist.

268
Query Results Ratings and Explanation

Is there a Navigational Result for this Query? No

Santa’s Professional Dog

Relevance Excellent
Grooming
603 Manley St, North Pole
AK, 99705 Name Accuracy Incorrect (Name Issue)
Category: Pet Services

Additional info: Address Accuracy Correct

Phone: (907) 480-9701
URL: http://
Pin Accuracy Perfect
www.paulasprofessionaldoggroom

269
 Query Results Ratings and Explanation
ing.com/
Use all the information provided , even
information that's not rated, like URL and
phone number, to research the result.
[santa's dog
grooming] Here, research shows that the address, url,
and phone number belong to Paula’s
User and fresh Professional Dog Grooming. Since the
viewport in North evidence points to Paula’s business actually
Pole, AK existing at this location, the name Santa’s
Professional Dog Grooming is considered
Incorrect.

When rating relevance, ignore all data issues

and rate the result as if the name and
category were correct. Because of the match
between the query and the incorrect name,
and how near the user is to the result in the
fresh viewport, this result is rated Excellent.

When rating other results, treat this one as

closed/does not exist and don’t demote the
relevance of other, more distant, results. This
is so that a valid result that’s farther away
won’t be demoted solely due to this naming
error.

Is there a Navigational Result for this Query? No

Shell Business/POI is closed/does not
☑
1221 S Main St, Wildwood, FL exist
34785
Category: Gas Station Relevance Excellent

270
 Query Shell Results Ratings and Explanation
1221 S Main St, Wildwood, FL
34785
Category: Gas Station

Additional info:
URL: www.shell.com
Phone: 1-352-748-6199 There are two gas stations in the immediate
area:

Mobil (purple pin)

1221 S Main St
Wildwood, FL 34785
Phone: 1-352-748-6199

Shell (red pin)

1001 S Main St
[gas station] Wildwood, FL 34785
User and fresh Phone: 1-352-748-8903
viewport in
Wildwood, FL
The result contains conflicting information
from two nearby businesses. The result name
and URL point to a Shell gas station a little
north of the result address listed, while the
result address and phone number point to a
Mobil gas station. If, after researching all of
the given information for a result no clear
consensus of what the result is pointing to can
be made, the result should be rated as
Closed/Does not exist.

You will still need to rate relevance as if the

business were open. The result matches the
query intent and is within the fresh viewport.
The result should receive a rating of
Excellent.

271
 Query Results Ratings and Explanation
Is there a Navigational Result for this Query? No
Dinosaur Inc Business/POI is closed/does not
☑
8655 Jones Rd exist
Houston,TX77065
Relevance Excellent
Additional info: Research on the result information shows
Phone: 1-281-531-6500 that the address belongs to Trails at
Category: n/a Corinthian Creek Apartments. The result
[dinosaur]
phone number belongs to a different
User and large apartment complex, Domain West
fresh viewport in Apartments. Finally, there is no evidence that
Texas the business Dinosaur Inc exists. Based on
the information listed, rate the result as
Closed/Does not exist.

You will still need to rate relevance as if the

business did exist. The result matches the
query intent and is within the large, fresh
viewport. The result should receive a rating of
Excellent.
Is there a Navigational Result for this Query? No
Business/POI is closed/does not
Lowongan Guru Kimia SMA SMU ☑
exist
Di Jakarta
[Salon di jalan Jalan Danau Sunter Utara No. 1 Relevance Bad (User Intent)
tebet timur Tanjung Priok
dalam raya DKI Jakarta 14350 The query is for a category with a specific
jakarta Indonesia location modifier. The result is for a job
selatan] Category: Education opening which does not satisfy the intent and
is not indicative of a POI. There is a
Translation: Result Title Translation: possibility the result is for a particular high
[Salons in [Job Opening for High School school, however, the result address is not
Jalan Tebet Chemistry Teacher in Jakarta] tied to any high school and no specific school
Timur Dalam Additional info: is named in the result title. Since the result
Raya, Jakarta Lat/long: -6.13799,106.86598 data is ambiguous and does not represent a
Selatan] or single POI this entity should be rated
[Category, Closed/Does not exist.
Street Name,
City Name] You will still need to rate relevance as if the
business were open. The user's intent is for a
Salon on a specific street. The result does
not match this intent and should be rated as
Bad.

272
11. Top Rating Tips
Solutions to the most common rating dilemmas.

11.1. What’s the relevance when query is business/POI name and address and the result is
only the address?

When a user types in the name of a business or a POI along with its address, they expect to see the
business mentioned in the query as part of the result. If the result is the address alone and does not
mention the business, the user cannot be certain that the query and the result refer to the same thing.
This is why the result’s relevance is Bad.

Address and pin accuracy can be rated as usual.

Query Results Ratings and Explanation

800 North Point Relevance Bad (User Intent)
800 North Point St, San Francisco,
California 94109 Name Accuracy n/a
[Gary Danko Category: New American Cuisine Address Accuracy Correct
800 North
Point St, San Pin Accuracy Perfect
Francisco, Result is for the address and not the
California business stated in the query, so the user
94109] cannot know whether or not the query and
result refer to the same thing. This is why the
result’s relevance is Bad.

11.2. What’s the relevance when the query is a street name and the result is a single business
on that street?

A query for an entire street is very broad. When the result is just one business or one address on that
street, it is too specific to satisfy the broad intent of the query.

Name, address, and pin accuracy can be rated as usual.

273
 Query Results Ratings and Explanation
Happy Lamb Hot Pot Relevance Bad (User Intent)
19062 Stevens Creek Blvd
Cupertino, CA, 95014 Name Accuracy Correct
Category: Hot Pot Address Accuracy Correct
Pin Accuracy Perfect
[Stevens
Creek Blvd]
The result is for a business on the street.
When the query is for a street, returning a
single business is too specific a result for the
broad query.

11.3. How do I know what the location intent is when user is outside/inside a fresh viewport?

When a viewport is fresh, the location intent for results depends on whether the user is inside or
outside the viewport:

• When a user is inside the viewport, that user’s location is the location intent

• When a user is outside the viewport, results are expected in or near the viewport, no matter
how near or far the user is from that viewport.

Viewport User Location Intent

When the user is within a fresh viewport, take the user
location as location intent. Results are generally expected in
Inside Viewport
or near the viewport, and results inside the area cannot be
rated Bad because of distance alone.

Fresh Results are expected in or near the viewport area. If no results

Outside Viewport can be found in or near the viewport, use the user location as
a secondary location intent.

When the user is missing, the viewport remains the location

Missing
intent.

274
11.4. How do I know what the location intent is when user is outside/inside a stale viewport?

When a viewport is stale, the user’s location is considered the location intent for the results whether
the user is inside or outside the viewport

Viewport User Location Intent

Inside Viewport
When the viewport is stale, consider only the user location as
Outside Viewport location intent.
Stale
Use the stale viewport as location intent when the user
Missing
location is missing.

11.5. This full address result does not exist. How do I rate it?

An address type result needs to be associated with a building or officially assigned to a plot of land in
order to be useful to a user. Sometimes your research will show that a queried address simply does
not exist. In cases like, this you’ll see one of three types of results:

• The closest verified address (on the same street, in the same city and state): Rate
relevance as Excellent.
• The same address as the query address: Rate relevance as Excellent and address as
Incorrect – Address does not exist. The pin will be rated Can’t Verify.
• The queried street without a street number: Rate relevance as Acceptable.

When a queried address does not exist, the answer to the question “Is there a navigational result for
this query” will always be No.

275
 Query Results Ratings and Explanation
Is there a Navigational Result for this Query? No

[2001 Duncan 1099 Duncan St Relevance Excellent

St, San 1099 Duncan St., San
Francisco, 94131, CA Name Accuracy n/a
Francisco]
Category: n/a Address Accuracy Correct
Pin Accuracy Perfect

If 2001 does not actually exist then selecting

1099, the closest number that does exist, is
considered correct. For a locale that uses the
rule of even numbers on one side of the street
and the odd numbers on the other, we need to
return the closest existing address on the same
side of the street as the query. If the rule of
odds and evens does not apply then the closest
numeric number will be considered correct.
The pin has to be correctly dropped on the
address presented in the result.

2001 Duncan St Relevance Excellent

2001 Duncan St., San Name Accuracy n/a
Francisco, 94131, CA
Category: n/a Incorrect – Address
Address Accuracy
does not exist
Pin Accuracy Can’t Verify
Duncan Street only goes up to number 1099,
but the query requests a non-existent number.
The result returned is the same as the queried
address and should be rated as Excellent.
The pin rating for addresses that do not exist is
Can’t Verify.

276
 Query Results Ratings and Explanation
Duncan St Relevance Acceptable (User Intent)
Duncan St, San Francisco,
94131, CA Name Accuracy n/a
Category: n/a Address Accuracy Correct
Pin Accuracy Perfect

The result returns only the street with a pin

dropped within the boundaries of the street.
This result technically satisfies the user intent,
but this intent is unlikely, so relevance should
be rated Acceptable.

Appendix: Release Survey

You may occasionally be unable to rate a survey. To skip it, click Release Survey at the top right of
the tool.

A pop-up window with a list of reasons for release will appear. You must choose one before you can
release the survey.

Note: If you change your mind and decide to rate the survey, click Cancel at the bottom left of the
pop-up window.

Reasons to Release

1. Adult Content

You are not comfortable rating queries and/or results that include businesses offering adult services
or entertainment, like strip clubs.

2. Technical Issue

Technical issues that prevent rating include:

• Map or map features not loading.

• Query field is empty.
• Tool is not interactive. You cannot zoom in and out or move the map at all.
• Cannot submit ratings.
• More than 5 pins in a row are missing or pins do not stay in place.
• Receiving 5 consecutive surveys with all results in China, if China, or Hong Kong, are not the
markets you are rating.

If you choose this option, leave a comment describing the issue before releasing the survey.

277