Cloud Observability Unified Query A UQL query is piped, made up of two or more stages that operate on the metric

of two or more stages that operate on the metric or span data returned by the
Fetch stage. Subsequent stages operate on the data returned from the previous stage.
Language (UQL) Cheat Sheet
Stage 1: Fetch Stage 2: Align Stage 3: Filter Stage 4: Group & Aggregate
Visit Cloud Observability Learning Portal for an intro to UQL
and reference documentation metric request | rate | filter service == checkout | group_by [operation], sum

Fetch (required) Latency values for services where the point value is greater
than 1 second metric [Link] | latest | group_by [], sum |
Returns the specified data type (metric or spans). Metric point value > 5
requires the metric name. Spans requires a computation type.
spans latency | delta | group_by [service],
sum | point percentile(value, 99) | Squared value of each point in [Link]
Fetching the rate (ops/s) of HTTP requests metric
point_filter value > 1000
metric [Link] | latest | group_by [], sum |
metric [Link] | rate | group_by [], sum
point pow(value, 2)
Requests where the the point value is less than 1,000 requests

Fetching the p95 latency of all spans Change in HTTP requests over previous hour
metric requests | delta | point_filter value < 1000
spans latency | delta | group_by [], sum | metric [Link] | delta 1h | group_by [],
point percentile(value, 95) Align (required) sum

Aligns points to regular time periods and can include an input Maximum value of each time series over previous hour
Allowed values: latency count [Link]
window and output period. Output periods are required and only
Fetching the count of all spans as a rate (ops/s) allowed on intermediate aligners. Delta and cumulative type metric [Link] | reduce 1h, max |
metrics typically use the rate and delta aligners. Gauge metrics group_by [], sum
spans count | rate | group_by [], sum typically use reduce. Latest can only be used with gauges.
Allowed values: max min mean sum distribution count
Rate (ops/s) of HTTP requests metric count_nonzero std_dev
Fetch the rate of all logs that contain an error in severity_text
grouped by service name (logs/s) Maximum change over 1 minute of requests in the previous hour
metric [Link] | rate | group_by [], sum

logs count metric [Link] | delta 1m, 1m | reduce

Allowed values: rate delta latest reduce
| filter severity_text == "error" 1h, max
| rate Rate (ops/s) of HTTP requests metric, averaged over previous 5 mins
| group_by [[Link]], sum Explicit output period required for intermediate aligners (final
aligners don't have any output period)
metric [Link] | rate 5m | group_by [],
Fetch all logs where the service equals web logs sum
Gauge points aligned to a consistent period

logs Allowed values: s m h d w

metric [Link] | latest | group_by [], sum
| filter [Link] == "web"
Change in HTTP request metric
Join
Filter metric [Link] | delta | group_by [], sum
Combines two or more sets of time series by matching
Drops data that doesn't match the predicate.
attributes, then applies arithmetic to create a single set of time
Group by (required for spans & logs time series. A join can result in a very large results set when labels
Rate (ops/s) of HTTP requests only if from the checkout
service and in the us-east-1 time zone
series queries) have high cardinality. Using group_by before a join reduces the
Separates time series into groups by the values of the given labels the join will match on.
metric [Link] | rate 1m | filter attributes, then for each value group, combines the time series Fraction of requests that were successful, broken down by
service == checkout && zone == us-east-1 | using a reducer. Queries for span and log time series data must HTTP method (for example, GET, PUT)
group_by [], sum include a group_by. To get a single time series, use an empty
group_by value, reduced by the sum. with
Allowed values: = == > >= < <= ! != && || defined successes = metric [Link] |
Time series for memory usage, by host
undefined contains phrase_match !~ (regex no match) delta 1m | group_by [[Link]], sum;
=~ (regex match) total = metric [Link] | delta
metric [Link] | latest |
1m | group_by [[Link]], sum;
p95 latency for spans from the iOS service group_by [host], sum
join successes/total, successes = 0

spans latency | delta | filter service == iOS | Allowed values: max min mean sum distribution count

group_by [], sum | point percentile(value, 95) count_nonzero std_dev Percent of kube memory limits currently being used by
dataingest, broken down by container and pod
Maximum value of the rate of requests per second over a two
p95 latency for spans from the iOS service and the customer minute period, grouped by the region and zone with
Packing Kings
usage = metric [Link] |
metric requests | rate 2m | group_by [region,
spans latency | delta | filter service == iOS latest | filter kube_app == dataingest |
zone], max
&& customer == "Packing Kings" | group_by [], group_by [pod_name, container_name], sum;
sum | point percentile(value, 95) limits = metric [Link] |
p95 latency for spans from the iOS service latest | filter kube_app == dataingest |
Quotes required for string values with spaces group_by [pod_name, container_name], sum;
spans latency | delta | filter service == iOS |
join usage / limits * 100
Time shift group_by [], sum | point percentile(value, 95)

Moves each point in the time series forwards by the amount of

Rate of spans per second on the crouton service, grouped by p99 latency for spans from the database-update operation on
time specified in the duration. A time_shift doesn't have to
customer the warehouse service
immediately follow a fetch operation, but it must come before a
window operation or group_by.
spans count | rate | filter service == crouton spans latency | delta | filter service =
Difference in request rate from one week prior | group_by [customer], sum warehouse && operation = database-update |
group_by [], sum | point percentile(value, 99)
with current = metric requests | rate | Point
group_by [customer, method], sum; last_week = Multiple latencies of the database-update operation on the
Takes the input value of each data point (called value) and warehouse service
metric requests | time_shift 7d | rate | applies an expression to it.
group_by [customer, method], sum;
spans latency | delta | filter service =
join current - last_week p95 value of the distribution metric [Link]
warehouse && operation = database-update |
group_by [], sum | point percentile(value,
metric [Link] | delta | group_by [], sum |
99.9), percentile(value, 99),
Point filter point percentile(value, 95)
percentile(value, 90)
Keeps all points that match the predicate and removes all other
Allowed values: * / + - pow(a,b) percentile(a,b)
points. This will produce gaps in the time series or remove time Boolean-valued time series where a point is true if the value of
series altogether that don’t match the predicate. [Link] is greater than 5 dist_sum(a) dist_count(a)
Cloud Observability Unified Query A UQL query is piped, made up of two or more stages that operate on the metric or span data returned by the
Fetch stage. Subsequent stages operate on the data returned from the previous stage.
Language (UQL) Cheat Sheet
Stage 1: Fetch Stage 2: Align Stage 3: Filter Stage 4: Group & Aggregate
Visit Cloud Observability Learning Portal for an intro to UQL
and reference documentation metric request | rate | filter service == checkout | group_by [operation], sum

Alerts Cookbook

Seasonality alerts Direct comparison alerts

Use this to get alerted on the historical comparison for <metric> between now and <lookback> Use this to get alerted on the comparison between <metricA> and <metricB>.
. For example,
duration ago. For example, to compare now and a week ago, use time_shift 7d . SQS messages in vs. out.

Allowed values for time_shift: s m h d w

Gauge metrics
<smoothingDuration> defines how much smoothing to apply to the historical data. The more
smoothing, the less likely spikes in historical data will cause your alert to trigger. Too large a value with
here may cause your seasonality to be lost. a = metric <metricA> | latest | group_by [], mean;
b = metric <metricB> | latest | group_by [], mean;
Gauge metrics join a - b

with
Delta/Cumulative scalar metrics
a = metric <metric> | reduce <smoothingDuration>, mean | group_by [],
mean; with
b = metric <metric> | time_shift <lookback> | reduce <smoothingDuration>, a = metric <metricA> | delta | group_by [], sum;
mean | group_by [], mean; b = metric <metricB> | delta | group_by [], sum;
join ((a-b)/b) * 100 join a - b

Delta/Cumulative scalar metrics

with
a = metric <metric> | rate <smoothingDuration> | group_by [], mean;
b = metric <metric> | time_shift <lookback> | rate <smoothingDuration> |
group_by [], mean;
join ((a-b)/b) * 100

Delta/Cumulative distribution metrics

with
a = metric <metric> | delta <smoothingDuration> | group_by [], sum |
point percentile(value, 95);
b = metric <metric> | time_shift <lookback> | delta <smoothingDuration> |
group_by [], sum | point percentile(value, 95);
join ((a-b)/b) * 100

© 2023 ServiceNow, Inc. All rights reserved. ServiceNow, the ServiceNow logo, Now, Now Platform, and other ServiceNow marks are trademarks and/or registered trademarks of ServiceNow, Inc. in the United States and/or other
countries. Other company names, product names, and logos may be trademarks of the respective companies with which they are associated.