For AI agents: A markdown version of this page is available at https://docs.datadoghq.com/api/latest/llm-observability/list-patterns-topics-with-clustered-points.md. A documentation index is available at /llms.txt.

List patterns topics with clustered points

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

GET https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-pointshttps://api.ap2.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-pointshttps://api.datadoghq.eu/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-pointshttps://api.ddog-gov.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-pointshttps://api.us2.ddog-gov.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-pointshttps://api.uk1.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-pointshttps://api.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-pointshttps://api.us3.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-pointshttps://api.us5.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-points

Overview

List the topics discovered by a patterns run, with the clustered points attached inline to each leaf topic. When no run is specified, the most recent completed run is used.

Arguments

Query Strings

Name

Type

Description

config_id [required]

string

The ID of the patterns configuration.

run_id

string

The ID of a specific patterns run. Defaults to the most recent completed run.

include_metrics

boolean

When true, enrich each clustered point with span metrics such as status, duration, token counts, estimated cost, and evaluations.

Response

OK

Response containing the topics, and the clustered points of their leaf topics, discovered by an LLM Observability patterns run.

Expand All

Field

Type

Description

data [required]

object

Data object of an LLM Observability patterns topics-with-clustered-points response.

attributes [required]

object

Attributes of an LLM Observability patterns topics-with-clustered-points response.

completed_at

date-time

Timestamp when the run completed. Null if the run has not completed.

config_id [required]

string

Identifier of the configuration that produced the run.

config_snapshot

object

Snapshot of the configuration used for a patterns run.

account_id

string

Integration account ID used for a bring-your-own-model run.

evp_query

string

Query that selected the spans for the run.

hierarchy_depth

int32

Depth of the topic hierarchy generated.

integration_provider

string

Integration provider used for a bring-your-own-model run.

model_name

string

Model name used for a bring-your-own-model run.

num_records

int32

Maximum number of records processed for the run.

sampling_ratio

double

Fraction of matching spans sampled for the run.

created_at [required]

date-time

Timestamp when the run was created.

previous_run_id [required]

string

Identifier of the run that completed immediately before this one. Empty if none.

run_id [required]

string

Identifier of the run that produced the topics.

topics [required]

[object]

List of discovered topics with their clustered points.

cluster_points

[object]

List of clustered points attached to a topic.

duration

double

Duration of the source span in nanoseconds. Included only when metrics are requested.

estimated_total_cost

double

Estimated total cost of the source span. Included only when metrics are requested.

evaluation

object

Evaluation results for the source span keyed by evaluation name. Included only when metrics are requested.

input_tokens

double

Number of input tokens of the source span. Included only when metrics are requested.

output_tokens

double

Number of output tokens of the source span. Included only when metrics are requested.

span_id [required]

string

Identifier of the source span.

status

string

Status of the source span. Included only when metrics are requested.

total_tokens

double

Total number of tokens of the source span. Included only when metrics are requested.

created_at [required]

date-time

Timestamp when the topic was created.

description [required]

string

Description of the topic.

first_seen_at [required]

date-time

Timestamp when the topic was first seen.

hierarchy_level [required]

int64

Level of the topic in the hierarchy. Level 0 is a leaf topic.

id [required]

string

Unique identifier of the topic.

is_validated [required]

boolean

Whether the topic has been validated.

name [required]

string

Name of the topic.

parent_topic_id [required]

string

Identifier of the parent topic. Empty for top-level topics.

point_count [required]

int64

Number of data points assigned to the topic.

run_id [required]

string

Identifier of the run that produced the topic.

id [required]

string

Identifier of the run the topics belong to.

type [required]

enum

Resource type of an LLM Observability patterns topics-with-clustered-points response. Allowed enum values: get_topics_with_cluster_points_response

{
  "data": {
    "attributes": {
      "completed_at": "2024-01-15T10:45:00Z",
      "config_id": "a7c8d9e0-1234-5678-9abc-def012345678",
      "config_snapshot": {
        "account_id": "1000000001",
        "evp_query": "@ml_app:support-bot",
        "hierarchy_depth": 2,
        "integration_provider": "openai",
        "model_name": "gpt-4o",
        "num_records": 1000,
        "sampling_ratio": 0.1
      },
      "created_at": "2024-01-15T10:30:00Z",
      "previous_run_id": "",
      "run_id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
      "topics": [
        {
          "cluster_points": [
            {
              "duration": 1500000,
              "estimated_total_cost": 0.0021,
              "evaluation": {},
              "input_tokens": 128,
              "output_tokens": 64,
              "span_id": "1234567890123456789",
              "status": "ok",
              "total_tokens": 192
            }
          ],
          "created_at": "2024-01-15T10:30:00Z",
          "description": "Questions about invoices, charges, and refunds.",
          "first_seen_at": "2024-01-15T10:30:00Z",
          "hierarchy_level": 0,
          "id": "5c1fae90-2b6d-4e3a-9f12-7a0c4d8e6b21",
          "is_validated": true,
          "name": "Billing questions",
          "parent_topic_id": "",
          "point_count": 125,
          "run_id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012"
        }
      ]
    },
    "id": "3fd6b5e0-8910-4b1c-a7d0-5b84de329012",
    "type": "get_topics_with_cluster_points_response"
  }
}

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Internal Server Error

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Code Example

                  # Required query arguments
export config_id="a7c8d9e0-1234-5678-9abc-def012345678"
# Curl command
curl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.uk1.datadoghq.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v1/topic-discovery-topics/with-cluster-points?config_id=${config_id}" \ -H "Accept: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}"
"""
List patterns topics with clustered points returns "OK" response
"""

from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi

configuration = Configuration()
configuration.unstable_operations["list_llm_obs_patterns_topics_with_clustered_points"] = True
with ApiClient(configuration) as api_client:
    api_instance = LLMObservabilityApi(api_client)
    response = api_instance.list_llm_obs_patterns_topics_with_clustered_points(
        config_id="config_id",
    )

    print(response)

Instructions

First install the library and its dependencies and then save the example to example.py and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" python3 "example.py"
# List patterns topics with clustered points returns "OK" response

require "datadog_api_client"
DatadogAPIClient.configure do |config|
  config.unstable_operations["v2.list_llm_obs_patterns_topics_with_clustered_points".to_sym] = true
end
api_instance = DatadogAPIClient::V2::LLMObservabilityAPI.new
p api_instance.list_llm_obs_patterns_topics_with_clustered_points("config_id")

Instructions

First install the library and its dependencies and then save the example to example.rb and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" rb "example.rb"
// List patterns topics with clustered points returns "OK" response

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"os"

	"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
	"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)

func main() {
	ctx := datadog.NewDefaultContext(context.Background())
	configuration := datadog.NewConfiguration()
	configuration.SetUnstableOperationEnabled("v2.ListLLMObsPatternsTopicsWithClusteredPoints", true)
	apiClient := datadog.NewAPIClient(configuration)
	api := datadogV2.NewLLMObservabilityApi(apiClient)
	resp, r, err := api.ListLLMObsPatternsTopicsWithClusteredPoints(ctx, "config_id", *datadogV2.NewListLLMObsPatternsTopicsWithClusteredPointsOptionalParameters())

	if err != nil {
		fmt.Fprintf(os.Stderr, "Error when calling `LLMObservabilityApi.ListLLMObsPatternsTopicsWithClusteredPoints`: %v\n", err)
		fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
	}

	responseContent, _ := json.MarshalIndent(resp, "", "  ")
	fmt.Fprintf(os.Stdout, "Response from `LLMObservabilityApi.ListLLMObsPatternsTopicsWithClusteredPoints`:\n%s\n", responseContent)
}

Instructions

First install the library and its dependencies and then save the example to main.go and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" go run "main.go"
// List patterns topics with clustered points returns "OK" response

import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.LlmObservabilityApi;
import com.datadog.api.client.v2.model.LLMObsPatternsTopicsWithClusteredPointsResponse;

public class Example {
  public static void main(String[] args) {
    ApiClient defaultClient = ApiClient.getDefaultApiClient();
    defaultClient.setUnstableOperationEnabled(
        "v2.listLLMObsPatternsTopicsWithClusteredPoints", true);
    LlmObservabilityApi apiInstance = new LlmObservabilityApi(defaultClient);

    try {
      LLMObsPatternsTopicsWithClusteredPointsResponse result =
          apiInstance.listLLMObsPatternsTopicsWithClusteredPoints(
              "a7c8d9e0-1234-5678-9abc-def012345678");
      System.out.println(result);
    } catch (ApiException e) {
      System.err.println(
          "Exception when calling LlmObservabilityApi#listLLMObsPatternsTopicsWithClusteredPoints");
      System.err.println("Status code: " + e.getCode());
      System.err.println("Reason: " + e.getResponseBody());
      System.err.println("Response headers: " + e.getResponseHeaders());
      e.printStackTrace();
    }
  }
}

Instructions

First install the library and its dependencies and then save the example to Example.java and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" java "Example.java"
// List patterns topics with clustered points returns "OK" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_llm_observability::LLMObservabilityAPI;
use datadog_api_client::datadogV2::api_llm_observability::ListLLMObsPatternsTopicsWithClusteredPointsOptionalParams;

#[tokio::main]
async fn main() {
    let mut configuration = datadog::Configuration::new();
    configuration
        .set_unstable_operation_enabled("v2.ListLLMObsPatternsTopicsWithClusteredPoints", true);
    let api = LLMObservabilityAPI::with_config(configuration);
    let resp = api
        .list_llm_obs_patterns_topics_with_clustered_points(
            "config_id".to_string(),
            ListLLMObsPatternsTopicsWithClusteredPointsOptionalParams::default(),
        )
        .await;
    if let Ok(value) = resp {
        println!("{:#?}", value);
    } else {
        println!("{:#?}", resp.unwrap_err());
    }
}

Instructions

First install the library and its dependencies and then save the example to src/main.rs and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run
/**
 * List patterns topics with clustered points returns "OK" response
 */

import { client, v2 } from "@datadog/datadog-api-client";

const configuration = client.createConfiguration();
configuration.unstableOperations[
  "v2.listLLMObsPatternsTopicsWithClusteredPoints"
] = true;
const apiInstance = new v2.LLMObservabilityApi(configuration);

const params: v2.LLMObservabilityApiListLLMObsPatternsTopicsWithClusteredPointsRequest =
  {
    configId: "config_id",
  };

apiInstance
  .listLLMObsPatternsTopicsWithClusteredPoints(params)
  .then((data: v2.LLMObsPatternsTopicsWithClusteredPointsResponse) => {
    console.log(
      "API called successfully. Returned data: " + JSON.stringify(data)
    );
  })
  .catch((error: any) => console.error(error));

Instructions

First install the library and its dependencies and then save the example to example.ts and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" tsc "example.ts"