Get SPA Recommendations

Note: This endpoint is in preview and may change in the future. It is not yet recommended for production use.

GET https://api.ap1.datadoghq.com/api/v2/spa/recommendations/{service}https://api.ap2.datadoghq.com/api/v2/spa/recommendations/{service}https://api.datadoghq.eu/api/v2/spa/recommendations/{service}https://api.ddog-gov.com/api/v2/spa/recommendations/{service}https://api.us2.ddog-gov.com/api/v2/spa/recommendations/{service}https://api.uk1.datadoghq.com/api/v2/spa/recommendations/{service}https://api.datadoghq.com/api/v2/spa/recommendations/{service}https://api.us3.datadoghq.com/api/v2/spa/recommendations/{service}https://api.us5.datadoghq.com/api/v2/spa/recommendations/{service}

Présentation

Cet endpoint est actuellement expérimental et réservé à un usage interne Datadog uniquement. Récupérer les recommandations de ressources pour un job Spark. L’appelant (Spark Gateway ou DJM UI) fournit un nom de service et SPA renvoie des recommandations structurées pour les ressources du driver et des executors. La version avec un shard doit être préférée dans la mesure du possible, car elle fournit des résultats plus précis.

Arguments

Paramètres du chemin

Nom

Type

Description

service [required]

string

The service name for a spark job.

Chaînes de requête

Nom

Type

Description

bypass_cache

string

The recommendation service should not use its metrics cache.

Réponse

OK

Expand All

Champ

Type

Description

data [required]

object

JSON:API resource object for SPA Recommendation. Includes type, optional ID, and resource attributes with structured recommendations.

attributes [required]

object

Attributes of the SPA Recommendation resource. Contains recommendations for both driver and executor components.

confidence_level

double

The confidence level of the recommendation, expressed as a value between 0.0 (low confidence) and 1.0 (high confidence).

driver [required]

object

Resource recommendation for a single Spark component (driver or executor). Contains estimation data used to patch Spark job specs.

estimation [required]

object

Recommended resource values for a Spark driver or executor, derived from recent real usage metrics. Used by SPA to propose more efficient pod sizing.

cpu

object

CPU usage statistics derived from historical Spark job metrics. Provides multiple estimates so users can choose between conservative and cost-saving risk profiles.

max

int64

Maximum CPU usage observed for the job, expressed in millicores. This represents the upper bound of usage.

p75

int64

75th percentile of CPU usage (millicores). Represents a cost-saving configuration while covering most workloads.

p95

int64

95th percentile of CPU usage (millicores). Balances performance and cost, providing a safer margin than p75.

ephemeral_storage

int64

Recommended ephemeral storage allocation (in MiB). Derived from job temporary storage patterns.

heap

int64

Recommended JVM heap size (in MiB).

memory

int64

Recommended total memory allocation (in MiB). Includes both heap and overhead.

overhead

int64

Recommended JVM overhead (in MiB). Computed as total memory - heap.

executor [required]

object

Resource recommendation for a single Spark component (driver or executor). Contains estimation data used to patch Spark job specs.

estimation [required]

object

Recommended resource values for a Spark driver or executor, derived from recent real usage metrics. Used by SPA to propose more efficient pod sizing.

cpu

object

CPU usage statistics derived from historical Spark job metrics. Provides multiple estimates so users can choose between conservative and cost-saving risk profiles.

max

int64

Maximum CPU usage observed for the job, expressed in millicores. This represents the upper bound of usage.

p75

int64

75th percentile of CPU usage (millicores). Represents a cost-saving configuration while covering most workloads.

p95

int64

95th percentile of CPU usage (millicores). Balances performance and cost, providing a safer margin than p75.

ephemeral_storage

int64

Recommended ephemeral storage allocation (in MiB). Derived from job temporary storage patterns.

heap

int64

Recommended JVM heap size (in MiB).

memory

int64

Recommended total memory allocation (in MiB). Includes both heap and overhead.

overhead

int64

Recommended JVM overhead (in MiB). Computed as total memory - heap.

id

string

Resource identifier for the recommendation. Optional in responses.

type [required]

enum

JSON:API resource type for Spark Pod Autosizing recommendations. Identifies the Recommendation resource returned by SPA. Allowed enum values: recommendation

default: recommendation

{
  "data": {
    "attributes": {
      "confidence_level": "number",
      "driver": {
        "estimation": {
          "cpu": {
            "max": "integer",
            "p75": "integer",
            "p95": "integer"
          },
          "ephemeral_storage": "integer",
          "heap": "integer",
          "memory": "integer",
          "overhead": "integer"
        }
      },
      "executor": {
        "estimation": {
          "cpu": {
            "max": "integer",
            "p75": "integer",
            "p95": "integer"
          },
          "ephemeral_storage": "integer",
          "heap": "integer",
          "memory": "integer",
          "overhead": "integer"
        }
      }
    },
    "id": "string",
    "type": "recommendation"
  }
}

JSON:API document containing a single Recommendation resource. Returned by SPA when the Spark Gateway requests recommendations.

Expand All

Champ

Type

Description

data [required]

object

JSON:API resource object for SPA Recommendation. Includes type, optional ID, and resource attributes with structured recommendations.

attributes [required]

object

Attributes of the SPA Recommendation resource. Contains recommendations for both driver and executor components.

confidence_level

double

The confidence level of the recommendation, expressed as a value between 0.0 (low confidence) and 1.0 (high confidence).

driver [required]

object

Resource recommendation for a single Spark component (driver or executor). Contains estimation data used to patch Spark job specs.

estimation [required]

object

Recommended resource values for a Spark driver or executor, derived from recent real usage metrics. Used by SPA to propose more efficient pod sizing.

cpu

object

CPU usage statistics derived from historical Spark job metrics. Provides multiple estimates so users can choose between conservative and cost-saving risk profiles.

max

int64

Maximum CPU usage observed for the job, expressed in millicores. This represents the upper bound of usage.

p75

int64

75th percentile of CPU usage (millicores). Represents a cost-saving configuration while covering most workloads.

p95

int64

95th percentile of CPU usage (millicores). Balances performance and cost, providing a safer margin than p75.

ephemeral_storage

int64

Recommended ephemeral storage allocation (in MiB). Derived from job temporary storage patterns.

heap

int64

Recommended JVM heap size (in MiB).

memory

int64

Recommended total memory allocation (in MiB). Includes both heap and overhead.

overhead

int64

Recommended JVM overhead (in MiB). Computed as total memory - heap.

executor [required]

object

Resource recommendation for a single Spark component (driver or executor). Contains estimation data used to patch Spark job specs.

estimation [required]

object

Recommended resource values for a Spark driver or executor, derived from recent real usage metrics. Used by SPA to propose more efficient pod sizing.

cpu

object

CPU usage statistics derived from historical Spark job metrics. Provides multiple estimates so users can choose between conservative and cost-saving risk profiles.

max

int64

Maximum CPU usage observed for the job, expressed in millicores. This represents the upper bound of usage.

p75

int64

75th percentile of CPU usage (millicores). Represents a cost-saving configuration while covering most workloads.

p95

int64

95th percentile of CPU usage (millicores). Balances performance and cost, providing a safer margin than p75.

ephemeral_storage

int64

Recommended ephemeral storage allocation (in MiB). Derived from job temporary storage patterns.

heap

int64

Recommended JVM heap size (in MiB).

memory

int64

Recommended total memory allocation (in MiB). Includes both heap and overhead.

overhead

int64

Recommended JVM overhead (in MiB). Computed as total memory - heap.

id

string

Resource identifier for the recommendation. Optional in responses.

type [required]

enum

JSON:API resource type for Spark Pod Autosizing recommendations. Identifies the Recommendation resource returned by SPA. Allowed enum values: recommendation

default: recommendation

{
  "data": {
    "attributes": {
      "confidence_level": "number",
      "driver": {
        "estimation": {
          "cpu": {
            "max": "integer",
            "p75": "integer",
            "p95": "integer"
          },
          "ephemeral_storage": "integer",
          "heap": "integer",
          "memory": "integer",
          "overhead": "integer"
        }
      },
      "executor": {
        "estimation": {
          "cpu": {
            "max": "integer",
            "p75": "integer",
            "p95": "integer"
          },
          "ephemeral_storage": "integer",
          "heap": "integer",
          "memory": "integer",
          "overhead": "integer"
        }
      }
    },
    "id": "string",
    "type": "recommendation"
  }
}

Bad Request

API error response.

Expand All

Champ

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Not Authorized

API error response.

Expand All

Champ

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Too many requests

API error response.

Expand All

Champ

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Exemple de code

                  # Path parameters
export service="CHANGE_ME"
# Curl command
curl -X GET "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.uk1.datadoghq.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/spa/recommendations/${service}" \ -H "Accept: application/json"
"""
Get SPA Recommendations returns "OK" response
"""

from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.spa_api import SpaApi

configuration = Configuration()
configuration.unstable_operations["get_spa_recommendations"] = True
with ApiClient(configuration) as api_client:
    api_instance = SpaApi(api_client)
    response = api_instance.get_spa_recommendations(
        service="service",
    )

    print(response)

Instructions

First install the library and its dependencies and then save the example to example.py and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" python3 "example.py"
# Get SPA Recommendations returns "OK" response

require "datadog_api_client"
DatadogAPIClient.configure do |config|
  config.unstable_operations["v2.get_spa_recommendations".to_sym] = true
end
api_instance = DatadogAPIClient::V2::SpaAPI.new
p api_instance.get_spa_recommendations("service")

Instructions

First install the library and its dependencies and then save the example to example.rb and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" rb "example.rb"
// Get SPA Recommendations returns "OK" response

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"os"

	"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
	"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)

func main() {
	ctx := datadog.NewDefaultContext(context.Background())
	configuration := datadog.NewConfiguration()
	configuration.SetUnstableOperationEnabled("v2.GetSPARecommendations", true)
	apiClient := datadog.NewAPIClient(configuration)
	api := datadogV2.NewSpaApi(apiClient)
	resp, r, err := api.GetSPARecommendations(ctx, "service", *datadogV2.NewGetSPARecommendationsOptionalParameters())

	if err != nil {
		fmt.Fprintf(os.Stderr, "Error when calling `SpaApi.GetSPARecommendations`: %v\n", err)
		fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
	}

	responseContent, _ := json.MarshalIndent(resp, "", "  ")
	fmt.Fprintf(os.Stdout, "Response from `SpaApi.GetSPARecommendations`:\n%s\n", responseContent)
}

Instructions

First install the library and its dependencies and then save the example to main.go and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" go run "main.go"
// Get SPA Recommendations returns "OK" response

import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.SpaApi;
import com.datadog.api.client.v2.model.RecommendationDocument;

public class Example {
  public static void main(String[] args) {
    ApiClient defaultClient = ApiClient.getDefaultApiClient();
    defaultClient.setUnstableOperationEnabled("v2.getSPARecommendations", true);
    SpaApi apiInstance = new SpaApi(defaultClient);

    try {
      RecommendationDocument result = apiInstance.getSPARecommendations("service");
      System.out.println(result);
    } catch (ApiException e) {
      System.err.println("Exception when calling SpaApi#getSPARecommendations");
      System.err.println("Status code: " + e.getCode());
      System.err.println("Reason: " + e.getResponseBody());
      System.err.println("Response headers: " + e.getResponseHeaders());
      e.printStackTrace();
    }
  }
}

Instructions

First install the library and its dependencies and then save the example to Example.java and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" java "Example.java"
// Get SPA Recommendations returns "OK" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_spa::GetSPARecommendationsOptionalParams;
use datadog_api_client::datadogV2::api_spa::SpaAPI;

#[tokio::main]
async fn main() {
    let mut configuration = datadog::Configuration::new();
    configuration.set_unstable_operation_enabled("v2.GetSPARecommendations", true);
    let api = SpaAPI::with_config(configuration);
    let resp = api
        .get_spa_recommendations(
            "service".to_string(),
            GetSPARecommendationsOptionalParams::default(),
        )
        .await;
    if let Ok(value) = resp {
        println!("{:#?}", value);
    } else {
        println!("{:#?}", resp.unwrap_err());
    }
}

Instructions

First install the library and its dependencies and then save the example to src/main.rs and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" cargo run
/**
 * Get SPA Recommendations returns "OK" response
 */

import { client, v2 } from "@datadog/datadog-api-client";

const configuration = client.createConfiguration();
configuration.unstableOperations["v2.getSPARecommendations"] = true;
const apiInstance = new v2.SpaApi(configuration);

const params: v2.SpaApiGetSPARecommendationsRequest = {
  service: "service",
};

apiInstance
  .getSPARecommendations(params)
  .then((data: v2.RecommendationDocument) => {
    console.log(
      "API called successfully. Returned data: " + JSON.stringify(data)
    );
  })
  .catch((error: any) => console.error(error));

Instructions

First install the library and its dependencies and then save the example to example.ts and run following commands:

    
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" tsc "example.ts"