Cet endpoint est actuellement expérimental et réservé à un usage interne Datadog uniquement. Récupérer les recommandations de ressources pour un job Spark. L’appelant (Spark Gateway ou DJM UI) fournit un nom de service et SPA renvoie des recommandations structurées pour les ressources du driver et des executors. La version avec un shard doit être préférée dans la mesure du possible, car elle fournit des résultats plus précis.
Arguments
Paramètres du chemin
Nom
Type
Description
service [required]
string
The service name for a spark job.
Chaînes de requête
Nom
Type
Description
bypass_cache
string
The recommendation service should not use its metrics cache.
JSON:API resource object for SPA Recommendation. Includes type, optional ID, and resource attributes with structured recommendations.
attributes [required]
object
Attributes of the SPA Recommendation resource. Contains recommendations for both driver and executor components.
confidence_level
double
The confidence level of the recommendation, expressed as a value between 0.0 (low confidence) and 1.0 (high confidence).
driver [required]
object
Resource recommendation for a single Spark component (driver or executor). Contains estimation data used to patch Spark job specs.
estimation [required]
object
Recommended resource values for a Spark driver or executor, derived from recent real usage metrics. Used by SPA to propose more efficient pod sizing.
cpu
object
CPU usage statistics derived from historical Spark job metrics. Provides multiple estimates so users can choose between conservative and cost-saving risk profiles.
max
int64
Maximum CPU usage observed for the job, expressed in millicores. This represents the upper bound of usage.
p75
int64
75th percentile of CPU usage (millicores). Represents a cost-saving configuration while covering most workloads.
p95
int64
95th percentile of CPU usage (millicores). Balances performance and cost, providing a safer margin than p75.
ephemeral_storage
int64
Recommended ephemeral storage allocation (in MiB). Derived from job temporary storage patterns.
heap
int64
Recommended JVM heap size (in MiB).
memory
int64
Recommended total memory allocation (in MiB). Includes both heap and overhead.
overhead
int64
Recommended JVM overhead (in MiB). Computed as total memory - heap.
executor [required]
object
Resource recommendation for a single Spark component (driver or executor). Contains estimation data used to patch Spark job specs.
estimation [required]
object
Recommended resource values for a Spark driver or executor, derived from recent real usage metrics. Used by SPA to propose more efficient pod sizing.
cpu
object
CPU usage statistics derived from historical Spark job metrics. Provides multiple estimates so users can choose between conservative and cost-saving risk profiles.
max
int64
Maximum CPU usage observed for the job, expressed in millicores. This represents the upper bound of usage.
p75
int64
75th percentile of CPU usage (millicores). Represents a cost-saving configuration while covering most workloads.
p95
int64
95th percentile of CPU usage (millicores). Balances performance and cost, providing a safer margin than p75.
ephemeral_storage
int64
Recommended ephemeral storage allocation (in MiB). Derived from job temporary storage patterns.
heap
int64
Recommended JVM heap size (in MiB).
memory
int64
Recommended total memory allocation (in MiB). Includes both heap and overhead.
overhead
int64
Recommended JVM overhead (in MiB). Computed as total memory - heap.
id
string
Resource identifier for the recommendation. Optional in responses.
type [required]
enum
JSON:API resource type for Spark Pod Autosizing recommendations. Identifies the Recommendation resource returned by SPA.
Allowed enum values: recommendation
JSON:API document containing a single Recommendation resource. Returned by SPA when the Spark Gateway requests recommendations.
Expand All
Champ
Type
Description
data [required]
object
JSON:API resource object for SPA Recommendation. Includes type, optional ID, and resource attributes with structured recommendations.
attributes [required]
object
Attributes of the SPA Recommendation resource. Contains recommendations for both driver and executor components.
confidence_level
double
The confidence level of the recommendation, expressed as a value between 0.0 (low confidence) and 1.0 (high confidence).
driver [required]
object
Resource recommendation for a single Spark component (driver or executor). Contains estimation data used to patch Spark job specs.
estimation [required]
object
Recommended resource values for a Spark driver or executor, derived from recent real usage metrics. Used by SPA to propose more efficient pod sizing.
cpu
object
CPU usage statistics derived from historical Spark job metrics. Provides multiple estimates so users can choose between conservative and cost-saving risk profiles.
max
int64
Maximum CPU usage observed for the job, expressed in millicores. This represents the upper bound of usage.
p75
int64
75th percentile of CPU usage (millicores). Represents a cost-saving configuration while covering most workloads.
p95
int64
95th percentile of CPU usage (millicores). Balances performance and cost, providing a safer margin than p75.
ephemeral_storage
int64
Recommended ephemeral storage allocation (in MiB). Derived from job temporary storage patterns.
heap
int64
Recommended JVM heap size (in MiB).
memory
int64
Recommended total memory allocation (in MiB). Includes both heap and overhead.
overhead
int64
Recommended JVM overhead (in MiB). Computed as total memory - heap.
executor [required]
object
Resource recommendation for a single Spark component (driver or executor). Contains estimation data used to patch Spark job specs.
estimation [required]
object
Recommended resource values for a Spark driver or executor, derived from recent real usage metrics. Used by SPA to propose more efficient pod sizing.
cpu
object
CPU usage statistics derived from historical Spark job metrics. Provides multiple estimates so users can choose between conservative and cost-saving risk profiles.
max
int64
Maximum CPU usage observed for the job, expressed in millicores. This represents the upper bound of usage.
p75
int64
75th percentile of CPU usage (millicores). Represents a cost-saving configuration while covering most workloads.
p95
int64
95th percentile of CPU usage (millicores). Balances performance and cost, providing a safer margin than p75.
ephemeral_storage
int64
Recommended ephemeral storage allocation (in MiB). Derived from job temporary storage patterns.
heap
int64
Recommended JVM heap size (in MiB).
memory
int64
Recommended total memory allocation (in MiB). Includes both heap and overhead.
overhead
int64
Recommended JVM overhead (in MiB). Computed as total memory - heap.
id
string
Resource identifier for the recommendation. Optional in responses.
type [required]
enum
JSON:API resource type for Spark Pod Autosizing recommendations. Identifies the Recommendation resource returned by SPA.
Allowed enum values: recommendation
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com"cargo run
/**
* Get SPA Recommendations returns "OK" response
*/import{client,v2}from"@datadog/datadog-api-client";constconfiguration=client.createConfiguration();configuration.unstableOperations["v2.getSPARecommendations"]=true;constapiInstance=newv2.SpaApi(configuration);constparams: v2.SpaApiGetSPARecommendationsRequest={service:"service",};apiInstance.getSPARecommendations(params).then((data: v2.RecommendationDocument)=>{console.log("API called successfully. Returned data: "+JSON.stringify(data));}).catch((error: any)=>console.error(error));