GET https://api.ap1.datadoghq.com/api/v2/obs-pipelines/pipelineshttps://api.ap2.datadoghq.com/api/v2/obs-pipelines/pipelineshttps://api.datadoghq.eu/api/v2/obs-pipelines/pipelineshttps://api.ddog-gov.com/api/v2/obs-pipelines/pipelineshttps://api.datadoghq.com/api/v2/obs-pipelines/pipelineshttps://api.us3.datadoghq.com/api/v2/obs-pipelines/pipelineshttps://api.us5.datadoghq.com/api/v2/obs-pipelines/pipelines
Overview
Retrieve a list of pipelines.
This endpoint requires the observability_pipelines_read permission.
Arguments
Query Strings
Size for a given page. The maximum allowed value is 100.
Specific page number to return.
Response
OK
Represents the response payload containing a list of pipelines and associated metadata.
Expand All
Defines the pipeline’s name and its components (sources, processors, and destinations).
Specifies the pipeline's configuration, including its sources, processors, and destinations.
A list of destination components where processed logs are sent.
The elasticsearch destination writes logs or metrics to an Elasticsearch cluster.
Supported pipeline types: logs, metrics
The Elasticsearch API version to use. Set to auto to auto-detect.
Allowed enum values: auto,v6,v7,v8
Authentication settings for the Elasticsearch destination.
When strategy is basic, use username_key and password_key to reference credentials stored in environment variables or secrets.
Name of the environment variable or secret that holds the Elasticsearch password (used when strategy is basic).
The authentication strategy to use.
Allowed enum values: basic,aws
Name of the environment variable or secret that holds the Elasticsearch username (used when strategy is basic).
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
The name of the index to write events to in Elasticsearch.
Compression configuration for the Elasticsearch destination.
The compression algorithm applied when sending data to Elasticsearch.
Allowed enum values: none,gzip,zlib,zstd,snappy
The compression level. Only applicable for gzip, zlib, and zstd algorithms.
Configuration options for writing to Elasticsearch Data Streams instead of a fixed index.
When true, automatically routes events to the appropriate data stream based on the event content.
The data stream dataset. This groups events by their source or application.
The data stream type. This determines how events are categorized within the data stream.
The data stream namespace. This separates events into different environments or domains.
When true, synchronizes data stream fields with the Elasticsearch index mapping.
Name of the environment variable or secret that holds the Elasticsearch endpoint URL.
The unique identifier for this component.
The name of the field used as the document ID in Elasticsearch.
A list of component IDs whose output is used as the input for this component.
The name of an Elasticsearch ingest pipeline to apply to events before indexing.
When true, retries failed partial bulk requests when some events in a batch fail while others succeed.
Configuration for enabling TLS encryption between the pipeline component and external services.
Path to the Certificate Authority (CA) file used to validate the server’s TLS certificate.
Path to the TLS client certificate file used to authenticate the pipeline component with upstream or downstream services.
Path to the private key file associated with the TLS client certificate. Used for mutual TLS authentication.
Name of the environment variable or secret that holds the passphrase for the private key file.
The destination type. The value should always be elasticsearch.
Allowed enum values: elasticsearch
default: elasticsearch
The http_client destination sends data to an HTTP endpoint.
Supported pipeline types: logs, metrics
HTTP authentication strategy.
Allowed enum values: none,basic,bearer
Compression configuration for HTTP requests.
Compression algorithm.
Allowed enum values: gzip
Name of the environment variable or secret that holds a custom header value (used with custom auth strategies).
Encoding format for log events.
Allowed enum values: json
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
Name of the environment variable or secret that holds the password (used when auth_strategy is basic).
Configuration for enabling TLS encryption between the pipeline component and external services.
Path to the Certificate Authority (CA) file used to validate the server’s TLS certificate.
Path to the TLS client certificate file used to authenticate the pipeline component with upstream or downstream services.
Path to the private key file associated with the TLS client certificate. Used for mutual TLS authentication.
Name of the environment variable or secret that holds the passphrase for the private key file.
Name of the environment variable or secret that holds the bearer token (used when auth_strategy is bearer).
The destination type. The value should always be http_client.
Allowed enum values: http_client
default: http_client
Name of the environment variable or secret that holds the HTTP endpoint URI.
Name of the environment variable or secret that holds the username (used when auth_strategy is basic).
The amazon_opensearch destination writes logs to Amazon OpenSearch.
Supported pipeline types: logs
Authentication settings for the Amazon OpenSearch destination.
The strategy field determines whether basic or AWS-based authentication is used.
The ARN of the role to assume (used with aws strategy).
External ID for the assumed role (used with aws strategy).
Session name for the assumed role (used with aws strategy).
The authentication strategy to use.
Allowed enum values: basic,aws
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
The index to write logs to.
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
The destination type. The value should always be amazon_opensearch.
Allowed enum values: amazon_opensearch
default: amazon_opensearch
The amazon_s3 destination sends your logs in Datadog-rehydratable format to an Amazon S3 bucket for archiving.
Supported pipeline types: logs
AWS authentication credentials used for accessing AWS services such as S3.
If omitted, the system’s default credentials are used (for example, the IAM role and environment variables).
The Amazon Resource Name (ARN) of the role to assume.
A unique identifier for cross-account role assumption.
A session identifier used for logging and tracing the assumed role session.
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Unique identifier for the destination component.
A list of component IDs whose output is used as the input for this component.
Optional prefix for object keys.
AWS region of the S3 bucket.
S3 storage class.
Allowed enum values: STANDARD,REDUCED_REDUNDANCY,INTELLIGENT_TIERING,STANDARD_IA,EXPRESS_ONEZONE,ONEZONE_IA,GLACIER,GLACIER_IR,DEEP_ARCHIVE
Configuration for enabling TLS encryption between the pipeline component and external services.
Path to the Certificate Authority (CA) file used to validate the server’s TLS certificate.
Path to the TLS client certificate file used to authenticate the pipeline component with upstream or downstream services.
Path to the private key file associated with the TLS client certificate. Used for mutual TLS authentication.
Name of the environment variable or secret that holds the passphrase for the private key file.
The destination type. Always amazon_s3.
Allowed enum values: amazon_s3
default: amazon_s3
The amazon_s3_generic destination sends your logs to an Amazon S3 bucket.
Supported pipeline types: logs
AWS authentication credentials used for accessing AWS services such as S3.
If omitted, the system’s default credentials are used (for example, the IAM role and environment variables).
The Amazon Resource Name (ARN) of the role to assume.
A unique identifier for cross-account role assumption.
A session identifier used for logging and tracing the assumed role session.
Maximum batch size in bytes.
Maximum number of seconds to wait before flushing the batch.
Compression algorithm applied to encoded logs.
The compression type. Always zstd.
Allowed enum values: zstd
default: zstd
The compression type. Always gzip.
Allowed enum values: gzip
default: gzip
The compression type. Always snappy.
Allowed enum values: snappy
default: snappy
Encoding format for the destination.
The encoding type. Always json.
Allowed enum values: json
default: json
The encoding type. Always parquet.
Allowed enum values: parquet
default: parquet
Unique identifier for the destination component.
A list of component IDs whose output is used as the input for this component.
Optional prefix for object keys.
AWS region of the S3 bucket.
S3 storage class.
Allowed enum values: STANDARD,REDUCED_REDUNDANCY,INTELLIGENT_TIERING,STANDARD_IA,EXPRESS_ONEZONE,ONEZONE_IA,GLACIER,GLACIER_IR,DEEP_ARCHIVE
The destination type. Always amazon_s3_generic.
Allowed enum values: amazon_s3_generic
default: amazon_s3_generic
The amazon_security_lake destination sends your logs to Amazon Security Lake.
Supported pipeline types: logs
AWS authentication credentials used for accessing AWS services such as S3.
If omitted, the system’s default credentials are used (for example, the IAM role and environment variables).
The Amazon Resource Name (ARN) of the role to assume.
A unique identifier for cross-account role assumption.
A session identifier used for logging and tracing the assumed role session.
Name of the Amazon S3 bucket in Security Lake (3-63 characters).
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
custom_source_name [required]
Custom source name for the logs in Security Lake.
Unique identifier for the destination component.
A list of component IDs whose output is used as the input for this component.
AWS region of the S3 bucket.
Configuration for enabling TLS encryption between the pipeline component and external services.
Path to the Certificate Authority (CA) file used to validate the server’s TLS certificate.
Path to the TLS client certificate file used to authenticate the pipeline component with upstream or downstream services.
Path to the private key file associated with the TLS client certificate. Used for mutual TLS authentication.
Name of the environment variable or secret that holds the passphrase for the private key file.
The destination type. Always amazon_security_lake.
Allowed enum values: amazon_security_lake
default: amazon_security_lake
The azure_storage destination forwards logs to an Azure Blob Storage container.
Supported pipeline types: logs
Optional prefix for blobs written to the container.
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Name of the environment variable or secret that holds the Azure Storage connection string.
container_name [required]
The name of the Azure Blob Storage container to store logs in.
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
The destination type. The value should always be azure_storage.
Allowed enum values: azure_storage
default: azure_storage
The cloud_prem destination sends logs to Datadog CloudPrem.
Supported pipeline types: logs
Name of the environment variable or secret that holds the CloudPrem endpoint URL.
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
The destination type. The value should always be cloud_prem.
Allowed enum values: cloud_prem
default: cloud_prem
The crowdstrike_next_gen_siem destination forwards logs to CrowdStrike Next Gen SIEM.
Supported pipeline types: logs
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Compression configuration for log events.
Compression algorithm for log events.
Allowed enum values: gzip,zlib
Encoding format for log events.
Allowed enum values: json,raw_message
Name of the environment variable or secret that holds the CrowdStrike endpoint URL.
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
Configuration for enabling TLS encryption between the pipeline component and external services.
Path to the Certificate Authority (CA) file used to validate the server’s TLS certificate.
Path to the TLS client certificate file used to authenticate the pipeline component with upstream or downstream services.
Path to the private key file associated with the TLS client certificate. Used for mutual TLS authentication.
Name of the environment variable or secret that holds the passphrase for the private key file.
Name of the environment variable or secret that holds the CrowdStrike API token.
The destination type. The value should always be crowdstrike_next_gen_siem.
Allowed enum values: crowdstrike_next_gen_siem
default: crowdstrike_next_gen_siem
The datadog_logs destination forwards logs to Datadog Log Management.
Supported pipeline types: logs
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
A list of routing rules that forward matching logs to Datadog using dedicated API keys.
Name of the environment variable or secret that stores the Datadog API key used by this route.
A Datadog search query that determines which logs are forwarded using this route.
Unique identifier for this route within the destination.
Datadog site where matching logs are sent (for example, us1).
The destination type. The value should always be datadog_logs.
Allowed enum values: datadog_logs
default: datadog_logs
The google_chronicle destination sends logs to Google Chronicle.
Supported pipeline types: logs
Google Cloud credentials used to authenticate with Google Cloud Storage.
credentials_file [required]
Path to the Google Cloud service account key file.
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
The Google Chronicle customer ID.
The encoding format for the logs sent to Chronicle.
Allowed enum values: json,raw_message
Name of the environment variable or secret that holds the Google Chronicle endpoint URL.
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
The log type metadata associated with the Chronicle destination.
The destination type. The value should always be google_chronicle.
Allowed enum values: google_chronicle
default: google_chronicle
The google_cloud_storage destination stores logs in a Google Cloud Storage (GCS) bucket.
It requires a bucket name, Google Cloud authentication, and metadata fields.
Supported pipeline types: logs
Access control list setting for objects written to the bucket.
Allowed enum values: private,project-private,public-read,authenticated-read,bucket-owner-read,bucket-owner-full-control
Google Cloud credentials used to authenticate with Google Cloud Storage.
credentials_file [required]
Path to the Google Cloud service account key file.
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Unique identifier for the destination component.
A list of component IDs whose output is used as the input for this component.
Optional prefix for object keys within the GCS bucket.
Custom metadata to attach to each object uploaded to the GCS bucket.
Storage class used for objects stored in GCS.
Allowed enum values: STANDARD,NEARLINE,COLDLINE,ARCHIVE
The destination type. Always google_cloud_storage.
Allowed enum values: google_cloud_storage
default: google_cloud_storage
The google_pubsub destination publishes logs to a Google Cloud Pub/Sub topic.
Supported pipeline types: logs
Google Cloud credentials used to authenticate with Google Cloud Storage.
credentials_file [required]
Path to the Google Cloud service account key file.
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Encoding format for log events.
Allowed enum values: json,raw_message
Name of the environment variable or secret that holds the Google Cloud Pub/Sub endpoint URL.
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
The Google Cloud project ID that owns the Pub/Sub topic.
Configuration for enabling TLS encryption between the pipeline component and external services.
Path to the Certificate Authority (CA) file used to validate the server’s TLS certificate.
Path to the TLS client certificate file used to authenticate the pipeline component with upstream or downstream services.
Path to the private key file associated with the TLS client certificate. Used for mutual TLS authentication.
Name of the environment variable or secret that holds the passphrase for the private key file.
The Pub/Sub topic name to publish logs to.
The destination type. The value should always be google_pubsub.
Allowed enum values: google_pubsub
default: google_pubsub
The kafka destination sends logs to Apache Kafka topics.
Supported pipeline types: logs
Name of the environment variable or secret that holds the Kafka bootstrap servers list.
Compression codec for Kafka messages.
Allowed enum values: none,gzip,snappy,lz4,zstd
Encoding format for log events.
Allowed enum values: json,raw_message
The field name to use for Kafka message headers.
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
The field name to use as the Kafka message key.
Optional list of advanced Kafka producer configuration options, defined as key-value pairs.
The name of the librdkafka configuration option to set.
The value assigned to the specified librdkafka configuration option.
Maximum time in milliseconds to wait for message delivery confirmation.
Duration in seconds for the rate limit window.
Maximum number of messages allowed per rate limit duration.
Specifies the SASL mechanism for authenticating with a Kafka cluster.
SASL mechanism used for Kafka authentication.
Allowed enum values: PLAIN,SCRAM-SHA-256,SCRAM-SHA-512
Name of the environment variable or secret that holds the SASL password.
Name of the environment variable or secret that holds the SASL username.
Socket timeout in milliseconds for network requests.
Configuration for enabling TLS encryption between the pipeline component and external services.
Path to the Certificate Authority (CA) file used to validate the server’s TLS certificate.
Path to the TLS client certificate file used to authenticate the pipeline component with upstream or downstream services.
Path to the private key file associated with the TLS client certificate. Used for mutual TLS authentication.
Name of the environment variable or secret that holds the passphrase for the private key file.
The Kafka topic name to publish logs to.
The destination type. The value should always be kafka.
Allowed enum values: kafka
default: kafka
The microsoft_sentinel destination forwards logs to Microsoft Sentinel.
Supported pipeline types: logs
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Azure AD client ID used for authentication.
Name of the environment variable or secret that holds the Azure AD client secret.
Name of the environment variable or secret that holds the Data Collection Endpoint (DCE) URI.
dcr_immutable_id [required]
The immutable ID of the Data Collection Rule (DCR).
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
The name of the Log Analytics table where logs are sent.
The destination type. The value should always be microsoft_sentinel.
Allowed enum values: microsoft_sentinel
default: microsoft_sentinel
The new_relic destination sends logs to the New Relic platform.
Supported pipeline types: logs
Name of the environment variable or secret that holds the New Relic account ID.
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
Name of the environment variable or secret that holds the New Relic license key.
The New Relic region.
Allowed enum values: us,eu
The destination type. The value should always be new_relic.
Allowed enum values: new_relic
default: new_relic
The opensearch destination writes logs to an OpenSearch cluster.
Supported pipeline types: logs
Authentication settings for the Elasticsearch destination.
When strategy is basic, use username_key and password_key to reference credentials stored in environment variables or secrets.
Name of the environment variable or secret that holds the Elasticsearch password (used when strategy is basic).
The authentication strategy to use.
Allowed enum values: basic,aws
Name of the environment variable or secret that holds the Elasticsearch username (used when strategy is basic).
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
The index to write logs to.
Configuration options for writing to OpenSearch Data Streams instead of a fixed index.
The data stream dataset for your logs. This groups logs by their source or application.
The data stream type for your logs. This determines how logs are categorized within the data stream.
The data stream namespace for your logs. This separates logs into different environments or domains.
Name of the environment variable or secret that holds the OpenSearch endpoint URL.
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
The destination type. The value should always be opensearch.
Allowed enum values: opensearch
default: opensearch
The rsyslog destination forwards logs to an external rsyslog server over TCP or UDP using the syslog protocol.
Supported pipeline types: logs
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Name of the environment variable or secret that holds the syslog server endpoint URL.
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
Optional socket keepalive duration in milliseconds.
Configuration for enabling TLS encryption between the pipeline component and external services.
Path to the Certificate Authority (CA) file used to validate the server’s TLS certificate.
Path to the TLS client certificate file used to authenticate the pipeline component with upstream or downstream services.
Path to the private key file associated with the TLS client certificate. Used for mutual TLS authentication.
Name of the environment variable or secret that holds the passphrase for the private key file.
The destination type. The value should always be rsyslog.
Allowed enum values: rsyslog
default: rsyslog
The sentinel_one destination sends logs to SentinelOne.
Supported pipeline types: logs
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
The SentinelOne region to send logs to.
Allowed enum values: us,eu,ca,data_set_us
Name of the environment variable or secret that holds the SentinelOne API token.
The destination type. The value should always be sentinel_one.
Allowed enum values: sentinel_one
default: sentinel_one
The socket destination sends logs over TCP or UDP to a remote server.
Supported pipeline types: logs
Name of the environment variable or secret that holds the socket address (host:port).
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Encoding format for log events.
Allowed enum values: json,raw_message
Framing method configuration.
Each log event is delimited by a newline character.
The definition of ObservabilityPipelineSocketDestinationFramingNewlineDelimitedMethod object.
Allowed enum values: newline_delimited
Event data is not delimited at all.
The definition of ObservabilityPipelineSocketDestinationFramingBytesMethod object.
Allowed enum values: bytes
Each log event is separated using the specified delimiter character.
A single ASCII character used as a delimiter.
The definition of ObservabilityPipelineSocketDestinationFramingCharacterDelimitedMethod object.
Allowed enum values: character_delimited
The unique identifier for this component.
A list of component IDs whose output is used as the input for this component.
Protocol used to send logs.
Allowed enum values: tcp,udp
TLS configuration. Relevant only when mode is tcp.
Path to the Certificate Authority (CA) file used to validate the server’s TLS certificate.
Path to the TLS client certificate file used to authenticate the pipeline component with upstream or downstream services.
Path to the private key file associated with the TLS client certificate. Used for mutual TLS authentication.
Name of the environment variable or secret that holds the passphrase for the private key file.
The destination type. The value should always be socket.
Allowed enum values: socket
default: socket
The splunk_hec destination forwards logs to Splunk using the HTTP Event Collector (HEC).
Supported pipeline types: logs
If true, Splunk tries to extract timestamps from incoming log events.
If false, Splunk assigns the time the event was received.
Configuration for buffer settings on destination components.
Options for configuring a disk buffer.
Maximum size of the disk buffer.
The type of the buffer that will be configured, a disk buffer.
Allowed enum values: disk
default: disk
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by byte size.
Maximum size of the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
Options for configuring a memory buffer by queue length.
Maximum events for the memory buffer.
The type of the buffer that will be configured, a memory buffer.
Allowed enum values: memory
default: memory
Behavior when the buffer is full (block and stop accepting new events, or drop new events)
Allowed enum values: block,drop_newest
default: block
