For AI agents: A markdown version of this page is available at https://docs.datadoghq.com/observability_pipelines/sources/websocket.md. A documentation index is available at /llms.txt.

WebSocket Source

This product is not supported for your selected Datadog site. ().
Available for:

Logs

Join the Preview!

The WebSocket source is in Preview. Contact your account manager to request access.

Overview

Use Observability Pipelines’ WebSocket source to connect the Observability Pipelines Worker as a client to an upstream ws:// or wss:// endpoint. The Worker ingests the messages as logs.

Prerequisites

To use Observability Pipelines’ WebSocket source, you must have the following information:

  • The full WebSocket URI that the Observability Pipelines Worker connects to, such as wss://example.com/stream.
  • If the endpoint requires authentication, the credentials for your chosen authorization strategy (a username and password, a bearer token, or a custom Authorization header value).

Setup

Set up this source when you set up a pipeline. You can set up a pipeline in the UI, using the API, or with Terraform. The instructions in this section are for setting up the source in the UI.

For Secrets Management: Only enter the identifiers for the WebSocket URI and, if applicable, your authorization strategy secrets and TLS key pass. Do not enter the actual values.
If you enter secret identifiers and then choose to use environment variables, the environment variable is the identifier entered and prepended with DD_OP_. For example, if you entered PASSWORD_1 for a password identifier, the environment variable for that password is DD_OP_PASSWORD_1.

After you select the WebSocket source in the pipeline UI:

  1. Enter the identifier for your WebSocket URI, which references the ws:// or wss:// endpoint that the Worker connects to. If you leave it blank, the default is used.

  2. Select your authorization strategy. If you selected:

    • None: No authentication is sent with the connection request. Use this when the endpoint does not require credentials.
    • Basic: Enter the identifiers for your WebSocket username and password. If you leave it blank, the default is used.
    • Bearer: Enter the identifier for your bearer token. The token is sent in the Authorization header as Bearer <token>. If you leave it blank, the default is used.
    • Custom: Enter the identifier for your custom Authorization header value. The Worker sends this value as the Authorization header exactly as provided. Use this when the endpoint expects an authentication scheme other than basic or bearer. If you leave it blank, the default is used.

  3. In the Decoding dropdown menu, select the decoder to apply to incoming messages. Messages received from the endpoint must be in the selected format.

    DecodingDescription
    bytesSend each message as raw bytes. The raw message is stored in the message field.
    gelfDecode each message as a Graylog Extended Log Format (GELF) event.
    jsonDecode each message as a JSON object.
    syslogDecode each message as a Syslog-formatted event.

    If your messages do not match one of the formats above, select bytes to ingest the raw message in the message field, then transform it with a Custom Processor.

Optional TLS settings

For wss:// endpoints, configure TLS so that data is encrypted in transit. The WebSocket source supports two TLS modes:

Enable TLS

Toggle the switch to Enable TLS. Because the Worker connects to the endpoint as a client, no certificate fields are required. Enabling TLS encrypts the connection and the endpoint must present a certificate signed by a trusted Certificate Authority (CA).

Enable TLS with a client certificate

To use mutual TLS (mTLS), where the Worker presents a client certificate to the endpoint, provide a client certificate in addition to enabling TLS:

  • Client Certificate Path: The path to the client certificate file in DER, PEM, or CRT (X.509) format. This field is required for mTLS.
  • CA Certificate Path (optional): The path to the Certificate Authority (CA) root file used to verify the endpoint, in DER, PEM, or CRT (X.509) format.
  • Client Key Path (optional): The path to the .key private key file that belongs to your client certificate, in DER, PEM, or CRT (PKCS #8) format.
  • If your client key is encrypted, enter the identifier for the key passphrase. If you leave it blank, the default is used.

Notes:

  • The configuration data directory /var/lib/observability-pipelines-worker/config/ is automatically appended to the file paths. See Advanced Worker Configurations for more information.
  • The certificate and key files must be readable by the observability-pipelines-worker group and user.

Secret defaults

These are the defaults used for secret identifiers and environment variables.

    • WebSocket URI identifier:
      • References the ws:// or wss:// endpoint that the Observability Pipelines Worker connects to and ingests log events from.
      • The default identifier is SOURCE_WEBSOCKET_URI.
    • WebSocket TLS passphrase identifier (when a client certificate key is encrypted):
      • The default identifier is SOURCE_WEBSOCKET_KEY_PASS.
    • If you are using basic authentication:
      • WebSocket username identifier:
        • The default identifier is SOURCE_WEBSOCKET_USERNAME.
      • WebSocket password identifier:
        • The default identifier is SOURCE_WEBSOCKET_PASSWORD.
    • If you are using bearer authentication:
      • WebSocket bearer token identifier:
        • The default identifier is SOURCE_WEBSOCKET_BEARER_TOKEN.
    • If you are using custom authentication:
      • WebSocket custom Authorization header identifier:
        • The default identifier is SOURCE_WEBSOCKET_HEADERS_AUTHORIZATION.
    • WebSocket URI:
      • The Observability Pipelines Worker connects to this endpoint and ingests the messages it receives as logs. For example, wss://example.com/stream.
      • The default environment variable is DD_OP_SOURCE_WEBSOCKET_URI.
    • WebSocket TLS passphrase (when a client certificate key is encrypted):
      • The default environment variable is DD_OP_SOURCE_WEBSOCKET_KEY_PASS.
    • If you are using basic authentication:
      • The username and password for the WebSocket endpoint.
      • The default environment variables are DD_OP_SOURCE_WEBSOCKET_USERNAME and DD_OP_SOURCE_WEBSOCKET_PASSWORD.
    • If you are using bearer authentication:
      • The bearer token for the WebSocket endpoint.
      • The default environment variable is DD_OP_SOURCE_WEBSOCKET_BEARER_TOKEN.
    • If you are using custom authentication:
      • The value of the Authorization header to send with the connection request.
      • The default environment variable is DD_OP_SOURCE_WEBSOCKET_HEADERS_AUTHORIZATION.