opampsupervisor

OpAMP Supervisor for the OpenTelemetry Collector

Status
Stability	alpha: metrics
Issues
Code Owners	@evan-bradley, @atoulme, @tigrannajaryan, @douglascamata, @dpaasman00

This is an implementation of an OpAMP Supervisor that runs a Collector instance using configuration provided from an OpAMP server. This implementation is following a design specified here. The design is still undergoing changes, and as such this implementation may change as well.

Binary and container images for the Supervisor are available under tags starting with cmd/opampsupervisor here.

More information.

If you'd like to learn more about OpAMP, see the OpAMP specification. For production-ready client and server implementations in Go, see the OpAMP Go repository.

If you have questions or would like to participate, we'd love to hear from you. There is a regular SIG meeting, see the meeting notes for more information. You can also get in touch with us in the #otel-opamp channel on the CNCF Slack workspace.

Installing the Supervisor

Binary

Binaries are available as downloadable assets from OpenTelemetry Collector releases with tags using the prefix cmd/opampsupervisor. You can download the file containing the binary for your OS and architecture, and install it on your machine manually.

Linux/macOS

Replace <VERSION> with your desired Supervisor version, and <OS> and <ARCH> with your platform-specific information:

curl --proto '=https' --tlsv1.2 -fL -o opampsupervisor \
"https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/cmd%2Fopampsupervisor%2Fv<VERSION>/opampsupervisor_<VERSION>_<OS>_<ARCH>"
chmod +x opampsupervisor

Windows

Replace <VERSION> with your desired Supervisor version:

Invoke-WebRequest -Uri "https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/cmd%2Fopampsupervisor%2Fv<VERSION>/opampsupervisor_<VERSION>_windows_amd64.exe" -OutFile "opampsupervisor.exe"
Unblock-File -Path "opampsupervisor.exe"

Container image

Container images are available for download from GitHub Container registry:

docker pull ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-opampsupervisor

Using the Supervisor

To use the Supervisor, you will need four things:

1. A Supervisor binary, which can be obtained through the installation step above.
2. A Collector binary that you would like to control through the Supervisor.
3. A Supervisor config file. See examples here.
4. A running OpAMP server.

To test the Supervisor with an example server, download the opamp-go repository, and run the OpAMP example server in the internal/examples/server directory.

git clone [email protected]:open-telemetry/opamp-go.git
cd opamp-go/internal/examples/server
go run .

Visit localhost:4321 to verify that the server is running.

In a separate directory, create a Supervisor configuration file named supervisor.yaml. Copy one of the existing examples or create a new configuration. The agent::executable should be set to the path of your Collector binary:

agent:
  executable: <COLLECTOR_BINARY>

Start the Supervisor:

./opampsupervisor --config=supervisor.yaml

The supervisor's own log encoding is configured under telemetry.logs.encoding. Use console for plain-text logs with ISO8601 timestamps. Leave the field unset (or set it to json) to keep the default JSON output.

telemetry:
  logs:
    encoding: console

Visit localhost:4321 again to verify that your Collector appears in the Agents list.

Persistent data storage

The supervisor persists some data to disk in order to mantain state between restarts. The directory where this data is stored may be specified via the supervisor configuration:

storage:
  directory: "/path/to/storage/dir"

By default, the supervisor will use /var/lib/otelcol/supervisor on posix systems, and %ProgramData%/Otelcol/Supervisor on Windows.

This directory will be created on supervisor startup if it does not exist.

Healthcheck

The Supervisor can be configured to expose a healthcheck endpoint that can be used to determine whether the Supervisor is running and healthy. This can be configured in the Supervisor configuration file:

healthcheck:
  endpoint: localhost:13133

At the moment this healtcheck checks that the following conditions are met:

• The Supervisor is able to persist state to disk.
• The Supervisor is able to generate the agent's configuration.

For more details on the healthcheck configuration, see the see the full list of ServerConfig options.

Note that the healthceck endpoint is not enabled by default. To enable it, you must explicitly set at least the endpoint field in the configuration.

Startup Fallback Configuration

The Supervisor supports a startup fallback configuration mechanism that provides resilience when the OpAMP server is unreachable at startup and there's no previous configuration state persisted in disk. This is useful for ensuring the Collector can start with a known-good configuration during network outages or server maintenance. When the Supervisor successfully connects to the OpAMP server, the regular configuration (indicated by agent::config_files) is restored and any potential remote configuration received from the OpAMP server is applied.

If, when the Supervisor starts, there's previous configuration state persisted in disk, the Supervisor will use it to start the Collector.

Configuration Options

The following options can be configured under the agent section:

agent:
  startup_fallback_configs:
  - /path/to/fallback_config.yaml

Important Notes

• The startup fallback configuration is a set of standalone configuration files. It is intentionally not merged with the agent::config_files setting. This ensures predictable behavior without dependencies on other configuration files.

• The OpAMP extension and "own telemetry" configuration are automatically added to the fallback configuration. This maintains the Collector's own telemetry flowing and the communication between Supervisor and Collector.

• The Supervisor will continue attempting to reconnect to the OpAMP server even while using the fallback configuration.

Example

See examples/supervisor_fallback.yaml for a complete example configuration with fallback enabled.

Status

The OpenTelemetry OpAMP Supervisor is intended to be the reference implementation of an OpAMP Supervisor, and as such will support all OpAMP capabilities. Additionally, it follows a design document for the features it intends to support.

For a list of open issues related to the Supervisor, see these issues.

Key:

✅: Fully implemented

⚠️: Implemented with caveats

OpAMP capabilities

Supervisor specification features