| Status | |
|---|---|
| Stability | development: profiles |
| beta: traces, metrics, logs | |
| Issues |   |
| Code Owners | @TylerHelmuth, @evan-bradley, @edmocosta, @bogdandrutu | Seeking more code owners! |
| Emeritus | @anuraaga, @kentquirk |
The OpenTelemetry Transformation Language (OTTL) is a small, domain-specific programming language intended to process data with OpenTelemetry-native concepts and constructs.
This package implements everything necessary to use OTTL in a Collector component or in another user-facing system.
An OTTL statement is made up of 2 parts:
- A function that transforms telemetry
- Optionally, a condition that determines whether the function is executed.
Here is an example OTTL statement:
set(span.attributes["test"], "pass") where span.attributes["test"] == nil
This statement sets a new span attribute named "test" with a value of "pass" whenever the span does not already
have an attribute named "test". In this example, the function is set, which uses the second parameter to set the value of the first parameter, and the condition is span.attributes["test"] == nil.
Within a statement you utilize OTTL Paths to access telemetry. The example uses the Path span.attributes to access
the span's attributes. For each Open Telemetry Signal, OTTL has a Path to every field (plus some extras to help make
interacting with the data easier).
To see a list of available Paths for each Open Telemetry Signal, checkout the links below.
| Telemetry | OTTL Context |
|---|---|
Resource |
Resource |
Instrumentation Scope |
Instrumentation Scope |
Span |
Span |
Span Event |
SpanEvent |
Metric |
Metric |
Datapoint |
DataPoint |
Log |
Log |
Profile |
Profile |
OTTL does not support cross-signal interactions at this time. That means you cannot write a statement like
set(span.attributes["log body"], log.body)
See OTTL Functions for a list of functions available for use in OTTL statements of most components.
To see more examples of OTTL statements, checkout the Transform Processor
There is a lot more OTTL can do, like nested functions, arithmetic, indexing, and enums. To explore it further check out OTTL's grammar doc.
- To modify your data as it passes through a pipeline, use the transform processor.
- To remove data from your pipeline, use the filter processor.
- To select spans to be sampled, use the tail sampling processor.
- To route data between pipelines, use the routing connector.
When using OTTL you can enable debug logging in the collector to print out useful information, such as the current Statement/Condition and the current TransformContext, to help you troubleshoot why a statement is not behaving as you expect. This feature is very verbose, but provides you an accurate view into how OTTL views the underlying data.
service:
telemetry:
logs:
level: debug2024-05-29T16:38:09.600-0600 debug [email protected]/parser.go:265 initial TransformContext {"kind": "processor", "name": "transform", "pipeline": "logs", "TransformContext": {"resource": {"attributes": {}, "dropped_attribute_count": 0}, "scope": {"attributes": {}, "dropped_attribute_count": 0, "name": "", "version": ""}, "log_record": {"attributes": {"log.file.name": "test.log"}, "body": "test", "dropped_attribute_count": 0, "flags": 0, "observed_time_unix_nano": 1717022289500721000, "severity_number": 0, "severity_text": "", "span_id": "", "time_unix_nano": 0, "trace_id": ""}, "cache": {}}}
2024-05-29T16:38:09.600-0600 debug [email protected]/parser.go:268 TransformContext after statement execution {"kind": "processor", "name": "transform", "pipeline": "logs", "statement": "set(resource.attributes[\"test\"], \"pass\")", "condition matched": true, "TransformContext": {"resource": {"attributes": {"test": "pass"}, "dropped_attribute_count": 0}, "scope": {"attributes": {}, "dropped_attribute_count": 0, "name": "", "version": ""}, "log_record": {"attributes": {"log.file.name": "test.log"}, "body": "test", "dropped_attribute_count": 0, "flags": 0, "observed_time_unix_nano": 1717022289500721000, "severity_number": 0, "severity_text": "", "span_id": "", "time_unix_nano": 0, "trace_id": ""}, "cache": {}}}
2024-05-29T16:38:09.600-0600 debug [email protected]/parser.go:268 TransformContext after statement execution {"kind": "processor", "name": "transform", "pipeline": "logs", "statement": "set(instrumentation_scope.attributes[\"test\"], [\"pass\"])", "condition matched": true, "TransformContext": {"resource": {"attributes": {"test": "pass"}, "dropped_attribute_count": 0}, "scope": {"attributes": {"test": ["pass"]}, "dropped_attribute_count": 0, "name": "", "version": ""}, "log_record": {"attributes": {"log.file.name": "test.log"}, "body": "test", "dropped_attribute_count": 0, "flags": 0, "observed_time_unix_nano": 1717022289500721000, "severity_number": 0, "severity_text": "", "span_id": "", "time_unix_nano": 0, "trace_id": ""}, "cache": {}}}
2024-05-29T16:38:09.601-0600 debug [email protected]/parser.go:268 TransformContext after statement execution {"kind": "processor", "name": "transform", "pipeline": "logs", "statement": "set(attributes[\"test\"], true)", "condition matched": true, "TransformContext": {"resource": {"attributes": {"test": "pass"}, "dropped_attribute_count": 0}, "scope": {"attributes": {"test": ["pass"]}, "dropped_attribute_count": 0, "name": "", "version": ""}, "log_record": {"attributes": {"log.file.name": "test.log", "test": true}, "body": "test", "dropped_attribute_count": 0, "flags": 0, "observed_time_unix_nano": 1717022289500721000, "severity_number": 0, "severity_text": "", "span_id": "", "time_unix_nano": 0, "trace_id": ""}, "cache": {}}}
These are previous conference presentations given about OTTL: