-
Notifications
You must be signed in to change notification settings - Fork 782
Description
Env:
opentelemetry-exporter-otel-proto-grpc 1.12.0rc1
python 3.10
Steps to reproduce
- Follow the example code: https://github.com/open-telemetry/opentelemetry-python/blob/main/docs/examples/logs/example.py
log_emitter_provider = LogEmitterProvider(
resource=Resource.create(
{
"service.name": "shoppingcart"
}
),
)
set_log_emitter_provider(log_emitter_provider)
exporter = OTLPLogExporter(insecure=True)
log_emitter_provider.add_log_processor(BatchLogProcessor(exporter))
log_emitter = log_emitter_provider.get_log_emitter(__name__, "0.1")
handler = LoggingHandler(level=logging.NOTSET, log_emitter=log_emitter)
root_logger = logging.getLogger()
root_logger.addHandler(handler)
root_logger.warning("test it out")
- Shutdown the gRPC destination service (local open telemetry collector, in this case)
- run the example code.
What is the expected behavior?
Given the fact the open telemetry collector is down. the gRPC should return StatusCode.UNAVAILABLE. It should keep retrying sending the log every a few sec.
What is the actual behavior?
The code went into an infinite recursive loop
Additional context
This is because the otel export is attached to the root logger.
In the opentleemetry/exporter/otlp/proto/grpc/exporter.py when gRPC error happens, it use the logger.warning() to send another log indicating the error, which triggered another gRPC call to send the error log, which will encounter another error, which will also be retried infinitely, this will go into infinite recursive loop.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status