Skip to content

Attaching OTLPLogExporter to the root logger will cause infinite recursive calls if the gRPC destination is down.  #2701

@bigqi0812

Description

@bigqi0812

Env:
opentelemetry-exporter-otel-proto-grpc 1.12.0rc1
python 3.10

Steps to reproduce

  1. Follow the example code: https://github.com/open-telemetry/opentelemetry-python/blob/main/docs/examples/logs/example.py
log_emitter_provider = LogEmitterProvider(
    resource=Resource.create(
        {
            "service.name": "shoppingcart"
        }
    ),
)
set_log_emitter_provider(log_emitter_provider)

exporter = OTLPLogExporter(insecure=True)
log_emitter_provider.add_log_processor(BatchLogProcessor(exporter))
log_emitter = log_emitter_provider.get_log_emitter(__name__, "0.1")
handler = LoggingHandler(level=logging.NOTSET, log_emitter=log_emitter)

root_logger = logging.getLogger()
root_logger.addHandler(handler)
root_logger.warning("test it out")
  1. Shutdown the gRPC destination service (local open telemetry collector, in this case)
  2. run the example code.

What is the expected behavior?
Given the fact the open telemetry collector is down. the gRPC should return StatusCode.UNAVAILABLE. It should keep retrying sending the log every a few sec.

What is the actual behavior?
The code went into an infinite recursive loop

Additional context
This is because the otel export is attached to the root logger.
In the opentleemetry/exporter/otlp/proto/grpc/exporter.py when gRPC error happens, it use the logger.warning() to send another log indicating the error, which triggered another gRPC call to send the error log, which will encounter another error, which will also be retried infinitely, this will go into infinite recursive loop.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingexportersloggingsdkAffects the SDK package.

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions