It looks like our logging requires a pseudo tty setup and stdin open in order to be seen, that is not the case (by default) in k8s.
With an interactive session (-i) and an allocated pseudo-tty (-t) we get the debug messages:
$ docker run -i -t --rm -e OTEL_LOG_LEVEL=trace -e ELASTIC_OTEL_OPAMP_ENDPOINT=http://127.0.0.1:4320 dice-app:latest
HTTPConnectionPool(host='127.0.0.1', port=4320): Max retries exceeded with url: /v1/opamp (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7b39e5e531d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Job b'\n\x10\x01\x9a\xe3\xf0\x98vs\xb2\xb0\xb1\xb4e\xdbzH\xe4\x1aY\n)\n\x0cservice.name\x12\x19\n\x17otel-example-python-app\n,\n\x1bdeployment.environment.name\x12\r\n\x0bdevelopment \x87`' failed attempt 1/10:
[2025-12-03 11:19:47 +0000] [7] [INFO] Starting gunicorn 23.0.0
[2025-12-03 11:19:47 +0000] [7] [INFO] Listening at: http://0.0.0.0:8050 (7)
[2025-12-03 11:19:47 +0000] [7] [INFO] Using worker: sync
[2025-12-03 11:19:47 +0000] [35] [INFO] Booting worker with pid: 35
^C[2025-12-03 11:19:47 +0000] [7] [INFO] Handling signal: int
[2025-12-03 11:19:47 +0000] [35] [INFO] Worker exiting (pid: 35)
DEBUG:opentelemetry._opamp.agent:Stopping OpAMPClient: sending AgentDisconnect
ERROR:opentelemetry._opamp.transport.requests:HTTPConnectionPool(host='127.0.0.1', port=4320): Max retries exceeded with url: /v1/opamp (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7b39d4644850>: Failed to establish a new connection: [Errno 111] Connection refused'))
DEBUG:opentelemetry._opamp.agent:Stopping OpAMPClient: failed to send AgentDisconnect message
DEBUG:opentelemetry._opamp.agent:Stopping OpAMPClient: cancelling jobs
DEBUG:opentelemetry._opamp.agent:OpAMPClient stopped
WARNING:opentelemetry.exporter.otlp.proto.grpc.exporter:Transient error StatusCode.UNAVAILABLE encountered while exporting metrics to localhost:4318, retrying in 0.94s.
HTTPConnectionPool(host='127.0.0.1', port=4320): Max retries exceeded with url: /v1/opamp (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7b39e40a3b90>: Failed to establish a new connection: [Errno 111] Connection refused'))
Without the options we don't see the debug messages from the opamp agent:
$ docker run --rm -e OTEL_LOG_LEVEL=trace -e ELASTIC_OTEL_OPAMP_ENDPOINT=http://127.0.0.1:4320 dice-app:latest
HTTPConnectionPool(host='127.0.0.1', port=4320): Max retries exceeded with url: /v1/opamp (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x70dcb4e1b890>: Failed to establish a new connection: [Errno 111] Connection refused'))
Job b'\n\x10\x01\x9a\xe3\xf0\xfe\xdbp\xb0\x9b\xc6\x8an\xcaPB\x9b\x1aY\n)\n\x0cservice.name\x12\x19\n\x17otel-example-python-app\n,\n\x1bdeployment.environment.name\x12\r\n\x0bdevelopment \x87`' failed attempt 1/10:
[2025-12-03 11:20:13 +0000] [7] [INFO] Starting gunicorn 23.0.0
[2025-12-03 11:20:13 +0000] [7] [INFO] Listening at: http://0.0.0.0:8050 (7)
[2025-12-03 11:20:13 +0000] [7] [INFO] Using worker: sync
[2025-12-03 11:20:13 +0000] [35] [INFO] Booting worker with pid: 35
HTTPConnectionPool(host='127.0.0.1', port=4320): Max retries exceeded with url: /v1/opamp (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x70dcb4e1b150>: Failed to establish a new connection: [Errno 111] Connection refused'))
Job b'\n\x10\x01\x9a\xe3\xf0\xfe\xdbp\xb0\x9b\xc6\x8an\xcaPB\x9b\x1aY\n)\n\x0cservice.name\x12\x19\n\x17otel-example-python-app\n,\n\x1bdeployment.environment.name\x12\r\n\x0bdevelopment \x87`' failed attempt 2/10:
^C
It looks like our logging requires a pseudo tty setup and stdin open in order to be seen, that is not the case (by default) in k8s.
With an interactive session (
-i) and an allocated pseudo-tty (-t) we get the debug messages:Without the options we don't see the debug messages from the opamp agent: