Description
Every line in the scheduler log pertaining to a particular task instance should be stamped with that task instance's identifying information. In the Kubernetes executor, some lines are stamped only with the pod name instead.
Use case/motivation
When trying to trace the lifecycle of a task in the kubernetes executor, you currently must search first for the name of the pod created by the task, then search for the pod name in the logs. This means you need to be pretty familiar with the structure of the scheduler logs in order to search effectively for the lifecycle of a task that had a problem.
Some log statements like Attempting to finish pod do have the annotations for the pod, which include dag name, task name, and run_id, but others do not. For instance, Event: podname-a2f2c1ac706 had an event of type DELETED has no such annotations.
Related issues
No response
Are you willing to submit a PR?
Code of Conduct
Description
Every line in the scheduler log pertaining to a particular task instance should be stamped with that task instance's identifying information. In the Kubernetes executor, some lines are stamped only with the pod name instead.
Use case/motivation
When trying to trace the lifecycle of a task in the kubernetes executor, you currently must search first for the name of the pod created by the task, then search for the pod name in the logs. This means you need to be pretty familiar with the structure of the scheduler logs in order to search effectively for the lifecycle of a task that had a problem.
Some log statements like
Attempting to finish poddo have the annotations for the pod, which include dag name, task name, and run_id, but others do not. For instance,Event: podname-a2f2c1ac706 had an event of type DELETEDhas no such annotations.Related issues
No response
Are you willing to submit a PR?
Code of Conduct