Running Dagu 2.3.7 on K8s (deployed using the Dagu helm chart)
I upgraded from Dagu 2.3.1 to 2.3.7 (using helm upgrade), also tried uninstall chart followed by install... same result.
Rolled back to 2.3.1 & everything works as per normal
The problem:
worker log:
time=2026-03-26T15:43:17.931+10:00 level=INFO msg="Task polled successfully" run-id=019d28ab-234b-7b89-9bd2-4f86ac6fd0c1 target=test worker-selector=map[] coordinator-id=dagu-coordinator-5f4485fc78-tt9gj@50055
time=2026-03-26T15:43:17.931+10:00 level=INFO msg="Task received" worker-id=dagu-worker-general-5c5f4db5b-8bxmk poller-id=3f7b261b-6179-42a6-b9eb-d472ffa1deee poller-index=93 root-dag-run-name=test root-dag-run-id=019d28ab-234b-7b89-9bd2-4f86ac6fd0c1 parent-dag-run-name="" parent-dag-run-id="" run-id=019d28ab-234b-7b89-9bd2-4f86ac6fd0c1
time=2026-03-26T15:43:17.931+10:00 level=INFO msg="Task received, starting execution" worker-id=dagu-worker-general-5c5f4db5b-8bxmk poller-index=93 run-id=019d28ab-234b-7b89-9bd2-4f86ac6fd0c1
time=2026-03-26T15:43:18.003+10:00 level=INFO msg="Executing task" operation=OPERATION_START target=test run-id=019d28ab-234b-7b89-9bd2-4f86ac6fd0c1 root-dag-run-id=019d28ab-234b-7b89-9bd2-4f86ac6fd0c1 parent-dag-run-id="" worker-id=dagu-worker-general-5c5f4db5b-8bxmk
time=2026-03-26T15:43:18.003+10:00 level=INFO msg="Creating temporary DAG file from definition" dag=test size=180
time=2026-03-26T15:43:18.004+10:00 level=INFO msg="Created temporary DAG file" file=/tmp/dagu/worker-dags/test-2886685920.yaml
Error: local execution already exists: dag-run ID 019d28ab-234b-7b89-9bd2-4f86ac6fd0c1 already exists for DAG test
time=2026-03-26T15:43:18.741+10:00 level=ERROR msg="Distributed task execution failed" operation=OPERATION_START target=/tmp/dagu/worker-dags/test-2886685920.yaml run-id=019d28ab-234b-7b89-9bd2-4f86ac6fd0c1 err="command failed: exit status 1\nstderr: Error: local execution already exists: dag-run ID 019d28ab-234b-7b89-9bd2-4f86ac6fd0c1 already exists for DAG test"
time=2026-03-26T15:43:18.742+10:00 level=ERROR msg="Task execution failed" worker-id=dagu-worker-general-5c5f4db5b-8bxmk poller-index=93 run-id=019d28ab-234b-7b89-9bd2-4f86ac6fd0c1 err="command failed: exit status 1\nstderr: Error: local execution already exists: dag-run ID 019d28ab-234b-7b89-9bd2-4f86ac6fd0c1 already exists for DAG test"
coordinator:
time=2026-03-26T15:41:36.989+10:00 level=INFO msg="Coordinator initialization" bind-address=0.0.0.0 advertise-address=dagu-coordinator.default.svc.k8s.cluster port=50055 instance-id=dagu-coordinator-5f4485fc78-tt9gj@50055
time=2026-03-26T15:41:36.990+10:00 level=INFO msg="Started zombie detector" interval=45s
time=2026-03-26T15:41:36.990+10:00 level=INFO msg="Starting service registry" service=coordinator service-id=dagu-coordinator-5f4485fc78-tt9gj@50055 host=dagu-coordinator.default.svc.k8s.cluster port=50055 status=active
time=2026-03-26T15:41:36.990+10:00 level=INFO msg="Starting health check server" service=coordinator port=8091
time=2026-03-26T15:41:37.077+10:00 level=INFO msg="Registered with service registry" service-id=dagu-coordinator-5f4485fc78-tt9gj@50055 configured-host=dagu-coordinator.default.svc.k8s.cluster port=50055 addr=[::]:50055
time=2026-03-26T15:41:37.077+10:00 level=INFO msg="Starting to serve on coordinator service" addr=[::]:50055
time=2026-03-26T15:42:54.373+10:00 level=INFO msg="Handler Dispatch called" run-id=019d28aa-c86f-78a9-881b-5f8fc90c53fd target=test operation=OPERATION_START
time=2026-03-26T15:42:54.771+10:00 level=INFO msg="Created DAGRun attempt for dispatched task" run-id=019d28aa-c86f-78a9-881b-5f8fc90c53fd target=test attempt-id=765a36 attempt-key=5344d16c94d6db0f:765a36
time=2026-03-26T15:43:17.579+10:00 level=INFO msg="Handler Dispatch called" run-id=019d28ab-234b-7b89-9bd2-4f86ac6fd0c1 target=test operation=OPERATION_START
time=2026-03-26T15:43:17.809+10:00 level=INFO msg="Created DAGRun attempt for dispatched task" run-id=019d28ab-234b-7b89-9bd2-4f86ac6fd0c1 target=test attempt-id=25a901 attempt-key=a136d67089d46b7f:25a901
scheduler:
time=2026-03-26T15:41:44.206+10:00 level=INFO msg="Scheduler initialization" dir=/data/dags log-format=text
time=2026-03-26T15:41:44.207+10:00 level=INFO msg="Starting service registry" service=scheduler service-id=dagu-scheduler-567b97c947-fzm9r-1-1774503704 host=dagu-scheduler-567b97c947-fzm9r port=8090 status=inactive
time=2026-03-26T15:41:44.288+10:00 level=INFO msg="Registered with service registry as inactive" service-id=dagu-scheduler-567b97c947-fzm9r-1-1774503704 host=dagu-scheduler-567b97c947-fzm9r port=8090
time=2026-03-26T15:41:44.288+10:00 level=INFO msg="Waiting to acquire scheduler lock"
time=2026-03-26T15:41:44.288+10:00 level=INFO msg="Starting health check server" service=scheduler port=8090
time=2026-03-26T15:41:44.315+10:00 level=INFO msg="Acquired scheduler lock"
time=2026-03-26T15:41:44.316+10:00 level=INFO msg="Updated scheduler status to active"
time=2026-03-26T15:41:44.317+10:00 level=INFO msg="Queue watcher setup complete" dir=/data/queue
time=2026-03-26T15:41:44.318+10:00 level=INFO msg="Loading DAGs" dir=/data/dags
time=2026-03-26T15:41:44.357+10:00 level=INFO msg="Loaded scheduler watermark" lastTick=2026-03-26T15:40:00.000+10:00 dagCount=3
time=2026-03-26T15:41:44.357+10:00 level=INFO msg="Scheduler started"
time=2026-03-26T15:41:44.357+10:00 level=INFO msg="Started zombie detector" interval=45s
time=2026-03-26T15:41:44.357+10:00 level=INFO msg="Started retry scanner" interval=30s retry_failure_window=24h0m0s
Running Dagu 2.3.7 on K8s (deployed using the Dagu helm chart)
I upgraded from Dagu 2.3.1 to 2.3.7 (using helm upgrade), also tried uninstall chart followed by install... same result.
Rolled back to 2.3.1 & everything works as per normal
The problem:
worker log:
coordinator:
scheduler: