fix: expand runtime vars in output paths by yottahmd · Pull Request #1957 · dagucloud/dagu

yottahmd · 2026-04-02T15:57:58Z

Summary

move step and handler env setup ahead of node preparation so stdout/stderr path evaluation sees the real runtime context
keep step stdout/stderr file vars as execution-time only while fixing prepare-time expansion for step envs, step refs, and handler status vars
add runtime and integration regressions for step and handler output path expansion

Testing

make fmt
go test ./internal/runtime ./internal/intg -run 'TestRunner/(SpecialVarsDAGRUNSTEPNAME|StdoutPathExpandsStepNameBeforePrepare|StdoutPathExpandsStepEnvBeforePrepare|StdoutPathExpandsUpstreamStepRefBeforePrepare|DAGRunStatusHandlerEnv)$|TestDAGExecution/(StdoutAndStderrPathsExpandPrepareTimeVars)$|TestHandlerOn/(StepOutputVars_NotAvailableInHandlers|SuccessHandler_StdoutPathExpandsDAGRunStatus|WaitHandler_StdoutPathExpandsWaitingSteps)$'

Closes #1955

Summary by CodeRabbit

Release Notes

New Features
- Added support for variable expansion in stdout and stderr path templates, allowing use of placeholders such as ${DAG_RUN_STEP_NAME}, ${LOG_SUFFIX}, ${DAG_RUN_STATUS}, and ${DAG_WAITING_STEPS} for both steps and handlers.
Tests
- Added comprehensive test coverage for stdout/stderr path templating functionality.

coderabbitai · 2026-04-02T15:58:17Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 59889b97-3724-4b11-92f4-07a998308c93

📥 Commits

Reviewing files that changed from the base of the PR and between 9022852 and 50450ad.

📒 Files selected for processing (5)

internal/intg/cfg_test.go
internal/intg/handler_test.go
internal/runtime/runner.go
internal/runtime/runner_helper_test.go
internal/runtime/runner_test.go

📝 Walkthrough

Walkthrough

This pull request fixes variable expansion in stdout/stderr path templates by reordering execution flow. The changes ensure that prepare-time variables (like ${DAG_RUN_STEP_NAME} and ${LOG_SUFFIX}) are expanded before step and handler preparation, rather than after. Handler execution now accepts extra environment variables to be merged during setup, and step variable initialization occurs within the per-step goroutine before preparation.

Changes

Cohort / File(s)	Summary
Integration & Unit Tests `internal/intg/cfg_test.go`, `internal/intg/handler_test.go`, `internal/runtime/runner_test.go`	Added comprehensive tests validating stdout/stderr path variable expansion for steps and handlers, including expansion of `${DAG_RUN_STEP_NAME}`, `${DAG_RUN_STATUS}`, `${DAG_WAITING_STEPS}`, and custom environment variables. Tests verify file creation with expanded paths and expected content.
Handler Execution Flow `internal/runtime/runner.go`	Modified `runEventHandler` and `setupEnvironEventHandler` to accept `extraEnvs` parameter, enabling environment setup before node preparation. Moved step variable setup from `runNodeExecution` into the per-step goroutine, ensuring variables are available during preparation.
Test Utilities `internal/runtime/runner_helper_test.go`	Added `stepOption` helper functions `withStdout()` and `withEnvVars()` to simplify test step construction.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

feat(spec): add init handler field #1455 — Both PRs modify handler execution in internal/runtime/runner.go, with this PR changing runEventHandler/setupEnvironEventHandler signatures and the related PR adding OnInit handler execution that calls these functions.
fix: resolve DAG env/params references in container env variables #1778 — Both PRs move environment-variable/template evaluation to runtime and adjust environment merging to ensure templates (e.g., stdout paths, container env) are expanded with complete runtime scope.
fix(runtime): pass DAG level variables to executor config evaluation properly #1448 — Both PRs adjust how environment variables are assembled and passed into template evaluation at runtime, with changes to environment preparation and handler/step setup.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: expand runtime vars in output paths' accurately summarizes the main change: enabling runtime variable expansion (like DAG_RUN_STEP_NAME) in stdout/stderr path templates.
Linked Issues check	✅ Passed	The PR successfully addresses issue `#1955` by moving environment setup before node preparation, enabling DAG_RUN_STEP_NAME and other runtime variables to expand correctly in step and handler output paths. Changes include handler logic updates, comprehensive test coverage for step/handler stdout expansion, and helper test utilities.
Out of Scope Changes check	✅ Passed	All changes are scoped to the PR objectives: runner environment setup refactoring, handler execution updates, integration and unit tests for output path expansion, and test helper utilities. No unrelated modifications were introduced.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/1955-step-output-path-env

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2026-04-02T16:13:50Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.08%. Comparing base (9022852) to head (50450ad).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1957      +/-   ##
==========================================
+ Coverage   68.07%   68.08%   +0.01%     
==========================================
  Files         475      475              
  Lines       61308    61310       +2     
==========================================
+ Hits        41735    41744       +9     
+ Misses      15609    15599      -10     
- Partials     3964     3967       +3

Files with missing lines	Coverage Δ
internal/runtime/runner.go	`85.75% <100.00%> (+0.03%)`	⬆️

... and 19 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9022852...50450ad. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

fix: expand runtime vars in output paths

50450ad

yottahmd merged commit 3b167ec into main Apr 2, 2026
5 checks passed

yottahmd deleted the fix/1955-step-output-path-env branch April 2, 2026 16:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: expand runtime vars in output paths#1957

fix: expand runtime vars in output paths#1957
yottahmd merged 1 commit intomainfrom
fix/1955-step-output-path-env

yottahmd commented Apr 2, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Apr 2, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

codecov Bot commented Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

yottahmd commented Apr 2, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Summary by CodeRabbit

Release Notes

Uh oh!

coderabbitai Bot commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

codecov Bot commented Apr 2, 2026

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

yottahmd commented Apr 2, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Apr 2, 2026 •

edited

Loading