Skip to content

feat: add configurable env passthrough for step execution#1925

Merged
yottahmd merged 3 commits intomainfrom
passshtourgh-env
Apr 1, 2026
Merged

feat: add configurable env passthrough for step execution#1925
yottahmd merged 3 commits intomainfrom
passshtourgh-env

Conversation

@yottahmd
Copy link
Copy Markdown
Collaborator

@yottahmd yottahmd commented Apr 1, 2026

Summary

  • add top-level env_passthrough and env_passthrough_prefixes to extend filtered base environment inheritance
  • preserve platform-aware exact-name and prefix matching in config loading and worker step execution
  • expose the feature in the Helm chart with config.envPassthrough, config.envPassthroughPrefixes, and shared extraEnv

Testing

Summary by CodeRabbit

Release Notes

  • New Features

    • Added configurable environment variable passthrough to control which host environment variables are exposed to workflow steps via exact name matching or prefix-based patterns.
    • Added support for injecting additional environment variables into Dagu containers.
  • Documentation

    • Added environment passthrough configuration guidance in chart README with usage examples.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 1, 2026

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 1196921a-1aca-49ac-9425-f5618d87a5eb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR extends the Dagu Helm chart and configuration system to support configurable environment variable passthrough. It adds schema validation, template helpers for injecting extra environment variables, configuration fields for exact env-name and prefix-based filtering, and new environment loading logic with corresponding tests.

Changes

Cohort / File(s) Summary
Helm Chart Version & Documentation
charts/dagu/Chart.yaml, charts/dagu/README.md
Updated chart version from 1.0.4 to 1.0.5. Added "Environment Passthrough" section documenting configuration options and usage of extraEnv, config.envPassthrough, and config.envPassthroughPrefixes.
Helm Schema & Values
charts/dagu/values.schema.json, charts/dagu/values.yaml
Introduced JSON schema defining chart values with config (containing envPassthrough and envPassthroughPrefixes arrays) and extraEnv (array of environment variable objects). Added corresponding default value entries in values.yaml.
Helm Templates & Helpers
charts/dagu/templates/_helpers.tpl, charts/dagu/templates/configmap.yaml, charts/dagu/templates/coordinator-deployment.yaml, charts/dagu/templates/scheduler-deployment.yaml, charts/dagu/templates/ui-deployment.yaml, charts/dagu/templates/worker-deployment.yaml
Added dagu.extraEnv helper template. Updated ConfigMap to conditionally render env_passthrough and env_passthrough_prefixes config entries. Extended all deployment templates (coordinator, scheduler, UI, worker) to inject extra environment variables into container env lists.
Configuration Structures
internal/cmn/config/config.go, internal/cmn/config/definition.go, internal/cmn/config/key_hints.go
Added EnvPassthrough and EnvPassthroughPrefixes fields to Core config struct and Definition struct with mapstructure tags. Added legacy camelCase key mappings in legacyToSnakeCaseKey for backward compatibility.
Environment Loading & Filtering
internal/cmn/config/env.go, internal/cmn/config/loader.go
Introduced LoadBaseEnvWithExtras(extraAllow, extraPrefixes) function supporting configurable environment allowlists and prefix filters. Added helper functions for normalizing, trimming, and de-duplicating environment entries. Updated LoadBaseEnv to delegate to LoadBaseEnvWithExtras. Modified loadCoreConfig to read passthrough settings and pass them to environment filtering.
Test Coverage
internal/cmn/config/env_test.go, internal/cmn/config/loader_test.go, internal/service/worker/handler_test.go
Added tests for LoadBaseEnvWithExtras covering exact allowlist and prefix-based filtering with whitespace normalization. Added loader tests validating passthrough configuration from both YAML and environment variables. Added worker handler tests verifying environment passthrough across task START and RETRY operations.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~35 minutes

Possibly related PRs

  • dagu-org/dagu#1924: Directly modifies internal/cmn/config/env.go BaseEnv filtering logic and related worker environment tests.
  • dagu-org/dagu#1448: Changes runtime environment composition and propagation via AllEnvs/AllEnvsMap to consume BaseEnv.
  • dagu-org/dagu#1860: Modifies how environment variables are preserved and propagated to subprocesses through environment resolution.
🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'feat: add configurable env passthrough for step execution' directly and clearly describes the main change: adding environment variable passthrough configuration for workflow step execution.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch passshtourgh-env

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
internal/cmn/config/loader_test.go (1)

461-479: Add one negative assertion in the env-based passthrough test.

Line 461 onward verifies positive inclusion but not exclusion of unrelated keys. Mirroring the YAML test’s negative check would harden regression coverage.

💡 Suggested test hardening
 func TestLoad_BaseEnvIncludesConfiguredEnvPassthroughFromEnv(t *testing.T) {
 	tempDir := t.TempDir()
 	configFile := filepath.Join(tempDir, "config.yaml")
 	preserveTZEnv(t)
 	t.Setenv("EXACT_FROM_ENV", "exact-value")
 	t.Setenv("PREFIX_FROM_ENV_TOKEN", "prefix-value")
+	t.Setenv("BLOCKED_FROM_ENV", "blocked-value")
 	t.Setenv("DAGU_ENV_PASSTHROUGH", " EXACT_FROM_ENV , EXACT_FROM_ENV ,,")
 	t.Setenv("DAGU_ENV_PASSTHROUGH_PREFIXES", " PREFIX_FROM_ENV_ , PREFIX_FROM_ENV_ ,,")
 	require.NoError(t, os.WriteFile(configFile, []byte("# minimal config"), 0o600))

 	cfg := testLoad(t, WithConfigFile(configFile), WithAppHomeDir(tempDir))
 	baseEnv := cfg.Core.BaseEnv.AsSlice()

 	require.Equal(t, []string{"EXACT_FROM_ENV"}, cfg.Core.EnvPassthrough)
 	require.Equal(t, []string{"PREFIX_FROM_ENV_"}, cfg.Core.EnvPassthroughPrefixes)
 	require.Contains(t, baseEnv, "EXACT_FROM_ENV=exact-value")
 	require.Contains(t, baseEnv, "PREFIX_FROM_ENV_TOKEN=prefix-value")
+	require.NotContains(t, baseEnv, "BLOCKED_FROM_ENV=blocked-value")
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/cmn/config/loader_test.go` around lines 461 - 479, Add a negative
assertion to TestLoad_BaseEnvIncludesConfiguredEnvPassthroughFromEnv to ensure
unrelated environment variables are not passed through: after computing baseEnv
(cfg.Core.BaseEnv.AsSlice()) assert that an unrelated key (e.g.,
"UNRELATED_ENV=some-value" or simply ensure "UNRELATED_ENV" is not present) is
not in baseEnv; use the same style as the existing positive assertions and
reference cfg.Core.EnvPassthrough / cfg.Core.EnvPassthroughPrefixes to mirror
the YAML test’s exclusion check so the test fails if non-configured keys slip
into BaseEnv.
internal/cmn/config/env.go (1)

136-145: dedupePreserveOrder can be linear-time with a seen map.

Line 139 repeatedly calls slices.Contains, making this O(n²). A map-backed check keeps order while reducing overhead.

♻️ O(n) dedupe preserving order
 func dedupePreserveOrder(values []string) []string {
-	var deduped []string
+	deduped := make([]string, 0, len(values))
+	seen := make(map[string]struct{}, len(values))
 	for _, value := range values {
-		if slices.Contains(deduped, value) {
+		if _, ok := seen[value]; ok {
 			continue
 		}
+		seen[value] = struct{}{}
 		deduped = append(deduped, value)
 	}
 	return deduped
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/cmn/config/env.go` around lines 136 - 145, The dedupePreserveOrder
function is currently O(n²) because it calls slices.Contains for each element;
change it to O(n) by using a map (e.g., map[string]struct{}) named seen to track
which strings were already encountered, iterate the input slice once, append
each value to deduped only if not present in seen, and mark it in seen to
preserve original order while avoiding repeated containment checks.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@internal/service/worker/handler_test.go`:
- Around line 369-450: The tests set environment variables with t.Setenv but
then reuse the pre-created th.Config and NewTaskHandler, so the config snapshot
won't include the new DAGU_ENV_PASSTHROUGH* values; reinitialize the
configuration or test harness after calling t.Setenv and before constructing the
handler. Concretely: after each block of t.Setenv calls, refresh th.Config (or
recreate th/test harness) so the loader reads the updated env, then call
NewTaskHandler(th.Config) and proceed to create tasks (see test names
HandleTaskStartPreservesConfiguredEnvPassthrough and
HandleTaskRetryPreservesConfiguredEnvPassthrough and usage sites of th.Config,
NewTaskHandler, runtimeexec.CreateTask) to ensure the passthrough settings are
applied.

---

Nitpick comments:
In `@internal/cmn/config/env.go`:
- Around line 136-145: The dedupePreserveOrder function is currently O(n²)
because it calls slices.Contains for each element; change it to O(n) by using a
map (e.g., map[string]struct{}) named seen to track which strings were already
encountered, iterate the input slice once, append each value to deduped only if
not present in seen, and mark it in seen to preserve original order while
avoiding repeated containment checks.

In `@internal/cmn/config/loader_test.go`:
- Around line 461-479: Add a negative assertion to
TestLoad_BaseEnvIncludesConfiguredEnvPassthroughFromEnv to ensure unrelated
environment variables are not passed through: after computing baseEnv
(cfg.Core.BaseEnv.AsSlice()) assert that an unrelated key (e.g.,
"UNRELATED_ENV=some-value" or simply ensure "UNRELATED_ENV" is not present) is
not in baseEnv; use the same style as the existing positive assertions and
reference cfg.Core.EnvPassthrough / cfg.Core.EnvPassthroughPrefixes to mirror
the YAML test’s exclusion check so the test fails if non-configured keys slip
into BaseEnv.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: eb9fdb77-6e5c-4f02-bb48-284191f9d10a

📥 Commits

Reviewing files that changed from the base of the PR and between 53c69a4 and bef41d0.

📒 Files selected for processing (18)
  • charts/dagu/Chart.yaml
  • charts/dagu/README.md
  • charts/dagu/templates/_helpers.tpl
  • charts/dagu/templates/configmap.yaml
  • charts/dagu/templates/coordinator-deployment.yaml
  • charts/dagu/templates/scheduler-deployment.yaml
  • charts/dagu/templates/ui-deployment.yaml
  • charts/dagu/templates/worker-deployment.yaml
  • charts/dagu/values.schema.json
  • charts/dagu/values.yaml
  • internal/cmn/config/config.go
  • internal/cmn/config/definition.go
  • internal/cmn/config/env.go
  • internal/cmn/config/env_test.go
  • internal/cmn/config/key_hints.go
  • internal/cmn/config/loader.go
  • internal/cmn/config/loader_test.go
  • internal/service/worker/handler_test.go

Comment thread internal/service/worker/handler_test.go
@yottahmd yottahmd merged commit fcf27b5 into main Apr 1, 2026
6 checks passed
@yottahmd yottahmd deleted the passshtourgh-env branch April 1, 2026 10:17
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 1, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.63%. Comparing base (14249a6) to head (aec9f31).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1925      +/-   ##
==========================================
+ Coverage   68.59%   68.63%   +0.04%     
==========================================
  Files         462      462              
  Lines       58222    58322     +100     
==========================================
+ Hits        39936    40032      +96     
- Misses      14602    14603       +1     
- Partials     3684     3687       +3     
Files with missing lines Coverage Δ
internal/cmn/config/config.go 72.68% <ø> (ø)
internal/cmn/config/env.go 94.44% <100.00%> (+5.25%) ⬆️
internal/cmn/config/key_hints.go 45.45% <ø> (ø)
internal/cmn/config/loader.go 81.97% <100.00%> (+0.07%) ⬆️

... and 14 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 14249a6...aec9f31. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant