Skip to content

feat: unify params and output schema handling#1869

Merged
yottahmd merged 2 commits intomainfrom
refactor-input-schema
Mar 28, 2026
Merged

feat: unify params and output schema handling#1869
yottahmd merged 2 commits intomainfrom
refactor-input-schema

Conversation

@yottahmd
Copy link
Copy Markdown
Collaborator

@yottahmd yottahmd commented Mar 28, 2026

Summary

  • unify params.schema and output.schema handling behind a shared schema declaration resolver
  • allow output.schema to use the same declaration shapes as params: string refs, inline objects, and boolean schemas
  • preserve legacy top-level params maps like params: { schema: prod } and params: { schema: true } by keeping schema-backed params mode opt-in only when the shape is unambiguous
  • clarify the schema-backed params compatibility rule and output schema resolution context

Why

The feature added in #1867 introduced output schema validation, but params and output still used different schema declaration surfaces and separate resolution logic. That made the behavior harder to reason about and left duplicated logic in the spec layer.

Impact

params.schema and output.schema now share one resolution path and one schema declaration contract. Existing DAGs keep working, including legacy named params that happen to use the key schema.

Validation

  • go test ./internal/core/spec ./internal/cmn/schema -count=1
  • go test ./internal/intg -run 'TestOutputValidation_|TestIssue1252_' -count=1
  • go test ./internal/runtime -run OutputValidation -count=1

Summary by CodeRabbit

  • New Features

    • Expanded schema validation support to accept inline schema objects, boolean schemas, and file/URL references for both parameters and step outputs, providing greater flexibility in schema declarations.
  • Tests

    • Added comprehensive test coverage for multiple schema declaration formats, schema resolution, and end-to-end validation scenarios.

@yottahmd yottahmd changed the title [codex] unify params and output schema handling feat: unify params and output schema handling Mar 28, 2026
@yottahmd yottahmd marked this pull request as ready for review March 28, 2026 12:03
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 28, 2026

📝 Walkthrough

Walkthrough

This PR extends DAG schema validation to support flexible schema declarations for parameters and step outputs: paths/URLs (strings), inline JSON Schema objects, or booleans. It introduces a shared jsonSchemaDeclaration definition and refactors parameter/output schema detection and resolution logic.

Changes

Cohort / File(s) Summary
Schema Definition Updates
internal/cmn/schema/dag.schema.json
Added new definitions.jsonSchemaDeclaration supporting string, object, and boolean schemas. Updated dagParamExternalSchema.schema and step output schema fields to reference this declaration type via $ref.
Schema Definition Tests
internal/cmn/schema/dag_schema_test.go
Added TestDAGSchemaStepOutputSchema validating that step output schemas can be inline objects, booleans, or string references.
Parameter Schema Detection
internal/core/spec/dag_params.go, internal/core/spec/params.go, internal/core/spec/dag_params_test.go
Replaced extractSchemaReference checks with extractParamsSchemaDeclaration boolean checks for schema-backed parameter detection. Added test verifying boolean schema: true in legacy params syntax.
Core Schema Resolution
internal/core/spec/schema.go, internal/core/spec/schema_test.go
Major refactor of resolveSchemaFromParams and schema extraction logic. New resolveSchemaDeclaration function handles string references, inline object schemas, and boolean schemas. Extracted schema resolution into resolveSchemaData helper. Renamed/repurposed extraction tests to cover new declaration types with comprehensive test cases for inline/boolean schema handling.
Step Output Schema
internal/core/spec/step.go, internal/core/spec/step_output_schema_test.go
Updated buildStepOutputSchema to use resolveSchemaDeclaration instead of JSON marshal/unmarshal roundtrip. Removed unused import. Updated tests to verify schema references resolve relative to DAG working directory.
Integration Tests
internal/intg/output_validation_e2e_test.go, internal/intg/params_validation_issue1252_test.go
Added e2e test for external schema file reference during output validation. Added integration test for inline schema declarations in params with default values and validation error scenarios.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • dagu-org/dagu#1867: Related through overlapping step output schema handling changes (dag.schema.json output.schema field, buildStepOutputSchema parsing, and output validation tests).
  • dagu-org/dagu#1770: Related through DAG params/schema refactoring, as both PRs modify schema-backed parameter detection and resolution logic alongside paramDefs and plan-building infrastructure.
🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 27.59% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat: unify params and output schema handling' accurately and concisely describes the main change—consolidating params and output schema handling into a unified approach.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor-input-schema

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
internal/core/spec/params.go (1)

154-161: ⚠️ Potential issue | 🟠 Major

Schema-mode detection here is too global and can drop step params silently.

parseParamValue is also used by step/sub-DAG params parsing, so this branch can reinterpret ordinary maps as schema-backed and return no pairs when values is absent. That changes behavior outside DAG-root params parsing and can lose user input without an explicit error.

💡 Suggested fix direction
-func parseParamValue(ctx BuildContext, input any) ([]paramPair, error) {
+func parseParamValue(ctx BuildContext, input any, allowSchemaDeclaration bool) ([]paramPair, error) {
...
-	case map[string]any:
-		if _, ok := extractParamsSchemaDeclaration(v); !ok {
+	case map[string]any:
+		if !allowSchemaDeclaration {
+			return parseMapParams(ctx, []any{v})
+		}
+		if _, ok := extractParamsSchemaDeclaration(v); !ok {
			return parseMapParams(ctx, []any{v})
		}

Then call DAG-root parsing with allowSchemaDeclaration=true, and step/sub-DAG parsing with false.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/core/spec/params.go` around lines 154 - 161, The current schema-mode
detection in parseParamValue (using extractParamsSchemaDeclaration and checking
v["values"]) is too global and can silently drop step/sub-DAG params; update
parseParamValue to accept an allowSchemaDeclaration bool and only treat maps as
schema-backed when allowSchemaDeclaration is true, leaving the existing branch
(extractParamsSchemaDeclaration + v["values"] check) guarded by that flag;
update callers that parse DAG-root params to pass allowSchemaDeclaration=true
and callers for step/sub-DAG params to pass allowSchemaDeclaration=false so
ordinary maps are not reinterpreted as schema declarations.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@internal/core/spec/params.go`:
- Around line 154-161: The current schema-mode detection in parseParamValue
(using extractParamsSchemaDeclaration and checking v["values"]) is too global
and can silently drop step/sub-DAG params; update parseParamValue to accept an
allowSchemaDeclaration bool and only treat maps as schema-backed when
allowSchemaDeclaration is true, leaving the existing branch
(extractParamsSchemaDeclaration + v["values"] check) guarded by that flag;
update callers that parse DAG-root params to pass allowSchemaDeclaration=true
and callers for step/sub-DAG params to pass allowSchemaDeclaration=false so
ordinary maps are not reinterpreted as schema declarations.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: cda9360c-7c0a-40c1-b6f1-c18ddb948c16

📥 Commits

Reviewing files that changed from the base of the PR and between 57f4f3a and 79eddef.

📒 Files selected for processing (11)
  • internal/cmn/schema/dag.schema.json
  • internal/cmn/schema/dag_schema_test.go
  • internal/core/spec/dag_params.go
  • internal/core/spec/dag_params_test.go
  • internal/core/spec/params.go
  • internal/core/spec/schema.go
  • internal/core/spec/schema_test.go
  • internal/core/spec/step.go
  • internal/core/spec/step_output_schema_test.go
  • internal/intg/output_validation_e2e_test.go
  • internal/intg/params_validation_issue1252_test.go

@yottahmd yottahmd merged commit 7205abf into main Mar 28, 2026
8 checks passed
@yottahmd yottahmd deleted the refactor-input-schema branch March 28, 2026 13:11
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 28, 2026

Codecov Report

❌ Patch coverage is 86.44068% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.13%. Comparing base (57f4f3a) to head (79eddef).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
internal/core/spec/schema.go 84.61% 5 Missing and 3 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1869      +/-   ##
==========================================
+ Coverage   69.06%   69.13%   +0.07%     
==========================================
  Files         445      445              
  Lines       54816    54837      +21     
==========================================
+ Hits        37859    37912      +53     
+ Misses      13589    13574      -15     
+ Partials     3368     3351      -17     
Files with missing lines Coverage Δ
internal/core/spec/dag_params.go 80.66% <100.00%> (ø)
internal/core/spec/params.go 93.14% <100.00%> (-0.04%) ⬇️
internal/core/spec/step.go 79.33% <100.00%> (+0.31%) ⬆️
internal/core/spec/schema.go 83.58% <84.61%> (+0.40%) ⬆️

... and 17 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 57f4f3a...79eddef. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant