Skip to content

Commit af2986e

Browse files
Orange-SummerSbloodySrickchengxzhongjiajie
authored
[Feature-14545][Master][UI]Cross workflow parameter passing (#14552)
* feat: cross-workflow parameter passing * refactor: remove useless parameters * refactor: remove useless code * refactor: code format * docs: update docs * docs: docs format * more effective information Co-authored-by: xiangzihao <[email protected]> * more effective information Co-authored-by: xiangzihao <[email protected]> * fix: wrong writing of log code * fix: perfect error log --------- Co-authored-by: xiangzihao <[email protected]> Co-authored-by: xiangzihao <[email protected]> Co-authored-by: Rick Cheng <[email protected]> Co-authored-by: Jay Chung <[email protected]>
1 parent 93c3871 commit af2986e

File tree

19 files changed

+164
-19
lines changed

19 files changed

+164
-19
lines changed

docs/docs/en/guide/parameter/context.md

Lines changed: 30 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,9 @@ DolphinScheduler allows parameter transfer between tasks. Currently, transfer di
1414
* [SQL](../task/sql.md)
1515
* [Procedure](../task/stored-procedure.md)
1616
* [Python](../task/python.md)
17+
* [SubProcess](../task/sub-process.md)
1718

18-
When defining an upstream node, if there is a need to transmit the result of that node to a dependency related downstream node. You need to set an `OUT` direction parameter to [Custom Parameters] of the [Current Node Settings]. At present, we mainly focus on the SQL and shell nodes to pass parameters downstream.
19+
When defining an upstream node, if there is a need to transmit the result of that node to a dependency related downstream node. You need to set an `OUT` direction parameter to [Custom Parameters] of the [Current Node Settings]. If it is a SubProcess node, there is no need to set a parameter in [Current Node Settings], but an `OUT` direction parameter needs to be set in the workflow definition of the subprocess.
1920

2021
The value of upstream parameter can be updated in downstream node in the same way as [setting parameter](#create-a-shell-task-and-set-parameters).
2122

@@ -60,7 +61,7 @@ When the SHELL task is completed, we can use the output passed upstream as the q
6061

6162
Click on the Save workflow icon and set the global parameters output and value.
6263

63-
![context-parameter03](../../../../img/new_ui/dev/parameter/context_parameter04.png)
64+
![context-parameter04](../../../../img/new_ui/dev/parameter/context_parameter04.png)
6465

6566
#### View results
6667

@@ -88,4 +89,30 @@ Use `print('${setValue(key=%s)}' % value)`, DolphinScheduler will capture the `$
8889

8990
For example
9091

91-
![img.png](../../../../img/new_ui/dev/parameter/python_context_param.png)
92+
![python_context_param](../../../../img/new_ui/dev/parameter/python_context_param.png)
93+
94+
#### Pass parameter from SubProcess task to downstream
95+
96+
In the workflow definition of the subprocess, define `OUT` direction parameters as output parameters, and these parameters can be passed to the downstream tasks of the subprocess node.
97+
98+
Create an A task in the workflow definition of the subprocess, add var1 and var2 parameters to the custom parameters, and write the following script:
99+
100+
![context-subprocess01](../../../../img/new_ui/dev/parameter/context-subprocess01.png)
101+
102+
Save the subprocess_example1 workflow and set the global parameters var1.
103+
104+
![context-subprocess02](../../../../img/new_ui/dev/parameter/context-subprocess02.png)
105+
106+
Create a sub_process task in a new workflow, and use the subprocess_example1 workflow as the sub-node.
107+
108+
![context-subprocess03](../../../../img/new_ui/dev/parameter/context-subprocess03.png)
109+
110+
Create a shell task as a downstream task of the sub_process task, and write the following script:
111+
112+
![context-subprocess04](../../../../img/new_ui/dev/parameter/context-subprocess04.png)
113+
114+
Save the workflow and run it. The result of the downstream task is as follows:
115+
116+
![context-subprocess05](../../../../img/new_ui/dev/parameter/context-subprocess05.png)
117+
118+
Although the two parameters var1 and var2 are output in the A task, only the `OUT` parameter var1 is defined in the workflow definition, and the downstream task successfully outputs var1. It proves that the var1 parameter is passed in the workflow with reference to the expected value.

docs/docs/en/guide/parameter/global.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,11 @@
22

33
## Scope
44

5-
Global parameters are parameters that are valid for all task nodes of the entire workflow. It can be configured on the workflow definition page.
5+
Global parameters can be configured on the workflow definition page.
6+
7+
The `IN` direction parameter is valid for all task nodes of the entire workflow.
8+
9+
The `OUT` direction parameter is the output parameter of the workflow and passed to the downstream task of the corresponding SubProcess task in the parent workflow.
610

711
## Usage
812

docs/docs/zh/guide/parameter/context.md

Lines changed: 29 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,9 @@ DolphinScheduler 允许在任务间进行参数传递,目前传递方向仅支
1414
* [SQL](../task/sql.md)
1515
* [Procedure](../task/stored-procedure.md)
1616
* [Python](../task/python.md)
17+
* [SubProcess](../task/sub-process.md)
1718

18-
当定义上游节点时,如果有需要将该节点的结果传递给有依赖关系的下游节点,需要在【当前节点设置】的【自定义参数】设置一个方向是 OUT 的变量。目前我们主要针对 SQL 和 SHELL 节点做了可以向下传递参数的功能
19+
当定义上游节点时,如果有需要将该节点的结果传递给有依赖关系的下游节点,需要在【当前节点设置】的【自定义参数】设置一个方向是 OUT 的变量。如果是 SubProcess 节点无需在【当前节点设置】中设置变量,需要在子流程的工作流定义中设置一个方向是 OUT 的变量
1920

2021
上游传递的参数可以在下游节点中被更新,更新方法与[设置参数](#创建-shell-任务并设置参数)相同。
2122

@@ -87,4 +88,30 @@ Node_mysql 运行结果如下:
8788
使用 `print('${setValue(key=%s)}' % value)`,DolphinScheduler会捕捉输出中的 `${setValue(key=value}`来进行参数捕捉,从而传递到下游
8889

8990
90-
![img.png](../../../../img/new_ui/dev/parameter/python_context_param.png)
91+
![python_context_param](../../../../img/new_ui/dev/parameter/python_context_param.png)
92+
93+
#### SubProcess 任务传递参数
94+
95+
在子流程的工作流定义中定义方向是 OUT 的变量作为输出参数,可以将这些参数传递到子流程节点的下游任务。
96+
97+
在子流程的工作流定义中创建 A 任务,在自定义参数中添加 var1 和 var2 参数,并编写如下脚本:
98+
99+
![context-subprocess01](../../../../img/new_ui/dev/parameter/context-subprocess01.png)
100+
101+
保存 subprocess_example1 工作流,设置全局参数 var1。
102+
103+
![context-subprocess02](../../../../img/new_ui/dev/parameter/context-subprocess02.png)
104+
105+
在新的工作流中创建 sub_process 任务,使用 subprocess_example1 工作流作为子节点。
106+
107+
![context-subprocess03](../../../../img/new_ui/dev/parameter/context-subprocess03.png)
108+
109+
创建一个 shell 任务作为 sub_process 任务的下游任务,并编写如下脚本:
110+
111+
![context-subprocess04](../../../../img/new_ui/dev/parameter/context-subprocess04.png)
112+
113+
保存该工作流并运行,下游任务运行结果如下:
114+
115+
![context-subprocess05](../../../../img/new_ui/dev/parameter/context-subprocess05.png)
116+
117+
虽然在 A 任务中输出 var1 和 var2 两个参数,但是工作流定义中只定义了 var1 的 OUT 变量,下游任务成功输出 var1,证明var1 参数参照预期的值在该工作流中传递。

docs/docs/zh/guide/parameter/global.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,11 @@
22

33
## 作用域
44

5-
全局参数是指针对**整个工作流**的所有任务节点都有效的参数,在工作流定义页面配置。
5+
全局参数在工作流定义页面配置。
6+
7+
方向是 IN 的变量是针对**整个工作流**的所有任务节点都有效的参数。
8+
9+
方向是 OUT 的变量作为该工作流的输出参数,传递到父工作流中对应 SubProcess 任务的下游任务。
610

711
## 使用方式
812

151 KB
Loading
137 KB
Loading
161 KB
Loading
170 KB
Loading
160 KB
Loading
19.4 KB
Loading

0 commit comments

Comments
 (0)