Skip to content

Introduce Builder annotations#1576

Merged
uranusjr merged 6 commits into
feature/java-sdkfrom
java-annotation
May 7, 2026
Merged

Introduce Builder annotations#1576
uranusjr merged 6 commits into
feature/java-sdkfrom
java-annotation

Conversation

@uranusjr

@uranusjr uranusjr commented May 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

(Unit tests to come later…)

This provides two annotations: DagBuilder and DagBuilder.Task that slightly reduces the boilerplate needed to define tasks in Java (about two lines per task, and two more lines for the dag).

The way this works is you do

@DagBuilder
public class MyDag {
  @DagBuilder.Task
  public void myTask(...) { ... }
}

and the compiler uses our annotation processor to generate a wrapper class named MyDagBuilder with the needed tasks and dependencies defined. You then can register the dag to the bundle via the builder by calling

MyDagBuilder.build()

in the BundleBuilder's getDags().

A lot of code for a little benefit for the moment, but this should make taskflow-style XCom a lot easier. (This is not implemented yet.)

For reference, this is the generated source file build/generated/sources/annotationProcessor/java/main/org/apache/airflow/example/AnnotationExampleBuilder.java

package org.apache.airflow.example;

import java.lang.Exception;
import java.lang.Override;
import org.apache.airflow.sdk.Client;
import org.apache.airflow.sdk.Dag;
import org.apache.airflow.sdk.Task;

public final class AnnotationExampleBuilder {
  public static Dag build() {
    var dag = new Dag("java_annotation_example");
    dag.addTask("extract", ExtractValue.class, java.util.List.of());
    dag.addTask("transform", TransformValue.class, java.util.List.of("extract"));
    dag.addTask("load", Load.class, java.util.List.of("transform"));
    return dag;
  }

  public static final class ExtractValue implements Task {
    @Override
    public void execute(Client client) throws Exception {
      new AnnotationExample().extractValue(client);
    }
  }

  public static final class TransformValue implements Task {
    @Override
    public void execute(Client client) throws Exception {
      new AnnotationExample().transformValue(client);
    }
  }

  public static final class Load implements Task {
    @Override
    public void execute(Client client) throws Exception {
      new AnnotationExample().load(client);
    }
  }
}

@uranusjr uranusjr force-pushed the java-annotation branch from 1d23910 to a1a3a92 Compare May 6, 2026 07:20
ashb
ashb reviewed May 6, 2026
View reviewed changes
Comment thread java-sdk/example/src/java/org/apache/airflow/example/AnnotationExample.java Outdated
ashb
ashb reviewed May 6, 2026
View reviewed changes
Comment thread java-sdk/example/src/java/org/apache/airflow/example/AnnotationExample.java Outdated
jason810496
jason810496 reviewed May 6, 2026
View reviewed changes
Comment thread java-sdk/example/src/java/org/apache/airflow/example/AnnotationExample.java Outdated
@uranusjr uranusjr mentioned this pull request May 6, 2026
Closed
@uranusjr
Introduce DagBuilder annotations
98056af 
This provides two annotations: DagBuilder and DagBuilder.Task that
*slightly* reduces the boilerplate needed to define tasks in Java (about
two lines per task, and two more lines for the dag).

The way this works is you do

    @DagBuilder
    public class MyDag {
      @DagBuilder.Task
      public void myTask(...) { ... }
    }

and the compiler uses our annotation processor to generate a wrapper
class named MyDagBuilder with the needed tasks and dependencies defined.
You then can register the dag to the bundle via the builder by calling

    MyDagBuilder.build()

in the BundleBuilder's getDags().

A lot of code for a little benefit for the moment, but this should make
taskflow-style XCom a lot easier. (This is not implemented yet.)
@uranusjr uranusjr force-pushed the java-annotation branch from a1a3a92 to 98056af Compare May 6, 2026 12:46
uranusjr added 3 commits May 7, 2026 05:33
@uranusjr
Push XCom from return value
d6208e5
@uranusjr
Support auto XCom set and get
697e0f3 
Generate extra code around the annotated function to pass in XCom
references, and set XCom from the return value.
@uranusjr
Move builder annotations to 'Builder'
e5ee5fe
@uranusjr uranusjr marked this pull request as ready for review May 7, 2026 08:35
@uranusjr uranusjr changed the title Introduce DagBuilder annotations Introduce Builder annotations May 7, 2026
@uranusjr uranusjr merged commit ae91075 into feature/java-sdk May 7, 2026
@uranusjr uranusjr deleted the java-annotation branch May 7, 2026 13:33
uranusjr added a commit that referenced this pull request May 27, 2026
@uranusjr
Introduce Builder annotations (#1576)
19de289 
* Introduce DagBuilder annotations

This provides two annotations: DagBuilder and DagBuilder.Task that
*slightly* reduces the boilerplate needed to define tasks in Java (about
two lines per task, and two more lines for the dag).

The way this works is you do

    @DagBuilder
    public class MyDag {
      @DagBuilder.Task
      public void myTask(...) { ... }
    }

and the compiler uses our annotation processor to generate a wrapper
class named MyDagBuilder with the needed tasks and dependencies defined.
You then can register the dag to the bundle via the builder by calling

    MyDagBuilder.build()

in the BundleBuilder's getDags().

A lot of code for a little benefit for the moment, but this should make
taskflow-style XCom a lot easier. (This is not implemented yet.)

* Push XCom from return value

* Support auto XCom set and get

Generate extra code around the annotated function to pass in XCom
references, and set XCom from the return value.

* Move builder annotations to 'Builder'

* Allow customizing generated builder class name

* Add tests for annotation processor
uranusjr added a commit that referenced this pull request May 27, 2026
@uranusjr
Introduce Builder annotations (#1576)
84a7645 
* Introduce DagBuilder annotations

This provides two annotations: DagBuilder and DagBuilder.Task that
*slightly* reduces the boilerplate needed to define tasks in Java (about
two lines per task, and two more lines for the dag).

The way this works is you do

    @DagBuilder
    public class MyDag {
      @DagBuilder.Task
      public void myTask(...) { ... }
    }

and the compiler uses our annotation processor to generate a wrapper
class named MyDagBuilder with the needed tasks and dependencies defined.
You then can register the dag to the bundle via the builder by calling

    MyDagBuilder.build()

in the BundleBuilder's getDags().

A lot of code for a little benefit for the moment, but this should make
taskflow-style XCom a lot easier. (This is not implemented yet.)

* Push XCom from return value

* Support auto XCom set and get

Generate extra code around the annotated function to pass in XCom
references, and set XCom from the return value.

* Move builder annotations to 'Builder'

* Allow customizing generated builder class name

* Add tests for annotation processor
uranusjr added a commit that referenced this pull request May 29, 2026
@uranusjr
Introduce Builder annotations (#1576)
ff7c522 
* Introduce DagBuilder annotations

This provides two annotations: DagBuilder and DagBuilder.Task that
*slightly* reduces the boilerplate needed to define tasks in Java (about
two lines per task, and two more lines for the dag).

The way this works is you do

    @DagBuilder
    public class MyDag {
      @DagBuilder.Task
      public void myTask(...) { ... }
    }

and the compiler uses our annotation processor to generate a wrapper
class named MyDagBuilder with the needed tasks and dependencies defined.
You then can register the dag to the bundle via the builder by calling

    MyDagBuilder.build()

in the BundleBuilder's getDags().

A lot of code for a little benefit for the moment, but this should make
taskflow-style XCom a lot easier. (This is not implemented yet.)

* Push XCom from return value

* Support auto XCom set and get

Generate extra code around the annotated function to pass in XCom
references, and set XCom from the return value.

* Move builder annotations to 'Builder'

* Allow customizing generated builder class name

* Add tests for annotation processor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ashb ashb ashb left review comments
@jason810496 jason810496 jason810496 left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@uranusjr @ashb @jason810496