Skip to content

Default to @task decorator for app task functions #121

@gpauloski

Description

@gpauloski

This is specifically motivated by wanting to better support TaskVine's serverless mode, where functions are associated with a library by their string name. The current process of wrapping tasks in TaskWrapper prevents this from being possible.

There are a few things to consider though:

  1. Can we maintain backwards compatibility with submitting functions directly to the engine? I think so, we just check if it's already decorated.
  2. How do we pass the TaskTransformer to the task if it's statically decorated? We could add an extra variable to the task for passing the transformer. This would result in the transformer being communicated more often for some executors, but if the transformer is cheaply serializable that's okay.
  3. How to maintain backwards compatibility with calling task functions outside of engine.submit()? We could exposed the wrapped function as an attribute (e.g., my_task_func.wrapped(*args, **kwargs)) and/or support two modes of calling. Here, if the extra variable is present (e.g., task flag or transformer), we execute it like a task with the extra functionality in the decorated, otherwise, we just execute it like a normal task. This could be correctly type-hinted with mypy using overloads.

Metadata

Metadata

Assignees

Labels

enhancementNew features or improvements to existing functionality

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions