queenbee.recipe.task module¶

DAGTask module.

class queenbee.recipe.task.DAGTask(*, type: ConstrainedStrValue = 'DAGTask', annotations: Dict[str, Any] = None, name: str, template: str, needs: List[str] = None, arguments: List[Union[TaskArgument, TaskPathArgument]] = None, loop: DAGTaskLoop = None, sub_folder: str = None, returns: List[Union[TaskReturn, TaskPathReturn]] = None)[source]¶

Bases: BaseModel

A single task in a DAG flow.

argument_by_name(name: str) → Union[TaskArgument, TaskPathArgument][source]¶: Find a task argument by name.

argument_by_ref_source(source) → List[Union[TaskArgument, TaskPathArgument]][source]¶

Retrieve a list of argument artifacts by their source type.

Parameters:

type (source {str} -- The source) – ‘dag’, ‘task’, ‘value’. dag refers to InputReference, InputFileReference, InputFolderReference and InputPathReference. ‘task’ refers to TaskReference, TaskFileReference, TaskFolderReference and TaskPathReference. ‘item’ refers to ItemReference. Finally ‘value’ refers to ValueReference, ValueFileReference and ValueFolder Reference.
of (one) – ‘dag’, ‘task’, ‘value’. dag refers to InputReference, InputFileReference, InputFolderReference and InputPathReference. ‘task’ refers to TaskReference, TaskFileReference, TaskFolderReference and TaskPathReference. ‘item’ refers to ItemReference. Finally ‘value’ refers to ValueReference, ValueFileReference and ValueFolder Reference.

Raises:

ValueError – The source type is not recognized

Returns:

List[TaskArguments] – A list of Argument Artifacts

arguments: List[Union[TaskArgument, TaskPathArgument]]¶

property artifact_arguments: List¶

Get artifact arguments. Artifacts are file, folder and path inputs.

Returns:: A list – A list of artifact arguments.

property artifact_returns: List¶

Get artifact returns. Artifacts are file, folder and path inputs.

Returns:: A list – A list of artifact returns.

static artifacts_by_ref_source(artifacts, source) → List[Union[TaskArgument, TaskPathArgument]][source]¶

Retrieve a list of argument artifacts by their source type.

Parameters:

type (source {str} -- The source) – ‘dag’, ‘task’, ‘value’. dag refers to InputFileReference, InputFolderReference and InputPathReference. ‘task’ refers to TaskFileReference, TaskFolderReference and TaskPathReference. Finally ‘value’ refers to ValueFileReference and ValueFolder Reference.
of (one) – ‘dag’, ‘task’, ‘value’. dag refers to InputFileReference, InputFolderReference and InputPathReference. ‘task’ refers to TaskFileReference, TaskFolderReference and TaskPathReference. Finally ‘value’ refers to ValueFileReference and ValueFolder Reference.

Raises:

ValueError – The source type is not recognized

Returns:

List[TaskPathArguments] – A list of Argument Artifacts

classmethod check_duplicate_argument_name(v, values)[source]¶

classmethod check_duplicate_return_name(v, values)[source]¶

classmethod check_item_references(v, values)[source]¶

classmethod check_loop_referenced_task(v, values)[source]¶

classmethod check_referenced_values(v, values)[source]¶

classmethod check_references(v, values)[source]¶

check_template(template)[source]¶

A function to check the inputs and outputs of a DAG task.

The DAG tasks should match a certain template.

Parameters:

{Union[DAG (template) –
template (Function]} -- A DAG or Function) –

Raises:

AssertionError – The task input and output values do not match the task template.

property is_root: bool¶

A root task does not have any dependencies

Returns:: bool – True if the task has no dependencies

loop: DAGTaskLoop¶

name: str¶

needs: List[str]¶

property parameter_arguments: List¶

Get parameter arguments. Parameters are file, folder and path inputs.

Returns:: A list – A list of parameter arguments.

property parameter_returns: List¶

Get parameter returns. Artifacts are file, folder and path inputs.

Returns:: A list – A list of artifact returns.

static parameters_by_ref_source(parameters, source) → List[Union[TaskArgument, TaskPathArgument]][source]¶

Retrieve an list of argument parameters by their source type.

Parameters:

type (source {str} -- The source) – ‘dag’, ‘task’, ‘item’, ‘value’
of (one) – ‘dag’, ‘task’, ‘item’, ‘value’

Raises:

ValueError – The source type is not recognized

Returns:

List[TaskArguments] – A list of Argument Parameters

return_by_name(name: str) → Union[TaskReturn, TaskPathReturn][source]¶: Find a task return by name.

returns: List[Union[TaskReturn, TaskPathReturn]]¶

sub_folder: str¶

template: str¶

type: ConstrainedStrValue¶

class queenbee.recipe.task.DAGTaskLoop(*, type: ConstrainedStrValue = 'DAGTaskLoop', annotations: Dict[str, Any] = None, **extra_data: Any)[source]¶

Bases: BaseModel

Loop configuration for the task.

This will run the template provided multiple times and in parallel relative to an input or task parameter which should be a list.

from_: Union[InputReference, TaskReference, ValueListReference]¶

type: ConstrainedStrValue¶