Runner
executes tasks in a sequential order. It is useful for debugging and testing the Workflow. Use this for simple workflows or for testing out functionality before you scale compute it with the QueueRunner
.
Runner
follows three steps:
num_retries
times"stage_1"
first, then successively emptying the queue for "stage_2"
. If you set the refresh_every
argument, the runner repolls both queues after emptying the initial set. This ensures data that arrived in the queue after the initial poll is picked up in the subsequent iteration. If an execution’s time already exceeds the refresh_every
threshold, the agent instantly polls for new tasks.
To illustrate the order of execution, see the pseudo-code below.
num_retries
times (default: 3). Changes to the label row are not rolled back.task_batch_size
)runner.run()
function.example.py
file that looks like this:
QueueRunner
run()
function.encord_agents.tasks.models.TaskCompletionResult
as a JSON string. This stringified JSON format is necessary for passing messages over queues, which typically do not support custom object types.Runner
”
QueueRunner
”
QueueRunner
and the sequential Runner
are:
Feature | Runner | QueueRunner |
---|---|---|
Execution Model | Executes tasks sequentially in a single process | Designed for distributed execution across multiple processes |
Project Hash | Optional at initialization | Required at initialization |
Function Wrapping | Executes your function directly with injected dependencies | Additionally wraps your function to handle JSON task specifications |
Execution Control | Handles task polling and execution | You control task distribution and execution through your queue system |
Scaling | Not suitable for scaling | Suitable for scaling |