QueueRunner
is designed for distributed task processing and parallel execution. It allows you to implement agent logic that can be executed across multiple workers or processes, making it ideal for scaling up task processing in production environments.
QueueRunner
follows these steps:
QueueRunner
is designed to work with various queue systems. Here are examples with popular frameworks:
QueueRunner
uses JSON strings to pass task specifications between processes:
TaskCompletionResult
containing:
QueueRunner
provides error handling at the task level:
stage
decorator accepts the same configuration options as the sequential Runner
:
Example:
QueueRunner
and the sequential Runner
are:
Feature | Runner | QueueRunner |
---|---|---|
Execution Model | Executes tasks sequentially in a single process | Designed for distributed execution across multiple processes |
Project Hash | Optional at initialization | Required at initialization |
Function Wrapping | Executes your function directly with injected dependencies | Additionally wraps your function to handle JSON task specifications |
Execution Control | Handles task polling and execution | You control task distribution and execution through your queue system |
Scaling | Not suitable for scaling | Suitable for scaling |
QueueRunner
when you need to: