Editor Agents allow you to integrate your own API endpoint with Encord, enhancing your annotation processes. For example, this could be a model hosted on a server or a cloud function. Annotators can trigger Editor Agents while annotating in the Label Editor.
Some common use-cases are:
Editor Agents are actions your annotators can trigger while they are labeling.
Editor Agents are API endpoints triggered on individual tasks within the Label Editor. They differ from [Task Agents](agents-documentation/Task Agents/index), which are Workflow components that activate on all tasks passing through the Agent stage.
Editor Agents work in the following way:
Use encord-agents
to define the logic for the “Editor Agent [custom API]” section of the diagram. You are responsible for programmatically determining what happens when your custom API receives a project_hash
, data_hash
, and potentially a frame
number.
We help with two different ways of building such Custom APIs:
Google run functions
which is Google’s way of building cloud functions.The encord-agents
library takes a lot of inspiration from [FastAPI]fastapi. Specifically, we have adopted the idea of [dependency injections]fastapi-dependency-injection from that library. While our injection scheme is not as sophisticated, it should feel familiar.
Google Cloud Run functions are ideal for lightweight operations, such as serving as proxies for model inference APIs or making minor label adjustments. In contrast, FastAPI and Modal apps are better suited for hosting your own models and handling resource-intensive tasks.
In the next section, we include a GCP example. If you need to build a FastAPI (or Modal) application, feel free to skip it.
This section details the interface for the EditorAgentPayload
, which is crucial for defining and implementing editor agents, whether you utilize the library’s built-in functionalities or create your own custom implementation.
Schema:
This schema closely mirrors the structure of FrameData
. It’s important to note the objectHashes
field, which is defined as string[]
. This indicates that the field can either be absent from the payload or, if present, will contain an array of strings representing object hashes.
Testing Your Agent with a Test Payload:
When registering your editor agent within the platform at the Editor Agents interface, you have the capability to test its functionality using a sample payload.
To facilitate secure testing, the platform employs the following mechanisms:
X-Encord-Editor-Agent
. Agents receiving this header are expected to respond appropriately for testing purposes.This testing feature serves several key objectives:
Error Handling:
To provide informative feedback to users, the platform leverages the AuthorisationError
handler. If your agent encounters any authorization issues, such as attempting to access a project it does not have permission for, the platform generates appropriate error responses.
Specifically, in the case of an authorization failure with the Encord platform, the body of the error response includes a message
field with the following structure:
This message
is then displayed within the platform’s user interface, offering intuitive guidance to users regarding any access-related problems with the agent.