We strongly recommend that highly technical users (examples: IT professionals, software developers, or system administrators) perform the steps outlined in this process.
Third-party products, such as OpenAI’s API, come with their own terms and conditions.

The following example uses ChatGPT-4o to automatically update classifications in Encord each time the Agent triggers. The example uses FastAPI as a server to host and run the code. The server returns an empty 200 response, indicating that the request has succeeded and causing the Label Editor to refresh.

Only HTTPS endpoints are supported.

Assumptions

This example makes the following assumptions:

  • The payload from Encord includes the project hash, data hash, and frame number in the following format:
{
  "dataHash": "038ed92d-dbe8-4991-a3aa-6ede35d6e62d",
  "projectHash": "027e0c65-c53f-426d-9f02-04fafe8bd80e",
  "frame": 10
}

Step 1: Create a directory to host your code

Create a directory containing the following files.

  • requirements.txt: Installs all the dependencies necessary for the example to work.
  • openai_api_agent.py: Makes the call to the OpenAI API. Ensure that you replace <classification_name> with the name of the classification you want GPT 4o to update.
  • data_models.py: Ensures that the response from OpenAI has the specific format required by Encord.
  • dependencies.py: Retrieves the information necessary for the query to be made and the label row to be updated.
  • main.py: Runs the program in the correct order.

Step 2: Test your Agent locally

We strongly recommend you test your Agent locally before deploying it to your server.

  1. Create a new virtual environment and install all the requirements using the following terminal commands:
$ python -m venv venv
$ source venv/bin/activate
$ python -m pip install -r requirements.txt
  1. Run your Agent and server locally using the following terminal command. Ensure that you replace:
  • <private_key_path> with the path to your private Encord API key.
  • <openai_key_path> with the path / key to your OpenAI API key.
ENCORD_SSH_KEY_FILE=\<private_key_path> OPENAI_API_KEY='<openai_key_path>' fastapi dev main.py
Keep this terminal window open as long as you want to run your server locally. The following steps should be performed in a new terminal window.
  1. In a new terminal window, point the server to a specific data asset in Encord. This step replicates the triggering of the Agent in the Encord platform. Ensure that you:
  • Replace <project_hash> with the hash of your Project.
  • Replace <data_hash> with the hash of the data unit you want to run your Agent on.

Both the <project_hash> and the <data_hash> can be found in the url of the Label Editor. The url is structured like this: https://app.encord.com/label_editor/<project_hash>\<data_hash>

curl localhost:8000 -H "Content-type: application/json" -d '{
    "projectHash": "<project_hash>",
    "dataHash": "<data_hash>",
    "frame": 0
}'

Step 3: Deploy the agent to your server

The Agent deploys to your server using a container image, such as Docker. The exact method of deployment varies depending on your choice of server.

  • Install the requirements from the requirements.txt file. Documentation for creating a Docker image for FastAPI can be found here.

  • Ensure that you set up environment variables to authenticate with Encord and OpenAI. Information on authenticating with OpenAI can be found in their documentation here.

    • ENCORD_SSH_KEY_FILE=/path/to/your/private_key_file
    • OPENAI_API_KEY=<your_openai_api_key>