Notifications

Use the Notifications tab in Settings to manage your push notifications.

This is where you create, filter and manage notifications via a

  • + Create notification button: this will initiate the notification creation flow.
  • Search bar: typing in the search bar will filter the notifications by title.
  • List of notifications: click the gear button to change a notification's settings and click the red bin button to delete it.

You can create a number of event-based or timing-based notifications to help monitor your workflow activity:

  • Email: send an email when certain events happen.
  • Slack: send a Slack message when certain events happen.
  • Digest: send regular emails giving a summary of activity over a given time period.
  • Webhook: send a JSON payload to a REST endpoint when certain events happen.

Creating webhooks on workflow stages

Notifications (known as webhooks) can be set up on individual workflow stages and will be triggered when tasks arrive at the stage in question. Watch our video tutorial on webhooks to learn more.

Video Tutorial - Creating webhooks in Workflow projects

Creating notifications

To create a new notification, click the + Create notification button in the Notifications view. This initiates the notification creation flow, a simple step-by-step process for creating push notifications.

1. Enter details

In the 'Enter details' screen, enter a notification title and optional description. A clear title and description helps keep your notifications organized and helps collaborators find what they are looking for. When you are happy, click Next.

2. Add projects

In the 'Add projects' screen, select the projects you would like to include in this notification. When you are happy, click Next.

3. Add channels

In the 'Add channels' screen, choose how you would like to receive the notification.

Slack channels

Click the Add Slack channel button to add a Slack channel.

Once redirected to the Slack web page, make sure to first select the correct workspace from the top right drop-down list. Then select the channel you would like to post Slack notifications to.

Add the channel and make sure to refresh the grid by clicking on the refresh icon above the Slack channels grid.
For a given user the available Slack channels are shared between all notifications.

Email addresses

Enter the email addresses of everyone who should receive the notifications in the entry box. A list of emails will appear above the box for each entered email.

ℹ️

Note

Encord attempts to reduce the number of outgoing emails.
New emails are not sent immediately, instead the email notifications are aggregated and sent in 5 minute intervals.
Consequently, you may not receive emails immediately on the triggering of an event.

Webhooks

Add a webhook by clicking the Add webhook button. Fill in all the relevant information.
You can create, change and delete webhooks. You cannot attach or detach them; to "detach" a webhook you need to delete it. There is currently no sharing of webhooks between notifications.

Click here for a list of all possible events.

4. Add triggers

In the 'Add triggers' screen, choose what type of events will trigger a notification. A notification can have multiple triggers. You can choose from 'Reactive events' and 'Digest events':

  • Reactive events: notification will be triggered in reaction to an event
  • Digest events: notification will be triggered at a regular time of day

Use the info icon next to the event type to get more detailed explanations on when these events are triggered.

5. Create the notification

Once you are happy with the triggers, click Create to create the push notification.


Example use cases for notifications

These are example use cases for different workflows using the notifications feature.

I want to notify my reviewers whenever annotations have finished

Have reviewers receive notifications whenever an annotation is finished, so reviewers can add their review without manually checking whether new annotations have been created.

  1. Create a new notification
  2. Add the projects you are interested in
  3. Add all the email addresses or Slack channels where your reviewer would expect them
  4. Enable the "Task submitted" trigger.

Whenever a task is submitted, a push notification will be sent to the reviewers. A relevant link will be included to direct them to the project page that they need to open.

I want to receive a daily notification about the annotator and reviewer progress on my projects

As a manager, you want the progress of annotations and reviews in your Project.

  1. Create a new notification.
  2. Select relevant Projects.
  3. Select your own email address or preferred Slack channel.
  4. Enable the "Full digest" trigger.

You will now receive a summary of the last 24h daily at 5pm UTC on your projects. This summary includes every participating annotator and reviewer and their participation statistics.

I want to create custom programmatic integrations after annotation tasks have been completed

To kick off a script or program once a given image sequence or video has been fully annotated and approved.

  1. Create a new notification.
  2. Select all relevant Projects.
  3. Add a webhook that points to the exact URL where you expect a payload to arrive.
  4. Enable the "Task completed" trigger.

Whenever the last label has been approved or a task has been completed without reviews being required, an HTTP message will be sent to the configured webhook endpoints. An explanation of the notification can be found below.


Webhook event payloads

Base payload

Every event will have the following top level fields:

KeyTypeDescription
uidstringUnique identifier of the event
versionintegerMajor version number to indicate backwards incompatible versions
sourcestringWill always be the string "Encord"
event_typestringThe event type. See below for more info.
event_created_timestampstringA string of the timestamp in the ISO 8601 format. For example 2022-04-13T14:35:11.791161+00:00
payloadobjectAn object with different fields depending on the type of event. See below for more info.

Individual event payloads

Task submitted payload

event_type: task_submitted_event

Payload

KeyTypeDescription
project_hashstringThe unique identifier for the Project.

Example payload:

{
  "uid": "687457ce-6489-4891-b7c8-544477d99e41",
  "version": 1,
  "source": "Encord",
  "event_type": "task_submitted_event",
  "event_created_timestamp": "2024-03-25T15:18:55.406529+00:00",
  "payload": {
    "project_hash": "751a3a3e-46b6-4aad-b0b8-19b169f91266",
    "data_hashes": [
      "f0b012de-5aea-4457-b515-936b0c185771"
    ],
    "label_hashes": [
      "fdb9a21f-d9a7-48b5-9dac-dfe3e0e2c57f"
    ]
  }
}

Task completed payload

event_type: task_completed_event

Payload

KeyTypeDescription
project_hashstringThe unique hash identifier for the Project.
label_hashstringThe unique hash identifier for the label.

Example payload:

{
  "uid": "950f9048-bd28-42e7-89cb-2538f27c1695",
  "version": 1,
  "source": "Encord",
  "event_type": "task_completed_event",
  "event_created_timestamp": "2024-03-25T15:13:40.456441+00:00",
  "payload": {
    "label_hash": "c25be5c7-2a7f-4de9-8afd-9e2f72064e84",
    "project_hash": "751a3a3e-46b6-4aad-b0b8-19b169f91266",
    "data_hash": "f5fb9c39-cbfc-4bf3-8fdb-95ee4da24c14"
  }
}

Full digest payload

event_type: digest_event

Payload

KeyTypeDescription
projects_statsarrayArray of ProjectStats objects

ProjectStats

Please note that the summary statistics are all for the time frame of the daily digest. This is currently 24h.

KeyTypeDescription
project_hashstringThe unique hash identifier for the project.
annotator_statsarrayArray of AnnotatorStats objects
reviewer_statsarrayArray of ReviewerStats objects

AnnotatorStats

KeyTypeDescription
user_emailstringEmail address of the user
submitted_labelsintegerNumber of labels the user has submitted
accepted_labelsintegerNumber of labels that were accepted by a reviewer
rejected_labelsintegerNumber of labels that were rejected by a reviewer
avg_time_for_addOptional[float]Average time it took for the annotator to add a label in milliseconds. This field is optional.
avg_time_for_editOptional[float]Average time it took for the annotator to edit a label in milliseconds. This field is optional.
rejection_ratefloatThe number of rejected labels over all rejected and approved labels.
total_add_or_edit_timefloatTotal time it took for adding and editing labels in milliseconds.

ReviewerStats

KeyTypeDescription
user_emailstringEmail address of the user
reject_countintegerNumber of labels rejected
accept_countintegerNumber of labels accepted
average_timefloatAverage time it took to process a single review
reviews_countintegerTotal number of reviews
total_timefloatTotal time spent on reviews
rejection_ratefloatRejected reviews over all reviews

Example payload:

{
  "uid": "f2d37310-19dd-4061-86b8-de20d932a8b1",
  "version": 1,
  "source": "Encord",
  "event_type": "digest_event",
  "event_created_timestamp": "2024-03-25T17:00:00.899873+00:00",
  "payload": {
    "projects_stats": [
      {
        "project_hash": "751a3a3e-46b6-4aad-b0b8-19b169f91266",
        "annotators_stats": [
          {
            "user_email": "[email protected]",
            "submitted_labels": 0,
            "accepted_labels": 1,
            "rejected_labels": 0,
            "avg_time_for_add": 495,
            "avg_time_for_edit": null,
            "rejection_rate": 0,
            "total_add_or_edit_time": 495
          }
        ],
        "reviewers_stats": [
          {
            "reject_count": 0,
            "accept_count": 1,
            "average_time": 1273,
            "user_email": "[email protected]",
            "reviews_count": 1,
            "total_time": 1273,
            "rejection_rate": 0
          }
        ]
      }
    ]
  }
}