Watch the video, or follow the step-by-step guide below to learn how to create annotation projects.

Creating annotation projects

Under in the 'Annotate' section in the navigation bar, select 'Annotation projects'. Click the + New annotation project button to create a new project.

1. Enter details

In the 'Enter details' screen, enter a project title and optional description. A clear title and description help keep your projects organized.

If you part of an organization, you will see an optional project tags drop-down. Project tags are useful for categorizing your projects. Select as many tags as are relevant to your project.

When you're ready to continue setting up your project, click Next.

2. Add datasets

The 'Add datasets' screen is where you attach datasets to your project. You can select from a list of datasets you have already created by clicking Add next to a dataset, or create a new dataset by clicking the + New dataset tab to initiate the dataset creation flow. Attached datasets will appear in a list on the right.

3. Select ontology

The 'Select ontology' screen is where you specify your ontology.

ℹ️
Note
DICOM customers might be more familiar with the term 'labeling protocol', which is equivalent to an ontology.

Encord offers support for objects within a frame, as well as frame level classifications, with nested attributes if need be.

You can reuse a pre-existing ontology by clicking Select next to the ontology, or create a new one by clicking the + New ontology tab. A preview of the label editor sandbox for your chosen ontology will be shown on the right side of the screen.

Click Next step to continue creating your project.

4. Select your QA type

Select what type of Quality Assurance (QA) you would like for your project.

Manual QA requires reviewers to manually review labels created by annotators. Use the 'Sampling rate' slider to adjust the percentage of labels that will be manually reviewed. Click here for more information on manual QA and its setup.
Automated QA uses an already existing manual QA project as a 'benchmark' to automatically review labels created by annotators. The manual QA 'benchmark project' needs to be complete, and use the same ontology as your automated QA project.
- Dynamic Benchmark gives you control over how labels in a frame are scored against the benchmark. [Click here] to learn more about setting up and editing dynamic benchmark scoring functions.
- Single frame-level classification only evaluates frame-level classifications, not labels within a frame.

5. Create the project

Once you are happy with your choice of datasets, ontology, and QA type - click Create project to complete the process.

Working with annotation projects

There are two types of annotation projects:

Annotation projects with manual quality assurance (QA) are projects where labels are reviewed manually by reviewers.
Annotation projects with automated quality assurance (QA) are be reviewed automatically. This method uses an already existing manual QA project to create a 'benchmark' to automatically review labels created by annotators. Please note that the benchmark project needs to have been completed, and use the same ontology you selected for your automated QA project.

Click here for a step-by-step guide to automated QA projects.

Roles and permissions

Roles and permissions vary based on the type of QA your project has.

Manual QA

Permission	Admin	Team Manager	Reviewer	Annotator	Annotator & Reviewer
Attach / Detach datasets	✅	❌	❌	❌	❌
Attach / Switch ontology	✅	❌	❌	❌	❌
Delete	✅	❌	❌	❌	❌
Invite team members	✅	✅	❌	❌	❌
Manage team permissions	✅	✅	❌	❌	❌
Manage admins	✅	❌	❌	❌	❌
Annotate & review tasks in task management system	✅	✅	Review only	Annotate only	✅
Confirm annotations outside of the task management system	✅	✅	❌	❌	❌
Control assignments and status in task management system	✅	✅	❌	❌	❌

Automated QA

Permission	Admin	Team Manager	Annotator
Attach / Detach datasets	✅	❌	❌
Attach / Switch ontology	✅	❌	❌
Benchmark QA	✅	❌	❌
Delete	✅	❌	❌
Invite team members	✅	✅	❌
Manage team permissions	✅	❌	❌
Manage admins	✅	❌	❌
Annotate tasks in the task management system	✅	✅	✅
Control assignments & status in the task management system	✅	✅	❌
View your team's performance	✅	✅	❌

Project dashboard

Video Tutorial - Monitoring annotation progress

Clicking on a project in the 'Projects overview' screen takes you to its 'Project dashboard'.

This is where you monitor and manage your project. For example, you can view your project's summary statistics, manage labeling tasks, view your team's productivity, train models and invite collaborators.

The dashboard is split into 7 tabs:

Summary: a high-level view of labeling and productivity statistics.
Explore: a quick way to explore the distribution of instances and labels across data assets in the project.
Labels: for managing all the project's labeling activity and tasks.
Performance: a more detailed view of your team's manual labeling and productivity.
Models: for administering models in this project.
Export: for exporting your labeling data.
Settings: editing project options, ontology, team collaborators, and other general project settings.

Summary - Manual QA

Clicking an annotation project takes you to its summary dashboard. This dashboard has 2 components and gives you a rich visual display of your project's progress at a high level.

A. Project task status overview: summary of your task statuses.
B. Instance label task status: summary of the labels in your tasks.

Project task status overview

Displays the number of annotation tasks that are in each state: Annotate, Review or Completed.

Annotate: The task is ready to be annotated.
Review: The task is ready to be reviewed.
Completed: The task has been annotated, and reviewed. There is no further action to be taken.

Instance label task status

Displays the number of labels / instances that have been created, and their assigned status.

Approved - The instance has been approved by a reviewer.
Returned for annotation - The instance has been returned to the annotator by the reviewer.
In review - The instance needs to be reviewed.

For a more comprehensive summary of how a task moves from annotation through instance review and full completion, reference the Status section below.

Summary - Auto QA

Clicking an annotation project takes you to its summary dashboard. This dashboard has 2 components and gives you a rich visual display of your project's progress at a high level.

'Project task status overview' displays the number of annotation tasks that are in either the Annotate, or Completed state.

Annotate: The task is ready to be annotated.
Completed: The task has been annotated, and automatically reviewed. There is no further action to be taken.

Explore

The 'Explore' tab provides interfaces to help you understand how project's annotations are distributed amongst the data assets at both an instance and label level. It allows a deeper exploration through attributes on objects, as well as frame-level classifications.

Instance statistics: Class distribution across data assets in the given project.
Label statistics: Label distributions within data assets, objects and classifications.

Instance statistics

This section provides the total count of all instances across the datasets in your project.

Project total: Shows total instances (both objects and classifications) across the project by default. To get instance statistics for individual data files, click the drop-down to select a data file.
Select class: Shows the total instances for a particular class. This is a summary of how a given class is
distributed across your project's data assets. The pie chart segments show a breakdown of how that class is split across the data assets.
Display timestamps: Flip the toggle to switch between frame numbers and timestamps for the labels.

Label statistics

This is a summary of how your labels are distributed across the project. The pie chart shows a breakdown of how many labels there are for a given class.

Project total: Shows the total number of labels across different datasets in the project. To get label stats for individual data files, click the drop-down to select a data file.
Objects: Click on the pie chart segment of a class to see the total number of labels and its attributes (sometimes called nested attributes) if available for that class.
Classifications: Shows the global classification at project or individual video level. For example, location, time of day, etc.

Quick definitions of classes, instances and labels

Class: Fundamental unit of the project's ontology. For example the ontology of a project annotating traffic videos could have classes such as Car, Truck, Bicycle, and so on. For more information on objects and classifications, see Ontologies Overview.
Instance: Specific occurrence of a class. Car(0) is an instance of the Car class, for example, it could be a specific black sedan. The single Car(0) instance can appear in a single frame or a range of frames. Therefore, instances may contain multiple labels across frames.
Label: An frame-specific annotation of an instance. For example the annotation of Car(0) on frame 201 is a label.

Labels

The labels page is your gateway to annotating, reviewing, and auditing the labels made against all the datasets in your project.

Access to each pane will depend on the user's project role. We quickly summarize the purpose of each tab, and the roles which can access each below.

The labels dashboard features the following tabs:

Activity: View all the tasks that are either In review or Completed, as well as confirm the status of
the labels within tasks by clicking View.
Queue: The Queue tab is where all labeling and review actions are initiated, regardless of a user's role within the project. It pulls tasks from the Task Management System, therefore ensuring collaborators' efforts don't overwrite each other.
Data: Use the Data tab to get an overview of every data asset in the project, regardless of status within the Task Management System.
Instances: The Instances tab lets you use the unique instance identifier to search the project for a specific instance, and jump directly into the editor to confirm the status of an annotation visually.

Access to each tab is associated with the various project roles as follows:

Role	Activity	Queue	Data	Instances
Annotator	✅	✅	❌	❌
Reviewer*	✅	✅	❌	❌
Annotator + Reviewer*	✅	✅	❌	❌
Team Manager	✅	✅	✅	✅
Admin	✅	✅	✅	✅

* Please note that projects with Auto QA do not have these project roles.

Activity

The activity screen allows you to quickly monitor annotation and review activity in your project by showing tasks and providing a summary interface. In manual QA projects the status of reviewed labels inside each task can also be seen. Tasks are displayed in most recently edited order from top to bottom.

A. File, Search, & Reopen: The name of the specific data unit or data asset. This is the same as the name in the dataset to which this data asset is a part of. Use the search box to filter the list by file name, and send tasks back to annotation using the 'Reopen' feature.
B. Dataset: The dataset the data asset belongs to.
C. Type: The type of the data, such as an image or video. For more on our supported data types, see our label editor documentation.
D. Status: The status of this task within the Task Management System.
E. Frames: The number of frames in the data asset. For a DICOM series, this will be the number of slices.
F. Reviews: How many annotation instances are in this data asset. This is only available for projects with manual QA.
G. Submitted: Indicates when the last submit action, whether for an annotation or review, was made against any of the labels in this data asset.
H. Submitted by: Who last submitted the annotations.
I. Reviewed by: Who submitted the most recent review.
J. Actions: Click the 'View' link to open the label editor. Note: this feature is only available to Team Managers and Administrators as an extra method of reviewing submissions outside the TMS. We advise extra caution if you decide to edit the labels from this interface. If significant work needs to be done, we strongly recommend to 'Reopen' the task to prevent possible errors from simultaneous edits.
K. Filter: Use the filter drop-down to only show tasks with the selected status.

File, Search, and Reopen

Please note that any references to 'Review' in this section apply to projects with manual QA only.

The file column shows the name of the data asset. For files uploaded via the GUI, they keep the name they were uploaded with. For files added from your cloud storage, this will normally be the path under the bucket they are stored on.

Use the search interface to quickly filter and display only those tasks with file names matching your desired text. Even partial matches will be shown. For example: searching "fly" will return file names containing "flyover" and "flyaround."

The 'Reopen' button allows Administrators and Team Managers to send tasks which are currently Completed or In review back to annotation. Select your target tasks using the checkboxes in the File column to select individual assets, or select the checkbox in the column header to select all tasks, and press the 'Reopen' button to move all selected tasks back to the annotation stage. Tasks reopened in this way will have the status 'Returned,' in the 'Queue' tab. No labels are lost by reopening a task. The 'Reopen' action is only applied to tasks which are both visible (i.e. not filtered out by the file search) and selected.

Status

This column shows the status of this task within the Task Management System. The Activity pane only shows assets which have had some action done on them, and therefore only reflects tasks with the following statuses:

In review (for manual QA projects only): The annotation task has been submitted but outstanding review tasks remain. In review task status is shown in blue.
Completed: The annotation task has been submitted and all reviews have been completed. Completed task status is shown in green.

For a comprehensive summary of the possible task states, see the status section of the Data tab, below.

Reviews

This column is only available for projects with Manual QA

The 'Reviews' column shows a count of how many instances have been reviewed in a given data asset. Click the number to open a panel which shows the last review action taken on each instance, as well as who originally created the annotation and when. Note that unless the review was done by an 'Expert Reviewer', all reviewed annotations must be either 'Approved' or 'Deleted' before a task can be 'Completed.' Read more about the Expert Review feature here.

Queue

The 'Queue' is where annotators and reviewers look to find their next task. The 'Start labeling' and 'Start reviewing' buttons visible throughout the project open the label editor with the next task in the queue according to the relevant task type.

This tab can be used to assess the number of tasks assigned to you as an annotator or reviewer and therefore estimate your likely workload. Administrators and Team Managers can also use it to quickly verify the current assignments per team member, and change assignments as necessary.

A. File, Search, & Assign: The name of the specific data unit or data asset. This is the same as the name in the dataset to which this data asset is a part of. Use the search box to filter the list by file name, and send tasks back to annotation using the 'Reopen' feature.
B. Dataset: The dataset the data asset belongs to.
C. Type: The type of the data, such as an image or video. For more on our supported data types, see our documentation here.
D. Status and Task: The status and category of this task within the Task Management System.
E. Last Edited: When the task was last edited
F. Reserved by: Who the task has been assigned to or reserved by
G. Actions: Depending on your collaborator role, you can initiate or reassign the task.
H. Filter: Use the filter drop-down to only show tasks of the selected status.

File, Search, and Assign

The 'Assign' button allows Administrators and Team Managers to allocate unassigned tasks to specific collaborators for annotation or review. Select your target tasks using the checkboxes in the File column to select individual assets, or select the checkbox in the column header to select all tasks, and press the 'Assign' button open the task assignment popup.

Confirm the selected tasks are as intended, then select the target collaborator from the drop-down and press assign. Tasks which have already been assigned to another collaborator, as indicated by the email in the 'Reserved by' column, can not be reassigned until they have first been released.

Status and Task

The 'Queue' tab only shows tasks which have remaining annotation or review work to be done within the Task Management System. Therefore, the stage of the task within the TMS is understood by reading the Status and Task columns together.

The two types of tasks are 'Annotate' and 'Review' which can be in any of the following states:

Queued: The task is ready for annotation or review. For an annotation tasks to be 'Queued' it must not be assigned to a user, and must have no submitted labels. It may have been previously assigned to a user, but subsequently released before any annotations were submitted.
Assigned: The annotation or review task is assigned to a specific user.
Returned: The annotation task was previously submitted, and either 'reopened' after completion by a Team Manager or Administrator, or rejected by the reviewer in manual QA projects.

Actions

There are two relevant actions that can be done on each task from the 'Queue' pane. Press 'Initiate' to open the label editor and proceed with annotation or review, depending on the task type.

Additionally, Administrators and Team Managers can click the three vertical dots to open the expanded menu, to access the 'Release task' function. Tasks must be explicitly released before they can be reassigned.

Data

The Data tab gives a complete overview of all the data asset tasks in the project, regardless of their progress through the Task Management System. Therefore, this is the first place Administrators and Team Managers should check if they want to confirm the status of a given task.

A. File & Search: The name of the specific data unit or data asset. This is the same as the name in the dataset to which this data asset is a part of. Use the search box to filter the list by file name.
B. Dataset: The dataset the data asset belongs to.
C. Type: The type of the data, such as an image or video. For more on our supported data types, see our documentation here for more details.
D. Status: The status of this task within the Task Management System.
E. Frames: The total frames in this data asset. This will apply to videos, image sequences and DICOM. Images always only have 1 frame.
F. FPS: the frames per second of the data asset. This only applies for data of type video. Others will show a dash (-).
G. Created: When the task was created. Tasks are created when the dataset containing the data asset is attached to the project.
H. Last edited by: the last collaborator to edit the task in any capacity (such as annotate or review), and when.
I. Actions: The 'Data' tab allows users to view the task in the label editor, as well as get a code snippet for using the SDK with this task, and confirming the edit actions via the Activity Log.
J. Filter by: Use the filter drop-down to view only tasks with the selected Status.

👍
Tip
Confused about the difference between image groups and image sequences? See our documentation here to learn about different data types in Encord.

File and Search

Status

The data tab provides the most comprehensive overview of all the tasks associated with each data asset in a given project. As such, this is the first place to check to see the status of various tasks.

Queued: The task is ready for annotation. For a task to be 'Queued' it must not be assigned to a user, and have no submitted labels. A queued task may have been previously assigned to a user, but subsequently released before any annotations were submitted. Queued tasks are shown in light orange.
Assigned: An annotation task has been assigned to a specific user. Assigned tasks are shown in aqua green.
In review (For manual QA projects only): The annotation task has been submitted but outstanding review tasks remain. In review task status is shown in blue.
Returned: The task was previously submitted, and either several of the annotations were rejected by the reviewer or it was 'reopened' after completion by a team manager or administrator.
Completed: The annotation task has been submitted and all reviews have been completed. Completed task status is shown in green.

Actions

Clicking View will drop you into the label editor to do a live audit of the annotations in this data asset. The 'Data' tab is only visible to Administrators and Team Managers and so grants great power to view any data asset, however appropriate care must be taken to ensure annotations are not simultaneously edited from the 'Queue' pane by an annotator or reviewer. Encord advises edit actions are NOT taken from the Data tab unless you have received confirmation no one else is concurrently editing the asset.

🚧
Caution
In order to prevent any possible issues of annotator work being overwritten, it's critical that all annotations are done via the Task Management System's Queue tab, and only the person assigned to the task makes annotations at any given time.

Other possible actions include 'API Details' which show a popup with sample code you can use to get started with our SDK to access this particular data asset, often known as a label row in the SDK. Click 'Activity log' to see a popup with a graphical summary of add / edit / delete actions on this data asset indexed by annotator or ontology class. Click 'Display logs' in the lower right to show all actions in reverse chronological order.

Instances

The 'Instances' tab allows Administrators and Team Managers to search within the data to directly find specific instances. Recall that an annotation instance correlates to a unique instantiation of a specific ontology class in a data asset.

For example, if you have the 'Person' class in your ontology, the first instance of a 'Person' in a given data asset will be indicated in the interface as 'Person (0)', the second as 'Person (1)' and so on. Instances, therefore, can exist in multiple frames of a data asset, and indicate the same object. Use the 'Instances' tab to search for specific instances of objects or classifications using their Identifier.

Instance identifiers are unique at the project scope, and can be found in any of the following ways:

From inside the label editor, by clicking on a particular instance, and then selecting 'Copy identifier' from the instance action menu.
From inside exported labels, where they are known as the objectHash or classificationHash as appropriate.
When uploading labels using the SDK, you may specify your own objectHash or classificationHash.

Once you have an identifier of interest, use the 'Search instance' interface to filter the instances by identifier to quickly find the instance you're interested in. This can be particularly handy when you want to visually confirm an annotation you may not have seen before, but for which you have the identifier.

After locating your instance of interest, click View from the 'Actions' column to jump deeply into the dataset straight to where the instance is first annotated.

Performance

You can switch between a summary, or detailed view of performance metrics using the toggle found at the top of the 'Performance' tab. Please note that the 'Details' tab will vary depending on the type of QA your project has - with both cases documented below.

Manual QA performance details
Auto QA performance details

Performance - Summary

The 'Summary' tab of the performance dashboard provides an overview of your team's manual labeling and productivity.

Task actions over time

View the number of tasks in a project that have been approved, rejected, and submitted for review over a given period of time.

The height of a bar represents the total number of tasks.
The height of each color within a bar represents the number of approved, rejected, and submitted tasks.

A: Set the time period you would like to see displayed by selecting a range of dates.
B: The Hide days without any actions toggle removes all days at which no actions were taken from the view.
C: Download a CSV file of the data.
D: Display the data as a bar chart, or a table. While the chart provides a clear visual representation, the table provides exact figures for a more detailed picture of your team's performance.

Instance Label actions over time

View the number of instance label actions in a project that have been approved, rejected, and submitted for review over a given period of time.

A: Set the time period you would like to see displayed by selecting a range of dates.
B: Download a CSV file of the data.
C: Display the data as a bar chart, or a table. While the chart provides a clear visual representation, the table provides exact figures for a more detailed picture of your team's performance.

Within your specified time period, you can choose which dates to display by using the slider located beneath the graph.

Team collaborators

The 'Team collaborators' section shows the duration of time each project collaborator spend working on a given file.

A. 'Data file' displays session time collaborators spent working on individual files. 'Project' displays session time collaborators have spent working on the project.

B. Table entries can be filtered according to dates by clicking the range of dates, and selecting the start and end date of the period you would like to see table entries displayed for.

C. Table entries can be downloaded in CSV format by clicking the Download CSV button.

D. When lots of entries are present they will be split across a number of different pages. The number of table entries per table can be adjusted.

Performance - Details (Manual QA)

The 'Details' tab of the performance dashboard gives a more detailed view of your team's labeling and productivity. This section will cover manual QA projects.

👍
Tip
You can specify a range of dates, as well as whether statics should be displayed for labels, or instances. More information on instances and labels can be found here.

Submissions chart

The submissions chart displays the number of submitted labels or instances over the specified time period. The chart can be filtered to show submissions for specific annotators or classes.

If you filter on both Annotators and Classes then the resulting chart will show the submission statistics for the selected annotators and for selected labels.

Reviews chart

The reviews chart displays the cumulative number of accepted and rejected labels or instances over the specified time period.

Annotators' table

The annotators' table displays all the relevant statistics for all annotators in a project. It can be filtered on classes to show annotator statistics only for the selected classes.

User: The annotator's email.
Rejection rate: Percentage of their labels or instances that have been rejected in the review process.
Submitted labels / instances: Number of labels or instances that the annotator has submitted for review
- Repeated submissions are not counted.
Accepted labels / instances: Number of labels or instances that the annotator created that passed the review process.
Rejected labels / instances: Number of labels or instances that the annotator created that we're rejected during the review process. Note that this can be higher than the number of submitted labels / instances since a label or instance can be rejected multiple times during the review process but the submission will only be logged once.
Total session time: Time spent labeling.

Reviewers table

User: The reviewers email.
Rejection rate: Percentage of labels or instances that they rejected in the review process.
Accepted labels / instances: Number of labels or instances that the reviewer accepted.
Rejected labels / instances: Number of labels or instances that the reviewer rejected.
Total session time: Time spent reviewing.

Objects and classifications table

Each row in the objects and classifications table can be expanded to show statistics on attributes.

Class: The class name.
Rejection rate: Percentage of labels or instances rejected in the review process.
Reviewed labels / instances: Number of labels or instances of the class that have gone through the review process.
Accepted labels / instances: Number of labels or instances of the class that have passed the review process.
Rejected labels / instances: Number of labels or instances of the class that failed the review process.
Avg. time to annotate: Average time spent annotating this class.

Performance - Details (Auto QA)

The 'Details' tab of the performance dashboard gives a more detailed view of your team's labeling and productivity.

👍
Tip
You can specify a range of dates, as well as whether statics should be displayed for labels, or instances. More information on instances and labels can be found here.

Training performance

The 'Training performance' section provides you with insights into labeling activities of annotators in the project.

Click the Re-calculate scores button to re-calculate all benchmark scores displayed on this page.
Click the Download CSV button to download a CSV file of these records.

Overview tab

The 'Overview' tab can be seen in the screenshot above. It lists the following information:

User: The user's email address.
Total benchmark tasks: The number of tasks in the benchmark project.
Submitted benchmark tasks: The number of tasks submitted belonging to the benchmark dataset.
Benchmark results: The user's percentage score against benchmark labels.
Progress: The percentage of benchmark tasks completed relative to the total.

Annotator submissions

👍
Tip
Click the + button next to a user's email address to reveal the Annotator submission details.

The 'Annotator submissions' tab lists the following information:

Annotator: The user's email address.
Data Unit: The task's file name.
Instance: The ontology instance being referred to.
Benchmark answer: The benchmark classification and label. To view the label, click .
Annotator answer: The annotator's classification and label. To view the label, click .
Score: A score measuring the annotator's label when compared to the benchmark label.

Annotator session time

The 'Annotator session time' table displays the duration of time each annotator spent labeling the tasks in their queue.

Data file: The task's file name.
User: The user's email address.
User role: The user's role.
Session time: Time spent labeling tasks.

Submissions

The 'Submissions' graph displays there number of submitted labels within a given range of time.

Automated labeling

For information on automated labeling, please see our documentation here.

Export

Use the 'Export' tab to export your data. Please see our exporting data page to learn how to do this.

Settings

The Settings tab allows you to make modifications to your project using the following tabs:

Options - Copy a project, modify datasets, modify ontology, upload annotation instructions, modify project tags, QA settings.
Team - Manage collaborators on a project.
Danger zone - Delete your project.

Options

Copy a project, modify datasets, modify ontology, upload annotation instructions, modify project tags, QA settings.

Copy a project

To copy a project, click on the Copy project button in the 'Options' tab under Settings. This will open the copy project window, where you can pick the various parts of your project you want to copy over into your new project.

1. Select copy options

Choose the parts of your project you want to copy.

You can copy any combination of the following assets:

Labels: this will copy the labels within videos and image sequences of your choice.
Models: this will copy all the models in your project along with their training logs.
Collaborators: copy all project users with their respective roles. Project admins are copied regardless of whether this is selected or not.
All datasets: all datasets will be copied, and new annotation tasks will be created for all videos and image sequences if their labels were not copied over (see next line).

👍
Tip
Confused about the difference between image groups and image sequences? See our documentation here to learn about different data types in Encord.

The new annotation project will use the same ontology as the original. This can be changed in the project settings if required.

If you don't want to copy labels, press Copy project. This will create the copy of your project, which you can then access in the 'Projects' tab.
If you choose to copy over labels, you will be asked to select the data assets for which you would like labels copied over. To begin the process, press Next: configure labels. Continue to step 2. below.

2. Select data assets

Select the files whose labels you want to copy over. After doing so, click Next.

3. Configure labels

Select the statuses of the files you want copied over into your new project.

ℹ️
Note
When a project is copied, the task status will not be copied.
This means that all tasks will be Annotate tasks, and their status will be Queued.
All tasks will have to be re-assigned after being copied.

Click the Copy project button to complete the process.

Upload annotation instructions

Video Tutorial - Uploading annotator instructions

Click the Add instructions button to upload instructions for your annotators in PDF format.
To ensure the best possible results, provide as much detail as possible about what you would like annotated and how precise bounding boxes should be drawn. For example, instead of saying 'person', consider defining what should constitute a person for your annotators - only a full person? A torso? Or should any part of a person in a frame be labeled as a 'person'?

ℹ️
Note
The more specific your annotator instructions, the higher the chances that your annotators will perform well.

Once uploaded, annotation instructions will be accessible within the Label Editor.

Project tags

ℹ️
Note
Tags are created and managed on the Organization level. Once created they can be added to individual projects.

You can add tags to a project if you are part of an organization.

Project tags allow you to:

Flexibly categorize and group your projects.
Filter your projects.

Adding and removing tags

You can add tags to your projects in:

When creating a project.
In the Settings page of a project. This process is described below.

To add tags to your projects in the Settings page, navigate to the 'Options' tab and click the 'Project tags' drop-down. Here you will see the available tags in your organization. Click on a tag to add it to a project. You can remove a tag from your project by clicking the same tag again, or clicking the x button next to its name.

Filtering projects by tags

You can filter your projects based on the tags they contain. To do so, click on the 'Projects tab' in the Navigation bar, click the 'Filter by tags' drop-down and select one or more project tags. This will result in only projects with the tags being selected being displayed.

Edit project ontology

You can view or switch the ontology attached to your project.

ℹ️
Note
Changing the ontology can render existing labels invalid and lead to data inconsistency.

Click the Switch ontology button to switch the ontology linked to your project.
The resulting pop-up allows you to choose an existing ontology from a list, or create a new ontology for this project.
Click the View ontology button to view the details of the ontology that is attached to the current project.

Edit datasets attached to a project

The 'Datasets' section allows you to attach or detach any number of datasets to your project. You will have to create a new dataset in the 'Datasets' section for it to become available in a project's settings.

Quality Assurance (Manual QA)

The 'Quality' section allows you to configure the way that manual quality assurance is implemented for a given project.

ℹ️
Note
For precise descriptions of all the options available to you, please see our Quality assurance section.

The Sampling rate slider determines the percentage of labels that will be manually reviewed. Clicking Configure sampling rate allows you to set the sampling rate for each label type, or annotator separately.
The Multi review assignment enabled toggle will assign all labels created for a given task to the same reviewer.
Default rejection reasons allows you to add commonly used reasons for rejecting a label, to make them available to your reviewers and save time when reviewing tasks.
Toggle Reviewer mapping and click Configure reviewer mapping to assign classes, or labels made my specific annotators, to a particular reviewer.
Toggle Expert reviewer rule to enable expert review.

Quality Assurance (Automated QA)

The 'Automatic QA' section in the 'Options' pane allows you to configure the way that quality assurance is implemented.

ℹ️
Note
For a precise description of automated QA please see our automated QA documentation.

The Benchmark project source allows you to see which project is being used as the gold standard benchmark function against which QA is done.
Click Edit next to the Benchmark scoring function to adjust the relative weight of the IoU and Category evaluation metrics. Click Save when you are done.

ℹ️
Note
The results of the Automated QA can be viewed on the Performance dashboard.

Team

To manage project collaborators, select the 'Team' pane in your project Settings.

Here you can invite collaborators to the project, and configure their roles.

1. Add a collaborator

To invite collaborators to your project, click the + Invite collaborators button. This will open a new window where you can enter email addresses of the people you would like to invite.

Once you have entered everybody you want to invite, press the Add button.

2. Add collaborators as a group

ℹ️
Note
To add collaborators as a group, your organization needs to have user groups. Navigate to our documentation on creating user groups for more information.

Collaborators can be added to a project as a group - which can save time as well as ensure that no individual is forgotten.

In the 'Groups' section of the page, click on Manage to make the 'Manage Groups' pop-up appear.

Click the 'Select group' drop-down and pick a group you would like to add as collaborators. After selecting a group, click the 'Select Role' drop-down to assign a role to the group of collaborators. Click Add to add the group.

The group you just added will appear under the 'Added groups' heading. Repeat the process if you'd like to add more groups with different roles to the project.

To delete a group from the project, simply click the button next to the group name.

2. Change collaborator role

A project admin can modify the different roles of collaborators, using the drop-down on the right.

You can assign the following roles to collaborators:

Annotator: annotators are responsible for labeling. This is the default role for all collaborators.
Reviewer: for reviewing labeled tasks.
Annotator & reviewer: a combination of annotator and reviewer.
Team manager: a team manager can assign tasks to other users, and add collaborators to the project.
Admin: gives this collaborator full administrative control over this project. Caution: this is an irreversible action.

* Please note that Reviewer and Annotator + Reviewer roles are only available for projects with manual QA.

Please confirm or cancel your selection when making a collaborator a project admin.

Danger zone

You can delete your project by going to the 'Danger zone' tab at the bottom of the menu, and clicking the red Delete project button, shown below.

Deleting your project does not delete the datasets in the project, but will delete the project's labels and ontology.