- Approving or rejecting labels
- Adding and editing labels
- Adding Issues for Annotators to address
- Approving or rejecting Tasks
- Depending on your company’s process to create and validate labels, your role as a Reviewer might not include adding or editing labels or creating Issues for Annotators.
- Reviewers have the ability to reject or approve tasks even if all a task has Issues or rejected labels.
- Reviewers cannot skip tasks. Tasks in a REVIEW block can only be approved or rejected.
Review Standard Projects
Standard Projects can have multiple Annotators. Each Task is labeled by a single Annotator or Agent. Reviewers perform reviews on the Tasks and Approve or Reject according to their company’s standards and processes.Review Images and Videos
Use the timeline to help when reviewing videos.Review Audio Files
Use the timeline to help when reviewing videos. Show and hide audio regions while reviewing.Review Point Cloud Data
Review Text and HTML
Review PDFs
Review Multi-modal Tasks
Multi-modal tasks can include any type of data unit type. We recommend referring to the other data unit types to understand how to review multi-modal data units.Review Consensus Projects
Consensus Projects have multiple Annotators. Each Task is labeled by 2 or more Annotators or Agents. Reviewers review and select all labels on the Tasks from all Annotators. Reviewers then Approve or Reject Tasks according to their company’s standards and processes. Consensus Projects belong to the following categories:- Review and Refine: In Review and Refine Projects, you can select the best labels from any annotator. You can also edit or create new labels on the data unit.
- Determine Consensus: In Determine Consensus Projects, you specify the annotators that are in consensus for all labels on the data unit, and then you select the annotator with the best labels to apply to the data unit. You cannot add or edit any labels on the data unit.