Ontologies

Ontologies are hierarchical structures that define the top-level concepts and categories in your data, along with nested attributes for detailed annotations. They consist of Classes at the top level, which can be either Objects or Classifications.

• Objects: Used to label specific locations in a frame, such as a car in a video frame.
• Classifications: Frame-level labels that do not have a specific location, such as indicating if an image is taken during the day or night.
• Attributes: These can be nested under objects, classifications, or other attributes to create detailed, hierarchical structures. For example, the object "horse" can have an attribute "color".

Ontology Structure

Objects

Objects are configured with a title, an object annotation type, and optional attributes. You can also change their color.

👍

Tip

All objects can be marked as Required.

Here are the object annotation types:

• Bounding box: A quick-to-draw box shape compatible with many advanced automated labeling techniques.
• Rotatable bounding box: A rotatable box for more accurate labels than standard bounding boxes.
• Polygon: Captures complex shapes that bounding boxes can't. Known as segmentations, polygons cannot be self-intersecting.
• Polyline: An unclosed polygon for representing long, thin annotations.
• Keypoint: A simple geometric point for tracking small objects or specific points on larger objects.
• Bitmask: Creates complex shapes with a brush tool, useful for parts of a frame or image. Multiple threshold filters can apply bitmasks to specific areas.
• Object Primitive: Previously called Skeleton template. A collection of connected geometric points, ideal for representing complex shapes like those in pose estimation. Object primitives are created when editing an Ontology, not when setting up an Ontology.

Classifications

Since classifications apply to the entire frame, there is no need for specific colors or shapes. Classification annotation types include:

• Checklist: Allows multiple values. For example, "Weather" could be both cloudy and rainy.
• Radio: Allows a single value. For example, "Time of day" could be "Day" or "Night."
• Text: Allows freeform input for each situation.

ℹ️

Note

Radio buttons can nest up to 7 layers deep. Check boxes and text fields do not support nesting.

Attributes

Attributes can be nested under objects or any classification with a Radio annotation type. To nest attributes, set the type to Radio, then click the Configure button next to the value where you want to add a new attribute.

👍

Tip

Attributes can be marked as Required, Dynamic, or Relation.

Required

Any object, classification, or attribute can be marked as Required. This means annotators must include at least one instance of the required feature in each task before submitting.

👍

Tip

Ontology objects and classifications marked as Required are always listed alongside the symbol.

If annotators try to submit a task without including a Required object or classification, they will see the following warning message.

Click View issue to open the Issues drawer. From there annotators can seamlessly resolve all Required issues before moving onto the next task.

👍

Tip

All Required objects and classifications appear as items in the issues drawer of the Label Editor.

Dynamic attributes

Top-level attributes on objects can be marked as Dynamic. Normally, attributes are static, but marking one as Dynamic allows it to change value during a video. This is useful for indicating temporary attributes of an object. For example, a single instance of a person can be marked as "moving" in one part of the video and "stationary" in another.

👍

Tip

Refer to our this documentation to learn how to use dynamic attributes in the Label Editor, and apply dynamic attributes to a range of frames in a video.

Relation attributes

The Relation attribute allows you link objects, and specify the relationship between them using text regardless of the annotation type used.

ℹ️

Note

Only one of the linked objects need a Relation attribute for the objects to be considered linked. For example, consider 2 Ontology objects: a chicken, and a chicken wing. To allow these objects to be linked, a Relation attribute must be created for the chicken or the chicken wing, while setting up the Ontology.

All Relation attributes must be text fields. They cannot be radio buttons or checklists. Relation attributes can be applied to object labels of any kind, but not classifications.

To create a Relation attribute, enable the Relation feature when creating an attribute during Ontology creation. The default name for all Relation attributes is #relation.

ℹ️

Note

Objects are linked in the Label Editor during annotation, not during Ontology creation.

Using Relation attributes in the Label Editor

Once an Ontology with Relation attributes has been set up, instances can be linked in the Label Editor during annotation.

1. Create both instance labels. In this example a chicken and its wing have been labeled using bounding boxes.

2. Click the Edit classifications button for the object with the Relation attribute - in this example the chicken wing, as seen below.

3. Click the Set relation... bar and select the instance you want to link the selected instance to. In this example the chicken and the wing appear on the same frame, and therefore appear under the This frame heading. Instances in different frames appear under the heading Rest.

4. Click Done. The instances are now linked. This is shown in the Instance labels section with the name of the linked instance being displayed.

Apply to new occurrences

The Apply to new occurrences checkbox in the Label Editor is available for the first instance of a label with a dynamic attribute. Selecting this option propagates the attribute to all future instance labels, meaning all labels created for this instance share the attribute value of the initial instance label.

The 'Apply to new occurrences' functionality also holds for instance labels created automatically using interpolation.

JSON Ontology structure

You can preview a JSON of your Ontology structure when you are setting up or editing your Ontology. Preview the JSON by enabling the Display JSON toggle

Ontology best practices

Creating an ontology is a crucial step in developing effective machine learning applications. Keep the following considerations in mind when designing your Ontologies:

• The Problem Domain: Ensure your ontology is exhaustive, with a class or representation for all important concepts. Consider the appropriate level of detail. For example, an application recognizing various animals might have top-level classes like "cat" and "dog," while one focused on dog breeds might use "German Shepherd" and "Border Collie" as top-level classes.

• The Team: Use terminology that is clear and communicable across your entire team, including annotators, reviewers, project managers, algorithm developers, and other stakeholders.

• The Workflow: Annotation can be difficult and time-consuming. Design your Ontology to represent classes and their attributes appropriately, but also aim for efficiency. Ensuring objects and scenes can be labeled both accurately and quickly will lead to a more efficient labeling process.