Ontology Structure

Ontologies are hierarchical structures that define the top-level concepts and categories in your data, along with nested attributes for detailed annotations. They consist of Classes at the top level, which can be either Objects or Classifications.

• Objects: Used to label specific locations in a frame, such as a car in a video frame.
• Classifications: Frame-level labels that do not have a specific location, such as indicating if an image is taken during the day or night.
• Attributes: These can be nested under objects, classifications, or other attributes to create detailed, hierarchical structures. For example, the object “horse” can have an attribute “color”.

​

Ontology Structure

​

Objects

Objects are configured with a title, an object annotation type, and optional attributes. You can also change their color.

The following object annotation types are available:

​

Classifications

Since classifications apply to the entire frame, there is no need for specific colors or shapes. Classification annotation types include:

• Checklist: Allows multiple values. For example, “Weather” could be both cloudy and rainy.
• Radio: Allows a single value. For example, “Time of day” could be “Day” or “Night.”
• Text: Allows freeform input for each situation.

​

Attributes

Attributes can be nested under objects or any classification with a Radio annotation type. To nest attributes, set the type to Radio, then click the Configure button next to the value where you want to add a new attribute.

​

Required Attributes

Any object, classification, or attribute can be marked as Required. This means annotators must include at least one instance of the required feature in each task before submitting.

If annotators try to submit a task without including a Required object or classification, they will see the following warning message.

Click View issue to open the Issues drawer. From there annotators can seamlessly resolve all Required issues before moving onto the next task.

​

Dynamic Attributes

Top-level attributes on objects can be marked as Dynamic. Normally, attributes are static, but marking one as Dynamic allows it to change value during a video. This is useful for indicating temporary attributes of an object. For example, a single instance of a person can be marked as “moving” in one part of the video and “stationary” in another.

​

Relation Attributes

The Relation attribute allows you link objects, and specify the relationship between them using text regardless of the annotation type used.

All Relation attributes must be text fields. They cannot be radio buttons or checklists. Relation attributes can be applied to object labels of any kind, but not classifications.

To create a Relation attribute, enable the Relation feature when creating an attribute during Ontology creation. The default name for all Relation attributes is #relation.

Using Relation attributes in the Label Editor:

Once an Ontology with Relation attributes has been set up, instances can be linked in the Label Editor during annotation.

1. Create both instance labels. In this example a chicken and its wing have been labeled using bounding boxes.

2. Click the Edit classifications button for the object with the Relation attribute - in this example the chicken wing, as seen below.

3. Click the Set relation… bar and select the instance you want to link the selected instance to. In this example the chicken and the wing appear on the same frame, and therefore appear under the This frame heading. Instances in different frames appear under the heading Rest.

4. Click Done. The instances are now linked. This is shown in the Instance labels section with the name of the linked instance being displayed.

​

Apply to new occurrences

The Apply to new occurrences checkbox in the Label Editor is available for the first instance of a label with a dynamic attribute. Selecting this option propagates the attribute to all future instance labels, meaning all labels created for this instance share the attribute value of the initial instance label.

The Apply to new occurrences functionality also holds for instance labels created automatically using interpolation.

​

Transcript Attributes

Transcript attributes are text fields specifically designed for transcribing audio files. Apply Transcript attributes to Audio Region objects and input the audio transcription directly into the text field. In the following example an Audio Region object called “Beautiful Voice” has the Transcription text attribute “Lyrics”, which can be used to transcribe song lyrics.

​

JSON Ontology Structure

You can preview a JSON of your Ontology structure when you are setting up or editing your Ontology. Preview the JSON by enabling the Display JSON toggle.

​

Ontology Best Practices

Creating an Ontology is a crucial step in developing effective machine learning applications. Keep the following considerations in mind when designing your Ontologies:

• The Problem Domain: Ensure your Ontology is exhaustive, with a class or representation for all important concepts. Consider the appropriate level of detail. For example, an application recognizing various animals might have top-level classes like “cat” and “dog,” while one focused on dog breeds might use “German Shepherd” and “Border Collie” as top-level classes.

• The Team: Use terminology that is clear and communicable across your entire team, including annotators, reviewers, project managers, algorithm developers, and other stakeholders.

• The Workflow: Annotation can be difficult and time-consuming. Design your Ontology to represent classes and their attributes appropriately, but also aim for efficiency. Ensuring objects and scenes can be labeled both accurately and quickly will lead to a more efficient labeling process.