The encord.objects.LabelRowV2 class is a wrapper around the Encord label row data format. It provides a convenient way to read, create, and manipulate labels.
To interact with Encord, you need to authenticate a client. You can find more details
here.
Specify a project
from pathlib import Path
from typing import List
from encord import EncordUserClient, Project
from encord.objects import (
AnswerForFrames,
Classification,
LabelRowV2,
Object,
ObjectInstance,
OntologyStructure,
RadioAttribute,
)
from encord.objects.common import Option
from encord.objects.coordinates import BoundingBoxCoordinates
from encord.objects.frames import Range
from encord.orm.project import Project as OrmProject
user_client = EncordUserClient.create_with_ssh_private_key(
ssh_private_key_path="<private_key_path>"
)
project_orm: OrmProject = next(
(
p["project"]
for p in user_client.get_projects(title_eq="Your project name")
)
)
project = user_client.get_project(project_orm.project_hash)
To inspect metadata for label rows (data units), such as the label hash, when the label was created, the corresponding data hash, or the creation date of the label.
label_rows: List[LabelRowV2] = project.list_label_rows_v2()
for label_row in label_rows:
print(f"Label hash: {label_row.label_hash}")
print(f"Label created at: {label_row.created_at}")
print(f"Annotation task status: {label_row.annotation_task_status}")
Inspect the filters in list_label_rows_v2() to only get a subset of the label rows.
You can find more examples around all the available read properties by inspecting the properties of the LabelRowV2 class.
Exporting labels
To export or download labels, or perform any other function that includes reading or writing labels, call the initialise_labels() method. This method downloads the current state of labels from the Encord server and create a label hash if none exists.
Once this method has been called, you can create your first label.
first_label_row: LabelRowV2 = label_rows[0]
first_label_row.initialise_labels()
Saving labels
Once initialise_labels() has been called, you can create your first label.
first_label_row: LabelRowV2 = label_rows[0]
first_label_row.initialise_labels()
...
first_label_row.save()
Editing labels
Editing labels after they have been created involves the following steps:
- Download existing labels.
- Find the attribute you are looking for, and change the attribute answer.
- Save the changes.
The script below shows how to update the location of an object’s bounding box, and attribute values. Make sure to substitute the following values:
\<private_key_path>
with the full path to your private key.
\<project_hash>
with the name of the project you want to update labels in.
<file_name>
with the name of the file / data unit you want to edit labels for.
<object_name>
with the name of the object.
<attribute_name>
with the name of the attribute.
<option_name>
with the name of the radio button option of the attribute.
from encord import EncordUserClient
user_client = EncordUserClient.create_with_ssh_private_key(
ssh_private_key_path="<private_key_path>"
)
project = user_client.get_project("<project_hash>")
first_label_row = project.list_label_rows_v2(
data_title_eq="<file_name>"
)[0]
first_label_row.initialise_labels()
ontology_structure = first_label_row.ontology_structure
first_obj_instance = first_label_row.get_object_instances()[0]
my_object = ontology_structure.get_child_by_title(title="<object_name>")
first_obj_instance.set_for_frames(
coordinates = BoundingBoxCoordinates(
height=0.1474,
width =0.1154,
top_left_x=0.1097,
top_left_y=0.3209
),
manual_annotation=False, overwrite=True
)
my_attribute = my_object.get_child_by_title(title="<attribute_name>", type_=RadioAttribute)
attribute_option = my_attribute.get_child_by_title(title="<option_name>", type_= Option)
first_object_instance.set_answer(attribute_option, overwrite=True)
first_label_row.save()
Creating/reading object instances
The encord.objects.LabelRowV2 class works with its corresponding Ontology. If you add object instances or classification instances, these will be created from the Ontology. You can read more about object instances here.
You can think of an object instance as a visual label in the Label Editor. One bounding box would be one object instance.
Finding the Ontology object
The LabelRowV2 is designed to work with its corresponding Ontology using the OntologyStructure. You must use the title or feature node hash to find the right objects, Classifications, attributes, or attribute options. See the example below to find the Ontology object for the demonstrative “Box of a human” object.
ontology_structure: OntologyStructure = first_label_row.ontology_structure
box_ontology_object: Object = ontology_structure.get_child_by_title(
title="Box of a human", type_=Object
)
Creating and saving an object instance
Creating and saving object instances is essential when importing annotations into Encord. Specify the label shape, the coordinates, and which frames the shape should occur on.
Inspecting an object instance
You can now get all the object instances that are part of the label row.
all_object_instances: List[
ObjectInstance
] = first_label_row.get_object_instances()
assert all_object_instances[0] == box_object_instance
assert all_object_instances[0].get_annotation(frame=0).manual_annotation is True
Adding object instances to multiple frames
Sometimes, you might want to work with a video where a single object instance is present in multiple frames. For example, you are tracking a car across multiple frames. In this case you would create one object instance and place it on all the frames where it is present. If objects are never present in multiple frames, you would always create a new object instance for a new frame.
coordinates_per_frame = {
3: BoundingBoxCoordinates(
height=0.5,
width=0.5,
top_left_x=0.2,
top_left_y=0.2,
),
4: BoundingBoxCoordinates(
height=0.5,
width=0.5,
top_left_x=0.3,
top_left_y=0.3,
),
5: BoundingBoxCoordinates(
height=0.5,
width=0.5,
top_left_x=0.4,
top_left_y=0.4,
),
}
box_object_instance_2: ObjectInstance = box_ontology_object.create_instance()
for frame_number, coordinates in coordinates_per_frame.items():
box_object_instance_2.set_for_frames(
coordinates=coordinates, frames=frame_number
)
box_object_instance_3: ObjectInstance = box_ontology_object.create_instance()
for frame_view in first_label_row.get_frame_views():
frame_number = frame_view.frame
if frame_number in coordinates_per_frame:
frame_view.add_object_instance(
object_instance=box_object_instance_3,
coordinates=coordinates_per_frame[frame_number],
)
Read access across multiple frames
As shown above with OPTION 1 and OPTION 2, you can think of the individual object instances and on which frames they are present or you can think of the individual frames and which objects they have. For a read access thinking of the individual frames can be particularly convenient.
for label_row_frame_view in first_label_row.get_frame_views():
frame_number = label_row_frame_view.frame
print(f"Frame number: {frame_number}")
object_instances_in_frame: List[
ObjectInstance
] = label_row_frame_view.get_object_instances()
for object_instance in object_instances_in_frame:
print(f"Object instance: {object_instance}")
annotation = object_instance.get_annotation(frame=frame_number)
print(f"Coordinates: {annotation.coordinates}")
Working with a classification instance
Creating a classification instance is similar to creating an object instance. The only differences are that you cannot create have more than one classification instance of the same type on the same frame and that there is no coordinates to be set for classification instances.
You can read more about classification instances here
Get the ontology classification
text_ontology_classification: Classification = (
ontology_structure.get_child_by_title(
title="Free text about the frame", type_=Classification
)
)
text_classification_instance = text_ontology_classification.create_instance()
Add the classification instance to the label row
text_classification_instance.set_answer(answer="This is a text classification.")
text_classification_instance.set_for_frames(frames=0)
first_label_row.add_classification_instance(text_classification_instance)
Read classification instances
all_classification_instances = first_label_row.get_classification_instances()
assert all_classification_instances[0] == text_classification_instance
Working with object/classification instance attributes
Both object instances and classification instances can have attributes. You can read more about examples here and here.
In the ontology you might have already configured text, radio, or checklist attributes for your object/classification. With the LabelRowV2, you can set or get the values of these attributes. Here, we refer to as “setting or getting an answer to an attribute”.
Answering classification instance attributes
The case for answering classification instance attributes is simpler, so let’s start with those.
You will again need to deal with the original ontology object to interact with answers to attributes. We have exposed convenient accessors to find the right attributes to get the attributes or the respective options by their title.
When working with attributes, you will see that the first thing to do is often to grab the ontology object. Usually, when calling the get_child_by_title the type_ is recommended, but still optional. However, for classifications this is often required.
The reason is that the classification title is always equal to the title of the top level attribute of this classification. Therefore, it is important to distinguish what exactly you’re trying to search for.
Text attributes
Answering text attributes is the simplest case and has already been shown in the section on classification instances above.
text_ontology_classification: Classification = ontology_structure.get_child_by_title(
title="Free text about the frame",
type_=Classification,
)
text_classification_instance = text_ontology_classification.create_instance()
text_classification_instance.set_answer(answer="This is a text classification.")
assert (
text_classification_instance.get_answer()
== "This is a text classification."
)
We encourage you to read the set_answer and get_answer docstrings to understand the different behaviours and possible options which you can set.
Checklist attributes
Assume we have a checklist with “all colours in the picture” which defines a bunch of colours that we can see in the image. You will need to get all the options from the checklist ontology that you would like to select as answers.
checklist_ontology_classification: Classification = ontology_structure.get_child_by_title(
title="All colours in the picture",
type_=Classification,
)
checklist_classification_instance = (
checklist_ontology_classification.create_instance()
)
green_option: Option = checklist_ontology_classification.get_child_by_title(
"Green", type_=Option
)
blue_option: Option = checklist_ontology_classification.get_child_by_title(
"Blue", type_=Option
)
checklist_classification_instance.set_answer([green_option, blue_option])
assert sorted(checklist_classification_instance.get_answer()) == sorted(
[green_option, blue_option]
)
Radio attributes
Let’s assume we have a radio classification called “Scenery” with the options “Mountains”, “Ocean”, and “Desert”.
scenery_ontology_classification: Classification = ontology_structure.get_child_by_title(
title="Scenery",
type_=Classification,
)
mountains_option = scenery_ontology_classification.get_child_by_title(
title="Mountains", type_=Option
)
scenery_classification_instance = (
scenery_ontology_classification.create_instance()
)
scenery_classification_instance.set_answer(mountains_option)
assert scenery_classification_instance.get_answer() == mountains_option
Radio attributes can be nested. You can read more about nested options here
If you have the Mountains scenery, there is an additional radio classification called “Mountains count” with the answers “One”, “Two”, and “Many”. Continuing the example above, you can set the nested answer like this:
mountains_count_attribute = mountains_option.get_child_by_title(
"Mountains count", type_=RadioAttribute
)
two_mountains_option = mountains_count_attribute.get_child_by_title(
"Two", type_=Option
)
scenery_classification_instance.set_answer(two_mountains_option)
assert (
scenery_classification_instance.get_answer(
attribute=mountains_count_attribute
)
== two_mountains_option
)
Answering object instance attributes
Setting answers on object instances is almost identical to setting answers on classification instances. You will need to possibly get the attribute, but also the answer options from the ontology.
car_ontology_object: Object = ontology_structure.get_child_by_title(
"Car", type_=Object
)
car_brand_attribute = car_ontology_object.get_child_by_title(
title="Car brand", type_=RadioAttribute
)
mercedes_option = car_brand_attribute.get_child_by_title(
title="Mercedes", type_=Option
)
car_object_instance = car_ontology_object.create_instance()
car_object_instance.set_answer(mercedes_option)
assert (
car_object_instance.get_answer(attribute=car_brand_attribute)
== mercedes_option
)
Setting answers for dynamic attributes
Dynamic attributes are attributes for object instances where the answer can change in each frame. You can read more about them here.
These behave very similarly to static attributes, however, they expect that a frame is passed to the set_answer which will set the answer for the specific frame.
The read access, however, behaves slightly different to show which answers have been set for which frames.
person_ontology_object: Object = ontology_structure.get_child_by_title(
"Person", type_=Object
)
position_attribute = person_ontology_object.get_child_by_title(
title="Position",
type_=RadioAttribute,
)
person_object_instance = person_ontology_object.create_instance()
person_object_instance.set_answer(
answer=position_attribute.get_child_by_title("Standing", type_=Option),
frames=Range(start=0, end=5),
)
person_object_instance.set_answer(
answer=position_attribute.get_child_by_title("Walking", type_=Option),
frames=Range(start=6, end=10),
)
assert person_object_instance.get_answer(attribute=position_attribute) == [
AnswerForFrames(
answer=position_attribute.get_child_by_title("Standing", type_=Option),
ranges=[Range(start=0, end=5)],
),
AnswerForFrames(
answer=position_attribute.get_child_by_title("Walking", type_=Option),
ranges=[Range(start=6, end=10)],
),
]
Dealing with numeric frames
In many places you can use encord.objects.frames.Range
to specify frames in a more flexible way. Use one of the many helpers around frames to conveniently transform between formats of a single frame, frame ranges, or a list of frames.