Model inference
Inference is the process in which we use trained models as automated labeling agents, which can help speed up the labeling process significantly. If you don't have any trained models yet, head over to Training to learn the training process, or head to Models to get started creating models.
Assuming you have at least one trained model, read on to learn about running inference on Encord.
Accessing your models
The inference process is started from inside the label editor. Navigate to the label editor for your data asset of interest, and open the Automated labelling drawer by clicking the button in the lower left of the label editor screen. The Automated labelling drawer will pop out from the right. You can find models trained in classification, object detection and segmentation from the relevant dropdowns. Consult the image below as a reference.
Classification
To run inference with a classification model, select the model you want to run from the list of models under 'Classification' in the Automated labelling drawer.
Before running the predictions, configure the inference parameters. Specify the Classification range to set an inclusive range over which the inference process is run. Set the Minimum confidence threshold to control the required confidence value of any given prediction necessary to create a label. Finally, use the Advanced settings to choose whether or not the inference process will run on a GPU or CPU. The choice may be limited by the execution environment.
Nested classifications are only predicted when the 'parent' classification has already been made. Consider the case where we have a classification structure that asks 'Is there a car in the frame?' and if the answer is yes, then we also have a nested question of 'Is the car red?'. If you run a model that trains on 'Is the car red?', it will only return inference results for frames that you have classified as having a car in the frame. This ensures conditional relationships are always enforced with automated labelling as well.
Object detection and segmentation
Inference for object detection and instance segmentation are accessed from the same drop-down. If you're unsure of the type of a model, return to the Models tab to confirm your selection.
The normal parameters for running object detection and instance segmentation are the same as those for classification models. However, the Advanced settings dropdown adds a new parameter Intersection over union. The parameter specifies that any boxes or polygons with an amount of overlap higher than the specified threshold should be deleted.
Run inference and review results
After you've configured the inference parameters, press Run model to start the inference process. On screen queues will keep you informed as the process progresses.
When inference is finished, labels generated via inference will be rendered in the editor. The confidence of the predictions is shown above the object boundary. You can inspect the labels for quality and if you find that labels below a certain confidence threshold are not satisfactory, you can use the Bulk label operations (click the wrench icon in top menu) to remove all labels below a specified confidence threshold.
