Heuristic

Work on images or individual video frames and are heuristic in the sense that they mostly depend on the image content without labels.

Title	Metric Type	Data Type
Area - Ranks images by their area (width/height).	image
Aspect Ratio - Ranks images by their aspect ratio (width/height).	image
Blue Values - Ranks images by how blue the average value of the image is.	image
Blur - Ranks images by their blurriness.	image
Brightness - Ranks images by their brightness.	image
Contrast - Ranks images by their contrast.	image
Green Values - Ranks images by how green the average value of the image is.	image
Inconsistent Object Classification and Track IDs - Looks for overlapping objects with different classes (across frames).	sequence	bounding box, polygon, rotatable bounding box
Missing Objects and Broken Tracks - Identifies missing objects and broken tracks based on object overlaps.	sequence	bounding box, polygon, rotatable bounding box
Object Count - Counts number of objects in the image	image	bounding box, checklist, point, polygon, polyline, radio, rotatable bounding box, skeleton, text
Random Values on Images - Assigns a random value between 0 and 1 to images	image
Random Values on Objects - Assigns a random value between 0 and 1 to objects	image	bounding box, polygon, rotatable bounding box
Red Values - Ranks images by how red the average value of the image is.	image
Sharpness - Ranks images by their sharpness.	image

Area

Ranks images by their area.

Area is computed as the product of image width and image height ($width \times height$).

Implementation on GitHub

Aspect Ratio

Ranks images by their aspect ratio.

Aspect ratio is computed as the ratio of image width to image height ($\frac{width}{height}$).

Implementation on GitHub

Blue Values

Ranks images by how blue the average value of the image is.

Implementation on GitHub

Blur

Ranks images by their blurriness.

Blurriness is computed by applying a Laplacian filter to each image and computing the variance of the output. In short, the score computes "the amount of edges" in each image. Note that this is $1 - \text{sharpness}$.

score = 1 - cv2.Laplacian(image, cv2.CV_64F).var()

Implementation on GitHub

Brightness

Ranks images their brightness.

Brightness is computed as the average (normalized) pixel value across each image.

Implementation on GitHub

Contrast

Ranks images by their contrast.

Contrast is computed as the standard deviation of the pixel values.

Implementation on GitHub

Green Values

Ranks images by how green the average value of the image is.

Implementation on GitHub

Inconsistent Object Classification and Track IDs

This algorithm looks for overlapping objects in consecutive frames that have different classes. Furthermore, if classes are the same for overlapping objects but have different track-ids, they will be flagged as potential inconsistencies in tracks.

Example 1:

      Frame 1                       Frame 2
┌───────────────────┐        ┌───────────────────┐
│                   │        │                   │
│  ┌───────┐        │        │  ┌───────┐        │
│  │       │        │        │  │       │        │
│  │ CAT:1 │        │        │  │ DOG:1 │        │
│  │       │        │        │  │       │        │
│  └───────┘        │        │  └───────┘        │
│                   │        │                   │
└───────────────────┘        └───────────────────┘

Dog:1 will be flagged as potentially wrong class, because it overlaps with CAT:1.

Example 2:

      Frame 1                       Frame 2
┌───────────────────┐        ┌───────────────────┐
│                   │        │                   │
│  ┌───────┐        │        │  ┌───────┐        │
│  │       │        │        │  │       │        │
│  │ CAT:1 │        │        │  │ CAT:2 │        │
│  │       │        │        │  │       │        │
│  └───────┘        │        │  └───────┘        │
│                   │        │                   │
└───────────────────┘        └───────────────────┘

Cat:2 will be flagged as potentially having a broken track, because track ids 1 and 2 doesn't match.

Implementation on GitHub

Missing Objects and Broken Tracks

Identifies missing objects by comparing object overlaps based on a running window.

Case 1: If an intermediate frame (frame $i$) doesn't include an object in the same region, as the two surrounding frames ($i-1$ and $i+1$), it is flagged.

      Frame i-1                     Frame i                      Frame i+1
┌───────────────────┐        ┌───────────────────┐        ┌───────────────────┐
│                   │        │                   │        │                   │
│  ┌───────┐        │        │                   │        │  ┌───────┐        │
│  │       │        │        │                   │        │  │       │        │
│  │ CAT:1 │        │        │                   │        │  │ CAT:1 │        │
│  │       │        │        │                   │        │  │       │        │
│  └───────┘        │        │                   │        │  └───────┘        │
│                   │        │                   │        │                   │
│                   │        │                   │        │                   │
└───────────────────┘        └───────────────────┘        └───────────────────┘

Frame $i$ will be flagged as potentially missing an object.

Case 2: If objects of the same class overlap in three consecutive frames ($i-1$, $i$, and $i+1$) but do not share object hash, they will be flagged as a potentially broken track.

      Frame i-1                     Frame i                      Frame i+1
┌───────────────────┐        ┌───────────────────┐        ┌───────────────────┐
│                   │        │                   │        │                   │
│  ┌───────┐        │        │  ┌───────┐        │        │  ┌───────┐        │
│  │       │        │        │  │       │        │        │  │       │        │
│  │ CAT:1 │        │        │  │ CAT:2 │        │        │  │ CAT:1 │        │
│  │       │        │        │  │       │        │        │  │       │        │
│  └───────┘        │        │  └───────┘        │        │  └───────┘        │
│                   │        │                   │        │                   │
│                   │        │                   │        │                   │
└───────────────────┘        └───────────────────┘        └───────────────────┘

CAT:2 will be marked as potentially having a wrong track id.

Implementation on GitHub

Object Count

Counts number of objects in the image.

Implementation on GitHub

Random Values on Images

Uses a uniform distribution to generate a value between 0 and 1 to each image

Implementation on GitHub

Random Values on Objects

Uses a uniform distribution to generate a value between 0 and 1 to each object

Implementation on GitHub

Red Values

Ranks images by how red the average value of the image is.

Implementation on GitHub

Sharpness

Ranks images by their sharpness.

Sharpness is computed by applying a Laplacian filter to each image and computing the variance of the output. In short, the score computes "the amount of edges" in each image.

score = cv2.Laplacian(image, cv2.CV_64F).var()

Implementation on GitHub