How to visualize cleaning issues¶

Problem statement¶

Exploratory data analysis (EDA) can be overwhelming. There are so many things to check. Duplicates in your dataset, bad/corrupted images in the set, blurred or bright/dark images, the list goes on.

DataEval created a data cleaning class to assist you with your EDA so you can start training your models on high quality data.

When to use¶

The cleaning class should be used during the initial EDA process or if you are trying to verify that you have the right data in your dataset.

What you will need¶

1. A dataset to analyze

2. A Python environment with the following packages installed:

   • dataeval

   • maite-datasets

Getting started¶

Let’s import the required libraries needed to set up a minimal working example

import polars as pl
from maite_datasets.image_classification import CIFAR10

from dataeval import Metadata
from dataeval.config import set_max_processes
from dataeval.quality import Outliers

set_max_processes(4)
_ = pl.Config.set_tbl_rows(-1)

Loading in the data¶

We are going to start by loading in the CIFAR-10 dataset.

The CIFAR-10 dataset contains 60,000 images - 50,000 in the train set and 10,000 in the test set. For the purposes of this demonstration, we are just going to use the test set.

# Load in the CIFAR10 dataset
testing_dataset = CIFAR10("./data", image_set="test", download=True)

# Create the metadata for the dataset
metadata = Metadata(testing_dataset)

Cleaning the dataset¶

Now we can begin finding those images which are significantly different from the rest of the data.

# Initialize the Outliers class
outliers = Outliers()

# Evaluate the data
results = outliers.evaluate(testing_dataset)

# Also evaluate the data classwise
results_classwise = results.classwise(metadata)

The results are a dictionary with the keys being the image that has an issue in one of the listed properties below:

• Brightness

• Blurriness

• Missing

• Zero

• Width

• Height

• Size

• Aspect Ratio

• Channels

• Depth

print(f"Total number of images with an issue: {len(results.aggregate_by_item())}")

Total number of images with an issue: 319

print(f"Total number of images with an issue (classwise): {len(results_classwise.aggregate_by_item())}")

Total number of images with an issue (classwise): 322

# View issues by metric
results.aggregate_by_metric()

# View issues by metric (classwise)
results_classwise.aggregate_by_metric()

# View issues by class
results.aggregate_by_class(metadata)

# View issues by class (classwise)
results_classwise.aggregate_by_class(metadata)

Related concepts¶

• Data Integrity

• Clustering

• Acting on Results

• Dataset Bias and Coverage

See also¶

How-to guides¶

• How to specify custom statistics on object detection datasets

• How to run clustering analysis

• How to add intrinsic factors to Metadata

Tutorials¶

• Introduction to data cleaning

• Detecting common augmentations as duplicates